Research Design
Learn
Research design is the blueprint for conducting scientific investigations. A well-designed study ensures that results are valid, reliable, and can answer the research question. Understanding research design is essential for evaluating scientific claims and conducting your own investigations.
Types of Research Studies
Experimental Studies
The researcher manipulates one or more independent variables and measures the effect on the dependent variable while controlling other factors. This is the only design that can establish cause-and-effect relationships.
Observational Studies
The researcher observes and measures variables without manipulating them. Types include:
- Cross-sectional: Data collected at one point in time
- Longitudinal: Same subjects followed over time
- Case-control: Compares groups with and without an outcome
- Cohort: Follows groups with different exposures over time
| Study Type | Can Establish Causation? | Researcher Control | Best Used For |
|---|---|---|---|
| Experimental | Yes | High | Testing specific hypotheses |
| Quasi-experimental | Limited | Moderate | Real-world interventions |
| Observational | No (correlation only) | Low | Describing patterns, generating hypotheses |
Key Components of Experimental Design
Variables
- Independent Variable (IV): The factor manipulated by the researcher
- Dependent Variable (DV): The outcome measured
- Control Variables: Factors held constant to prevent confounding
- Confounding Variables: Uncontrolled factors that may affect results
Hypothesis
A testable prediction about the relationship between variables.
- Null Hypothesis (H₀): States there is no effect or relationship
- Alternative Hypothesis (H₁): States there is an effect or relationship
Control and Comparison
Control Group
A group that does not receive the experimental treatment, used as a baseline for comparison. Without a control group, we cannot determine if observed effects are due to the treatment.
Placebo and Placebo Effect
A placebo is an inactive treatment (like a sugar pill) given to the control group. The placebo effect occurs when subjects improve simply because they believe they are receiving treatment.
Reducing Bias
| Technique | Description | What It Controls |
|---|---|---|
| Random Assignment | Subjects randomly placed in groups | Selection bias, confounding variables |
| Random Sampling | Subjects randomly selected from population | Sample bias, improves generalizability |
| Single-Blind | Subjects don't know their group | Placebo effect, subject bias |
| Double-Blind | Neither subjects nor researchers know groups | Placebo effect, researcher bias |
| Replication | Study repeated multiple times | Random error, increases reliability |
Sample Size and Statistical Power
Sample Size
The number of subjects in a study. Larger samples:
- Reduce the effect of random variation
- Increase statistical power (ability to detect real effects)
- Provide more precise estimates
- Improve generalizability to the population
The Problem with Small Samples
Small samples are more susceptible to random variation and may produce misleading results. A study with 10 subjects might show a large effect by chance, while a study with 1,000 subjects gives more reliable estimates of the true effect size.
Validity and Reliability
Internal Validity
The degree to which a study accurately establishes a cause-and-effect relationship. Threats include confounding variables, selection bias, and experimental mortality (subjects dropping out).
External Validity
The degree to which results can be generalized to other populations, settings, and times. Threats include unrepresentative samples and artificial laboratory conditions.
Reliability
The consistency of measurements. A reliable study produces similar results when repeated under the same conditions.
Ethical Considerations
Research Ethics Principles
- Informed Consent: Participants must understand and agree to the study
- Minimal Harm: Risks must be minimized and justified by benefits
- Confidentiality: Participant data must be protected
- Right to Withdraw: Participants can leave at any time
- Debriefing: Participants informed of study purpose afterward
ACT Science Connection
The ACT Science section frequently tests your ability to:
- Identify independent, dependent, and control variables
- Evaluate experimental design choices
- Recognize potential sources of error or bias
- Understand the purpose of control groups
- Interpret results in context of the study design
Examples
Example 1: Identifying Variables
Problem: A researcher wants to test whether a new fertilizer increases tomato plant growth. She grows 50 plants with the new fertilizer and 50 plants with standard fertilizer, measuring plant height after 8 weeks. Identify the independent variable, dependent variable, and two control variables.
Solution:
Independent Variable: Type of fertilizer (new vs. standard) - this is what the researcher manipulates.
Dependent Variable: Plant height after 8 weeks - this is what is measured as the outcome.
Control Variables (examples):
- Amount of water given to each plant (should be equal)
- Amount of sunlight (same location or conditions)
- Type of soil used
- Initial plant size/age
- Temperature conditions
These factors must be kept constant so that any difference in height can be attributed to the fertilizer, not other variables.
Example 2: Evaluating Study Design
Problem: A pharmaceutical company tests a new headache medication. They give the medication to 200 people with headaches and find that 70% report relief within one hour. The company claims the medication is highly effective. What is wrong with this study design?
Solution:
This study has several critical flaws:
1. No Control Group: Without a group receiving a placebo, we cannot know if the 70% relief rate is due to the medication or other factors (placebo effect, natural recovery, etc.).
2. No Blinding: Participants knew they were receiving medication, which could inflate the perceived effectiveness (placebo effect).
3. No Random Assignment: We don't know how participants were selected or assigned.
4. No Comparison to Baseline: What is the natural rate of headache relief within one hour without treatment?
Better Design: Randomly assign participants to receive either the medication or an identical-looking placebo, with neither participants nor researchers knowing who received which (double-blind), then compare relief rates between groups.
Example 3: Correlation vs. Causation
Problem: A study finds that students who eat breakfast have higher test scores than students who skip breakfast. A news headline claims "Eating breakfast improves academic performance." Is this claim justified? What alternative explanations exist?
Solution:
The claim is not justified based on this observational study. Correlation does not imply causation.
Alternative Explanations:
1. Reverse Causation: Perhaps higher-performing students are more organized and thus more likely to eat breakfast.
2. Confounding Variables:
- Socioeconomic status: Families with more resources may provide both breakfast and better educational support
- Parental involvement: Parents who ensure breakfast may also help with homework
- Overall health habits: Breakfast eaters may also sleep more, exercise more, etc.
- School attendance: Students who eat breakfast may be less likely to be tardy or absent
To Establish Causation: Would need a randomized controlled experiment where students are randomly assigned to eat or skip breakfast, with all other factors controlled.
Example 4: Sample Selection
Problem: A researcher wants to study the average sleep duration of American teenagers. She surveys 500 students at her local high school. What are the limitations of this sample? How could the study be improved?
Solution:
Limitations:
1. Lack of Random Sampling: Students from one school may not represent all American teenagers.
2. Geographic Bias: The school's location (urban/rural, time zone, climate) may affect sleep patterns.
3. Socioeconomic Bias: School demographics may not match national demographics.
4. Selection Bias: If participation was voluntary, students with certain sleep habits may be more/less likely to respond.
5. Limited External Validity: Results cannot be confidently generalized to all American teenagers.
Improvements:
- Use stratified random sampling from schools across different regions
- Include students from various socioeconomic backgrounds
- Account for school schedules (start times vary significantly)
- Use objective measures (sleep tracking) rather than self-report
- Collect data during different seasons (sleep varies by daylight)
Example 5: Designing an Experiment
Problem: Design an experiment to test whether background music affects concentration during studying. Include your hypothesis, variables, groups, and methods to reduce bias.
Solution:
Hypothesis: Students who study with background music will perform differently on a comprehension test compared to students who study in silence.
Variables:
- Independent Variable: Presence of background music (music vs. silence)
- Dependent Variable: Score on comprehension test
- Control Variables: Study material, time allowed, room temperature, lighting, time of day, difficulty of test
Groups:
- Experimental Group: Studies with instrumental background music
- Control Group: Studies in silence
Procedure:
- Recruit 100 participants from the same population
- Randomly assign 50 to each group
- All participants study the same material for the same duration
- Administer identical comprehension tests
- Compare mean scores between groups
Bias Reduction:
- Random assignment ensures groups are equivalent
- Standardized materials and conditions control confounding variables
- Objective test scoring (multiple choice) reduces researcher bias
- Participants unaware of hypothesis reduces demand characteristics
Practice
1. In an experiment testing whether caffeine improves reaction time, what is the dependent variable?
A) Amount of caffeine consumed B) Reaction time C) Age of participants D) Time of day
2. A study finds that people who own dogs live longer than those who don't. Which conclusion is most appropriate?
A) Dog ownership causes longer life B) Longer-lived people prefer dogs C) There is a correlation between dog ownership and longevity D) The study proves dogs improve health
3. In a double-blind drug trial, who does NOT know which treatment each participant receives?
A) Only the participants B) Only the researchers C) Both participants and researchers administering/evaluating D) The pharmaceutical company
4. A researcher studies the effect of sleep on memory by having participants sleep either 4 hours or 8 hours, then take a memory test. Which is NOT a control variable?
A) The memory test used B) Hours of sleep C) Time of day the test is given D) Testing environment
5. Why is random assignment important in experiments?
A) It makes the study more interesting B) It ensures groups are equivalent, reducing confounding variables C) It increases sample size D) It makes results statistically significant
6. A study with 10 participants found that a supplement doubled muscle growth. A study with 500 participants found only a 5% increase. Which result is more reliable and why?
A) The 10-person study, because the effect was larger B) The 500-person study, because larger samples reduce random variation C) Both are equally reliable D) Cannot be determined without more information
7. A researcher wants to know if a teaching method improves test scores. She uses the new method with her morning class and the old method with her afternoon class. What is the main problem?
A) The sample is too small B) There is no control group C) Time of day is a confounding variable D) The study lacks a hypothesis
8. Which study design can establish a cause-and-effect relationship?
A) Cross-sectional survey B) Longitudinal observation C) Randomized controlled experiment D) Case study
9. The placebo effect refers to:
A) When researchers accidentally bias results B) When participants improve because they believe they're receiving treatment C) When medication has unexpected side effects D) When the control group performs better than expected
10. A researcher surveys college students about study habits and finds that students who study in groups report higher satisfaction. The study's external validity is threatened because:
A) There was no control group B) The results may not generalize to non-college populations C) Satisfaction cannot be measured D) Group studying causes satisfaction
Click to reveal answers
- B) Reaction time - The dependent variable is what is measured as the outcome. Caffeine amount is the independent variable being manipulated.
- C) There is a correlation between dog ownership and longevity - Observational studies can only establish correlation, not causation. Many confounding variables could explain this relationship.
- C) Both participants and researchers administering/evaluating - In a double-blind study, neither the participants nor the researchers who interact with them or evaluate outcomes know group assignments.
- B) Hours of sleep - Hours of sleep is the independent variable (what is manipulated), not a control variable. Control variables are held constant across groups.
- B) It ensures groups are equivalent, reducing confounding variables - Random assignment distributes individual differences and potential confounding variables equally across groups.
- B) The 500-person study, because larger samples reduce random variation - Small samples are more susceptible to random variation and extreme results. Large samples provide more reliable estimates.
- C) Time of day is a confounding variable - Students' alertness and performance may differ between morning and afternoon, confounding the effect of teaching method.
- C) Randomized controlled experiment - Only experiments with random assignment and manipulation of variables can establish causation.
- B) When participants improve because they believe they're receiving treatment - The placebo effect is a psychological phenomenon where belief in treatment produces real improvements.
- B) The results may not generalize to non-college populations - External validity concerns whether results apply to other groups. College students may differ from the general population in study habits and satisfaction.
Check Your Understanding
1. Explain why "correlation does not imply causation" and give an example of a correlation that is clearly not causal.
Show answer
Correlation means two variables are related - when one changes, the other tends to change. However, this relationship doesn't prove that one causes the other. There could be a third variable causing both (confounding), the causal direction could be reversed, or the correlation could be coincidental. Example: Ice cream sales and drowning deaths are positively correlated (both increase in summer), but ice cream doesn't cause drowning. The confounding variable is temperature - hot weather increases both ice cream consumption and swimming, which increases drowning risk. Another example: Shoe size correlates with reading ability in children, but bigger feet don't cause better reading - age is the confounding variable that affects both.
2. Why are control groups necessary in experiments? What happens if we don't include one?
Show answer
Control groups provide a baseline for comparison, allowing us to determine whether observed effects are due to the treatment or other factors. Without a control group, we cannot distinguish between: (1) Effects of the treatment, (2) Natural improvement or change over time, (3) Placebo effects from expecting improvement, (4) Effects of simply being in a study (Hawthorne effect), (5) Random variation. For example, if we give a new cold medicine to 100 people and 60% recover within a week, this seems effective - but without a control group, we don't know that about 60% of colds resolve on their own within a week anyway. The control group reveals the baseline rate, allowing us to see if the treatment provides any benefit beyond what would happen naturally.
3. What is the difference between random sampling and random assignment? Why is each important?
Show answer
Random Sampling is selecting participants from a population so that every member has an equal chance of being chosen. It affects external validity - how well results generalize to the broader population. Without random sampling, the sample may not represent the population (e.g., surveying only people at a gym about exercise habits).
Random Assignment is placing participants into experimental groups by chance after they've been selected. It affects internal validity - how confidently we can attribute effects to the treatment. Without random assignment, groups may differ systematically (e.g., if volunteers choose their group, motivated people might choose the treatment group).
A study can have one without the other: A study could randomly sample from the population but not randomly assign to groups (observational), or randomly assign but use a convenience sample (limits generalizability but can still show causation within that sample).
4. How does sample size affect the reliability and validity of research findings? What are the consequences of using too small a sample?
Show answer
Effects of Sample Size:
Statistical Power: Larger samples increase the ability to detect real effects. Small samples may miss true effects (false negatives) or be fooled by random variation into showing effects that don't exist (false positives).
Precision: Larger samples provide more precise estimates with narrower confidence intervals. A mean from 1,000 people is more trustworthy than from 10 people.
Representativeness: Larger samples are more likely to include the diversity present in the population.
Consequences of Small Samples:
- Results heavily influenced by outliers or unusual participants
- Effect sizes appear larger than they really are (due to publication bias for significant results)
- Findings often fail to replicate
- Wide confidence intervals make conclusions uncertain
- May lead to incorrect conclusions that waste resources or cause harm
This is why replication with larger samples is essential before accepting research findings as reliable.
🚀 Next Steps
- Review any concepts that felt challenging
- Move on to the next lesson when ready
- Return to practice problems periodically for review