Study design and its trade-offs
In this lesson
By the end, you’ll be able to
- Match a question to a study design and know what each design can and cannot support
- Identify confounding, selection bias, and reverse causation
- Use a causal diagram to decide what to control for, and what never to
Sign in to save your progress
You can keep reading without an account, but completed lessons won’t be saved.
The design of a study fixes the ceiling on what it can claim, before a single data point is collected. No amount of clever analysis rescues a design that cannot answer the question. So the first act of doing research is choosing the instrument, and knowing, honestly, what that instrument can and cannot see.
The first fork is experimental vs observational: does the researcher assign the exposure, or only watch? In a randomized controlled trial, random assignment balances confounders, known and unknown, so it can support causal claims. Observational designs watch: a cohort follows exposed and unexposed groups forward; a case-control starts from the outcome and looks back at exposures (efficient for rare outcomes); a cross-sectional study takes a single snapshot. Each is right for some questions and cannot support the claims of the others. A cross-sectional survey can show an association; it cannot establish that one thing came first, let alone caused the other.
Categorize
Match each study to its design.
RCT = researcher randomizes. Cohort = follow forward by exposure. Case-control = start from outcome, look back. Cross-sectional = one snapshot.
Three threats explain most wrong observational conclusions. Confounding: a third variable causes both the exposure and the outcome, manufacturing an association that isn't causal. Selection bias: who ends up in the sample distorts the result; a survey of a runners' forum tells you little about everyone else. Reverse causation: the outcome may be driving the exposure, not the other way around (anxious people may seek the app, not be harmed by it). Naming which threat is in play is half of appraising a study.
Select all that apply
Towns that sell more ice cream report more drownings. Select every variable that is a plausible confounder of this association.
A confounder is a common cause of BOTH ice-cream sales and drowning. What raises both at once?
A causal diagram (a DAG) is a picture of your assumptions about what causes what, and it is a thinking tool, not decoration. Once you draw it, it tells you what to adjust for: block the confounding paths, leave the mediators alone, and never condition on a collider (a common effect of two things), because doing so opens a spurious path and can create an association from nothing. Controlling for 'everything you measured' is not caution; it is how careful people introduce bias.
Signature interactive
DAG Builder
You want the effect of hours on a coding-practice app (X) on exam score (Y). Here is the causal story. Choose what to adjust for, then run the estimate and watch what each choice does.
A stylized simulation: 900 samples, effect estimated by ordinary least squares. Constructed example.
Written response
Lab (constructed study). Researchers surveyed 2,000 adults once, recording daily coffee intake and self-reported anxiety, and found heavier coffee drinkers reported more anxiety. Name the design, state the strongest claim it can support, and one claim it cannot, and say why.
One time point + survey = which design? Then: association yes; causation why not?
0/50 words
Checkpoint · item 1 of 5
A study finds people who use mental-health apps report worse anxiety. Before concluding the apps cause anxiety, what should you consider?
Could the outcome be causing the exposure?
Reflection