- Document the estimand, causal assumptions and adjustment rationale.
Define a meaningful contrast
Specify the intervention, population, comparison and outcome time. An association between attendance and grades differs from the effect of a defined attendance program. Without a clear contrast, a claim that X affects Y is underspecified.
Consider time and variable roles
Record when exposure, covariates and outcomes occur. Confounders, mediators and selection variables have different roles. Adjusting for every available variable may introduce bias; justify the set using subject knowledge and explicit assumptions.
Make assumptions reviewable
Causal diagrams can expose assumptions and bias paths but cannot establish that those assumptions are true. Appropriate methods require attention to exchangeability, intervention consistency and comparison support. Some identifying assumptions cannot be verified from the observed dataset alone.
Design matters more than a coefficient
Randomization helps control some confounding when implemented appropriately; attrition, adherence and measurement still matter. Observational inference needs a defensible design and assumptions. Significance or predictive accuracy alone is insufficient for a causal claim.
Report limitations and sensitivity
Describe the target effect, identification strategy, adjustment choices and sensitivity to assumptions. Use association language when the design supports only association. Educational guidance cannot substitute for specialist review of the actual study.
Practical research checklist
- Define intervention, population and timing.
- Justify covariate roles.
- Disclose identification assumptions and bias.
- Match claims to the design.
Worked case and implementation decisions
The following is a fictional teaching case. Do not use its numbers or wording as actual study findings.
Prior motivation may affect both support use and later participation in an observational example, so their association need not be the support effect. A post-treatment variable may be a mediator, and adjusting for it can change the estimand. Select adjustment using causal assumptions and measurement timing rather than each covariate’s p-value. Define the intervention, population and time, and explain exchangeability, overlap and consistency. Data alone cannot fully prove these assumptions.
| Stage | Teaching example | Verification question |
|---|---|---|
| Target | Effect of a defined intervention | Distinct from descriptive association? |
| Timing | Prior motivation and later outcome | Is temporal order credible? |
| Adjustment | Confounders under a causal model | Are mediators/colliders handled deliberately? |
| Coverage | Comparable groups | Are overlap and selection assessed? |
| Sensitivity | Unmeasured confounding | Are claims explicitly limited? |
Exercise output: Document the estimand, causal assumptions and adjustment rationale.
Sources and further reading
Official sources for verification and further reading
- Hernán & Robins — Causal Inference: What If (author-hosted book, 2024 version) Source verification: 2026-10-06
- CASP — Critical appraisal checklists