When is the Story in the Subgroups?: Strategies for Interpreting and Reporting Intervention Effects for Subgroups
The DOI matches its Crossref record; 11 of the 12 references it cites were found in a registry or an index.
Abstract
This paper examines strategies for interpreting and reporting estimates of intervention effects for subgroups of a study sample. The paper considers: why and how subgroup findings are important for applied research, alternative ways to define subgroups, different research questions that motivate subgroup analyses, the importance of pre-specifying subgroups before analyses are conducted, the importance of using existing theory and prior research to distinguish between subgroups for whom study findings are confirmatory (hypothesis testing) as opposed to exploratory (hypothesis generating), and the conditions under which study findings should be considered confirmatory. Each issue is illustrated by selected empirical examples.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Historically, the scientific methods used to answer questions about the heterogeneity of intervention effects have been underdeveloped and underused21,24,25.”
From A national experiment reveals where a growth mindset improves achievement (2019) · cites this paper as [24]
Citations
Cites
Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing
Benjamini, Y., & Hochberg, Y. (1995) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1995.tb02031.x · Cited by 109,482
Journal of the Royal Statistical Society.
(1923) · The Economic Journal · doi:10.1093/ej/33.130.269 · Cited by 3,219
Antidepressant Drug Effects and Depression Severity: A Patient-Level Meta-analysis
Fournier, J. C., DeRubeis, R. J., Hollon, S. D., Dimidjian, S., Amsterdam, J. D., Shelton, R. C., & Fawcett, J. (2010) · JAMA · doi:10.1001/jama.2009.1943 · Cited by 2,009
Wells, K. B., Sherbourne, C., Schoenbaum, M., Duan, N., Meredith, L., Unützer, J., Miranda, J., Carney, M. F., & Rubenstein, L. V. (2000) · JAMA · doi:10.1001/jama.283.2.212 · Cited by 1,021
Subgroup analysis in randomised controlled trials: importance, indications, and interpretation
Rothwell, P. M. (2005) · The Lancet · doi:10.1016/s0140-6736(05)17709-5 · Cited by 942
Wells, K., Sherbourne, C., Schoenbaum, M., Ettner, S., Duan, N., Miranda, J., Unützer, J., & Rubenstein, L. (2004) · Archives of General Psychiatry · doi:10.1001/archpsyc.61.4.378 · Cited by 225
Technical Methods Report: Guidelines for Multiple Testing in Impact Evaluations. NCEE 2008-4018.
Schochet, P. Z. (2008) · National Center for Education Evaluation and Regional Assistance · Cited by 97
Bloom, H. S., Thompson, S. L., & Unterman, R. (2010) · SSRN Electronic Journal · Cited by 63
Self-citationRedcross, C., Bloom, D., Azurdia, G., Zweig, J. M., & Pindus, N. M. (2009) · SSRN Electronic Journal · Cited by 62
Redcross, C., Zweig, J. M., & Azurdia, G. (2007) · Issue Lab (Candid) · Cited by 50
Research works printed but not linked: 4
- Council, N. R. (2007). Crime, parole, desistance from crime, and community integration. Committee on Community Supervision and Desistance from Crime. Washington, DC: The National Academies Press.
- Kim, S., Leblanc, A., & Michalopoulos, C. (2009). Working toward Wellness: Early results from a telephonic care management program for medicaid recipients with depression. New York: MDRC.
- Kim, S., Leblanc, A., Simon, G., Walter, J., & Michalopoulos, C. (2010). Working toward Wellness: Eighteen-month results from a telephonic care management program for medicaid recipients with depression. New York: MDRC.
- Michalopoulos, C., & Schwartz, C. (2000). What works best for whom: Impacts of 20 welfare-to-work programs by subgroup. Washington, DC: U.S. Department of Health and Human Services, Office of the Assistant Secretary for Planning and Evaluation and Administration for Children and Families, and U.S. Department of Education.
Cited by
Palinkas, L. A., Horwitz, S. M., Green, C. A., Wisdom, J. P., Duan, N., & Hoagwood, K. (2015) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-013-0528-y · Cited by 12,038
A national experiment reveals where a growth mindset improves achievement
Yeager, D. S., Hanselman, P., Walton, G. M., Murray, J. S., Crosnoe, R., Muller, C., Tipton, E., Schneider, B., Hulleman, C. S., Hinojosa, C. P., et al. (2019) · Nature · doi:10.1038/s41586-019-1466-y · Cited by 1,513
Walton, G. M., & Yeager, D. S. (2020) · Current Directions in Psychological Science · doi:10.1177/0963721420904453 · Cited by 298
Greenberg, M. T., & Abenavoli, R. (2017) · Journal of Research on Educational Effectiveness · doi:10.1080/19345747.2016.1246632 · Cited by 255
“Because of these concerns, it has been suggested that researchers clearly differentiate between exploratory versus confirmatory analyses when conducting and reporting subgroup analyses in RCTs (Bloom & Michalopoulos, 2013; Rothwell, 2005).”
BackgroundRivers, S. E., Brackett, M. A., Reyes, M. R., Elbertson, N. A., & Salovey, P. (2013) · Prevention Science · doi:10.1007/s11121-012-0305-2 · Cited by 200
Metadata
- Authors
- Bloom, H. S., & Michalopoulos, C.
- Affiliations
- Manpower Demonstration Research Corporation (US)
- Corresponding author
- Bloom, Howard S.
- Venue
- Prevention Science
- Published
- 2011-01-29
- Volume
- 14
- Issue
- 2
- Pages
- 179-188
- ISSN
- 1389-4986 (ISSN-L)
- 1573-6695
- OpenAlex
- W2152154155
- Cited by
- 83
- References
- 12
- Citations by year
- 2026: 6 · 2025: 5 · 2024: 8 · 2023: 4 · 2022: 4 · 2021: 3 · 2020: 5 · 2019: 6 · 2018: 4 · 2017: 2 · 2016: 8 · 2015: 4 · 2014: 10 · 2013: 7 · 2012: 6
- FWCI
- 6.74
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- www.springer.com/tdm (tdm)
- Locations
- Prevention Science (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Keywords
- subgroup analysis, intervention effects, hypothesis testing, applied research
- MeSH terms
- Data Interpretation, Statistical
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: When is the Story in the Subgroups?: Strategies for Interpreting and Reporting Intervention Effects for Subgroups. https://app.cochat.ai/papers/10.1007/s11121-010-0198-x
- The paper
- Bloom, H. S., & Michalopoulos, C. (2013). When is the Story in the Subgroups?: Strategies for Interpreting and Reporting Intervention Effects for Subgroups. Prevention Science. https://doi.org/10.1007/s11121-010-0198-x
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.