The DOI matches its Crossref record; 26 of the 26 references it cites were found in a registry or an index.
Effect Sizes in Cluster-Randomized Designs
Abstract
Multisite research designs involving cluster randomization are becoming increasingly important in educational and behavioral research. Researchers would like to compute effect size indexes based on the standardized mean difference to compare the results of cluster-randomized studies (and corresponding quasi-experiments) with other studies and to combine information across studies in meta-analyses. This article addresses the problem of defining effect sizes in multilevel designs and computing estimates of those effect sizes and their standard errors from information that is likely to be reported in journal articles. Three effect sizes are defined corresponding to different standardizations. Estimators of each effect size index are also presented along with their sampling distributions (including standard errors).
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Distribution Theory for Glass's Estimator of Effect Size and Related Estimators
Hedges, L. V. (1981) · Journal of Educational Statistics · doi:10.2307/1164588 · Cited by 5,695
Self-citationSearle, S. R., & Searle, S. R. (1971) · Cited by 3,229
Gill, J. (2005) · Encyclopedia of Social Measurement · doi:10.1016/b0-12-369398-5/00166-3 · Cited by 3,114
Searle, S. R., Casella, G., & McCulloch, C. E. (1992) · Wiley Series in Probability and Statistics · doi:10.1002/9780470316856 · Cited by 1,926
Distributions in Statistics: Continuous Multivariate Distributions.
Fisk, P. R., Johnson, N. L., & Kotz, S. (1973) · Journal of the Royal Statistical Society. Series A (General) · doi:10.2307/2344756 · Cited by 1,543
Statistical methods: Applied to experiments in agriculture and biology (4th ed.).
Snedecor, G. W. (1950) · Iowa State University Press eBooks · doi:10.1037/13268-000 · Cited by 1,523
Box, G. E. P. (1954) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177728786 · Cited by 1,513
Design and Analysis of Cluster Randomization Trials in Health Research
DONNER, A. P., & Klar, N. S. (2000) · Cited by 1,236
An Extension of Box's Results on the Use of the $F$ Distribution in Multivariate Analysis
Geisser, S., & Greenhouse, S. W. (1958) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177706545 · Cited by 1,098
Distributions in Statistics: Continuous Univariate Distributions
Buckland, W. R., Johnson, N. L., & Kotz, S. (1971) · Revue de l'Institut International de Statistique / Review of the International Statistical Institute · doi:10.2307/1402281 · Cited by 1,008
Cited by
Interpreting effect sizes of education interventions
Kraft, M. A. (2020) · Educational Researcher · doi:10.3102/0013189x20912798 · Cited by 1,281
Digital Games, Design, and Learning: A Systematic Review and Meta-Analysis
Clark, D. B., Tanner-Smith, E. E., & Killingsworth, S. S. (2016) · Review of Educational Research · doi:10.3102/0034654315582065 · Cited by 1,274
Bradshaw, C. P., Mitchell, M. M., & Leaf, P. J. (2010) · Journal of Positive Behavior Interventions · doi:10.1177/1098300709334798 · Cited by 868
Intraclass Correlation Values for Planning Group-Randomized Trials in Education
Hedges, L. V., & Hedberg, E. C. (2007) · Educational Evaluation and Policy Analysis · doi:10.3102/0162373707299706 · Cited by 847
Self-citationSchool-based programs to reduce bullying and victimization: (528362010-001)
Farrington, D. P., & Ttofi, M. M. (2009) · PsycEXTRA Dataset · doi:10.1037/e528362010-001 · Cited by 667
Korpershoek, H., Harms, T., de Boer, H., van Kuijk, M., & Doolaard, S. (2016) · Review of Educational Research · doi:10.3102/0034654315626799 · Cited by 633
“In these cases, we recomputed the class/school-level effect sizes by multiplying them by the square root of the intraclass correlation, as Hedges (2007) prescribes.”
MethodMethodological guidance paper: High-quality meta- analysis in a systematic review
Pigott, T. D., & Polanin, J. R. (2020) · Review of Educational Research · doi:10.3102/0034654319877153 · Cited by 568
“Though Hedges and Pigott (2004) discuss power analysis for effect size models, research has not been conducted on power for moderator models with”
BackgroundMethodWriting to Read: A Meta-Analysis of the Impact of Writing and Writing Instruction on Reading
Graham, S., & Hebert, M. (2011) · Harvard Educational Review · doi:10.17763/haer.81.4.t2k0m13756113566 · Cited by 563
Han, S., Capraro, R., & Capraro, M. M. (2015) · International Journal of Science and Mathematics Education · doi:10.1007/s10763-014-9526-0 · Cited by 496
Bradshaw, C. P., Koth, C. W., Thornton, L. A., & Leaf, P. J. (2009) · Prevention Science · doi:10.1007/s11121-008-0114-9 · Cited by 448
Does stereotype threat influence performance of girls in stereotyped domains? A meta-analysis
Flore, P. C., & Wicherts, J. M. (2015) · Journal of School Psychology · doi:10.1016/j.jsp.2014.10.002 · Cited by 410
(2012) · TigerPrints (Clemson University) · Cited by 402
Bradshaw, C. P., Koth, C. W., Bevans, K. B., Ialongo, N., & Leaf, P. J. (2008) · School Psychology Quarterly · doi:10.1037/a0012883 · Cited by 389
Graham, S., & Santangelo, T. (2014) · Reading and Writing · doi:10.1007/s11145-014-9517-0 · Cited by 368
Testing for funnel plot asymmetry of standardized mean differences
Pustejovsky, J. E., & Rodgers, M. A. (2019) · Research Synthesis Methods · doi:10.1002/jrsm.1332 · Cited by 354
Formative Assessment and Writing: A Meta-Analysis
Graham, S., Hebert, M., & Harris, K. R. (2015) · The Elementary School Journal · doi:10.1086/681947 · Cited by 340
A standardized mean difference effect size for single case designs
Hedges, L. V., Pustejovsky, J. E., & Shadish, W. R. (2012) · Research Synthesis Methods · doi:10.1002/jrsm.1052 · Cited by 305
Self-citationThe impact of shared book reading on children's language skills: A meta-analysis
Noble, C., Sala, G., Peter, M., Lingwood, J., Rowland, C., Gobet, F., & Pine, J. (2019) · Educational Research Review · doi:10.1016/j.edurev.2019.100290 · Cited by 299
Moving From the Lab to the Field: The Role of Fidelity and Achieved Relative Intervention Strength
Hulleman, C. S., & Cordray, D. S. (2009) · Journal of Research on Educational Effectiveness · doi:10.1080/19345740802539325 · Cited by 291
“Indices of the strength of the intervention were dose and participant responsiveness.2 1This effect size is calculated using Hedges’s g and corrected for clustering in the classroom (Hedges, 2007).”
MethodInfluentialThe Effects of Writing on Learning in Science, Social Studies, and Mathematics: A Meta-Analysis
Graham, S., Kiuhara, S. A., & MacKay, M. (2020) · Review of Educational Research · doi:10.3102/0034654320914744 · Cited by 292
De La Rue, L., Polanin, J. R., Espelage, D. L., & Pigott, T. D. (2017) · Review of Educational Research · doi:10.3102/0034654316632061 · Cited by 292
Dietrichson, J., Bøg, M., Filges, T., & Klint Jørgensen, A.-M. (2017) · Review of Educational Research · doi:10.3102/0034654316687036 · Cited by 288
Teacher classroom management practices: effects on disruptive or aggressive student behavior
Oliver, R. M., Wehby, J. H., & Reschly, D. J. (2011) · Campbell Systematic Reviews · doi:10.4073/csr.2011.4 · Cited by 286
Nonindependence and sensitivity analyses in ecological and evolutionary meta‐analyses
Noble, D. W. A., Lagisz, M., O'dea, R. E., & Nakagawa, S. (2017) · Molecular Ecology · doi:10.1111/mec.14031 · Cited by 267
Conn, K. M. (2017) · Review of Educational Research · doi:10.3102/0034654317712025 · Cited by 263
Sikander, S., Ahmad, I., Atif, N., Zaidi, A., Vanobberghen, F., Weiss, H. A., Nisar, A., Tabana, H., Ain, Q. U., Bibi, A., et al. (2019) · The Lancet Psychiatry · doi:10.1016/s2215-0366(18)30467-x · Cited by 256
Mullender-Wijnsma, M. J., Hartman, E., de Greeff, J. W., Doolaard, S., Bosker, R. J., & Visscher, C. (2016) · Pediatrics · doi:10.1542/peds.2015-2743 · Cited by 187
Wilson, S. J., Tanner-Smith, E. E., Lipsey, M. W., Steinka-Fry, K., & Morrison, J. (2011) · Campbell Systematic Reviews: Better Evidence for a Better World · doi:10.4073/csr.2011.8 · Cited by 173
Prado, E. L., Larson, L. M., Cox, K., Bettencourt, K., Kubes, J. N., & Shankar, A. H. (2019) · The Lancet Global Health · doi:10.1016/s2214-109x(19)30361-4 · Cited by 173
Vitiello, M. V., McCurry, S. M., Shortreed, S. M., Balderson, B. H., Baker, L. D., Keefe, F. J., Rybarczyk, B. D., & Von Korff, M. (2013) · Journal of the American Geriatrics Society · doi:10.1111/jgs.12275 · Cited by 165
“correlations (ICCs) calculated using postintervention assessment data are reported.(42) Baseline and analysis of session surveys were performed in SAS 9.”
MethodHagelskamp, C., Brackett, M. A., Rivers, S. E., & Salovey, P. (2013) · American Journal of Community Psychology · doi:10.1007/s10464-013-9570-x · Cited by 160
Didion, L., Toste, J. R., & Filderman, M. J. (2020) · Journal of Research on Educational Effectiveness · doi:10.1080/19345747.2019.1670884 · Cited by 151
de Boer, H., Timmermans, A. C., & van der Werf, M. P. C. (2018) · Educational Research and Evaluation · doi:10.1080/13803611.2018.1550834 · Cited by 144
NIH peer review: Criterion scores completely account for racial disparities in overall impact scores
Erosheva, E. A., Grant, S., Chen, M.-C., Lindner, M. D., Nakamura, R. K., & Lee, C. J. (2020) · Science Advances · doi:10.1126/sciadv.aaz4868 · Cited by 125
What Are Effect Sizes and Why Do We Need Them?
Hedges, L. V. (2008) · Child Development Perspectives · doi:10.1111/j.1750-8606.2008.00060.x · Cited by 123
Self-citationLotzke, H., Brisby, H., Gutke, A., Hägg, O., Jakobsson, M., Smeets, R., & Lundberg, M. (2019) · Physical Therapy · doi:10.1093/ptj/pzz020 · Cited by 112
“Hedges g for between-group effect sizes of the primary and secondary outcomes at each time point was computed using SEs from the mixed-model output to derive pooled and weighted SDs. 57 Effect sizes (ESs) were categorized as small ( d ≈ 0.20 to 0.50), medium ( d ≥ 0.50 to 0.80), and large ( d ≥ 0.80) 58 and presented…”
InfluentialAhlen, J., Lenhard, F., & Ghaderi, A. (2015) · The Journal of Primary Prevention · doi:10.1007/s10935-015-0405-4 · Cited by 95
School-based interventions for reducing disciplinary school exclusion: a systematic review
Valdebenito, S., Eisner, M., Farrington, D. P., Ttofi, M. M., & Sutherland, A. (2018) · Campbell Systematic Reviews: Better Evidence for a Better World · doi:10.4073/csr.2018.1 · Cited by 93
Wei, P., Wang, X., & Dong, H. (2023) · Frontiers in Psychology · doi:10.3389/fpsyg.2023.1249991 · Cited by 89
Voluntary work for the physical and mental health of older volunteers: A systematic review
Filges, T., Siren, A., Fridberg, T., & Nielsen, B. C. V. (2020) · Campbell Systematic Reviews: Better Evidence for a Better World · doi:10.1002/cl2.1124 · Cited by 79
Wilson, S. J., & Tanner-Smith, E. E. (2013) · Journal of the Society for Social Work and Research · doi:10.5243/jsswr.2013.22 · Cited by 66
McCurry, S. M., Shortreed, S. M., Von Korff, M., Balderson, B. H., Baker, L. D., Rybarczyk, B. D., & Vitiello, M. V. (2014) · Sleep · doi:10.5665/sleep.3402 · Cited by 60
Cole (2014) · doi:10.1177/1086296x14552179 · Cited by 57
Dietrichson, J., Filges, T., Seerup, J. K., Klokker, R. H., Viinholt, B. C. A., Bøg, M., & Eiberg, M. (2021) · Campbell Systematic Reviews: Better Evidence for a Better World · doi:10.1002/cl2.1152 · Cited by 54
Imaging, Behavior and Endocrine Analysis of “Jealousy” in a Monogamous Primate
Maninger, N., Mendoza, S. P., Williams, D. R., Mason, W. A., Cherry, S. R., Rowland, D. J., Schaefer, T., & Bales, K. L. (2017) · Frontiers in Ecology and Evolution · doi:10.3389/fevo.2017.00119 · Cited by 51
Salazar de Pablo, G., De Micheli, A., Solmi, M., Oliver, D., Catalan, A., Verdino, V., Di Maggio, L., Bonoldi, I., Radua, J., Baccaredda Boy, O., et al. (2021) · Harvard Review of Psychiatry · doi:10.1097/hrp.0000000000000294 · Cited by 51
Garrett, R., Citkowicz, M., & Williams, R. (2019) · Review of Research in Education · doi:10.3102/0091732x19830634 · Cited by 48
Wolf, B., & Harbatkin, E. (2023) · Journal of Research on Educational Effectiveness · doi:10.1080/19345747.2022.2071364 · Cited by 45
“The WWC uses the standard deviation, r̂T , which includes both withinand between-cluster variation for cluster studies (Hedges, 2007).”
MethodEffects of a data-based decision making intervention on student achievement
Staman, L. (., Timmermans, A. C. (., & Visscher, A. J. (. (2017) · Studies in Educational Evaluation · doi:10.1016/j.stueduc.2017.07.002 · Cited by 38
Dietrichson, J., Filges, T., Klokker, R. H., Viinholt, B. C. A., Bøg, M., & Jensen, U. H. (2020) · Campbell Systematic Reviews: Better Evidence for a Better World · doi:10.1002/cl2.1081 · Cited by 33
García-Escalera, J., Chorot, P., Sandín, B., Ehrenreich-May, J., Prieto, A., & Valiente, R. M. (2019) · Child & Youth Care Forum · doi:10.1007/s10566-018-9471-0 · Cited by 29
Proctor, B. J., Moghaddam, N. G., Evangelou, N., & das Nair, R. (2018) · Journal of Contextual Behavioral Science · doi:10.1016/j.jcbs.2018.07.006 · Cited by 26
“To produce standardised estimates of effect for the linear mixed models, we applied Cohen's d calculation to the relevant data for estimated marginal means and their pooled standard deviation (Hedges, 2007); we additionally computed 95% confidence intervals around point-estimates of effect-size.”
MethodVan Dyke, B. P., Newman, A. K., Moraís, C. A., Burns, J. W., Eyer, J. C., & Thorn, B. E. (2019) · The Journal of Pain · doi:10.1016/j.jpain.2019.04.006 · Cited by 25
Amiri, S., Mahmood, N., Junaidi, S., & Khan, M. A. (2024) · Journal of Education and Health Promotion · doi:10.4103/jehp.jehp_1156_23 · Cited by 21
A meta-analysis of the impact of COVID-19 on student achievement
Storey, N., & Zhang, Q. (2024) · Educational Research Review · doi:10.1016/j.edurev.2024.100624 · Cited by 18
Educational and wellbeing outcomes of an anxiety and depression prevention program for adolescents
García-Escalera, J., Valiente, R. M., Sandín, B., Ehrenreich-May, J., & Chorot, P. (2020) · Revista de Psicodidáctica (English ed.) · doi:10.1016/j.psicoe.2020.05.003 · Cited by 7
Hooker, A. R., Sagui-Henson, S. J., Daubenmier, J., Moran, P. J., Hartogensis, W., Acree, M., Kristeller, J., Epel, E. S., Mason, A. E., & Hecht, F. M. (2022) · Mindfulness · doi:10.1007/s12671-022-01951-2 · Cited by 6
“Effect sizes were quantified as Cohen’s d using standard deviations derived from our mixed model output (Hedges, 2007).”
MethodRubel, J., Quest, J., Pruessner, L., Timm, C., Hartmann, S., Barnow, S., Rittmeyer, L., Rosenbaum, D., & Lalk, C. (2023) · JMIR Research Protocols · doi:10.2196/41440 · Cited by 2
“Hedges [50] and Westfall et al [51] propose an effect size based on the Cohen d, which is also used for power analysis [47]:”
MethodPredicting emotional valence in autism: a preregistered study in the Bayesian Brain framework
Plank, I. S., Pior, A., Yurova, A., Nowak, J., Shi, Z., & Falter-Wagner, C. M. (2026) · Molecular Autism · doi:10.1186/s13229-026-00730-3 · Cited by 2
“We computed effect sizes δ by dividing the effect by the square root of the sum of all squared variances of the respective model [50].”
Metadata
- Authors
- Hedges, L. V.
- Venue
- Journal of Educational and Behavioral Statistics
- Published
- 2007-12
- Volume
- 32
- Issue
- 4
- Pages
- 341-370
- ISSN
- 1076-9986 (ISSN-L)
- 1935-1054
- OpenAlex
- W2132744755
- Cited by
- 567
- References
- 26
- FWCI
- 10.59
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.