Do Published Studies Yield Larger Effect Sizes than Unpublished Studies in Education and Special Education? A Meta-review
The DOI matches its Crossref record; 72 of the 75 references it cites were found in a registry or an index.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“This finding is consistent with other meta-review research that found the same trend (Chow & Ekholm, 2018).”
From The state of evidence for social and emotional learning: A contemporary meta-analysis of universal school-based SEL interventions (2023) · cites this paper as Chow & Ekholm, 2018
Citations
Cites
Bias in meta-analysis detected by a simple, graphical test
Egger, M., Smith, G. D., Schneider, M., & Minder, C. (1997) · BMJ · doi:10.1136/bmj.315.7109.629 · Cited by 56,722
Operating characteristics of a rank correlation test for publication bias
Begg, C. B., & Mazumdar, M. (1994) · Biometrics · doi:10.2307/2533446 · Cited by 16,955
Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2009) · doi:10.1002/9780470743386 · Cited by 14,225
Duval, S., & Tweedie, R. (2000) · Biometrics · doi:10.1111/j.0006-341x.2000.00455.x · Cited by 14,099
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · PLoS Medicine · doi:10.1371/journal.pmed.0020124 · Cited by 10,505
Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2021) · doi:10.1002/9781119558378 · Cited by 9,799
(2000) · Cited by 9,198
The file drawer problem and tolerance for null results.
Rosenthal, R. (1979) · Psychological Bulletin · doi:10.1037/0033-2909.86.3.638 · Cited by 9,068
Cochrane Handbook for Systematic Reviews of Interventions Version 5.0.1. The Cochrane Collaboration
Higgins, J. P. T., Green, S. R., & Higgins, J. P. T. (2008) · Cited by 7,226
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · CHANCE · doi:10.1080/09332480.2005.10722754 · Cited by 4,702
Funnel plots for detecting bias in meta-analysis
Sterne, J. A. C., & Egger, M. (2001) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(01)00377-8 · Cited by 3,779
Publication Bias in Meta‐Analysis: Prevention, Assessment and Adjustments
(2005) · doi:10.1002/0470870168 · Cited by 2,368
Robust variance estimation in meta‐regression with dependent effect size estimates
Hedges, L. V., Tipton, E., & Johnson, M. C. (2010) · Research Synthesis Methods · doi:10.1002/jrsm.5 · Cited by 2,086
A Fail-Safe N for Effect Size in Meta-Analysis
Orwin, R. G. (1983) · Journal of Educational Statistics · doi:10.2307/1164923 · Cited by 2,064
A meta-analysis of the cognitive and motivational effects of serious games.
Wouters, P., van Nimwegen, C., van Oostendorp, H., & van der Spek, E. D. (2013) · Journal of Educational Psychology · doi:10.1037/a0031311 · Cited by 1,864
Does discovery-based instruction enhance learning?
Alfieri, L., Brooks, P. J., Aldrich, N. J., & Tenenbaum, H. R. (2011) · Journal of Educational Psychology · doi:10.1037/a0021017 · Cited by 1,338
Viechtbauer, W. (2007) · Psychometrika · doi:10.1007/s11336-006-1450-y · Cited by 1,250
Seventy-five trials and eleven systematic reviews a day: how will we ever keep up?
Bastian, H., Glasziou, P., & Chalmers, I. (2010) · PLoS Medicine · doi:10.1371/journal.pmed.1000326 · Cited by 1,188
Publication bias in clinical trials due to statistical significance or direction of trial results
Hopewell, S., Loudon, K., Clarke, M. J., Oxman, A. D., & Dickersin, K. (2009) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.mr000006.pub3 · Cited by 898
Intelligent tutoring systems and learning outcomes: A meta-analysis
Ma, W., Adesope, O. O., Nesbit, J. C., & Liu, Q. (2014) · Journal of Educational Psychology · doi:10.1037/a0037123 · Cited by 840
Tenenbaum, H. R., & Ruck, M. D. (2007) · Journal of Educational Psychology · doi:10.1037/0022-0663.99.2.253 · Cited by 828
Dent, A. L., & Koenka, A. C. (2016) · Educational Psychology Review · doi:10.1007/s10648-015-9320-8 · Cited by 802
The power of statistical tests in meta-analysis
Hedges, L. V., & Pigott, T. D. (2001) · Psychological Methods · doi:10.1037/1082-989x.6.3.203 · Cited by 716
Effects of Systematic Formative Evaluation: A Meta-Analysis
Fuchs, L. S., & Fuchs, D. (1986) · Exceptional Children · doi:10.1177/001440298605300301 · Cited by 668
Ferguson, C. J., & Heene, M. (2012) · Perspectives on Psychological Science · doi:10.1177/1745691612459059 · Cited by 603
Grey literature in meta-analyses of randomized trials of health care interventions
Hopewell, S., McDonald, S., Clarke, M. J., & Egger, M. (2007) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.mr000010.pub3 · Cited by 598
Tanner-Smith, E. E., & Tipton, E. (2014) · Research Synthesis Methods · doi:10.1002/jrsm.1091 · Cited by 588
Classroom management self-efficacy and burnout: A multivariate meta-analysis
Aloe, A. M., Amo, L. C., & Shanahan, M. E. (2014) · Educational Psychology Review · doi:10.1007/s10648-013-9244-0 · Cited by 560
A Meta-analysis of the Effects of Computer Technology on School Students’ Mathematics Learning
Li, Q., & Ma, X. (2010) · Educational Psychology Review · doi:10.1007/s10648-010-9125-8 · Cited by 558
Parenting Stress in Families of Children With ADHD: A Meta-Analysis
Theule, J., Wiener, J., Tannock, R., & Jenkins, J. M. (2013) · Journal of Emotional and Behavioral Disorders · doi:10.1177/1063426610387433 · Cited by 539
How Methodological Features Affect Effect Sizes in Education
Cheung, A. C. K., & Slavin, R. E. (2016) · Educational Researcher · doi:10.3102/0013189x16656615 · Cited by 529
Creativity and academic achievement: A meta-analysis
Gajda, A., Karwowski, M., & Beghetto, R. A. (2017) · Journal of Educational Psychology · doi:10.1037/edu0000133 · Cited by 515
Vasquez, A. C., Patall, E. A., Fong, C. J., Corrigan, A. S., & Pine, L. (2016) · Educational Psychology Review · doi:10.1007/s10648-015-9329-z · Cited by 506
A meta-analysis of the efficacy of teaching mathematics with concrete manipulatives.
Carbonneau, K. J., Marley, S. C., & Selig, J. P. (2013) · Journal of Educational Psychology · doi:10.1037/a0031084 · Cited by 500
Ferguson, C. J., & Brannick, M. T. (2012) · Psychological Methods · doi:10.1037/a0024445 · Cited by 493
Publication and other reporting biases in cognitive sciences: detection, prevalence, and prevention
Ioannidis, J. P. A., Munafò, M. R., Fusar-Poli, P., Nosek, B. A., & David, S. P. (2014) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2014.02.010 · Cited by 471
Effects of computer-based teaching on secondary school students.
Kulik, J. A., Bangert, R. L., & Williams, G. W. (1983) · Journal of Educational Psychology · doi:10.1037//0022-0663.75.1.19 · Cited by 469
Relationship Between School Suspension and Student Outcomes: A Meta-Analysis
Noltemeyer, A. L., Ward, R. M., & Mcloughlin, C. (2015) · School Psychology Review · doi:10.17105/spr-14-0008.1 · Cited by 451
Pinquart, M. (2016) · Educational Psychology Review · doi:10.1007/s10648-015-9338-y · Cited by 403
Maeda, Y., & Yoon, S. Y. (2013) · Educational Psychology Review · doi:10.1007/s10648-012-9215-x · Cited by 398
A Meta-Analysis of Co-Teaching Research: Where Are the Data?
Murawski, W. W., & Lee Swanson, H. (2001) · Remedial and Special Education · doi:10.1177/074193250102200501 · Cited by 374
Searching for studies: a guide to information retrieval for Campbell systematic reviews
Kugley, S., Wade, A., Thomas, J., Mahood, Q., Jørgensen, A. K., Hammerstrøm, K., & Sathe, N. (2017) · Campbell Systematic Reviews · doi:10.4073/cmg.2016.1 · Cited by 371
Learning Disabilities and Anxiety: A Meta-Analysis
Nelson, J. M., & Harwood, H. (2011) · Journal of Learning Disabilities · doi:10.1177/0022219409359939 · Cited by 350
Sterne, J. A. C., Becker, B. J., & Egger, M. (2005) · Publication Bias in Meta‐Analysis · doi:10.1002/0470870168.ch5 · Cited by 341
Reading Research for Students with LD: A Meta-Analysis of Intervention Outcomes
Swanson, H. L. (1999) · Journal of Learning Disabilities · doi:10.1177/002221949903200605 · Cited by 325
Huang, C. (2012) · Journal of Educational Psychology · doi:10.1037/a0026223 · Cited by 319
Learning Through Case Comparisons: A Meta-Analytic Review
Alfieri, L., Nokes-Malach, T. J., & Schunn, C. D. (2013) · Educational Psychologist · doi:10.1080/00461520.2013.775712 · Cited by 318
Publication Bias: Recognizing the Problem, Understanding Its Origins and Scope, and Preventing Harm
Dickersin, K. (2005) · Publication Bias in Meta‐Analysis · doi:10.1002/0470870168.ch2 · Cited by 291
Should Unpublished Data Be Included in Meta-analyses?: Current Convictions and Controversies
Cook, D. J. (1993) · JAMA · doi:10.1001/jama.1993.03500210049030 · Cited by 259
Estimating the Difference Between Published and Unpublished Effect Sizes: A Meta-Review
Polanin, J. R., Tanner-Smith, E. E., & Hennessy, E. A. (2016) · Review of Educational Research · doi:10.3102/0034654315582067 · Cited by 234
Cooper, H., & Koenka, A. C. (2012) · American Psychologist · doi:10.1037/a0027119 · Cited by 196
Failsafe N or File‐Drawer Number
Becker, B. J. (2005) · Publication Bias in Meta‐Analysis · doi:10.1002/0470870168.ch7 · Cited by 179
Academic self-handicapping and achievement: A meta-analysis.
Schwinger, M., Wirthwein, L., Lemmer, G., & Steinmayr, R. (2014) · Journal of Educational Psychology · doi:10.1037/a0035832 · Cited by 178
Lee, J., & Yoon, S. Y. (2017) · Journal of Learning Disabilities · doi:10.1177/0022219415605194 · Cited by 176
Steenbergen-Hu, S., & Cooper, H. (2013) · Journal of Educational Psychology · doi:10.1037/a0032447 · Cited by 175
PUBLICATION BIAS: THE ACHILLES’ HEEL OF SYSTEMATIC REVIEWS?
Torgerson, C. J. (2006) · British Journal of Educational Studies · doi:10.1111/j.1467-8527.2006.00332.x · Cited by 175
Verbal redundancy in multimedia learning environments: A meta-analysis.
Adesope, O. O., & Nesbit, J. C. (2012) · Journal of Educational Psychology · doi:10.1037/a0026147 · Cited by 161
Physical Activity and School Engagement in Youth: A Systematic Review and Meta-Analysis
Owen, K. B., Parker, P. D., Van Zanden, B., MacMillan, F., Astell-Burt, T., & Lonsdale, C. (2016) · Educational Psychologist · doi:10.1080/00461520.2016.1151793 · Cited by 153
Internalizing Problems of Adults With Learning Disabilities: A Meta-Analysis
Klassen, R. M., Tze, V. M. C., & Hannok, W. (2011) · Journal of Learning Disabilities · doi:10.1177/0022219411422260 · Cited by 139
Designing Instructional Text in a Conversational Style: A Meta-analysis
Ginns, P., Martin, A. J., & Marsh, H. W. (2013) · Educational Psychology Review · doi:10.1007/s10648-013-9228-0 · Cited by 138
Overviews in Education Research: A Systematic Review and Analysis
Polanin, J. R., Maynard, B. R., & Dell, N. A. (2017) · Review of Educational Research · doi:10.3102/0034654316631117 · Cited by 131
Grey Literature and Systematic Reviews
Hopewell, S., Clarke, M., & Mallett, S. (2005) · Publication Bias in Meta‐Analysis · doi:10.1002/0470870168.ch4 · Cited by 128
Using Dissertations to Examine Potential Bias in Child and Adolescent Clinical Trials.
McLeod, B. D., & Weisz, J. R. (2004) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.72.2.235 · Cited by 125
Wilson, D. B. (2009) · Journal of Experimental Criminology · doi:10.1007/s11292-009-9085-5 · Cited by 86
Null Effects and Publication Bias in Special Education Research
Cook, B. G., & Therrien, W. J. (2017) · Behavioral Disorders · doi:10.1177/0198742917709473 · Cited by 82
A Teacher's Guide to Meta-Analysis
Banda, D. R., & Therrien, W. J. (2008) · Teaching Exceptional Children · doi:10.1177/004005990804100208 · Cited by 50
Kromrey, J. D., & Rendina-Gobioff, G. (2006) · Educational and Psychological Measurement · doi:10.1177/0013164405278585 · Cited by 49
Torgerson, C. J., Porthouse, J., & Brooks, G. (2003) · Journal of Research in Reading · doi:10.1111/1467-9817.00200 · Cited by 41
Dunst, C. J. (2014) · Exceptional Children · doi:10.1177/001440291408000201 · Cited by 35
When do words hurt? A multiprocess view of the effects of verbalization on visual memory.
Brown, C., Brandimonte, M. A., Wickham, L. H. V., Bosco, A., & Schooler, J. W. (2014) · Journal of Experimental Psychology Learning Memory and Cognition · doi:10.1037/a0037222 · Cited by 17
Best Practices and Interventions in Special Education: How Do We Know What Works?.
Spaulding, L. S. (2009) · Scholars Crossing (Liberty University) · Cited by 12
Fuchs, L. S., & Fuchs, D. (1984) · Teaching Exceptional Children · doi:10.1177/004005998401600318 · Cited by 1
Research works printed but not linked: 1
- Rothstein, H. R., Bornstein, M., & Sutton, A. J. (2006). Publication bias in meta-analysis. Prevention, assessment, and adjustments. Chichester: Wiley.
Cited by
Systematic Reviews in Educational Research: Methodology, Perspectives and Application
Newman, M., & Gough, D. (2020) · Systematic Reviews in Educational Research · doi:10.1007/978-3-658-27602-7_1 · Cited by 591
Cipriano, C., Strambler, M. J., Naples, L. H., Ha, C., Kirk, M., Wood, M., Sehgal, K., Zieher, A. K., Eveleigh, A., McCarthy, M., et al. (2023) · Child Development · doi:10.1111/cdev.13968 · Cited by 636
“Although these findings are consistent with other recent meta-analyses of school-based interventions (Chow & Ekholm, 2018), further examination is warranted to consider if published studies of USB SEL effects are more likely to be from commercialized USB SEL interventions.”
Self-citationTheobald, M. (2021) · Contemporary Educational Psychology · doi:10.1016/j.cedpsych.2021.101976 · Cited by 478
Liu, J., Peng, P., Zhao, B., & Luo, L. (2022) · Educational Psychology Review · doi:10.1007/s10648-022-09689-y · Cited by 269
“…a powerful approach to examine the relations between SES and academic achievement, because of its advantages in statistical power, robust point estimation across different samples and measures, and correlational data with less publication bias (e.g., Chow & Ekholm, 2018; Lipsey & Wilson, 2001).”
BackgroundLeavy, A., Dick, L. K., Meletiou-Mavrotheris, Μ., Paparistodemou, E., & Stylianou, E. (2023) · Journal of Computer Assisted Learning · doi:10.1111/jcal.12806 · Cited by 254
Metadata
- Authors
- Chow, J. C., & Ekholm, E.
- Affiliations
- Virginia Commonwealth University (US)
- Corresponding author
- Chow, Jason C.
- Venue
- Educational Psychology Review
- Published
- 2018-04-06
- Volume
- 30
- Issue
- 3
- Pages
- 727-744
- ISSN
- 1040-726X (ISSN-L)
- 1573-336X
- OpenAlex
- W2795794738
- Cited by
- 80
- References
- 75
- Citations by year
- 2026: 10 · 2025: 7 · 2024: 6 · 2023: 14 · 2022: 8 · 2021: 14 · 2020: 12 · 2019: 6 · 2018: 3
- FWCI
- 6.67
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- www.springer.com/tdm (tdm)
- Locations
- Educational Psychology Review (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
- Social Sciences › Library and Information Sciences › scientometrics and bibliometrics research
- Keywords
- publication bias, effect size, unpublished research, meta-analysis, special education, educational research
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.