The DOI matches its Crossref record; 32 of the 32 references it cites were found in a registry or an index.
Outcome-Reporting Bias in Education Research
Open accessAbstract
Outcome-reporting bias occurs when primary studies do not include information about all outcomes measured in a study. When studies omit findings on important measures, efforts to synthesize the research using systematic review techniques will be biased and interpretations of individual studies will be incomplete. Outcome-reporting bias has been well documented in medicine and has been shown to lead to inaccurate assessments of the effects of medical treatments and, in some cases, to omission of reports of harms. This study examines outcome-reporting bias in educational research by comparing the reports of educational interventions from dissertations to their published versions. We find that nonsignificant outcomes were 30% more likely to be omitted from a published study than statistically significant ones.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“The inclusion of unpublished literature, in particular, is important to limit the risk of publication bias”
From Effects of after-school programs with at-risk youth on attendance and externalizing behaviors: A systematic review and meta-analysis (2015) · cites this paper as Pigott et al. 2013
In the citing paper
“We also suspect that outcome reporting bias may be an issue in after-school program intervention research as it has been found to be problematic in education research (Pigott et al. 2013).”
From Effects of after-school programs with at-risk youth on attendance and externalizing behaviors: A systematic review and meta-analysis (2015) · cites this paper as Pigott et al. 2013
Citations
Cites
CONSORT 2010 Statement: updated guidelines for reporting parallel group randomised trials
the CONSORT Group, Schulz, K. F., Altman, D. G., & Moher, D. (2010) · BMC Medicine · doi:10.1186/1741-7015-8-18 · Cited by 13,371
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · PLoS Medicine · doi:10.1371/journal.pmed.0020124 · Cited by 10,505
CONSORT 2010 Statement: updated guidelines for reporting parallel group randomised trials
Schulz, K. F., Altman, D. G., Moher, D., & for the CONSORT Group (2010) · BMJ · doi:10.1136/bmj.c332 · Cited by 9,329
Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011) · Psychological Science · doi:10.1177/0956797611417632 · Cited by 6,567
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · CHANCE · doi:10.1080/09332480.2005.10722754 · Cited by 4,702 · cited 8×
“Ioannidis, J. P. A. (2005). Why most published research findings are false. PLoS Medicine, 2, 696–701. doi:10.1371/journal .pmed.0020124 John, L. K., Loewenstein, G., & Prelec, D. (2012). Measuring the prevalence of questionable research practices with incentives for truth telling. Psychological Science, 23, 524–32. d…”
BackgroundResultInfluentialThe Handbook of Research Synthesis and Meta-Analysis
(2019) · doi:10.7758/9781610448864 · Cited by 4,553
Selective publication of antidepressant trials and its influence on apparent efficacy
Turner, E. H., Matthews, A. M., Linardatos, E., Tell, R. A., & Rosenthal, R. (2008) · New England Journal of Medicine · doi:10.1056/nejmsa065779 · Cited by 2,435
Publication Bias in Meta‐Analysis: Prevention, Assessment and Adjustments
(2005) · doi:10.1002/0470870168 · Cited by 2,368
Measuring the Prevalence of Questionable Research Practices With Incentives for Truth Telling
John, L. K., Loewenstein, G., & Prelec, D. (2012) · Psychological Science · doi:10.1177/0956797611430953 · Cited by 2,239
Chan, A.-W., Hróbjartsson, A., Haahr, M. T., Gøtzsche, P. C., & Altman, D. G. (2004) · JAMA · doi:10.1001/jama.291.20.2457 · Cited by 1,845
The statistical power of abnormal-social psychological research: A review.
Cohen, J. (1962) · The Journal of Abnormal and Social Psychology · doi:10.1037/h0045186 · Cited by 1,824
Suñé, P., Suñé, J. M., & Montoro, J. B. (2013) · PLoS ONE · doi:10.1371/journal.pone.0054583 · Cited by 1,687
How Many Studies Do You Need?: A Primer on Statistical Power for Meta-Analysis
Valentine, J. C., Pigott, T. D., & Rothstein, H. R. (2010) · Journal of Educational and Behavioral Statistics · doi:10.3102/1076998609346961 · Cited by 1,294
Self-citationPublication bias in clinical trials due to statistical significance or direction of trial results
Hopewell, S., Loudon, K., Clarke, M. J., Oxman, A. D., & Dickersin, K. (2009) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.mr000006.pub3 · Cited by 898
Estimation of a Common Effect Parameter from Sparse Follow-Up Data
Greenland, S., & Robins, J. M. (1985) · Biometrics · doi:10.2307/2530643 · Cited by 798
Do studies of statistical power have an effect on the power of studies?
Sedlmeier, P., & Gigerenzer, G. (1989) · Psychological Bulletin · doi:10.1037/0033-2909.105.2.309 · Cited by 748
Chan, A.-W., & Altman, D. G. (2005) · BMJ · doi:10.1136/bmj.38356.424606.8f · Cited by 613 · cited 7×
“However, note that the Chan and Altman study found evidence of selective outcome reporting among outcomes labeled “important” – they just found more of it for outcomes that were perceived to be less important. Given the documented presence of outcome reporting bias in medical and social science research, we are intere…”
BackgroundResultInfluentialReporting standards for research in psychology: why do we need them? What might they be?
APA Publications and Communications Board Working Group on Journal Article Reporting Standards (2008) · American Psychologist · doi:10.1037/0003-066x.63.9.839 · Cited by 571
Robins, J. M., Breslow, N. E., & Greenland, S. (1986) · Biometrics · doi:10.2307/2531052 · Cited by 476
Outcome reporting bias in randomized trials funded by the Canadian Institutes of Health Research
Chan, A.-W. (2004) · Canadian Medical Association Journal · doi:10.1503/cmaj.1041086 · Cited by 444
Outcome Reporting in Industry-Sponsored Trials of Gabapentin for Off-Label Use
Vedula, S. S., Bero, L. A., Scherer, R. W., & Dickersin, K. (2009) · New England Journal of Medicine · doi:10.1056/nejmsa0906126 · Cited by 360
Statistical power of psychological research: What have we gained in 20 years?
Rossi, J. S. (1990) · Journal of Consulting and Clinical Psychology · doi:10.1037//0022-006x.58.5.646 · Cited by 340
Cooper, H., & Patall, E. A. (2009) · Psychological Methods · doi:10.1037/a0015565 · Cited by 339
Statistical power of psychological research: What have we gained in 20 years?
Rossi, J. S. (1990) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.58.5.646 · Cited by 323
Rethinking and reframing the carnegie classification
McCormick, A. C., & Zhao, C. (2005) · Change The Magazine of Higher Learning · doi:10.3200/chng.37.5.51-57 · Cited by 240
Statistical Power and Effect Sizes of Clinical Neuropsychology Research
Bezeau, S., & Graves, R. E. (2001) · Journal of Clinical and Experimental Neuropsychology · doi:10.1076/jcen.23.3.399.1181 · Cited by 161
Finding the missing science: The fate of studies submitted for review by a human subjects committee.
Cooper, H., DeNeve, K., & Charlton, K. (1997) · Psychological Methods · doi:10.1037/1082-989x.2.4.447 · Cited by 144
Valentine, J. C., & Thompson, S. G. (2013) · Research Synthesis Methods · doi:10.1002/jrsm.1064 · Cited by 130
Self-citationEffects of deficient reporting on meta-analysis: A conceptual framework and reanalysis.
Orwin, R. G., & Cordray, D. S. (1985) · Psychological Bulletin · doi:10.1037/0033-2909.97.1.134 · Cited by 102
Spybrook, J. K., & Raudenbush, S. W. (2009) · Educational Evaluation and Policy Analysis · doi:10.3102/0162373709339524 · Cited by 88
Combining individual participant and aggregated data in a meta‐analysis with correlational studies
Pigott, T., Williams, R. T., & Polanin, J. R. (2012) · Research Synthesis Methods · doi:10.1002/jrsm.1051 · Cited by 27
Self-citationFrancis, D. P. (2012) · International Journal of Cardiology · doi:10.1016/j.ijcard.2011.12.018 · Cited by 23
Research works printed but not linked: 3
- 10.1037//1082-989X.2.4.447 · DOI · Paper page
- Francis D. P., International Journal of Cardiology
- 10.1037//0033-2909.97.1.134 · DOI
Cited by
Sorted by citations
Facts Are More Important Than Novelty: Replication in the Education Sciences
Makel, M. C., & Plucker, J. A. (2014) · Educational Researcher · doi:10.3102/0013189x14545513 · Cited by 609
Låg, T., & Sæle, R. G. (2019) · AERA Open · doi:10.1177/2332858419870489 · Cited by 307
Estimating the Difference Between Published and Unpublished Effect Sizes: A Meta-Review
Polanin, J. R., Tanner-Smith, E. E., & Hennessy, E. A. (2016) · Review of Educational Research · doi:10.3102/0034654315582067 · Cited by 234
Self-citationEditorial: Evidence on Questionable Research Practices: The Good, the Bad, and the Ugly
Banks, G. C., Rogelberg, S. G., Woznyj, H. M., Landis, R. S., & Rupp, D. E. (2016) · Journal of Business and Psychology · doi:10.1007/s10869-016-9456-7 · Cited by 176
Questions About Questionable Research Practices in the Field of Management: A Guest Commentary
Banks, G. C., O’Boyle, E. H., Pollack, J. M., White, C. D., Batchelor, J. H., Whelpley, C. E., Abston, K. A., Bennett, A. A., & Adkins, C. L. (2016) · Journal of Management · doi:10.1177/0149206315619011 · Cited by 163
Metadata
- Authors
- Pigott, T. D., Valentine, J. C., Polanin, J. R., Williams, R. T., & Canada, D. D.
- Affiliations
- Loyola University Chicago, IL
- University of Louisville (US)
- Vanderbilt University (US)
- University of Memphis (US)
- Boston College (US)
- Venue
- Educational Researcher
- Published
- 2013-11
- Volume
- 42
- Issue
- 8
- Pages
- 424-432
- ISSN
- 0013-189X (ISSN-L)
- 1935-102X
- OpenAlex
- W2122723488
- Cited by
- 127
- References
- 32
- Citations by year
- 2026: 2 · 2025: 12 · 2024: 11 · 2023: 12 · 2022: 12 · 2021: 23 · 2020: 7 · 2019: 11 · 2018: 7 · 2017: 8 · 2016: 10 · 2015: 7 · 2014: 4 · 2013: 1
- FWCI
- 7.98
- Type
- journal article
- Language
- en
- Open access
- green
- ecommons.luc.edu
- Licence
- Locations
- Educational Researcher (publishedVersion)
- Loyola eCommons (Loyola University of Chicago) (submittedVersion, cc-by-nc-nd)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Education › Educational Research and Methods
- Social Sciences › Education › Academic integrity and plagiarism
- Keywords
- selective reporting, educational intervention, dissertations, publication bias, non-significant results, evidence synthesis, educational research
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
