Deficiencies of Reporting in Meta-Analyses and Some Remedies
The DOI matches its Crossref record; 9 of the 9 references it cites were found in a registry or an index.
Abstract
There is general agreement that meta-analysis is an important tool for synthesizing study results in quantitative educational research. Yet, a shared feature of many meta-analyses is a failure to report sufficient information for readers to fully judge the reported findings, such as the populations to which generalizations are to be made, inclusion and exclusion criteria for studies, the rationale underlying coded variables, including their scaling, relationship to study goals, and the sensitivity of conclusions to the chosen methodology. The authors provide (a) evidence of the prevalence of deficiencies of reporting and (b) suggestions about the kind of information that meta-analysts should typically provide. Providing readers with that information should significantly improve readers' ability to evaluate the credibility of a meta-analysis.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Intraclass correlations: Uses in assessing rater reliability.
Shrout, P. E., & Fleiss, J. L. (1979) · Psychological Bulletin · doi:10.1037/0033-2909.86.2.420 · Cited by 24,532
Gender differences in mathematics performance: A meta-analysis.
Hyde, J. S., Fennema, E., & Lamon, S. J. (1990) · Psychological Bulletin · doi:10.1037/0033-2909.107.2.139 · Cited by 1,732
Student Peer Assessment in Higher Education: A Meta-Analysis Comparing Peer and Teacher Marks
Falchikov, N., & Goldfinch, J. (2000) · Review of Educational Research · doi:10.3102/00346543070003287 · Cited by 1,170
The Effects of Summer Vacation on Achievement Test Scores: A Narrative and Meta-Analytic Review
Cooper, H., Nye, B., Charlton, K., Lindsay, J., & Greathouse, S. (1996) · Review of Educational Research · doi:10.3102/00346543066003227 · Cited by 991
The power of statistical tests in meta-analysis
Hedges, L. V., & Pigott, T. D. (2001) · Psychological Methods · doi:10.1037/1082-989x.6.3.203 · Cited by 716
The power of statistical tests for moderators in meta-analysis
Hedges, L. V., & Pigott, T. D. (2004) · Psychological Methods · doi:10.1037/1082-989x.9.4.426 · Cited by 672
Incidental Word Learning While Reading: A Meta-Analysis
Swanborn, M. S. L., & de Glopper, K. (1999) · Review of Educational Research · doi:10.3102/00346543069003261 · Cited by 359
Cooper, H., Valentine, J. C., Charlton, K., & Melson, A. (2003) · Review of Educational Research · doi:10.3102/00346543073001001 · Cited by 138
Published educational research: An empirical study of its quality
Hall, B. W., Ward, A. W., & Comer, C. B. (1988) · The Journal of Educational Research · doi:10.1080/00220671.1988.10885820 · Cited by 34
Cited by
Rethinking the Use of Tests: A Meta-Analysis of Practice Testing
Adesope, O. O., Trevisan, D. A., & Sundararajan, N. (2017) · Review of Educational Research · doi:10.3102/0034654316689306 · Cited by 544
Enhancing mathematical problem posing competence: a meta-analysis of intervention studies
Zhang, L., Stylianides, G. J., & Stylianides, A. J. (2024) · International Journal of STEM Education · doi:10.1186/s40594-024-00507-1 · Cited by 51
Treatment of tinnitus: A scoping review.
Makar, S. K., Mukundan, G., & Gore, G. (2017) · The International Tinnitus Journal · doi:10.5935/0946-5448.20170027 · Cited by 40
Random or Fixed? An Empirical Examination of Meta-Analysis Model Choices
Slaney, K. L., Tafreshi, D., & Hohn, R. (2018) · Review of General Psychology · doi:10.1037/gpr0000140 · Cited by 13
Metadata
- Authors
- Harwell, M., & Maeda, Y.
- Venue
- The Journal of Experimental Education
- Publisher
- Informa UK Limited
- Published
- 2008-07
- Volume
- 76
- Issue
- 4
- Pages
- 403-430
- ISSN
- 0022-0973 (ISSN-L)
- 1940-0683
- OpenAlex
- W2117339006
- Cited by
- 24
- References
- 9
- FWCI
- 1.12
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.