The DOI matches its Crossref record.
Summing up: The Science of Reviewing Research
Abstract
1. Introduction 2. Organizing a Reviewing Strategy 3. Quantitative Procedures 4. Numbers and Narrative: The Division of Labor 5. What We Have Learned 6. A Checklist for Evaluating Reviews Reference Index
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cited by
Bias in meta-analysis detected by a simple, graphical test
Egger, M., Smith, G. D., Schneider, M., & Minder, C. (1997) · BMJ · doi:10.1136/bmj.315.7109.629 · Cited by 56,722
Conducting Meta-Analyses in R with the metafor Package
Viechtbauer, W. (2010) · Journal of Statistical Software · doi:10.18637/jss.v036.i03 · Cited by 20,464
Operating characteristics of a rank correlation test for publication bias
Begg, C. B., & Mazumdar, M. (1994) · Biometrics · doi:10.2307/2533446 · Cited by 16,955
Duval, S., & Tweedie, R. (2000) · Biometrics · doi:10.1111/j.0006-341x.2000.00455.x · Cited by 14,099
A meta-analytic test of intergroup contact theory.
Pettigrew, T. F., & Tropp, L. R. (2006) · Journal of personality and social psychology · doi:10.1037/0022-3514.90.5.751 · Cited by 9,708
No adjustments are needed for multiple comparisons.
Rothman, K. J. (1990) · Epidemiology · doi:10.1097/00001648-199001000-00010 · Cited by 5,965
Achenbach, T. M., McConaughy, S. H., & Howell, C. T. (1987) · Psychological Bulletin · doi:10.1037/0033-2909.101.2.213 · Cited by 5,436
The Handbook of Research Synthesis and Meta-Analysis
(2019) · doi:10.7758/9781610448864 · Cited by 4,553
Pharmacotherapy for neuropathic pain in adults: a systematic review and meta-analysis
Finnerup, N. B., Attal, N., Haroutounian, S., McNicol, E., Baron, R., Dworkin, R. H., Gilron, I., Haanpää, M., Hansson, P., Jensen, T. S., et al. (2015) · The Lancet Neurology · doi:10.1016/s1474-4422(14)70251-0 · Cited by 3,980
Funnel plots for detecting bias in meta-analysis
Sterne, J. A. C., & Egger, M. (2001) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(01)00377-8 · Cited by 3,779
Sin, N. L., & Lyubomirsky, S. (2009) · Journal of Clinical Psychology · doi:10.1002/jclp.20593 · Cited by 2,955
A Nonparametric “Trim and Fill” Method of Accounting for Publication Bias in Meta-Analysis
Duval, S., & Tweedie, R. (2000) · Journal of the American Statistical Association · doi:10.1080/01621459.2000.10473905 · Cited by 2,955
Deeks, J. J., Macaskill, P., & Irwig, L. (2005) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2005.01.016 · Cited by 2,910
Maternal Depression and Child Psychopathology: A Meta-Analytic Review
Goodman, S. H., Rouse, M. H., Connell, A. M., Broth, M. R., Hall, C. M., & Heyward, D. (2011) · Clinical Child and Family Psychology Review · doi:10.1007/s10567-010-0080-1 · Cited by 2,621
Quantitative Synthesis in Systematic Reviews
Lau, J., Ioannidis, J. P. A., & Schmid, C. H. (1997) · Annals of Internal Medicine · doi:10.7326/0003-4819-127-9-199711010-00008 · Cited by 2,517
Correlation between Fitness and Genetic Diversity
Reed, D. H., & Frankham, R. (2003) · Conservation Biology · doi:10.1046/j.1523-1739.2003.01236.x · Cited by 2,428
Effectiveness of L2 instruction: A research synthesis and quantitative meta-analysis
Norris, J. M., & Ortega, L. (2000) · Language Learning · doi:10.1111/0023-8333.00136 · Cited by 2,569
Publication and related bias in meta-analysis
Sterne, J. A. C., Gavaghan, D., & Egger, M. (2000) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(00)00242-0 · Cited by 2,282
Purposeful Sampling in Qualitative Research Synthesis
Suri, H. (2011) · Qualitative Research Journal · doi:10.3316/qrj1102063 · Cited by 2,284
Investigating and dealing with publication and other biases in meta-analysis
Sterne, J. A. C., Egger, M., & Smith, G. D. (2001) · BMJ · doi:10.1136/bmj.323.7304.101 · Cited by 2,225
Writing narrative literature reviews for peer-reviewed journals: secrets of the trade
Green, B. N., Johnson, C. D., & Adams, A. (2006) · Journal of Chiropractic Medicine · doi:10.1016/s0899-3467(07)60142-6 · Cited by 2,224
The repeatability of behaviour: a meta-analysis
Bell, A. M., Hankison, S. J., & Laskowski, K. L. (2009) · Animal Behaviour · doi:10.1016/j.anbehav.2008.12.022 · Cited by 2,128
Peters, J. L., Sutton, A. J., Jones, D. R., Abrams, K. R., & Rushton, L. (2008) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2007.11.010 · Cited by 2,053
Gershoff, E. T. (2002) · Psychological Bulletin · doi:10.1037/0033-2909.128.4.539 · Cited by 2,165
Popay, J., Roberts, H., Sowden, A., Petticrew, M., Arai, L., Rodgers, M., Britten, N., Roen, K., & Duffy, S. (2006) · Lancaster University · doi:10.13140/2.1.1018.4643 · Cited by 2,070
Newbert, S. L. (2007) · Strategic Management Journal · doi:10.1002/smj.573 · Cited by 2,053
Exposure and affect: Overview and meta-analysis of research, 1968–1987.
Bornstein, R. F. (1989) · Psychological Bulletin · doi:10.1037/0033-2909.106.2.265 · Cited by 1,970
Synthesizing information systems knowledge: A typology of literature reviews
Paré, G., Trudel, M.-C., Jaana, M., & Kitsiou, S. (2015) · Information & Management · doi:10.1016/j.im.2014.08.008 · Cited by 1,987
A meta‐analysis of trait differences between invasive and non‐invasive plant species
Van Kleunen, M., Weber, E., & Fischer, M. (2010) · Ecology Letters · doi:10.1111/j.1461-0248.2009.01418.x · Cited by 1,876
Meta-Analysis: Recent Developments in Quantitative Methods for Literature Reviews
Rosenthal, R., & DiMatteo, M. R. (2001) · Annual Review of Psychology · doi:10.1146/annurev.psych.52.1.59 · Cited by 1,780
The case of the misleading funnel plot
Lau, J., Ioannidis, J. P. A., Terrin, N., Schmid, C. H., & Olkin, I. (2006) · BMJ · doi:10.1136/bmj.333.7568.597 · Cited by 1,766
Chang, S.-H., Stoll, C. R. T., Song, J., Varela, J. E., Eagon, C. J., & Colditz, G. A. (2014) · JAMA Surgery · doi:10.1001/jamasurg.2013.3654 · Cited by 1,688
Systematic Reviews: Rationale for systematic reviews
Mulrow, C. D. (1994) · BMJ · doi:10.1136/bmj.309.6954.597 · Cited by 1,677
Hofmann, W., Gawronski, B., Gschwendner, T., Le, H., & Schmitt, M. (2005) · Personality and Social Psychology Bulletin · doi:10.1177/0146167205275613 · Cited by 1,642
Publication bias in the social sciences: Unlocking the file drawer
Franco, A., Malhotra, N., & Simonovits, G. (2014) · Science · doi:10.1126/science.1255484 · Cited by 1,542
meta-analysis bias in location and selection of studies
Egger, M., & Smith, G. D. (1998) · BMJ · doi:10.1136/bmj.316.7124.61 · Cited by 1,374
Mays, N., Pope, C., & Popay, J. (2005) · Journal of Health Services Research & Policy · doi:10.1258/1355819054308576 · Cited by 1,375
A comparison of methods to detect publication bias in meta‐analysis
Macaskill, P., Walter, S. D., & Irwig, L. (2001) · Statistics in Medicine · doi:10.1002/sim.698 · Cited by 1,286
The Relationship Between Self‐Compassion and Well‐Being: A Meta‐Analysis
Zessin, U., Dickhäuser, O., & Garbade, S. (2015) · Applied Psychology: Health and Well-Being · doi:10.1111/aphw.12051 · Cited by 1,237
Parents' differential socialization of boys and girls: a meta-analysis
Lytton, H., & Romney, D. M. (1991) · Psychological Bulletin · doi:10.1037/0033-2909.109.2.267 · Cited by 1,210
The Power of Feedback Revisited: A Meta-Analysis of Educational Feedback Research
Wisniewski, B., Zierer, K., & Hattie, J. (2020) · Frontiers in Psychology · doi:10.3389/fpsyg.2019.03087 · Cited by 1,199
Owen, C. G., Martin, R. M., Whincup, P. H., Smith, G. D., & Cook, D. G. (2005) · Pediatrics · doi:10.1542/peds.2004-1176 · Cited by 1,168
Guidelines for systematic review in Conservation and environmental management
PULLIN, A. S., & STEWART, G. B. (2006) · Conservation Biology · doi:10.1111/j.1523-1739.2006.00485.x · Cited by 1,137
A Meta-Analysis of Local Adaptation in Plants
Leimu, R., & Fischer, M. (2008) · PLoS ONE · doi:10.1371/journal.pone.0004010 · Cited by 1,155
Reinforcement, Reward, and Intrinsic Motivation: A Meta-Analysis
Cameron, J., & Pierce, W. D. (1994) · Review of Educational Research · doi:10.3102/00346543064003363 · Cited by 1,153
A systematic assessment of the empirical support for transaction cost economics
David, R. J., & Han, S. (2004) · Strategic Management Journal · doi:10.1002/smj.359 · Cited by 1,127
Fluency: A review of developmental and remedial practices.
Kuhn, M. R., & Stahl, S. A. (2003) · Journal of Educational Psychology · doi:10.1037/0022-0663.95.1.3 · Cited by 1,124
Writing meta-analytic reviews.
Rosenthal, R. (1995) · Psychological Bulletin · doi:10.1037/0033-2909.118.2.183 · Cited by 1,108
STATISTICAL ISSUES IN ECOLOGICAL META-ANALYSES
Gurevitch, J., & Hedges, L. V. (1999) · Ecology · doi:10.1890/0012-9658(1999)080[1142:siiema]2.0.co;2 · Cited by 1,081
Marital quality and personal well-being: a meta- analysis
Proulx, C. M., Helms, H. M., & Buehler, C. (2007) · Journal of Marriage and Family · doi:10.1111/j.1741-3737.2007.00393.x · Cited by 1,068
Classical fear-conditioning in the anxiety disorders: a meta-analysis
Lissek, S., Powers, A. S., McClure, E. B., Phelps, E. A., Woldehawariat, G., Grillon, C., & Pine, D. S. (2005) · Behaviour Research and Therapy · doi:10.1016/j.brat.2004.10.007 · Cited by 1,034
Qualitative metasynthesis: Issues and techniques
Sandelowski, M., Docherty, S., & Emden, C. (1997) · Research in Nursing & Health · doi:10.1002/(sici)1098-240x(199708)20:4<365::aid-nur9>3.0.co;2-e · Cited by 1,046
Tamim, R. M., Bernard, R. M., Borokhovski, E., Abrami, P. C., & Schmid, R. F. (2011) · Review of Educational Research · doi:10.3102/0034654310393361 · Cited by 1,029
Linde, K., Clausius, N., Ramirez, G., Melchart, D., Eitel, F., Hedges, L. V., & Jonas, W. B. (1997) · The Lancet · doi:10.1016/s0140-6736(97)02293-9 · Cited by 979
Best evidence synthesis: An intelligent alternative to meta-analysis
Slavin, R. E. (1995) · Journal of Clinical Epidemiology · doi:10.1016/0895-4356(94)00097-a · Cited by 949
Connell, A. M., & Goodman, S. H. (2002) · Psychological Bulletin · doi:10.1037/0033-2909.128.5.746 · Cited by 943
Best Practices in Quantitative Methods
Osborne, J. (2008) · Sage eBooks · doi:10.4135/9781412995627 · Cited by 942
The psychological effects of meditation: a meta-analysis
Sedlmeier, P., Eberth, J., Schwarz, M., Zimmermann, D., Haarig, F., Jaeger, S., & Kunze, S. (2012) · Psychological Bulletin · doi:10.1037/a0028168 · Cited by 910
The Handbook of Research Synthesis
Ziegel, E. R., Cooper, H., & Hedges, L. (1995) · Technometrics · doi:10.2307/1269651 · Cited by 927
Evaluative conditioning in humans: A meta-analysis.
Hofmann, W., De Houwer, J., Perugini, M., Baeyens, F., & Crombez, G. (2010) · Psychological Bulletin · doi:10.1037/a0018916 · Cited by 923
How general are positive relationships between plant population size, fitness and genetic variation?
LEIMU, R., MUTIKAINEN, P., KORICHEVA, J., & FISCHER, M. (2006) · Journal of Ecology · doi:10.1111/j.1365-2745.2006.01150.x · Cited by 904
Depression and memory impairment: A meta-analysis of the association, its pattern, and specificity.
Burt, D. B., Zembar, M. J., & Niederehe, G. (1995) · Psychological Bulletin · doi:10.1037/0033-2909.117.2.285 · Cited by 903
Co-Teaching in Inclusive Classrooms: A Metasynthesis of Qualitative Research
Scruggs, T. E., Mastropieri, M. A., & McDuffie, K. A. (2007) · Exceptional Children · doi:10.1177/001440290707300401 · Cited by 903
Self-threat magnifies the self-serving bias: A meta-analytic integration
Campbell, W. K., & Sedikides, C. (1999) · Review of General Psychology · doi:10.1037/1089-2680.3.1.23 · Cited by 873
Shame, guilt, and depressive symptoms: A meta-analytic review.
Kim, S., Thibodeau, R., & Jorgensen, R. S. (2011) · Psychological Bulletin · doi:10.1037/a0021466 · Cited by 866
A Meta-Analysis of Competition in Field Experiments
Gurevitch, J., Morrow, L. L., Wallace, A., & Walsh, J. S. (1992) · The American Naturalist · doi:10.1086/285428 · Cited by 865
The rules of the game called psychological science
Bakker, M., van Dijk, A., & Wicherts, J. M. (2012) · Perspectives on Psychological Science · doi:10.1177/1745691612459060 · Cited by 800
(1997) · Cited by 856
The Effects of Television Violence on Antisocial Behavior: A Meta-Analysis1
PAIK, H., & COMSTOCK, G. (1994) · Communication Research · doi:10.1177/009365094021004004 · Cited by 856
Hohenstein, N.-O., Feisel, E., Hartmann, E., & Giunipero, L. (2015) · International Journal of Physical Distribution & Logistics Management · doi:10.1108/ijpdlm-05-2013-0128 · Cited by 848
Nguyen, H.-H. D., & Ryan, A. M. (2008) · Journal of Applied Psychology · doi:10.1037/a0012702 · Cited by 913
Psychological predictors of heart disease: A quantitative review.
Booth-Kewley, S., & Friedman, H. S. (1987) · Psychological Bulletin · doi:10.1037//0033-2909.101.3.343 · Cited by 835
Adjusting for publication bias in the presence of heterogeneity
Terrin, N., Schmid, C. H., Lau, J., & Olkin, I. (2003) · Statistics in Medicine · doi:10.1002/sim.1461 · Cited by 809
Hedges, L. V., Laine, R. D., & Greenwald, R. (1994) · Educational Researcher · doi:10.3102/0013189x023003005 · Cited by 819
Chen, M. J., Fan, X., & Moe, S. T. (2002) · Journal of Sports Sciences · doi:10.1080/026404102320761787 · Cited by 819
Publication bias—a problem in interpreting medical data
Begg, C. B., & Berlin, J. A. (1988) · Journal of the Royal Statistical Society. Series A (Statistics in Society) · doi:10.2307/2982993 · Cited by 814
HOW CLOSELY CORRELATED ARE MOLECULAR AND QUANTITATIVE MEASURES OF GENETIC VARIATION? A META-ANALYSIS
Reed, D. H., & Frankham, R. (2001) · Evolution · doi:10.1111/j.0014-3820.2001.tb00629.x · Cited by 803
The Effects of School-Based Writing-to-Learn Interventions on Academic Achievement: A Meta-Analysis
Bangert-Drowns, R. L., Hurley, M. M., & Wilkinson, B. (2004) · Review of Educational Research · doi:10.3102/00346543074001029 · Cited by 802
Is there publication bias in the reporting of cancer risk in Barrett's esophagus?
Shaheen, N. J., Crosby, M. A., Bozymski, E. M., & Sandler, R. S. (2000) · Gastroenterology · doi:10.1053/gast.2000.9302 · Cited by 758
An exploratory test for an excess of significant findings
Ioannidis, J. P., & Trikalinos, T. A. (2007) · Clinical Trials · doi:10.1177/1740774507079441 · Cited by 751
Physical symptoms in primary care. Predictors of psychiatric disorders and functional impairment
Kroenke, K. (1994) · Archives of Family Medicine · doi:10.1001/archfami.3.9.774 · Cited by 742
Cognitive impairment in preclinical Alzheimer's disease: a meta-analysis
Bäckman, L., Jones, S., Berger, A.-K., Laukka, E. J., & Small, B. J. (2005) · Neuropsychology · doi:10.1037/0894-4105.19.4.520 · Cited by 735
Mood-congruent recall of affectively toned stimuli: A meta-analytic review
Matt, G. E., Vázquez, C., & Campbell, W. K. (1992) · Clinical Psychology Review · doi:10.1016/0272-7358(92)90116-p · Cited by 726
The effect of error correction on learners’ ability to write accurately
Truscott, J. (2007) · Journal of Second Language Writing · doi:10.1016/j.jslw.2007.06.003 · Cited by 719
Smart homes and home health monitoring technologies for older adults: A systematic review
Liu, L., Stroulia, E., Nikolaidis, I., Miguel-Cruz, A., & Rios Rincon, A. (2016) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2016.04.007 · Cited by 713
Effects of self-assessment on self-regulated learning and self-efficacy: Four meta-analyses
Panadero, E., Jonsson, A., & Botella, J. (2017) · Educational Research Review · doi:10.1016/j.edurev.2017.08.004 · Cited by 712
Blood Pressure and Stroke: An Overview of Published Reviews
Lawes, C. M. M., Bennett, D. A., Feigin, V. L., & Rodgers, A. (2004) · Stroke · doi:10.1161/01.str.0000116869.64771.5a · Cited by 705
Best-Evidence Synthesis: An Alternative to Meta-Analytic and Traditional Reviews
Slavin, R. E. (1986) · Educational Researcher · doi:10.2307/1174711 · Cited by 698
Nurmohamed, M. T., Buller, H. R., Dekker, E., Hommes, D. W., Rosendaal, F. R., Briet, E., Rosendaal, F. R., & Vandenbroucke, J. P. (1992) · The Lancet · doi:10.1016/0140-6736(92)93223-a · Cited by 668
Community coalitions for prevention and health promotion
Butterfoss, F. D., Goodman, R. M., & Wandersman, A. (1993) · Health Education Research · doi:10.1093/her/8.3.315 · Cited by 648
Cognitive gender differences are disappearing.
Feingold, A. (1988) · American Psychologist · doi:10.1037//0003-066x.43.2.95 · Cited by 635
Depression, anxiety, and resting frontal EEG asymmetry: A meta-analytic review.
Thibodeau, R., Jorgensen, R. S., & Kim, S. (2006) · Journal of Abnormal Psychology · doi:10.1037/0021-843x.115.4.715 · Cited by 620
Publication bias in reports of animal stroke studies leads to major overstatement of efficacy
Sena, E. S., van der Worp, H. B., Bath, P. M. W., Howells, D. W., & Macleod, M. R. (2010) · PLoS Biology · doi:10.1371/journal.pbio.1000344 · Cited by 600
Psychological impairment in the wake of disaster: The disaster-psychopathology relationship.
Rubonis, A. V., & Bickman, L. (1991) · Psychological Bulletin · doi:10.1037//0033-2909.109.3.384 · Cited by 612
Effects of paternal depression on fathers' parenting behaviors: a meta-analytic review
Wilson, S., & Durbin, C. E. (2010) · Clinical Psychology Review · doi:10.1016/j.cpr.2009.10.007 · Cited by 603
Synthesizing Research on Language Learning and Teaching
(2006) · Language Learning & Language Teaching · doi:10.1075/lllt.13 · Cited by 608
Motivation Interventions in Education: A Meta-Analytic Review
Lazowski, R. A., & Hulleman, C. S. (2016) · Review of Educational Research · doi:10.3102/0034654315617832 · Cited by 607
Meta-analysis of Grade Retention Research: Implications for Practice in the 21st Century
Jimerson, S. R. (2001) · School Psychology Review · doi:10.1080/02796015.2001.12086124 · Cited by 597
Pervasive negative effects of rewards on intrinsic motivation: The myth continues
Cameron, J., Banko, K. M., & Pierce, W. D. (2001) · The Behavior Analyst · doi:10.1007/bf03392017 · Cited by 574
Does monitoring goal progress promote goal attainment? A meta-analysis of the experimental evidence.
Harkin, B., Webb, T. L., Chang, B. P. I., Prestwich, A., Conner, M., Kellar, I., Benn, Y., & Sheeran, P. (2016) · Psychological Bulletin · doi:10.1037/bul0000025 · Cited by 564
Metadata
- Authors
- Wilkinson, I., Light, R. J., & Pillemer, D. B.
- Venue
- Journal of Educational Statistics
- Publisher
- JSTOR
- Published
- 1987
- Volume
- 12
- Issue
- 3
- Pages
- 302
- ISSN
- 0362-9791 (ISSN-L)
- OpenAlex
- W2038886783
- Cited by
- 1,471
- References
- 0
- FWCI
- 74.91
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.