The DOI matches its Crossref record; 40 of the 40 references it cites were found in a registry or an index.
A Nonparametric “Trim and Fill” Method of Accounting for Publication Bias in Meta-Analysis
Abstract
Meta-analysis collects and synthesizes results from individual studies to estimate an overall effect size. If published studies are chosen, say through a literature review, then an inherent selection bias may arise, because, for example, studies may tend to be published more readily if they are statistically significant, or deemed to be more “interesting” in terms of the impact of their outcomes. We develop a simple rank-based data augmentation technique, formalizing the use of funnel plots, to estimate and adjust for the numbers and outcomes of missing studies. Several nonparametric estimators are proposed for the number of missing studies, and their properties are developed analytically and through simulations. We apply the method to simulated and epidemiological datasets and show that it is both effective and consistent with other criteria in the literature.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Publication bias was investigated by applying the trim and fill method, where effect sizes are recalculated to adjust for supposed publication biases (Duval & Tweedie, 2000).”
From Taking stock: A meta-analysis of the effects of foreign language enjoyment (2022) · cites this paper as Duval & Tweedie, 2000
In the citing paper
“The trim‐and‐fill procedure determined that nine studies should be added to create symmetry.”
From Problem-oriented policing for reducing crime and disorder: An updated systematic review and meta-analysis (2020) · cites this paper as Duval and Tweedie (2000)
In the citing paper
“Like many systematic reviews, our meta‐analyses used the trim‐and‐fill procedure to explore whether publication bias might be affecting the results and to estimate how the reported effects would change if the bias were to be removed (Duval & Tweedie, 2000; Duval, 2005).”
From Hot spots policing of small geographic areas effects on crime (2019) · cites this paper as Duval & Tweedie, 2000
In the citing paper
“When we encountered publication bias, we conducted a trim and fill adjusted analysis 25 to remove the most extreme small studies from the positive side of the funnel plot, and recalculated the effect size iteratively, until the funnel plot was symmetrical around the (new) effect size.”
From Diabetes mellitus in people with schizophrenia, bipolar disorder and major depressive disorder: a systematic review and large scale meta‐analysis (2016) · cites this paper as [25]
In the citing paper
“Variations in the risk for glucose abnormalities are evident in the literature, with the highest risk being associated with clozapine, olanzapine and quetiapine in carefully designed studies 25,34,35 .”
From Diabetes mellitus in people with schizophrenia, bipolar disorder and major depressive disorder: a systematic review and large scale meta‐analysis (2016) · cites this paper as [25]
Citations
Cites
Bias in meta-analysis detected by a simple, graphical test
Egger, M., Smith, G. D., Schneider, M., & Minder, C. (1997) · BMJ · doi:10.1136/bmj.315.7109.629 · Cited by 56,722
Meta-analysis in clinical trials
DerSimonian, R., & Laird, N. (1986) · Controlled Clinical Trials · doi:10.1016/0197-2456(86)90046-2 · Cited by 39,552
Operating characteristics of a rank correlation test for publication bias
Begg, C. B., & Mazumdar, M. (1994) · Biometrics · doi:10.2307/2533446 · Cited by 16,955
Duval, S., & Tweedie, R. (2000) · Biometrics · doi:10.1111/j.0006-341x.2000.00455.x · Cited by 14,099
Self-citationWasserman, S. S., Hedges, L. V., & Olkin, I. (1988) · Journal of Educational Statistics · doi:10.2307/1164953 · Cited by 10,377
Casella, G., & Berger, R. L. (1993) · Chapman & Hall/CRC eBooks · doi:10.1201/9781003456285 · Cited by 5,835
The Handbook of Research Synthesis.
Howel, D., Cooper, H., & Hedges, L. V. (1995) · Biometrics · doi:10.2307/2533295 · Cited by 3,657
Publication bias in clinical research
Easterbrook, P. J., Gopalan, R., Berlin, J. A., & Matthews, D. R. (1991) · The Lancet · doi:10.1016/0140-6736(91)90201-y · Cited by 3,109 · cited 1×
“The negative binomial forms here are reminiscent of those used by Eberly and Casella (1999) and Givens et al.”
MethodSumming up: The Science of Reviewing Research
Wilkinson, I., Light, R. J., & Pillemer, D. B. (1987) · Journal of Educational Statistics · doi:10.2307/1164691 · Cited by 1,486
Light, R. J., & Pillemer, D. B. (1984) · Harvard University Press eBooks · doi:10.4159/9780674040243 · Cited by 1,422
Research works printed but not linked: 10
- 1983, British Medical Journal, 280, 508
- Cooper H., 1998, Synthesizing Research
- Dear K., 1997, Statistical Science, 12, 245
- DuMouchel W., 1997, Statistical Science, 12, 244
- Duval S. J., 1999, “Effects of Publication Bias in Meta-Analysis,”
- Duval S. J., “Robustness of the ‘Trim and Fill’ publication bias approach to different mechanisms for study suppression,”
- Eberly L. E., 1999, Journal of Official Statistics, 15
- Hedges L., 1985, Statistical Methods for Meta-Analysis
- Sugita M., 1994, Tokai Journal of Experimental and Clinical Medicine, 19, 29
- Sutton A. J., 1999, “The Impact of Publication Bias on Meta-Analyses Within the Cochrane Database of Systematic Reviews,”
Cited by
Sorted by citations
Conducting Meta-Analyses in R with the metafor Package
Viechtbauer, W. (2010) · Journal of Statistical Software · doi:10.18637/jss.v036.i03 · Cited by 20,464
Hao (2015) · doi:10.1002/14651858 · Cited by 11,321
A meta-analytic test of intergroup contact theory.
Pettigrew, T. F., & Tropp, L. R. (2006) · Journal of personality and social psychology · doi:10.1037/0022-3514.90.5.751 · Cited by 9,708
Social Relationships and Mortality Risk: A Meta-analytic Review
Holt-Lunstad, J., Smith, T. B., & Layton, J. B. (2010) · PLoS Medicine · doi:10.1371/journal.pmed.1000316 · Cited by 7,272
“Second, we employed the ‘‘trim and fill’’ methodology described by Duval and Tweedie [178,179] to estimate the number of studies missing due to publication bias, but this analysis failed to reveal any studies that would need to be created on the opposite side of the distribution, meaning that adjustment to the omnibus…”
MethodHow to perform a meta-analysis with R: a practical tutorial
Balduzzi, S., Rücker, G., & Schwarzer, G. (2019) · Evidence Based Mental Health · doi:10.1136/ebmental-2019-300117 · Cited by 5,827
Perceived discrimination and health: A meta-analytic review.
Pascoe, E. A., & Smart Richman, L. (2009) · Psychological Bulletin · doi:10.1037/a0016059 · Cited by 4,427
Roberts, B. W., Walton, K. E., & Viechtbauer, W. (2006) · Psychological Bulletin · doi:10.1037/0033-2909.132.1.1 · Cited by 3,318
Lorenz, M. W., Markus, H. S., Bots, M. L., Rosvall, M., & Sitzer, M. (2007) · Circulation · doi:10.1161/circulationaha.106.628875 · Cited by 2,977
Racism as a Determinant of Health: A Systematic Review and Meta-Analysis
Paradies, Y., Ben, J., Denson, N., Elias, A., Priest, N., Pieterse, A., Gupta, A., Kelaher, M., & Gee, G. (2015) · PLOS ONE · doi:10.1371/journal.pone.0138511 · Cited by 2,610
“Where we detected publication bias, we used the trim and fill method to estimate and adjust for missing (un-reported) studies to estimate what the effect size would have been in the absence of bias [66, 67].”
MethodDo Psychosocial and Study Skill Factors Predict College Outcomes? A Meta-Analysis.
Robbins, S. B., Lauver, K., Le, H., Davis, D., Langley, R., & Carlstrom, A. (2004) · Psychological Bulletin · doi:10.1037/0033-2909.130.2.261 · Cited by 2,479
Varese, F., Smeets, F., Drukker, M., Lieverse, R., Lataster, T., Viechtbauer, W., Read, J., van Os, J., & Bentall, R. P. (2012) · Schizophrenia Bulletin · doi:10.1093/schbul/sbs050 · Cited by 2,436
Effective techniques in healthy eating and physical activity interventions: A meta-regression.
Michie, S., Abraham, C., Whittington, C., McAteer, J., & Gupta, S. (2009) · Health Psychology · doi:10.1037/a0016136 · Cited by 2,229
Perceived organizational support: a meta-analytic evaluation of organizational support theory.
Kurtessis, J. N., Eisenberger, R., Ford, M. T., Buffardi, L. C., Stewart, K. A., & Adis, C. S. (2017) · Journal of Management · doi:10.1177/0149206315575554 · Cited by 2,354
Comparison of two methods to detect publication bias in meta-analysis.
Peters, J. L. (2006) · JAMA · doi:10.1001/jama.295.6.676 · Cited by 2,278
A modified test for small‐study effects in meta‐analyses of controlled trials with binary endpoints
Harbord, R. M., Egger, M., & Sterne, J. A. C. (2006) · Statistics in Medicine · doi:10.1002/sim.2380 · Cited by 2,267
Peters, J. L., Sutton, A. J., Jones, D. R., Abrams, K. R., & Rushton, L. (2008) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2007.11.010 · Cited by 2,053
Anderson, C. A., Shibuya, A., Ihori, N., Swing, E. L., Bushman, B. J., Sakamoto, A., Rothstein, H. R., & Saleem, M. (2010) · Psychological Bulletin · doi:10.1037/a0018251 · Cited by 2,048
A Global Perspective on Child Sexual Abuse: Meta-Analysis of Prevalence Around the World
Stoltenborgh, M., van IJzendoorn, M. H., Euser, E. M., & Bakermans-Kranenburg, M. J. (2011) · Child Maltreatment · doi:10.1177/1077559511403920 · Cited by 2,010
Effect of Aerobic Exercise on Blood Pressure: A Meta-Analysis of Randomized, Controlled Trials
Whelton, S. P., Chin, A., Xin, X., & He, J. (2002) · Annals of Internal Medicine · doi:10.7326/0003-4819-136-7-200204020-00006 · Cited by 1,970
Snyder, H. R. (2013) · Psychological Bulletin · doi:10.1037/a0028727 · Cited by 1,667
Quantifying publication bias in meta-analysis
Lin, L., & Chu, H. (2018) · Biometrics · doi:10.1111/biom.12817 · Cited by 1,674
Effects of occupational stress management intervention programs: A meta-analysis.
Richardson, K. M., & Rothstein, H. R. (2008) · Journal of Occupational Health Psychology · doi:10.1037/1076-8998.13.1.69 · Cited by 1,374
Does culture influence what and how we think? Effects of priming individualism and collectivism.
Oyserman, D., & Lee, S. W. S. (2008) · Psychological Bulletin · doi:10.1037/0033-2909.134.2.311 · Cited by 1,390
Andrews, G., Cuijpers, P., Craske, M. G., McEvoy, P., & Titov, N. (2010) · PLoS ONE · doi:10.1371/journal.pone.0013196 · Cited by 1,311
Mind-sets matter: A meta-analytic review of implicit theories and self-regulation.
Burnette, J. L., O'Boyle, E. H., VanEpps, E. M., Pollack, J. M., & Finkel, E. J. (2013) · Psychological Bulletin · doi:10.1037/a0029531 · Cited by 1,367
Fearon, R. P., Bakermans-Kranenburg, M. J., Van IJzendoorn, M. H., Lapsley, A.-M., & Roisman, G. I. (2010) · Child Development · doi:10.1111/j.1467-8624.2009.01405.x · Cited by 1,326
Fitting three-level meta-analytic models in R: A step-by-step tutorial
Assink, M., & Wibbelink, C. J. M. (2016) · The Quantitative Methods for Psychology · doi:10.20982/tqmp.12.3.p154 · Cited by 1,330
Kyu, H. H., Bachman, V. F., Alexander, L. T., Mumford, J. E., Afshin, A., Estep, K., Veerman, J. L., Delwiche, K., Iannarone, M. L., Moyer, M. L., et al. (2016) · BMJ · doi:10.1136/bmj.i3857 · Cited by 1,272
Social Psychiatry and Psychiatric Epidemiology
(2013) · Social Psychiatry and Psychiatric Epidemiology · doi:10.1007/127.1433-9285 · Cited by 1,330
To read or not to read: a meta-analysis of print exposure from infancy to early adulthood
Mol, S. E., & Bus, A. G. (2011) · Psychological Bulletin · doi:10.1037/a0021890 · Cited by 1,299
Rosenberg, M. S. (2005) · Evolution · doi:10.1111/j.0014-3820.2005.tb01004.x · Cited by 1,285
Vancampfort, D., Stubbs, B., Mitchell, A. J., De Hert, M., Wampers, M., Ward, P. B., Rosenbaum, S., & Correll, C. U. (2015) · World Psychiatry · doi:10.1002/wps.20252 · Cited by 1,271
Global ecological impacts of invasive species in aquatic ecosystems
Gallardo, B., Clavero, M., Sánchez, M. I., & Vilà, M. (2016) · Global Change Biology · doi:10.1111/gcb.13004 · Cited by 1,245
Mezulis, A. H., Abramson, L. Y., Hyde, J. S., & Hankin, B. L. (2004) · Psychological Bulletin · doi:10.1037/0033-2909.130.5.711 · Cited by 1,369
Ownby, R. L., Crocco, E., Acevedo, A., John, V., & Loewenstein, D. (2006) · Archives of General Psychiatry · doi:10.1001/archpsyc.63.5.530 · Cited by 1,493
Digital Games, Design, and Learning: A Systematic Review and Meta-Analysis
Clark, D. B., Tanner-Smith, E. E., & Killingsworth, S. S. (2016) · Review of Educational Research · doi:10.3102/0034654315582065 · Cited by 1,274
Chu, C., Buchman-Schmitt, J. M., Stanley, I. H., Hom, M. A., Tucker, R. P., Hagan, C. R., Rogers, M. L., Podlogar, M. C., Chiurliza, B., Ringer, F. B., et al. (2017) · Psychological Bulletin · doi:10.1037/bul0000123 · Cited by 1,231
Culturally adapted mental health intervention: a meta-analytic review
Griner, D., & Smith, T. B. (2006) · Psychotherapy: Theory, Research, Practice, Training · doi:10.1037/0033-3204.43.4.531 · Cited by 1,236
Men and things, women and people: A meta-analysis of sex differences in interests.
Su, R., Rounds, J., & Armstrong, P. I. (2009) · Psychological Bulletin · doi:10.1037/a0017364 · Cited by 1,307
Field, A. P., & Gillett, R. (2010) · British Journal of Mathematical and Statistical Psychology · doi:10.1348/000711010x502733 · Cited by 1,162
Smith, T. B., McCullough, M. E., & Poll, J. (2003) · Psychological Bulletin · doi:10.1037/0033-2909.129.4.614 · Cited by 1,150
Kyrgiou, M., Koliopoulos, G., Martin-Hirsch, P., Arbyn, M., Prendiville, W., & Paraskevaidis, E. (2006) · The Lancet · doi:10.1016/s0140-6736(06)68181-6 · Cited by 1,120
Patall, E. A., Cooper, H., & Robinson, J. C. (2008) · Psychological Bulletin · doi:10.1037/0033-2909.134.2.270 · Cited by 1,101
Andrews, G., Basu, A., Cuijpers, P., Craske, M. G., McEvoy, P., English, C. L., & Newby, J. M. (2018) · Journal of Anxiety Disorders · doi:10.1016/j.janxdis.2018.01.001 · Cited by 922
Elder abuse prevalence in community settings: a systematic review and meta-analysis
Yon, Y., Mikton, C. R., Gassoumis, Z. D., & Wilber, K. H. (2017) · The Lancet Global Health · doi:10.1016/s2214-109x(17)30006-2 · Cited by 1,074
Smetana, G. W., Lawrence, V. A., & Cornell, J. E. (2006) · Annals of Internal Medicine · doi:10.7326/0003-4819-144-8-200604180-00009 · Cited by 1,040
Methodological issues and advances in biological meta-analysis
Nakagawa, S., & Santos, E. S. A. (2012) · Evolutionary Ecology · doi:10.1007/s10682-012-9555-5 · Cited by 1,032
Outdoor Particulate Matter Exposure and Lung Cancer: A Systematic Review and Meta-Analysis
Hamra, G. B., Guha, N., Cohen, A., Laden, F., Raaschou‐Nielsen, O., Samet, J. M., Vineis, P., Forastiere, F., Saldiva, P. H. N., Yorifuji, T., et al. (2014) · Environmental Health Perspectives · doi:10.1289/ehp/1408092 · Cited by 1,152
Atkinson, R. W., Kang, S., Anderson, H. R., Mills, I., & Walton, H. (2014) · Thorax · doi:10.1136/thoraxjnl-2013-204492 · Cited by 1,044
Vazquez, G., Duval, S., Jacobs, D. R., & Silventoinen, K. (2007) · Epidemiologic Reviews · doi:10.1093/epirev/mxm008 · Cited by 976
Self-citationPostmenopausal Hormone Replacement Therapy: Scientific Review
Nelson, H. D., Humphrey, L. L., Nygren, P., Teutsch, S. M., & Allan, J. D. (2002) · JAMA · doi:10.1001/jama.288.7.872 · Cited by 990
Fischer, P., Krueger, J. I., Greitemeyer, T., Vogrincic, C., Kastenmüller, A., Frey, D., Heene, M., Wicher, M., & Kainbacher, M. (2011) · Psychological Bulletin · doi:10.1037/a0023304 · Cited by 983
Added value of dialogic parent–child book readings: A meta-analysis
Mol, S. E., Bus, A. G., de Jong, M. T., & Smeets, D. J. H. (2008) · Early Education and Development · doi:10.1080/10409280701838603 · Cited by 973
Stanley, T. D., & Doucouliagos, H. (2014) · Research Synthesis Methods · doi:10.1002/jrsm.1095 · Cited by 916
Cao, Z. J., & Lumineau, F. (2014) · Journal of Operations Management · doi:10.1016/j.jom.2014.09.009 · Cited by 996
Rueger, S. Y., Malecki, C. K., Pyun, Y., Aycock, C., & Coyle, S. (2016) · Psychological Bulletin · doi:10.1037/bul0000058 · Cited by 946
Effects of fibrates on cardiovascular outcomes: a systematic review and meta-analysis
Jun, M., Foote, C., Lv, J., Neal, B., Patel, A., Nicholls, S. J., Grobbee, D. E., Cass, A., Chalmers, J., & Perkovic, V. (2010) · The Lancet · doi:10.1016/s0140-6736(10)60656-3 · Cited by 927
Niche breadth predicts geographical range size: a general ecological pattern
Slatyer, R., Hirst, M. J., & Sexton, J. P. (2013) · Ecology Letters · doi:10.1111/ele.12140 · Cited by 938
Meta-analyses of factors motivating climate change adaptation behaviour
van Valkengoed, A. M., & Steg, L. (2019) · Nature Climate Change · doi:10.1038/s41558-018-0371-y · Cited by 940
Make, Buy, or Ally: A Transaction Cost Theory Meta-Analysis
Geyskens, I., Steenkamp, J. E. M., & Kumar, N. (2006) · Academy of Management Journal · doi:10.5465/amj.2006.21794670 · Cited by 925
Adam, E. K., Quinn, M. E., Tavernier, R., McQuillan, M. T., Dahlke, K. A., & Gilbert, K. E. (2017) · Psychoneuroendocrinology · doi:10.1016/j.psyneuen.2017.05.018 · Cited by 905
Kelley, J. M., Kraft‐Todd, G., Schapira, L., Kossowsky, J., & Riess, H. (2014) · PLoS ONE · doi:10.1371/journal.pone.0094207 · Cited by 968
How Effective Are Mentoring Programs for Youth? A Systematic Assessment of the Evidence
DuBois, D. L., Portillo, N., Rhodes, J. E., Silverthorn, N., & Valentine, J. C. (2011) · Psychological Science in the Public Interest · doi:10.1177/1529100611414806 · Cited by 889
The relationship between materialism and personal well-being: A meta-analysis
Dittmar, H., Bond, R., Hurst, M., & Kasser, T. (2014) · Journal of Personality and Social Psychology · doi:10.1037/a0037409 · Cited by 890
Racial/ethnic discrimination and well-being during adolescence: A meta-analytic review.
Benner, A. D., Wang, Y., Shen, Y., Boyle, A. E., Polk, R., & Cheng, Y. (2018) · American Psychologist · doi:10.1037/amp0000204 · Cited by 960
Meng, X., & D’Arcy, C. (2012) · PLoS ONE · doi:10.1371/journal.pone.0038268 · Cited by 964
Contingency management for treatment of substance use disorders: a meta‐analysis
Prendergast, M., Podus, D., Finney, J., Greenwell, L., & Roll, J. (2006) · Addiction · doi:10.1111/j.1360-0443.2006.01581.x · Cited by 877
Leucocyte telomere length and risk of cardiovascular disease: systematic review and meta-analysis
Haycock, P. C., Heydon, E. E., Kaptoge, S., Butterworth, A. S., Thompson, A., & Willeit, P. (2014) · BMJ · doi:10.1136/bmj.g4227 · Cited by 856
Hadfield, J. D., & Nakagawa, S. (2010) · Journal of Evolutionary Biology · doi:10.1111/j.1420-9101.2009.01915.x · Cited by 829
Correll, C. U., Galling, B., Pawar, A., Krivko, A., Bonetto, C., Ruggeri, M., Craig, T. J., Nordentoft, M., Srihari, V. H., Guloksuz, S., et al. (2018) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2018.0623 · Cited by 826
Adjusting for publication bias in the presence of heterogeneity
Terrin, N., Schmid, C. H., Lau, J., & Olkin, I. (2003) · Statistics in Medicine · doi:10.1002/sim.1461 · Cited by 809
Recognition of Emotions in Autism: A Formal Meta-Analysis
Uljarevic, M., & Hamilton, A. (2013) · Journal of Autism and Developmental Disorders · doi:10.1007/s10803-012-1695-5 · Cited by 824
Sexual assault victimization and psychopathology: A review and meta-analysis
Dworkin, E. R., Menon, S. V., Bystrynski, J., & Allen, N. E. (2017) · Clinical Psychology Review · doi:10.1016/j.cpr.2017.06.002 · Cited by 825
Shi, L., & Lin, L. (2019) · Medicine · doi:10.1097/md.0000000000015987 · Cited by 818
Dent, A. L., & Koenka, A. C. (2015) · Educational Psychology Review · doi:10.1007/s10648-015-9320-8 · Cited by 1,011
Social influence approaches to encourage resource conservation: a meta-analysis
Abrahamse, W., & Steg, L. (2013) · Global Environmental Change · doi:10.1016/j.gloenvcha.2013.07.029 · Cited by 802
p -Curve and Effect Size: Correcting for Publication Bias Using Only Significant Results
Simonsohn, U., Nelson, L. D., & Simmons, J. P. (2014) · Perspectives on Psychological Science · doi:10.1177/1745691614553988 · Cited by 717
The neglect of child neglect: A meta-analytic review of the prevalence of neglect
Stoltenborgh, M., Bakermans-Kranenburg, M. J., & van IJzendoorn, M. H. (2013) · Social Psychiatry and Psychiatric Epidemiology · doi:10.1007/s00127-012-0549-y · Cited by 736
Yap, M. B. H., Pilkington, P. D., Ryan, S., & Jorm, A. F. (2013) · Journal of Affective Disorders · doi:10.1016/j.jad.2013.11.007 · Cited by 848
Long-term opioid management for chronic noncancer pain
Noble, M., Treadwell, J. R., Tregear, S., Coates, V. H., Wiffen, P. J., Akafomo, C., Schoelles, K. M., & Chou, R. (2010) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd006605.pub2 · Cited by 748
Childhood maltreatment and obesity: systematic review and meta-analysis
Danese, A., & Tan, M. (2014) · Molecular Psychiatry · doi:10.1038/mp.2013.54 · Cited by 709
Peters, J. L., Sutton, A. J., Jones, D. R., Abrams, K. R., & Rushton, L. (2007) · Statistics in Medicine · doi:10.1002/sim.2889 · Cited by 716
O’Dea, R. E., Lagisz, M., Jennions, M. D., Koricheva, J., Noble, D. W. A., Parker, T. H., Gurevitch, J., Page, M. J., Stewart, G. B., Moher, D., et al. (2021) · Biological reviews/Biological reviews of the Cambridge Philosophical Society · doi:10.1111/brv.12721 · Cited by 825
Astill, R. G., Van der Heijden, K. B., Van IJzendoorn, M. H., & Van Someren, E. J. W. (2012) · Psychological Bulletin · doi:10.1037/a0028204 · Cited by 707
Does Homework Improve Academic Achievement? A Synthesis of Research, 1987–2003
Cooper, H., Robinson, J. C., & Patall, E. A. (2006) · Review of Educational Research · doi:10.3102/00346543076001001 · Cited by 1,230
Meta-Analysis of the Reasoned Action Approach (RAA) to Understanding Health Behaviors
McEachan, R., Taylor, N. J., Harrison, R., Lawton, R., Gardner, P. H., & Conner, M. T. (2016) · Annals of Behavioral Medicine · doi:10.1007/s12160-016-9798-4 · Cited by 706
Food cue reactivity and craving predict eating and weight gain: A meta-analytic review
Boswell, R. G., & Kober, H. (2016) · Obesity Reviews · doi:10.1111/obr.12354 · Cited by 703
Molendijk, M. L., Spinhoven, P., Polak, M., Bus, B. A. A., Penninx, B. W. J. H., & Elzinga, B. M. (2014) · Molecular Psychiatry · doi:10.1038/mp.2013.105 · Cited by 702
The Effects of Hot Spots Policing on Crime: An Updated Systematic Review and Meta-Analysis
Braga, A. A., Papachristos, A. V., & Hureau, D. M. (2012) · Justice Quarterly · doi:10.1080/07418825.2012.673632 · Cited by 727
Meta‐analysis of human connection to nature and proenvironmental behavior
Whitburn, J., Linklater, W. L., & Abrahamse, W. (2019) · Conservation Biology · doi:10.1111/cobi.13381 · Cited by 700
Gardening is beneficial for health: A meta-analysis
Soga, M., Gaston, K. J., & Yamaura, Y. (2017) · Preventive Medicine Reports · doi:10.1016/j.pmedr.2016.11.007 · Cited by 688
“If publication biaswas indicated by Egger's test, we performed a trim and fill analysis (with the R0 estimator) (Duval and Tweedie, 2000).”
MethodInterpretation of tests of heterogeneity and bias in meta-analysis
Ioannidis, J. P. A. (2008) · Journal of Evaluation in Clinical Practice · doi:10.1111/j.1365-2753.2008.00986.x · Cited by 696
Personality and prosocial behavior: A theoretical framework and meta-analysis.
Thielmann, I., Spadaro, G., & Balliet, D. P. (2019) · Psychological Bulletin · doi:10.1037/bul0000217 · Cited by 743
Risk of gastrointestinal haemorrhage with long term use of aspirin: meta-analysis
Derry, S. (2000) · BMJ · doi:10.1136/bmj.321.7270.1183 · Cited by 675
Lench, H. C., Flores, S. A., & Bench, S. W. (2011) · Psychological Bulletin · doi:10.1037/a0024244 · Cited by 766
Macnamara, B. N., Hambrick, D. Z., & Oswald, F. L. (2014) · Psychological Science · doi:10.1177/0956797614535810 · Cited by 656
Cabral, R. R., & Smith, T. B. (2011) · Journal of Counseling Psychology · doi:10.1037/a0025266 · Cited by 651
Vancampfort, D., Correll, C. U., Galling, B., Probst, M., De Hert, M., Ward, P. B., Rosenbaum, S., Gaughran, F., Lally, J., & Stubbs, B. (2016) · World Psychiatry · doi:10.1002/wps.20309 · Cited by 659
The Effects of Vocabulary Intervention on Young Children’s Word Learning
Marulis, L. M., & Neuman, S. B. (2010) · Review of Educational Research · doi:10.3102/0034654310377087 · Cited by 668
Polanin, J. R., Espelage, D. L., & Pigott, T. (2012) · School Psychology Review · doi:10.1080/02796015.2012.12087375 · Cited by 696
Metadata
- Authors
- Duval, S., & Tweedie, R.
- Affiliations
- Colorado State University (US)
- University of Minnesota (US)
- ORCID
- Duval, Sue
- Venue
- Journal of the American Statistical Association
- Publisher
- Informa UK Limited
- Published
- 2000-03
- Volume
- 95
- Issue
- 449
- Pages
- 89-98
- ISSN
- 0162-1459 (ISSN-L)
- 1537-274X
- OpenAlex
- W2043565183
- Cited by
- 2,955
- References
- 40
- Citations by year
- 2026: 106 · 2025: 167 · 2024: 189 · 2023: 171 · 2022: 233 · 2021: 275 · 2020: 259 · 2019: 179 · 2018: 184 · 2017: 160 · 2016: 142 · 2015: 149 · 2014: 128 · 2013: 91 · 2012: 101
- FWCI
- 21.09
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Journal of the American Statistical Association (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Social Sciences › Library and Information Sciences › scientometrics and bibliometrics research
- Keywords
- publication bias, trim and fill, missing studies, funnel plot, nonparametric estimators, selection bias, meta-analysis, effect size estimation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
