Finding the missing science: The fate of studies submitted for review by a human subjects committee.

Cooper, Harris; DeNeve, Kristina; Charlton, Kelly

doi:10.1037/1082-989x.2.4.447

Cited by 145Psychology Research and BibliometricsMeta-analysis and systematic reviewsAcademic Publishing and Open Access

Finding the missing science: The fate of studies submitted for review by a human subjects committee.

Open access

Cooper, Harris, DeNeve, Kristina, Charlton, Kelly·1997·Psychological Methods

DOI 10.1037/1082-989x.2.4.447

Open this paper in CoChat — free account

Abstract

Publication bias, including prejudice against the null hypothesis, and other biasing filters may operate on researchers as well as journal editors and reviewers. A survey asked 33 psychology researchers to describe the fate of 159 studies approved by their departmental human subjects review committee. About two thirds of completed studies did not result in published summaries. About half of the unpublished studies fell out of the process for reasons other than methodological quality. Among these, lack of interest and aims that did not include publication were cited more often than nonsignificant results as the reasons why publication was not pursued. However, significant findings were more likely than nonsignificant findings to be submitted for meeting presentation or publication. These results indicate attention needs to be paid to improving how psychological scientists communicate, especially to the creation of prospective research registers.

Integrity signals

Registry

Crossref · 10.1037/1082-989x.2.4.447

Cites

4 of 4 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Publication bias can result from journals not publishing non-significant primary outcomes or authors not submitting such studies.
“Publication bias may be the result of journals choosing not to publish papers with non-significant primary outcomes (Hopewell et al. 2009) or study authors choosing not to submit the study for publication (Cooper et al. 1997).”
From Effects of after-school programs with at-risk youth on attendance and externalizing behaviors: A systematic review and meta-analysis (2015) · cites this paper as Cooper et al. 1997

Consequences of prejudice against the null hypothesis.
Greenwald, A. G. (1975) · Psychological Bulletin · doi:10.1037/h0076157 · Cited by 1,022 · cited 1× · see citing works
“Greenwald (1975) demonstrated that prejudice against the null hypothesis is not restricted to journal reviewers and editors only.”
Background
Publication bias: the case for an international registry of clinical trials.
Simes, R. J. (1986) · Journal of Clinical Oncology · doi:10.1200/jco.1986.4.10.1529 · Cited by 747 · see citing works
Minimizing the Three Stages of Publication Bias
Chalmers, T. C. (1990) · JAMA · doi:10.1001/jama.1990.03440100104016 · Cited by 226 · see citing works
Cooper, Harris, and Larry V. Hedges, eds., The Handbook of Research Synthesis. New York: Russell Sage Foundation, 1994.
Cooper, H. M., & Hedges, L. V. (1994) · Journal of International Crisis and Risk Communication Research · Cited by 169 · see citing works
Self-citation

Flagged first, then by citations

Sorted by citations

Effectiveness of L2 instruction: A research synthesis and quantitative meta-analysis
Norris, J. M., & Ortega, L. (2000) · Language Learning · doi:10.1111/0023-8333.00136 · Cited by 2,414 · see citing works
Systematic Review of the Empirical Evidence of Study Publication Bias and Outcome Reporting Bias
Dwan, K., Altman, D. G., Arnaiz, J. A., Bloom, J., Chan, A.-W., Cronin, E., Decullier, E., Easterbrook, P. J., Von Elm, E., Gamble, C., et al. (2008) · PLoS ONE · doi:10.1371/journal.pone.0003081 · Cited by 1,546 · see citing works
Publication bias in the social sciences: Unlocking the file drawer
Franco, A., Malhotra, N., & Simonovits, G. (2014) · Science · doi:10.1126/science.1255484 · Cited by 1,542 · see citing works
Systematic Review of the Empirical Evidence of Study Publication Bias and Outcome Reporting Bias — An Updated Review
Dwan, K., Gamble, C., Williamson, P. R., Kirkham, J. J., & the Reporting Bias Group (2013) · PLoS ONE · doi:10.1371/journal.pone.0066844 · Cited by 1,127 · see citing works
Dissemination and publication of research findings: an updated review of related biases
Song, F., Parekh, Hooper, L., Loke, Y., Ryder, J., Sutton, A., Hing, C., Kwok, C., Pang, C., & Harvey, I. (2010) · Health Technology Assessment · doi:10.3310/hta14080 · Cited by 1,039 · see citing works
The rules of the game called psychological science
Bakker, M., van Dijk, A., & Wicherts, J. M. (2012) · Perspectives on Psychological Science · doi:10.1177/1745691612459060 · Cited by 800 · see citing works
The Scientific Status of Projective Techniques
Lilienfeld, S. O., Wood, J. M., & Garb, H. N. (2000) · Gothic.net · doi:10.1111/1529-1006.002 · Cited by 759 · see citing works
Degrees of Freedom in Planning, Running, Analyzing, and Reporting Psychological Studies: A Checklist to Avoid p-Hacking
Wicherts, J. M., Veldkamp, C. L. S., Augusteijn, H. E. M., Bakker, M., van Aert, R. C. M., & van Assen, M. A. L. M. (2016) · Frontiers in Psychology · doi:10.3389/fpsyg.2016.01832 · Cited by 754 · see citing works
The Empirical Status of Social Learning Theory: A Meta‐Analysis
Pratt, T. C., Cullen, F. T., Sellers, C. S., Winfree, L. T., Madensen, T. D., Daigle, L. E., Fearn, N. E., & Gau, J. M. (2009) · Justice Quarterly · doi:10.1080/07418820903379610 · Cited by 681 · see citing works
On doing better science: From thrill of discovery to policy implications
Antonakis, J. (2017) · The Leadership Quarterly · doi:10.1016/j.leaqua.2017.01.006 · Cited by 483 · see citing works
Publication Bias in Research Synthesis: Sensitivity Analysis Using A Priori Weight Functions.
Vevea, J. L., & Woods, C. M. (2005) · Psychological Methods · doi:10.1037/1082-989x.10.4.428 · Cited by 461 · see citing works
SELF‐CONTROL AND VICTIMIZATION: A META‐ANALYSIS
Pratt, T. C., Turanovic, J. J., Fox, K. A., & Wright, K. A. (2013) · Criminology · doi:10.1111/1745-9125.12030 · Cited by 430 · see citing works
Grey Literature in Meta-Analyses
Conn, V. S., Valentine, J. C., Cooper, H. M., & Rantz, M. J. (2003) · Nursing Research · doi:10.1097/00006199-200307000-00008 · Cited by 400 · see citing works
Self-citation
Publication Bias in Psychology: A Diagnosis Based on the Correlation between Effect Size and Sample Size
Kühberger, A., Fritz, A., & Scherndl, T. (2014) · PLoS ONE · doi:10.1371/journal.pone.0105825 · Cited by 399 · see citing works
Quasi-Replication and the Contract of Error: Lessons from Sex Ratios, Heritabilities and Fluctuating Asymmetry
Palmer, A. R. (2000) · Annual Review of Ecology and Systematics · doi:10.1146/annurev.ecolsys.31.1.441 · Cited by 385 · see citing works
Publication bias examined in meta-analyses from psychology and medicine: A meta-meta-analysis
van Aert, R. C. M., Wicherts, J. M., & van Assen, M. A. L. M. (2019) · PLoS ONE · doi:10.1371/journal.pone.0215052 · Cited by 345 · see citing works
Disentangling the effects of low self-esteem and stressful events on depression: Findings from three longitudinal studies.
Orth, U., Robins, R. W., & Meier, L. L. (2009) · Journal of Personality and Social Psychology · doi:10.1037/a0015645 · Cited by 273 · see citing works
Extent of Non-Publication in Cohorts of Studies Approved by Research Ethics Committees or Included in Trial Registries
Schmucker, C., Schell, L. K., Portalupi, S., Oeller, P., Cabrera, L., Bassler, D., Schwarzer, G., Scherer, R. W., Antes, G., von Elm, E., et al. (2014) · PLoS ONE · doi:10.1371/journal.pone.0114023 · Cited by 269 · see citing works
Meta-analysis using effect size distributions of only statistically significant studies.
van Assen, M. A. L. M., van Aert, R. C. M., & Wicherts, J. M. (2015) · Psychological Methods · doi:10.1037/met0000025 · Cited by 265 · see citing works
Comparison of protocols and registry entries to published reports for randomised controlled trials
Dwan, K., Altman, D. G., Cresswell, L., Blundell, M., Gamble, C. L., & Williamson, P. R. (2011) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.mr000031.pub2 · Cited by 243 · see citing works
An Agenda for Open Science in Communication
Dienlin, T., Johannes, N., Bowman, N. D., Masur, P. K., Engesser, S., Kümpel, A. S., Lukito, J., Bier, L. M., Zhang, R., Johnson, B. K., et al. (2020) · Journal of Communication · doi:10.1093/joc/jqz052 · Cited by 221 · see citing works
Taking Stock of the Relationship Between Gang Membership and Offending
Pyrooz, D. C., Turanovic, J. J., Decker, S. H., & Wu, J. (2015) · Criminal Justice and Behavior · doi:10.1177/0093854815605528 · Cited by 202 · see citing works
Meta-analysis of the relationship between teachers’ self-efficacy and attitudes toward inclusive education
Yada, A., Leskinen, M., Savolainen, H. K., & Schwab, S. (2021) · Teaching and Teacher Education · doi:10.1016/j.tate.2021.103521 · Cited by 246 · see citing works
Reducing bias, increasing transparency and calibrating confidence with preregistration
Hardwicke, T. E., & Wagenmakers, E. (2023) · Nature Human Behaviour · doi:10.1038/s41562-022-01497-2 · Cited by 190 · see citing works
Extent of publication bias in different categories of research cohorts: a meta-analysis of empirical studies
Song, F., Parekh-Bhurke, S., Hooper, L., Loke, Y. K., Ryder, J. J., Sutton, A. J., Hing, C. B., & Harvey, I. (2009) · BMC Medical Research Methodology · doi:10.1186/1471-2288-9-79 · Cited by 175 · see citing works
Iatrogenic Effects of Group Treatment for Antisocial Youths.
Weiss, B., Caron, A., Ball, S., Tapp, J., Johnson, M., & Weisz, J. R. (2005) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.73.6.1036 · Cited by 170 · see citing works
Self-Concept of Students with Learning Disabilities: A Meta-Analysis
Bear, G. G., Minke, K. M., & Manning, M. A. (2002) · School Psychology Review · doi:10.1080/02796015.2002.12086165 · Cited by 166 · see citing works
A historical review of publication bias
Marks‐Anglin, A., & Chen, Y. (2020) · Research Synthesis Methods · doi:10.1002/jrsm.1452 · Cited by 165 · see citing works
Conducting Meta-analyses of Proportions in R
Wang, N. (2023) · Journal of Behavioral Data Science · doi:10.35566/jbds/v3n2/wang · Cited by 143 · see citing works
Effects of after-school programs with at-risk youth on attendance and externalizing behaviors: A systematic review and meta-analysis
Kremer, K. P., Maynard, B. R., Polanin, J. R., Vaughn, M. G., & Sarteschi, C. M. (2015) · Journal of Youth and Adolescence · doi:10.1007/s10964-014-0226-4 · Cited by 132 · see citing works
Outcome-Reporting Bias in Education Research
Pigott, T. D., Valentine, J. C., Polanin, J. R., Williams, R. T., & Canada, D. D. (2013) · Educational Researcher · doi:10.3102/0013189x13507104 · Cited by 127 · see citing works
School Resource Officers and Exclusionary Discipline in U.S. High Schools: A Systematic Review and Meta-analysis
Fisher, B. W., & Hennessy, E. A. (2016) · Adolescent Research Review · doi:10.1007/s40894-015-0006-8 · Cited by 124 · see citing works
Underreporting in Psychology Experiments
Franco, A., Malhotra, N. A., & Simonovits, G. (2015) · Social Psychological and Personality Science · doi:10.1177/1948550615598377 · Cited by 119 · see citing works
Why Are Medical and Health-Related Studies Not Being Published? A Systematic Review of Reasons Given by Investigators
Song, F., Loke, Y. K., & Hooper, L. (2014) · PLoS ONE · doi:10.1371/journal.pone.0110418 · Cited by 117 · see citing works
Correcting for Publication Bias in a Meta-Analysis with the P-uniform* MethodPreprint
van Aert, R. C. M., & van Assen, M. A. L. M. (2018) · doi:10.31222/osf.io/zqjr9 · Cited by 108 · see citing works
Publication bias as a function of study characteristics.
Coburn, K. M., & Vevea, J. L. (2015) · Psychological Methods · doi:10.1037/met0000046 · Cited by 107 · see citing works
Psychological Bulletin: Editorial.
Cooper, H. (2003) · Psychological Bulletin · doi:10.1037/0033-2909.129.1.3 · Cited by 102 · see citing works
Self-citation
Systematic Reviews With Meta-Analysis
Crocetti, E. (2015) · Emerging Adulthood · doi:10.1177/2167696815617076 · Cited by 91 · see citing works
Improving transparency and reproducibility through registration: The status of intervention trials published in clinical psychology journals.
Cybulski, Ł., Mayo‐Wilson, E., & Grant, S. (2016) · Journal of Consulting and Clinical Psychology · doi:10.1037/ccp0000115 · Cited by 91 · see citing works
Maternal personality and psychopathology as determinants of parenting behavior: A quantitative integration of two parenting literatures.
McCabe, J. E. (2013) · Psychological Bulletin · doi:10.1037/a0034835 · Cited by 91 · see citing works
Missing a critical piece of the pie: simple document search strategies inadequate for systematic reviews
Wilson, D. B. (2009) · Journal of Experimental Criminology · doi:10.1007/s11292-009-9085-5 · Cited by 86 · see citing works
On gender and philosophical intuition: Failure of replication and other negative results
Seyedsayamdost, H. (2015) · Philosophical Psychology · doi:10.1080/09515089.2014.893288 · Cited by 52 · see citing works
A critical review of research on executive functions in sport and exercise
Furley, P., Schütz, L.-M., & Wood, G. (2023) · International Review of Sport and Exercise Psychology · doi:10.1080/1750984x.2023.2217437 · Cited by 84 · see citing works
Heterogeneity in direct replications in psychology and its association with effect size.
Olsson-Collentine, A., Wicherts, J. M., & van Assen, M. A. L. M. (2020) · Psychological Bulletin · doi:10.1037/bul0000294 · Cited by 78 · see citing works
The Effect of Publication Bias on the Q Test and Assessment of Heterogeneity
Augusteijn, H., van Aert, R. V., & van Assen, M. (2019) · Psychological methods · doi:10.1037/met0000197 · Cited by 69 · see citing works
The Replication Paradox: Combining Studies can Decrease Accuracy of Effect Size Estimates
Nuijten, M., van Assen, M., Veldkamp, C. L. S., & Wicherts, J. (2015) · Review of General Psychology · doi:10.1037/gpr0000034 · Cited by 60 · see citing works
Advancing the science of psychological assessment: The Rorschach Inkblot Method as exemplar.
Weiner, I. B. (2001) · Psychological Assessment · doi:10.1037/1040-3590.13.4.423 · Cited by 68 · see citing works
Success of collaboration for sustainable agriculture: a case study meta-analysis
Velten, S., Jager, N. W., & Newig, J. (2021) · Environment Development and Sustainability · doi:10.1007/s10668-021-01261-y · Cited by 66 · see citing works
Toward a resolution of the Rorschach controversy.
Garb, H. N., Wood, J. M., Nezworski, M. T., Grove, W. M., & Stejskal, W. J. (2001) · Psychological Assessment · doi:10.1037/1040-3590.13.4.433 · Cited by 61 · see citing works
Toward more rigorous and informative nutritional epidemiology: The rational space between dismissal and defense of the status quo
Brown, A. W., Aslibekyan, S. W., Bier, D. M., da Silva, R. F., Hoover, A. W., Klurfeld, D. M., Loken, E., Mayo‐Wilson, E., Menachemi, N., Pavela, G., et al. (2021) · Critical Reviews in Food Science and Nutrition · doi:10.1080/10408398.2021.1985427 · Cited by 57 · see citing works
Meta‐Analysis in Criminal Justice and Criminology: What It is, When It's Useful, and What to Watch Out for
Pratt, T. C. (2010) · Journal of Criminal Justice Education · doi:10.1080/10511251003693678 · Cited by 57 · see citing works
Arrested Development: Misguided Directions in Deterrence Theory and Policy
Pickett, J. T., & Roche, S. P. (2016) · Criminology & Public Policy · doi:10.1111/1745-9133.12217 · Cited by 58 · see citing works
Accuracy of Effect Size Estimates from Published Psychological Research
Brand, A., Bradley, M. T., Best, L. A., & Stoica, G. (2008) · Perceptual and Motor Skills · doi:10.2466/pms.106.2.645-649 · Cited by 52 · see citing works
Revisiting the Happy-Productive Worker Thesis from a Eudaimonic Perspective: A Systematic Review
Peiró, J., Montesa, D., Soriano, A., Kożusznik, M. W., Villajos, E., Magdaleno, J., Djourova, N. P., & Ayala, Y. (2021) · Sustainability · doi:10.3390/su13063174 · Cited by 45 · see citing works
Meta-analysis selection bias in marketing research
Eisend, M., & Tarrahi, F. (2014) · International Journal of Research in Marketing · doi:10.1016/j.ijresmar.2014.03.006 · Cited by 53 · see citing works
The Weak Spots in Contemporary Science (and How to Fix Them)
Wicherts, J. M. (2017) · Animals · doi:10.3390/ani7120090 · Cited by 53 · see citing works
Social-Emotional Learning Interventions for Students With Special Educational Needs: A Systematic Literature Review
Hassani, S., & Schwab, S. (2021) · Frontiers in Education · doi:10.3389/feduc.2021.808566 · Cited by 53 · see citing works
Times are changing, bias isn’t: A meta-meta-analysis on publication bias detection practices, prevalence rates, and predictors in industrial/organizational psychology.
Siegel, M., Eder, J. S. N., Wicherts, J. M., & Pietschnig, J. (2021) · Journal of Applied Psychology · doi:10.1037/apl0000991 · Cited by 55 · see citing works
Studying Cognitive Adaptations in the Field of Sport: Broad or Narrow Transfer? A Comment on Allen, Fioratou, and McGeorge (2011)
Furley, P., & Memmert, D. (2011) · Perceptual and Motor Skills · doi:10.2466/05.23.pms.113.5.481-488 · Cited by 52 · see citing works
Is there evidence of publication biases in JDM research?
Renkewitz, F., Fuchs, H. M., & Fiedler, S. (2011) · Judgment and Decision Making · doi:10.1017/s1930297500004289 · Cited by 49 · see citing works
Connecting Research to Action: Perspectives on Research Utilisation
Gray, M., Sharland, E., Heinsch, M., & Schubert, L. (2014) · The British Journal of Social Work · doi:10.1093/bjsw/bcu089 · Cited by 50 · see citing works
Sharing successes and hiding failures: ‘reporting bias’ in learning and teaching research
Dawson, P., & Dawson, S. L. (2016) · Studies in Higher Education · doi:10.1080/03075079.2016.1258052 · Cited by 48 · see citing works
African-American crack abusers and drug treatment initiation: barriers and effects of a pretreatment intervention
Wechsberg, W. M., Zule, W. A., Riehman, K. S., Luseno, W. K., & Lam, W. K. K. (2007) · Substance Abuse Treatment Prevention and Policy · doi:10.1186/1747-597x-2-10 · Cited by 47 · see citing works
Optimizing Research Payoff
Miller, J., & Ulrich, R. (2016) · Perspectives on Psychological Science · doi:10.1177/1745691616649170 · Cited by 34 · see citing works
Experimental Manipulations of Personal Control do Not Increase Illusory Pattern Perception
van Elk, M., & Lodder, P. (2018) · Collabra Psychology · doi:10.1525/collabra.155 · Cited by 40 · see citing works
The Effects of One versus Two Years of Intensive Reading Intervention Implemented with Late Elementary Struggling Readers
Miciak, J., Roberts, G. J., Taylor, W. P., Solis, M. R., Ahmed, Y., Vaughn, S. R., & Fletcher, J. M. (2017) · Learning Disabilities Research and Practice · doi:10.1111/ldrp.12159 · Cited by 38 · see citing works
Interpreting t-Statistics Under Publication Bias: Rough Rules of Thumb
Winship, C., & Zhuo, X. (2018) · Journal of Quantitative Criminology · doi:10.1007/s10940-018-9387-8 · Cited by 40 · see citing works
To Be or Not to Be: The Entrepreneur in Neo-Schumpeterian Growth Theory
Henrekson, M., Johansson, D., & Karlsson, J. (2023) · Entrepreneurship Theory and Practice · doi:10.1177/10422587221141679 · Cited by 36 · see citing works
Practical tools and strategies for researchers to increase replicability
Nuijten, M. B. (2018) · Developmental Medicine & Child Neurology · doi:10.1111/dmcn.14054 · Cited by 36 · see citing works
Transparent, Open, and Reproducible Prevention Science
Grant, S., Wendt, K. E., Leadbeater, B. J. R., Supplee, L. H., Mayo‐Wilson, E., Gardner, F. E. M., & Bradshaw, C. P. (2022) · Prevention Science · doi:10.1007/s11121-022-01336-w · Cited by 36 · see citing works
An unexpected influence of widely used significance thresholds on the distribution of reported P‐values
Ridley, J., Kolm, N., FRECKELTON, R. P., & Gage, M. J. G. (2007) · Journal of Evolutionary Biology · doi:10.1111/j.1420-9101.2006.01291.x · Cited by 34 · see citing works
Review of grey literature on drug prevention among young people.
McGrath, Y., Sumnall, H. R., Edmonds, K., McVeigh, J., & Bellis, M. A. (2006) · Cited by 33 · see citing works
Evidence for, and challenges to, sensory recruitment models of visual working memory
Adam, K. C. S., Rademaker, R. L., & Serences, J. T. (2022) · Routledge eBooks · doi:10.4324/9781003158134-2 · Cited by 34 · see citing works
Implications of the open science era for educational psychology research syntheses
Patall, E. A. (2021) · Educational Psychologist · doi:10.1080/00461520.2021.1897009 · Cited by 34 · see citing works
Degrees of freedom in planning, running, analyzing, and reporting psychological studies A checklist to avoid p-hackingPreprint
Wicherts, J. M., Veldkamp, C. L. S., Augusteijn, H. E. M., Bakker, M., van Aert, R. C. M., & van Assen, M. A. L. M. (2016) · doi:10.31219/osf.io/umq8d · Cited by 30 · see citing works
Meta-Analysis of Research in Social and Personality Psychology
Johnson, B. T., & Eagly, A. H. (2014) · Cambridge University Press eBooks · doi:10.1017/cbo9780511996481.031 · Cited by 27 · see citing works
Favorable and publicly funded studies are more likely to be published: a systematic review and meta-analysis
Canestaro, W. J., Hendrix, N., Bansal, A., Sullivan, S. D., Devine, E. B., & Carlson, J. J. (2017) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2017.08.004 · Cited by 27 · see citing works
The Research Reproducibility Crisis and Economics of Science
Maniadis, Z., & Tufano, F. (2017) · The Economic Journal · doi:10.1111/ecoj.12526 · Cited by 27 · see citing works
How best to quantify replication success? A simulation study on the comparison of replication success metrics
Muradchanian, J., Hoekstra, R., Kiers, H. A. L., & van Ravenzwaaij, D. (2021) · Royal Society Open Science · doi:10.1098/rsos.201697 · Cited by 33 · see citing works
On the Robustness of the Modality Effect: Attempting to Replicate a Basic Finding
Lindow, S., Fuchs, H. M., Fürstenberg, A., Kleber, J., Schweppe, J., & Rummer, R. (2011) · Zeitschrift für Pädagogische Psychologie · doi:10.1024/1010-0652/a000049 · Cited by 26 · see citing works
Determinants of selective reporting: A taxonomy based on content analysis of a random selection of the literature
van der Steen, J. T., van den Bogert, C. A., van Soest-Poortvliet, M. V., Farsani, S. F., Otten, R., ter Riet, G., & Bouter, L. (2018) · PLoS ONE · doi:10.1371/journal.pone.0188247 · Cited by 26 · see citing works
A retrospective analysis of dissemination biases in the brief alcohol intervention literature.
Tanner-Smith, E. E., & Polanin, J. R. (2015) · Psychology of Addictive Behaviors · doi:10.1037/adb0000014 · Cited by 26 · see citing works
“Prospective designs, such as those used by Cooper, DeNeve, and Charlton (1997), are preferred because the theoretical population of potentially published studies is known at the outset.”
Method
Criminal responsibility: Meta‐analysis and study space
Kois, L. E., & Chauhan, P. (2018) · Behavioral Sciences & the Law · doi:10.1002/bsl.2343 · Cited by 25 · see citing works
Sorting the File Drawer: A Typology for Describing Unpublished Studies
Lishner, D. A. (2021) · Perspectives on Psychological Science · doi:10.1177/1745691620979831 · Cited by 14 · see citing works
Methods for addressing publication bias in school psychology journals: A descriptive review of meta-analyses from 1980 to 2019
McClain, M. B., Callan, G. L., Harris, B., Floyd, R. G., Haverkamp, C. R., Golson, M. E., Longhurst, D. N., & Benallie, K. J. (2021) · Journal of School Psychology · doi:10.1016/j.jsp.2020.11.002 · Cited by 23 · see citing works
Establishing Journalistic Standards for the Publication of Negative Results
Kittelman, A., Gion, C., Horner, R. H., Levin, J. R., & Kratochwill, T. R. (2018) · Remedial and Special Education · doi:10.1177/0741932517745491 · Cited by 23 · see citing works
Metascience on Peer Review: Testing the Effects of a Study’s Originality and Statistical Significance in a Field Experiment
Elson, M., Huff, M., & Utz, S. (2020) · Advances in Methods and Practices in Psychological Science · doi:10.1177/2515245919895419 · Cited by 22 · see citing works
Mathematics achievement and orientation: A systematic review and meta-analysis of education technology
Rakes, C. R., Ronau, R. N., Bush, S. B., Driskell, S. O. S., Niess, M. L., & Pugalee, D. (2020) · Educational Research Review · doi:10.1016/j.edurev.2020.100337 · Cited by 29 · see citing works
The role of results in deciding to publish: A direct comparison across authors, reviewers, and editors based on an online survey
Muradchanian, J., Hoekstra, R., Kiers, H., & van Ravenzwaaij, D. (2023) · PLoS ONE · doi:10.1371/journal.pone.0292279 · Cited by 25 · see citing works
Effect Size Estimation From t-Statistics in the Presence of Publication Bias: A Brief Review of Existing Approaches With Some Extensions
Ulrich, R., Miller, J., & Erdfelder, E. (2018) · Zeitschrift für Psychologie · doi:10.1027/2151-2604/a000319 · Cited by 15 · see citing works
Schizotypy and hemispheric asymmetry: Results from two Chapman scales, the O-LIFE questionnaire, and two laterality measures
Schofield, K. L., & Möhr, C. (2013) · Laterality Asymmetries of Body Brain and Cognition · doi:10.1080/1357650x.2013.789883 · Cited by 18 · see citing works
Meta-analysis
Wilson, D. B. (2009) · Springer eBooks · doi:10.1007/978-0-387-77650-7_10 · Cited by 16 · see citing works
A Trim and Fill Examination of the Extent of Publication Bias in Communication Research
Carpenter, C. J. (2012) · Communication Methods and Measures · doi:10.1080/19312458.2011.651347 · Cited by 15 · see citing works
Standard Errors in Quantitative Criminology: Taking Stock and Looking Forward
Sweeten, G. (2020) · Journal of quantitative criminology · doi:10.1007/s10940-020-09463-9 · Cited by 9 · see citing works
Steady-state visually evoked potentials and feature-based attention: Pre-registered null results and a focused review of methodological considerations
Adam, K. C. S., Chang, L., Rangan, N., & Serences, J. (2020) · bioRxiv · doi:10.1162/jocn_a_01665 · Cited by 8 · see citing works
How much evidence is p > .05? Stimulus pre-testing and null primary outcomes in violent video games research.
Hilgard, J., Engelhardt, C. R., Bartholow, B. D., & Rouder, J. N. (2015) · Psychology of Popular Media Culture · doi:10.1037/ppm0000102 · Cited by 11 · see citing works
Publication Bias in the Social Sciences
Malhotra, N. A., & Zigerell, L. J. (2022) · Oxford University Press eBooks · doi:10.1093/oso/9780190938550.003.0013 · Cited by 10 · see citing works
Correcting for outcome reporting bias in a meta-analysis: A meta-regression approach
van Aert, R. C. M., & Wicherts, J. M. (2023) · Behavior Research Methods · doi:10.3758/s13428-023-02132-2 · Cited by 11 · see citing works
Four editorial proposals to improve social sciences research and publication
Joseph, J., & Baldwin, S. (2000) · International Journal of Risk & Safety in Medicine · doi:10.3233/jrs-2000-218 · Cited by 10 · see citing works
Guest Editor’s Introduction to Special Issue: The Science and Practice of Research Synthesis
Littell, J. H. (2013) · Journal of the Society for Social Work and Research · doi:10.5243/jsswr.2013.19 · Cited by 9 · see citing works

Authors: Cooper, H., DeNeve, K., & Charlton, K.
Affiliations: University of Missouri (US)
ORCID: Cooper, Harris
Venue: Psychological Methods
Publisher: American Psychological Association (APA)
Published: 1997-12
Volume: 2
Issue: 4
Pages: 447-452
ISSN: 1082-989X (ISSN-L)
1939-1463
DOI: 10.1037/1082-989x.2.4.447
OpenAlex: W2120579488
Cited by: 144
References: 4
Citations by year: 2026: 2 · 2025: 3 · 2024: 3 · 2023: 10 · 2022: 4 · 2021: 16 · 2020: 10 · 2019: 3 · 2018: 11 · 2017: 10 · 2016: 7 · 2015: 9 · 2014: 12 · 2013: 8 · 2012: 3
FWCI: 3.13
Type: journal article
Language: en
Open access: green
hdl.handle.net
Licence: cc-by-nc-nd
Locations: Psychological Methods (publishedVersion)
DukeSpace (Duke University) (submittedVersion, cc-by-nc-nd)
Topics: Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
Social Sciences › Decision Sciences › Information Systems and Management › Academic Publishing and Open Access
Keywords: publication bias, missing studies, unpublished research, null hypothesis significance testing, non-significant results, research register, scholarly communication, psychology research practice

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Finding the missing science: The fate of studies submitted for review by a human subjects committee.

Abstract

Integrity signals

Claims the paper makes

Claims other papers make about this one

Cites

Cited by

Related

Record

About these checks