Publication prejudices: An experimental study of confirmatory bias in the peer review system
The DOI matches its Crossref record; 28 of the 29 references it cites were found in a registry or an index.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“journal peer reviewers are more likely to recommend publication of reports with statistically positive results.”
From Reporting research results: a moral obligation for all researchers. (2007) · cites this paper as [7]
Experimental — accuracy not yet measuredIn the citing paper
“The bias against negative results appears to be a consequence of both editors and reviewers finding them less compelling for publication and from authors themselves not trying to publish them 8-12”
From Initial evidence of research quality of registered reports compared with the standard publishing model (2021) · cites this paper as [10]
In the citing paper
“Positive results -- finding that a treatment is effective or that there is a relationship between two variables -- are more likely to be published than negative results 6-12 .”
From Initial evidence of research quality of registered reports compared with the standard publishing model (2021) · cites this paper as [10]
Citations
Cites
Social Theory and Social Structure
Ross, E. J., & Merton, R. K. (1958) · The American Catholic Sociological Review · doi:10.2307/3709394 · Cited by 10,315
The American Psychological Association
Langfeld, H. S. (1919) · Science · doi:10.1126/science.49.1254.51 · Cited by 7,475
(1975) · Medical Entomology and Zoology · Cited by 7,245
Social Theory and Social Structure.
Bock, K. E., & Merton, R. K. (1969) · American Sociological Review · doi:10.2307/2091967 · Cited by 6,660
Effects of externally mediated rewards on intrinsic motivation.
Deci, E. L. (1971) · Journal of Personality and Social Psychology · doi:10.1037/h0030644 · Cited by 4,170
Lehman, H. (1972) · Philosophy of Science · doi:10.1086/288419 · Cited by 2,998
Cognitive Therapy and the Emotional Disorders
Ince, L. P. (1977) · American Journal of Psychotherapy · doi:10.1176/appi.psychotherapy.1977.31.4.633 · Cited by 2,892
Lepper, M. R., Greene, D., & Nisbett, R. E. (1973) · Journal of Personality and Social Psychology · doi:10.1037/h0035519 · Cited by 2,559
Lambert, N. M., Ketzenberger, K. E., & Kemp, C. A. (2014) · Encyclopedia of Special Education · doi:10.1002/9781118660584.ese0122 · Cited by 2,220
Social Stratification in Science
Cole, J. R., Cole, S., & deB. Beaver, D. (1974) · American Journal of Physics · doi:10.1119/1.1987897 · Cited by 1,135
Research works printed but not linked: 10
- Adams, J. A.Human memory. New York: McGraw-Hill, 1967.
- American Psychological Association.Standards for educational and psychological tests and manuals. Washington, D.C.: American Psychological Association, 1966.
- Bacon, F.Novum organum. New York: Bobbs-Merrill, 1960. (Originally published, 1621.)
- Dixon, B.What is science for? New York: Harper & Row, 1973.
- Lakatos, I., & Musgrave, A. (Eds.),Criticism and the growth of knowledge. London: Cambridge University Press, 1970.
- Mahoney, M. J., & DeMonbreun, B. G. Confirmatory bias in scientists and non-scientists.Cognitive Therapy and Research, 1977 (in press).
- Mahoney, M. J., & Kimper, T. P. From ethics to logic: A survey of scientists. In M. J. Mahoney,Scientist as subject. Cambridge, Massachusetts: Ballinger, 1976. Pp. 187?193.
- Neisser, V.Cognitive psychology. New York: Appleton-Century-Crofts, 1967.
- Popper, K. R.Objective knowledge: An evaluative approach. London: Oxford University Press, 1972.
- Weimer, W. B.Psychology and the conceptual foundations of science. Hillsdale, New Jersey; Lawrence Erlbaum Associates, 1977.
Cited by
Confirmation Bias: A Ubiquitous Phenomenon in Many Guises
Nickerson, R. S. (1998) · Review of General Psychology · doi:10.1037/1089-2680.2.2.175 · Cited by 7,267
Lord, C. G., Ross, L., & Lepper, M. R. (1979) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.37.11.2098 · Cited by 4,583
Rabin, M. (1997) · Journal of Economic Literature · Cited by 3,080
Why do humans reason? Arguments for an argumentative theory
Mercier, H., & Sperber, D. (2011) · Behavioral and Brain Sciences · doi:10.1017/s0140525x10000968 · Cited by 2,254
The preregistration revolution
Nosek, B. A., Ebersole, C. R., DeHaven, A. C., & Mellor, D. T. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1708274114 · Cited by 1,998
Metadata
- Authors
- Mahoney, M. J.
- Affiliations
- Pennsylvania State University (US)
- Corresponding author
- Mahoney, Michael J.
- ORCID
- Mahoney, Michael J.
- Venue
- Cognitive Therapy and Research
- Published
- 1977-06
- Volume
- 1
- Issue
- 2
- Pages
- 161-175
- ISSN
- 0147-5916 (ISSN-L)
- 1573-2819
- OpenAlex
- W2164136002
- Semantic Scholar
- d8cf5c798397b6a0be1b41f18f979f0988f1ece7
- Cited by
- 1,013
- References
- 29
- Citations by year
- 2026: 22 · 2025: 18 · 2024: 25 · 2023: 28 · 2022: 52 · 2021: 45 · 2020: 52 · 2019: 40 · 2018: 31 · 2017: 51 · 2016: 38 · 2015: 43 · 2014: 41 · 2013: 51 · 2012: 23
- FWCI
- 5.62
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- www.springer.com/tdm (tdm)
- Locations
- Cognitive Therapy and Research (publishedVersion)
- Topics
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Library and Information Sciences › Publishing and Scholarly Communication
- Keywords
- confirmation bias, peer review, publication bias, scientific journals, manuscript review, interobserver agreement, negative results, epistemology of science
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.