Publication prejudices: An experimental study of confirmatory bias in the peer review system
The DOI matches its Crossref record; 28 of the 29 references it cites were found in a registry or an index.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“journal peer reviewers are more likely to recommend publication of reports with statistically positive results.”
From Reporting research results: a moral obligation for all researchers. (2007) · cites this paper as [7]
Experimental — accuracy not yet measuredIn the citing paper
“The bias against negative results appears to be a consequence of both editors and reviewers finding them less compelling for publication and from authors themselves not trying to publish them 8-12”
From Initial evidence of research quality of registered reports compared with the standard publishing model (2021) · cites this paper as [10]
In the citing paper
“Positive results -- finding that a treatment is effective or that there is a relationship between two variables -- are more likely to be published than negative results 6-12 .”
From Initial evidence of research quality of registered reports compared with the standard publishing model (2021) · cites this paper as [10]
Citations
Cites
Social Theory and Social Structure
Ross, E. J., & Merton, R. K. (1958) · The American Catholic Sociological Review · doi:10.2307/3709394 · Cited by 10,315
The American Psychological Association
Langfeld, H. S. (1919) · Science · doi:10.1126/science.49.1254.51 · Cited by 7,475
(1975) · Medical Entomology and Zoology · Cited by 7,245
Social Theory and Social Structure.
Bock, K. E., & Merton, R. K. (1969) · American Sociological Review · doi:10.2307/2091967 · Cited by 6,660
Effects of externally mediated rewards on intrinsic motivation.
Deci, E. L. (1971) · Journal of Personality and Social Psychology · doi:10.1037/h0030644 · Cited by 4,170
Lehman, H. (1972) · Philosophy of Science · doi:10.1086/288419 · Cited by 2,998
Cognitive Therapy and the Emotional Disorders
Ince, L. P. (1977) · American Journal of Psychotherapy · doi:10.1176/appi.psychotherapy.1977.31.4.633 · Cited by 2,892
Lepper, M. R., Greene, D., & Nisbett, R. E. (1973) · Journal of Personality and Social Psychology · doi:10.1037/h0035519 · Cited by 2,559
Lambert, N. M., Ketzenberger, K. E., & Kemp, C. A. (2014) · Encyclopedia of Special Education · doi:10.1002/9781118660584.ese0122 · Cited by 2,220
Social Stratification in Science
Cole, J. R., Cole, S., & deB. Beaver, D. (1974) · American Journal of Physics · doi:10.1119/1.1987897 · Cited by 1,135
Research works printed but not linked: 10
- Adams, J. A.Human memory. New York: McGraw-Hill, 1967.
- American Psychological Association.Standards for educational and psychological tests and manuals. Washington, D.C.: American Psychological Association, 1966.
- Bacon, F.Novum organum. New York: Bobbs-Merrill, 1960. (Originally published, 1621.)
- Dixon, B.What is science for? New York: Harper & Row, 1973.
- Lakatos, I., & Musgrave, A. (Eds.),Criticism and the growth of knowledge. London: Cambridge University Press, 1970.
- Mahoney, M. J., & DeMonbreun, B. G. Confirmatory bias in scientists and non-scientists.Cognitive Therapy and Research, 1977 (in press).
- Mahoney, M. J., & Kimper, T. P. From ethics to logic: A survey of scientists. In M. J. Mahoney,Scientist as subject. Cambridge, Massachusetts: Ballinger, 1976. Pp. 187?193.
- Neisser, V.Cognitive psychology. New York: Appleton-Century-Crofts, 1967.
- Popper, K. R.Objective knowledge: An evaluative approach. London: Oxford University Press, 1972.
- Weimer, W. B.Psychology and the conceptual foundations of science. Hillsdale, New Jersey; Lawrence Erlbaum Associates, 1977.
Cited by
Items 401–500 of 1,000
Replication in music psychology
Frieler, K., Müllensiefen, D., Fischinger, T., Schlemmer, K., Jakubowski, K., & Lothwesen, K. (2013) · Musicae Scientiae · doi:10.1177/1029864913495404 · Cited by 26
“Kahneman (2011) and others (e.g., Mahoney, 1977) have demonstrated that scientists are prone to cognitive biases as well.”
BackgroundWhat's Wrong with Research Literatures? And how to Make Them Right
Howard, G. S., Hill, T. L., Maxwell, S. E., Baptista, T. M., Farias, M. H., Coelho, C., Coulter-Kern, M., & Coulter-Kern, R. (2009) · Review of General Psychology · doi:10.1037/a0015319 · Cited by 22
“313) Gigerenzer’s (1993) summary reflects an enormous literature (e.g., Atkinson, Furlong, & Wampold, 1982; Chase & Chase, 1976; Greenwald, 1975; Kerr, Tolliver, & Petree, 1977; Mahoney, 1977; Sterling, 1959) that demonstrates editors’ and reviewers’ preference for significant results (and their bias against nonsignif…”
The underlying process generating Lotka's Law and the statistics of exceedances
Huber, J. C. (1998) · Information Processing & Management · doi:10.1016/s0306-4573(98)00013-2 · Cited by 23
Arad, S., & Carnevale, P. J. D. (1994) · Journal of Conflict Resolution · doi:10.1177/0022002794038003004 · Cited by 28
Competency testing for reviewers and editors
Yalow, R. S. (1982) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00011729 · Cited by 26
Stall, L. M., & Petrocelli, J. V. (2022) · Applied Cognitive Psychology · doi:10.1002/acp.3998 · Cited by 24
The Meta-Science of Adult Statistical Word Segmentation: Part 1
Hartshorne, J. K., Skorb, L., Dietz, S. L., Garcia, C., Iozzo, G., Lamirato, K., Ledoux, J. R., Mu, J., Murdock, K., Ravid, J., et al. (2019) · Collabra Psychology · doi:10.1525/collabra.181 · Cited by 25
“How reliable is peer review (Mahoney, 1977; Newton, 2010; Olson et al., 2002)?”
BackgroundPsychologists’ psychologies of psychologists in a time of crisis.
Morawski, J. G. (2020) · History of Psychology · doi:10.1037/hop0000140 · Cited by 23
Langfeldt, L. (2022) · Springer eBooks · doi:10.1007/978-3-030-75263-7_13 · Cited by 23
Pharmacotherapy of impulsive aggression: A quality comparison of controlled studies
Felthous, A. R., Lake, S. L., Rundle, B. K., & Stanford, M. S. (2013) · International Journal of Law and Psychiatry · doi:10.1016/j.ijlp.2013.04.017 · Cited by 23
Report of the Ad Hoc Committee on Double-Blind Refereeing
Cox, D., Gleser, L., Perlman, M., Reid, N., & Roeder, K. (1993) · Statistical Science · doi:10.1214/ss/1177010904 · Cited by 21
Inconsistent-handed advantage in episodic memory extends to paragraph-level materials
Prichard, E. C., & Christman, S. D. (2016) · Memory · doi:10.1080/09658211.2016.1257725 · Cited by 21
“Due to the tendency for psychological journals to accept less ambiguous and positive results (Mahoney, 1977), the earliest publications describing an experimental paradigm are likely to present the cleanest results.”
BackgroundHartgerink, C. H. J., & van Zelst, M. (2018) · Publications · doi:10.3390/publications6020021 · Cited by 21
“, statistical significance increasing the perceived quality of methods; [11]).”
BackgroundA STUDY OF THE PUBLICATION OF SCHOLARLY OUTPUT IN ECONOMICS JOURNALS
Piette, M. J., & Ross, K. L. (2016) · RePEc: Research Papers in Economics · Cited by 21
Reviewers’ Perceptions of the Peer Review Process in Journalism and Mass Communication
Curtin, P. A., Russial, J. T., & Tefertiller, A. C. (2017) · Journalism & Mass Communication Quarterly · doi:10.1177/1077699017736031 · Cited by 23
“Two social science studies employed journal editors as confederates in experimental treatments involving deception to test for reviewer bias (Epstein, 1990; Mahoney, 1977).”
BackgroundHolman, B. (2018) · Synthese · doi:10.1007/s11229-018-1690-2 · Cited by 25
“1979) and that this phenomena occurs when scientists evaluate evidence (Mahoney 1977; Koehler 1993; Hergovich et al. 2010).”
BackgroundA large scale randomized controlled trial on herding in peer-review discussions
Stelmakh, I., Rastogi, C., Shah, N. B., Singh, A., & Daumé, H. (2023) · PLoS ONE · doi:10.1371/journal.pone.0287443 · Cited by 30
Yarkoni, T. (2012) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2012.00072 · Cited by 27
“Outside of a sense of duty to one’s profession and peers, and perhaps a pragmatic desire to curry favor with editors, scientists have little to gain by volunteering their time as reviewers, let alone by turning in high-quality reviews on time (Mahoney, 1977; Hojat et al., 2003).”
BackgroundWhen nonreliability of reviews indicates solid science
Eckberg, D. L. (1991) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00065791 · Cited by 24
Sigelman, L. (1999) · Political Analysis · doi:10.1093/oxfordjournals.pan.a029813 · Cited by 23
Nabavi, A., Safari, F., Shmoury, A. H., Tabet, S., Perdomo-Luna, C., & Celi, L. A. (2026) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2026.106418 · Cited by 24
Cavalcanti, M. F. R., & Silva, A. L. (2024) · Management Learning · doi:10.1177/13505076231220121 · Cited by 21
“This is frequent in publication ‘guidelines’ which, for example, are beneficial to some audiences and researchers, and harmful and/or costly to the less powerful (Blackburn and Hakel, 2006; Mahoney, 1977; Roumbanis, 2017).”
BackgroundCurrent and Emerging Ethical Issues for Counseling Psychology
Goodyear, R. K., & Sinnett, E. R. (1984) · The Counseling Psychologist · doi:10.1177/0011000084123009 · Cited by 20
La théorie argumentative du raisonnement
Mercier, H. (2009) · HAL · Cited by 32
Heim, A., Ravaud, P., Baron, G., & Boutron, I. (2018) · BMC Medicine · doi:10.1186/s12916-018-1167-7 · Cited by 19
Rodenburg, K. S., Rowan, M. J., Nixon, A., & Hughes, J. C. (2022) · Sustainability · doi:10.3390/su14159598 · Cited by 19
Descriptive Analysis of the 2014 Race-Based Healthcare Disparities Measurement Literature
Goodman, M. S., Gilbert, K. L., Hudson, D., Milam, L. A., & Colditz, G. A. (2016) · Journal of Racial and Ethnic Health Disparities · doi:10.1007/s40615-016-0281-5 · Cited by 19
“While the lack of evidence for disparity might be highly informative, or if statistically underpowered, addressed through systematic review and meta-analysis, peer reviewers and editors may be less likely to accept/publish findings that are not statistically significant [18, 21].”
Inter-study and intra-study replications in leading marketing journals: a longitudinal analysis
Kwon, E. S., Yan, S., Lee, J. S., & Reid, L. N. (2017) · European Journal of Marketing · doi:10.1108/ejm-07-2015-0450 · Cited by 20
A model of the editorial process in academic journals
Bayar, O., & Chemmanur, T. J. (2021) · Research Policy · doi:10.1016/j.respol.2021.104339 · Cited by 20
Houghton, F. T. (2022) · Journal of the Medical Library Association JMLA · doi:10.5195/jmla.2022.1441 · Cited by 21
Peer Review and Evaluation of R&D Impacts
Bozeman, B. L. (1993) · Springer eBooks · doi:10.1007/978-1-4757-5182-6_5 · Cited by 19
Novel predictions arise from contradictions
Yanai, I., & Lercher, M. J. (2021) · Genome biology · doi:10.1186/s13059-021-02371-6 · Cited by 21
“…Table 1 The contingency table for students in the two groups (expecting either a positive or a negative correlation between wealth and happiness) and what they observed [2] with a reviewer ’ s own assumptions are reviewed much more harshly than those that support the reviewer ’ s beliefs [21 – 23].”
ResultTechnology assisted research assessment: algorithmic bias and transparency issues
Thelwall, M. A., & Kousha, K. (2023) · Aslib Journal of Information Management · doi:10.1108/ajim-04-2023-0119 · Cited by 19
When all you have is a hammer: how social justice distorts what we know about racial disparities
Iceland, J., & Silver, E. (2024) · Theory and Society · doi:10.1007/s11186-024-09568-1 · Cited by 18
“Peer reviewers provide more positive reviews of articles that support their beliefs, theoretical orientations, and political views (Abramowitz et al., 1975 ; Koehler, 1993; Mahoney, 1977 ).”
Authors and reviewers who suffer from confirmatory bias
García, J. A., Sánchez, R. M. R., & Fdez-Valdivia, J. (2016) · Scientometrics · doi:10.1007/s11192-016-2079-y · Cited by 18
“A large and growing body of psychological research suggests that people have a cognitive bias that leads them to misinterpret new information as supporting previously held hypotheses (Lewicka 1998; Maccoun 1998; Mahoney 1977; Nickerson 1998; Oswald and Grosjean 2004).”
BackgroundMethodInfluentialLa Psicología Social de la Ciencia: Revisión y discusión de una nueva área de investigación
Íñiguez-Rueda, L., & Pallí, C. (2002) · Anales de Psicología · Cited by 38
Lee, H., Lindquist, J. D., & Acito, F. (1997) · Journal of Business Research · doi:10.1016/s0148-2963(96)00278-0 · Cited by 22
Reflections on the peer review process
Marsh, H. W., & Ball, S. (1991) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00065912 · Cited by 18
A review of the review process: manuscript peer-review in biomedical research
Kumar, M. N. (2009) · Biology and medicine · Cited by 24
Scientific Misconduct as a Dilemma for Nursing
Hawley, D. J., & Jeffers, J. M. (1992) · Journal of Nursing Scholarship · doi:10.1111/j.1547-5069.1992.tb00699.x · Cited by 18
Publication of research on controversial topics: The early acceptance procedure
Armstrong, J. S. (1996) · International Journal of Forecasting · doi:10.1016/0169-2070(95)00626-5 · Cited by 18
“The scientist” as role model for “the psychologist”
John, I. D. (1986) · Australian Psychologist · doi:10.1080/00050068608256183 · Cited by 21
Cumulative advantage and citation performance of repeat authors in scholarly journals
Siler, K., Vincent‐Lamarre, P., Sugimoto, C. R., & Larivière, V. (2022) · PLoS ONE · doi:10.1371/journal.pone.0265831 · Cited by 21
“Confirmation bias has been documented in science, where evaluators prefer work that reflects the status quo [80].”
BackgroundTempest in a teacup: An analysis of p-Hacking in organizational research
Gupta, A., & Bosco, F. A. (2023) · PLoS ONE · doi:10.1371/journal.pone.0281938 · Cited by 19
The Justification of Journal Rankings – A Pilot Study
Zehrer, A. (2007) · Scandinavian Journal of Hospitality and Tourism · doi:10.1080/15022250701339775 · Cited by 17
“Journal editors seeking market feedback on their article selection process can also benefit from such evaluations’’ (Brooks et al., 1991; Mahoney, 1977; Van Vught, 2000).”
BackgroundImplications of ideological bias in social psychology on clinical practice.
Silander, N. C., Geczy, B., Marks, O., & Mather, R. D. (2020) · Clinical Psychology Science and Practice · doi:10.1111/cpsp.12312 · Cited by 21
“Yet other studies indicate that responses to certain papers are guided by confirmatory bias, of which political persuasion is a component (Koehler, 1993; Mahoney, 1977).”
BackgroundBack to the Future: Can Counterhistory Accelerate Theoretical Advancement in Management?
Bylund, P. L., & Packard, M. D. (2021) · Academy of Management Perspectives · doi:10.5465/amp.2018.0113 · Cited by 17
It's Not Us, It's You: Why Isn't Research on Minority Workers Appearing in Our “Top-Tier” Journals?
Diaz, I., & Bergman, M. E. (2013) · Industrial and Organizational Psychology · doi:10.1111/iops.12010 · Cited by 18
“It is clear that research where the null is rejected tends to be evaluated more positively and published more often than research where the null is accepted (Dickersin, 1990; Mahoney, 1977).”
BackgroundBias in peer review: a case study
Walker, R. D., Barros, B., Conejo, R., Neumann, K., & Telefont, M. (2015) · F1000Research · doi:10.12688/f1000research.6012.1 · Cited by 20
“…categories of papers (e.g. studies challenging conventional wisdom 20 ; replication studies 21,22 and studies reporting negative results 12,23 ), to the unreliability of the review process 23–25 ), to its inability to detect errors and fraud 26 , and to unethical practices by editors and…”
Practical Suggestions for Improving Scholarly Peer Review Quality and Reducing Cycle Times
Ralph, P. (2016) · Communications of the Association for Information Systems · doi:10.17705/1cais.03813 · Cited by 17
Auspurg, K., & Hinz, T. (2011) · Jahrbücher für Nationalökonomie und Statistik · doi:10.1515/jbnst-2011-5-607 · Cited by 20
“These used similar versions of manuscripts with on the one hand positive (hypothesis-confirming) and on the other hand negative (hypothesis-rejecting) results (Epstein 1990; Mahoney 1977).”
MethodThe Death of Organization Science
Lounsbury, M. D. (2003) · Journal of Management Inquiry · doi:10.1177/1056492603257726 · Cited by 17
“Astley’s argument is not novel; he builds on well-established research in the sociology of knowledge that has shown how social structure shapes how kinds of research and knowledge become accepted as well as what gets published where (e.g., Lodahl & Gordon, 1972; Mahoney, 1977; Zuckerman & Merton, 1971).”
BackgroundReviewer “bias”: Do Peters and Ceci protest too much?
Perlman, D. (1982) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00011572 · Cited by 17
Reviewer ratings and agreement on manuscripts reviewed for the Journal of Counseling Psychology.
Munley, P. H., Sharkin, B. S., & Gelso, C. J. (1988) · Journal of Counseling Psychology · doi:10.1037//0022-0167.35.2.198 · Cited by 17
Statistical Power and Research in Occupational Therapy
Ottenbacher, K. J. (1982) · The Occupational Therapy Journal of Research · doi:10.1177/153944928200200102 · Cited by 17
Precision in the manuscript review process: Hargens and Herting revisited
Lindsey, D. (1991) · Scientometrics · doi:10.1007/bf02020004 · Cited by 17
Can Scientists Rationally Assess Conditional Inferences?
Tweney, R. D., & Yachanin, S. A. (1985) · Social Studies of Science · doi:10.1177/030631285015001006 · Cited by 17
Psychologists update their beliefs about effect sizes after replication studies
McDiarmid, A. D., Tullett, A. M., Whitt, C. M., Vazire, S., Smaldino, P. E., & Stephens, J. E. (2021) · Nature Human Behaviour · doi:10.1038/s41562-021-01220-7 · Cited by 16
Contemporary HIV/AIDS research: Insights from knowledge management theory
Callaghan, C. W. (2017) · SAHARA-J Journal of Social Aspects of HIV/AIDS · doi:10.1080/17290376.2017.1375426 · Cited by 16
Polyzos, N. P., Valachis, A., Patavoukas, E., Papanikolaou, E. G. E., Messinis, I. E., Tarlatzis, B. C., & Devroey, P. (2011) · Human Reproduction · doi:10.1093/humrep/der044 · Cited by 18
Pasgaard, M., Fold, N., Meilby, H., & Kalvig, P. (2021) · Geografisk Tidsskrift-Danish Journal of Geography · doi:10.1080/00167223.2020.1869574 · Cited by 16
“A focus on the negative impacts of tourism could reflect a possible confirmatory bias (e.g. Mahoney, 1977) favouring a common criticism of (mass) tourism in relation to natural resource degradation (e.g. Hernandez & Leon, 2013; Kariminia et al., 2013; Kizielewicz & Lukovic, 2015).4 The undesirable…”
BackgroundBias and Reasoning: Haidt’s Theory of Moral Judgment
Liao, S. M. (2011) · Palgrave Macmillan UK eBooks · doi:10.1057/9780230305885_7 · Cited by 16
“Also, there are studies that show that peer review may be subject to the kinds of motivated reasoning that are seen in individuals (Mahoney, 1977).”
BackgroundScience, politics, and rationality in a partisan era
Kirchner, J. W. (2017) · Water Resources Research · doi:10.1002/2017wr020882 · Cited by 16
Hoffman, A. J. (2021) · Research Ethics · doi:10.1177/17470161211051230 · Cited by 16
“Mahoney (1977) describes these phenomena in mental health practice as “confirmatory bias” (p. 161) and notes that individuals suffering from depression may often confirm maladaptive belief systems while ignoring positive events that exist in their lives.”
BackgroundAusmaß und Risikofaktoren des Publication Bias in der deutschen Soziologie
Auspurg, K., Hinz, T., & Schneck, A. (2014) · KZfSS Kölner Zeitschrift für Soziologie und Sozialpsychologie · doi:10.1007/s11577-014-0284-3 · Cited by 21
The fate of published articles, submitted again
Bartko, J. J. (1982) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00011213 · Cited by 18
Bonaccorsi, A. J., Cicero, T., Ferrara, A., & Malgarini, M. (2015) · F1000Research · doi:10.12688/f1000research.6478.1 · Cited by 16
Strategies to Address Clinical Bias in the Child Custody Evaluation Process
Robb, A. (2006) · Journal of Child Custody · doi:10.1300/j190v03n02_03 · Cited by 16
“The confirmatory bias is noted as persistent amongst mental health professionals, and has even been studied and shown to exist within the peer review system of professional journals (Epstein, 1990; Mahoney, 1977).”
BackgroundConfusion between reviewer reliability and wise editorial and funding decisions
Kiesler, C. A. (1991) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00065869 · Cited by 16
A Generalized Publication Bias ModelPreprint
Schönemann, P. H., & Scargle, J. D. (2008) · arXiv (Cornell University) · doi:10.6129/cjp.2008.5001.02 · Cited by 18
“For example, in the first paper published on this topic in Behavioral and Brain Sciences, (BBS), Rosenthal & Rubin (1978) reported a FSN of 65,122 for 345 published studies. The authors concluded that "It certainly seems unlikely that there are file drawers crammed with the unpublished results of over 65,000 studies o…”
BackgroundMethodInfluentialEditorial responsibilities in manuscript review
Crandall, R. (1982) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00011316 · Cited by 20
Reflections from the peer review mirror
Cicchetti, D. V. (1991) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00066024 · Cited by 16
Standard Errors in Quantitative Criminology: Taking Stock and Looking Forward
Sweeten, G. (2020) · Journal of Quantitative Criminology · doi:10.1007/s10940-020-09463-9 · Cited by 19
“In an intriguing experiment, Mahoney (1977) tested the role of confirmatory bias among reviewers by experimentally manipulating the contents of a paper supposedly under review, simultaneously sent to 75 reviewers.”
MethodStar Wars: The Empirics Strike BackPreprint
Brodeur, A., Lé, M., Sangnier, M., & Zylberberg, Y. (2013) · SSRN Electronic Journal · doi:10.2139/ssrn.2238281 · Cited by 26
Bradley, J. V. (1982) · Bulletin of the Psychonomic Society · doi:10.3758/bf03330255 · Cited by 16
Yatırımcı Davranışında Bilişsel Önyargılar ve Hevristikler
KARAKUŞ, T. F. (2022) · Uluslararası Davranış Sürdürülebilirlik ve Yönetim Dergisi · doi:10.54709/iisbf.1135271 · Cited by 16
Machuca, D. E. (2024) · Cambridge University Press eBooks · doi:10.1017/9781009324458 · Cited by 15
Understanding and supporting anonymity policies in peer review
Nobarany, S., & Booth, K. S. (2016) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.23711 · Cited by 17
“While such biases are easy to operationalize and study, the potential effects of other types of bias such as biases in favor of positive results (Mahoney, 1977) or of results that match the reviewers’ stance (Abramowitz, Gomes, & Abramowitz, 1975) or of results that corroborate previous work (Ernst & Resch, 1994) are…”
ResultEve, M. P., Neylon, C., O’Donnell, D. P., Moore, S., Gadie, R., Odeniyi, V., & Parvin, S. (2020) · Cambridge University Press eBooks · doi:10.1017/9781108783521 · Cited by 17
Be alarmed. Some reflections about the COVID-19 risk communication in Germany
Wiedemann, P. M., & Dorl, W. (2020) · Journal of Risk Research · doi:10.1080/13669877.2020.1825984 · Cited by 17
Manoeuvring towards research decline
Saunders, J., & Wong, V. (2011) · European Journal of Marketing · doi:10.1108/03090561111111307 · Cited by 15
“Controversial findings are rejected on methodological grounds (Mahoney et al., 1977), while papers adopting the same methodology, but supporting the established thinking, are accepted.”
BackgroundScholar as E-Publisher: The Future Role of [Anonymous] Peer Review within Online Publishing
Gould, T. H. P. (2010) · Journal of Scholarly Publishing · doi:10.3138/jsp.41.4.428 · Cited by 15
Deviant scientists and scientific deviance
Bechtel, H. K., & Pearson, W. (1985) · Deviant Behavior · doi:10.1080/01639625.1985.9967676 · Cited by 15
Manwell, C., & Baker, C. M. A. (1982) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00011493 · Cited by 15
Decision-making in expert panels evaluating research: Constraints, processes and bias
Langfeldt, L. (2002) · NORA - Norwegian Open Research Archives · Cited by 15
Academic Rankings and Research GovernancePreprint
Osterloh, M., & Frey, B. S. (2010) · SSRN Electronic Journal · doi:10.5167/uzh-51741 · Cited by 18
Introduction: an interdisciplinary criminology of international criminal law
Bantekas, I. (2014) · Cambridge University Press eBooks · doi:10.1017/cbo9781107446700.002 · Cited by 14
Sun, Z. (2024) · Journal of Informetrics · doi:10.1016/j.joi.2024.101501 · Cited by 17
Sociopolitical Diversity in Psychology: The Case for Pluralism
Redding, R. E. (2013) · Routledge eBooks · doi:10.4324/9780203956229-25 · Cited by 14
Political Outcome Bias in Grading: Identifying Problems and Suggesting Solutions
Rom, M. C., & Musgrave, P. (2014) · Journal of Political Science Education · doi:10.1080/15512169.2014.894352 · Cited by 14
Biases in research evaluation: Inflated assessment, oversight, or error-type weighting?
Reich, D. A., Green, M. C., Brock, T. C., & Tetlock, P. E. (2006) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2006.06.001 · Cited by 14
A new paradigm of knowledge management: Crowdsourcing as emergent research and development
Callaghan, C. W. (2019) · Southern African Business Review · doi:10.25159/1998-8125/6041 · Cited by 16
“For Mahoney (1977), this is fundamentally at odds with Popper’s (1972) notions of falsification.”
BackgroundIdentifying Confirmatory Bias in the Field
Andrews, R. J., Logan, T. D., & Sinkey, M. (2015) · Journal of Sports Economics · doi:10.1177/1527002515617511 · Cited by 14
“Confirmatory bias has been well documented in the psychology literature—examples include Mahoney (1977); Lord, Ross, and Lepper (1979); Darley and Gross (1983); and Nickerson (1998).”
Adverse selection of reviewers
García, J. A., Sánchez, R. M. R., & Fdez-Valdivia, J. (2014) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.23249 · Cited by 14
“The problem with this system is the presence of bias (Lee, Sugimoto, Zhang, & Cronin, 2013); for example, conservativism in peer review that may suppress dissent against “mainstream” theories (Campanario & Martin, 2004; “Coping with Peer Rejection,” 2003; Martin, 1997), reviewer bias against conclusions that contradic…”
BackgroundReclaiming the subject: Decommodifying marketing knowledge?
Desmond, J. (1995) · Journal of Marketing Management · doi:10.1080/0267257x.1995.9964384 · Cited by 14
How do experts update beliefs? Lessons from a non-market environment
Sinkey, M. (2015) · Journal of Behavioral and Experimental Economics · doi:10.1016/j.socec.2015.04.003 · Cited by 14
Pride and prejudice in peer review
Kemper, K. J. (1991) · Journal of Clinical Epidemiology · doi:10.1016/0895-4356(91)90072-h · Cited by 14
Effect of Recommendations from Reviewers Suggested or Excluded by Authors
Moore, J. L., Neilson, E. G., & Siegel, V. (2011) · Journal of the American Society of Nephrology · doi:10.1681/asn.2011070643 · Cited by 14
Issues and Challenges in Coding Interventions for Meta-Analysis of Prevention Research
Devine, E. C. (1997) · PsycEXTRA Dataset · doi:10.1037/e495592006-007 · Cited by 14
Reliability and bias in peer-review practices
Rosenthal, R. (1982) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00011614 · Cited by 16
Metadata
- Authors
- Mahoney, M. J.
- Affiliations
- Pennsylvania State University (US)
- Corresponding author
- Mahoney, Michael J.
- ORCID
- Mahoney, Michael J.
- Venue
- Cognitive Therapy and Research
- Published
- 1977-06
- Volume
- 1
- Issue
- 2
- Pages
- 161-175
- ISSN
- 0147-5916 (ISSN-L)
- 1573-2819
- OpenAlex
- W2164136002
- Semantic Scholar
- d8cf5c798397b6a0be1b41f18f979f0988f1ece7
- Cited by
- 1,013
- References
- 29
- Citations by year
- 2026: 22 · 2025: 18 · 2024: 25 · 2023: 28 · 2022: 52 · 2021: 45 · 2020: 52 · 2019: 40 · 2018: 31 · 2017: 51 · 2016: 38 · 2015: 43 · 2014: 41 · 2013: 51 · 2012: 23
- FWCI
- 5.62
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- www.springer.com/tdm (tdm)
- Locations
- Cognitive Therapy and Research (publishedVersion)
- Topics
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Library and Information Sciences › Publishing and Scholarly Communication
- Keywords
- confirmation bias, peer review, publication bias, scientific journals, manuscript review, interobserver agreement, negative results, epistemology of science
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.