Publication prejudices: An experimental study of confirmatory bias in the peer review system
The DOI matches its Crossref record; 28 of the 29 references it cites were found in a registry or an index.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“journal peer reviewers are more likely to recommend publication of reports with statistically positive results.”
From Reporting research results: a moral obligation for all researchers. (2007) · cites this paper as [7]
Experimental — accuracy not yet measuredIn the citing paper
“The bias against negative results appears to be a consequence of both editors and reviewers finding them less compelling for publication and from authors themselves not trying to publish them 8-12”
From Initial evidence of research quality of registered reports compared with the standard publishing model (2021) · cites this paper as [10]
In the citing paper
“Positive results -- finding that a treatment is effective or that there is a relationship between two variables -- are more likely to be published than negative results 6-12 .”
From Initial evidence of research quality of registered reports compared with the standard publishing model (2021) · cites this paper as [10]
Citations
Cites
Social Theory and Social Structure
Ross, E. J., & Merton, R. K. (1958) · The American Catholic Sociological Review · doi:10.2307/3709394 · Cited by 10,315
The American Psychological Association
Langfeld, H. S. (1919) · Science · doi:10.1126/science.49.1254.51 · Cited by 7,475
(1975) · Medical Entomology and Zoology · Cited by 7,245
Social Theory and Social Structure.
Bock, K. E., & Merton, R. K. (1969) · American Sociological Review · doi:10.2307/2091967 · Cited by 6,660
Effects of externally mediated rewards on intrinsic motivation.
Deci, E. L. (1971) · Journal of Personality and Social Psychology · doi:10.1037/h0030644 · Cited by 4,170
Lehman, H. (1972) · Philosophy of Science · doi:10.1086/288419 · Cited by 2,998
Cognitive Therapy and the Emotional Disorders
Ince, L. P. (1977) · American Journal of Psychotherapy · doi:10.1176/appi.psychotherapy.1977.31.4.633 · Cited by 2,892
Lepper, M. R., Greene, D., & Nisbett, R. E. (1973) · Journal of Personality and Social Psychology · doi:10.1037/h0035519 · Cited by 2,559
Lambert, N. M., Ketzenberger, K. E., & Kemp, C. A. (2014) · Encyclopedia of Special Education · doi:10.1002/9781118660584.ese0122 · Cited by 2,220
Social Stratification in Science
Cole, J. R., Cole, S., & deB. Beaver, D. (1974) · American Journal of Physics · doi:10.1119/1.1987897 · Cited by 1,135
Research works printed but not linked: 10
- Adams, J. A.Human memory. New York: McGraw-Hill, 1967.
- American Psychological Association.Standards for educational and psychological tests and manuals. Washington, D.C.: American Psychological Association, 1966.
- Bacon, F.Novum organum. New York: Bobbs-Merrill, 1960. (Originally published, 1621.)
- Dixon, B.What is science for? New York: Harper & Row, 1973.
- Lakatos, I., & Musgrave, A. (Eds.),Criticism and the growth of knowledge. London: Cambridge University Press, 1970.
- Mahoney, M. J., & DeMonbreun, B. G. Confirmatory bias in scientists and non-scientists.Cognitive Therapy and Research, 1977 (in press).
- Mahoney, M. J., & Kimper, T. P. From ethics to logic: A survey of scientists. In M. J. Mahoney,Scientist as subject. Cambridge, Massachusetts: Ballinger, 1976. Pp. 187?193.
- Neisser, V.Cognitive psychology. New York: Appleton-Century-Crofts, 1967.
- Popper, K. R.Objective knowledge: An evaluative approach. London: Oxford University Press, 1972.
- Weimer, W. B.Psychology and the conceptual foundations of science. Hillsdale, New Jersey; Lawrence Erlbaum Associates, 1977.
Cited by
Items 201–300 of 1,000
Discovery and communication of important marketing findings
Armstrong, J. S. (2003) · Journal of Business Research · doi:10.1016/s0148-2963(02)00386-7 · Cited by 83
Research Governance in Academia: Are There Alternatives to Academic Rankings?Preprint
Osterloh, M., & Frey, B. S. (2009) · SSRN Electronic Journal · doi:10.5167/uzh-51875 · Cited by 84
“Reviewers find methodological shortcomings in 71 percent of papers contradicting the mainstream, compared to only 25 percent of papers supporting the mainstream (Mahoney, 1977).”
Microaggressions: Mega problems or micro issues? A meta‐analysis
Costa, P. L., McDuffie, J. W., Brown, S. E. V., He, Y., Ikner, B. N., Sabat, I. E., & Miner, K. N. (2022) · Journal of Community Psychology · doi:10.1002/jcop.22885 · Cited by 77
Peer review and competition in the Art Exhibition Game
Balietti, S., Goldstone, R. L., & Helbing, D. (2016) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1603723113 · Cited by 71
“Proc Natl Acad Sci USA 111(16):5773–5777.”
BackgroundKelly‐Woessner, A., & Woessner, M. (2006) · PS Political Science & Politics · doi:10.1017/s104909650606080x · Cited by 78
“Researchers have found that, when evaluating a colleague’s research, college professors are more critical of work that contradicts their own views ~Mahoney 1977!.”
BackgroundCallaghan, C. W. (2016) · International Journal of Disaster Risk Reduction · doi:10.1016/j.ijdrr.2016.05.004 · Cited by 82
“…dissemination which have been found to constrain theoretical innovation [45], or (iii) per-petuate bias [18,29], including con fi rmation bias [36] resulting in subjective bias as empirical evidence is sought out to support core tenets, and to discon fi rm opposing tenets [35], which belies…”
BackgroundOverconfidence in ignorant experts
Bradley, J. V. (1981) · Bulletin of the Psychonomic Society · doi:10.3758/bf03333674 · Cited by 71
We Value What Values Us: The Appeal of Identity‐Affirming Science
Morton, T. A., Haslam, S. A., Postmes, T., & Ryan, M. K. (2006) · Political Psychology · doi:10.1111/j.1467-9221.2006.00539.x · Cited by 76
“Just as is the case for the population at large, studies suggest that scientists’ judgments about the qualities of research are equally colored by their biases (e.g., Mahoney, 1977, 1979; Peters & Ceci, 1982; see also Starbuck, 2003).”
BackgroundKliewer, M. A., Freed, K. S., DeLong, D. M., Pickhardt, P. J., & Provenzale, J. M. (2005) · American Journal of Roentgenology · doi:10.2214/ajr.184.6.01841731 · Cited by 67
“This phenomenon has been referred to as “confirmatory bias” [8, 9].”
BackgroundCatastrophism, Uniformitarianism, and a Scientific Realism Debate That Makes a Difference
Stanford, P. K. (2015) · Philosophy of Science · doi:10.1086/683325 · Cited by 69
“What little experimental evidence we have on closely related questions seems simply to reinforce this concern: Mahoney (1977), for example, found that referees rated a fictitious manuscript’s methodology, data presentation, and overall scientific contribution as significantly higher and were more…”
BackgroundScience and Pseudoscience in Law Enforcement: A User-Friendly Primer
Lilienfeld, S. O., & Landfield, K. (2008) · Criminal Justice and Behavior · doi:10.1177/0093854808321526 · Cited by 77
“For example, research on the peer review process shows that clinical psychologists are more likely to accept an article if the results agree rather than disagree with their favored theoretical orientation (e.g., behavioral vs. nonbehavioral; Mahoney, 1977).”
BackgroundProfiles in Patient Safety: Confirmation Bias in Emergency Medicine
Pines, J. M. (2006) · Academic Emergency Medicine · doi:10.1111/j.1553-2712.2006.tb00990.x · Cited by 95
Peer Review and Scholarly Originality: Let 1,000 Flowers Bloom, but Don’t Step on Any
Siler, K., & Strang, D. (2016) · Science Technology & Human Values · doi:10.1177/0162243916656919 · Cited by 66
An introduction to meta-analysis for psychotherapy outcome research
Berkeljon, A., & Baldwin, S. A. (2009) · Psychotherapy Research · doi:10.1080/10503300802621172 · Cited by 68
“An important threat to the validity of meta-analysis is publication bias, which refers to the fact that journal editors, reviewers, and authors are less willing to publish studies with null or negative findings (Mahoney, 1977; Rosenthal, 1979; Begg, 1994).”
BackgroundStarbuck, W. H. (2004) · Organization Studies · doi:10.1177/0170840604046361 · Cited by 68
“For example, Mahoney (1977, 1979) found that reviewers are much more likely to recommend acceptance or minor revision for manuscripts that agree with the reviewers’ previous publications.”
Creative destruction in science
Tierney, W., Hardy, J. H., Ebersole, C. R., Leavitt, K., Viganola, D., Clemente, E. G., Gordon, M., Dreber, A., Johannesson, M., Pfeiffer, T., et al. (2020) · Organizational Behavior and Human Decision Processes · doi:10.1016/j.obhdp.2020.07.002 · Cited by 66
“A host of related findings speak to the influence of prior convictions on information processing (Koriat, Lichtenstein, & Fischhoff, 1980; Mahoney, 1977; Pitz, 1969; Ross, Lepper, & Hubbard, 1975), which is arguably rationally defensible in Bayesian terms (Baron & Jost, 2019; Krueger & Funder, 2004).”
BackgroundFallacies and student discourse: Conceptualizing the role of critical thinking in science education
Zeidler, D. L., Lederman, N. G., & Taylor, S. C. (1992) · Science Education · doi:10.1002/sce.3730760407 · Cited by 67
Hypothesis validity of clinical research.
Wampold, B. E., Davis, B., & Good, R. H. (1990) · Journal of Consulting and Clinical Psychology · doi:10.1037//0022-006x.58.3.360 · Cited by 66
Peer Review of Term Papers in Graduate Psychology Courses
Haaga, D. A. F. (1993) · Teaching of Psychology · doi:10.1207/s15328023top2001_5 · Cited by 66
Hilgartner, S. (1997) · Science Technology & Human Values · doi:10.1177/016224399702200404 · Cited by 77
Squazzoni, F., & Gandelli, C. (2012) · Journal of Informetrics · doi:10.1016/j.joi.2011.12.005 · Cited by 74
An increasing problem in publication ethics: Publication bias and editors’ role in avoiding it
Ekmekçi, P. E. (2017) · Medicine Health Care and Philosophy · doi:10.1007/s11019-017-9767-0 · Cited by 68
“Moreover, positive manuscripts are more likely to be recommended for publication with minor revisions, while peer reviewers are prone to reject or ask for major revisions for studies with negative or null results (Mahoney 1977).”
BackgroundAttitudes of referees in a multidisciplinary journal: An empirical analysis
Casnici, N., Grimaldo, F., Gilbert, N., & Squazzoni, F. (2016) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.23665 · Cited by 65
“, Bornmann, Weymuth, & Daniel, 2010; Lamont, 2009; Garc ıa, RodriguezS anchez & Fdez-Valdivia, 2015); it could also inform us about the nature of the social norms of reviewing and so help to counterbalance possible bias (e.g., Bornmann & Daniel, 2009a; Demarest, Zhang, & Sugimoto, 2014; Lee, Sugimoto, Zhang, & Cronin,…”
BackgroundDoes single blind peer review hinder newcomers?
Seeber, M., & Bacchelli, A. (2017) · Scientometrics · doi:10.1007/s11192-017-2264-7 · Cited by 70
“Quest for anonymity were backed by the Mertonian norms of Science1 and, in particular, by the norm of universalism, stating that scientific claims should be evaluated according to the same impersonal criteria’, regardless of personal or social attributes of the author (Merton 1973).”
BackgroundInfluentialThe Social and Psychological Costs of Peer Review: Stress and Coping With Manuscript Rejection
Horn, S. (2015) · Journal of Management Inquiry · doi:10.1177/1056492615586597 · Cited by 99
“, in terms of a publication track record in “accepted” outlets), the less likely the actual or symbolic membership of one’s particular peer group will be (Mahoney, 1977).”
BackgroundDebates—Hypothesis testing in hydrology: Theory and practice
Pfister, L., & Kirchner, J. W. (2017) · Water Resources Research · doi:10.1002/2016wr020116 · Cited by 66
“Indeed, studies have shown that reviewers have their own confirmation biases, rating papers more favorably when they confirm previously held beliefs [e.g., Mahoney, 1977; Koehler, 1993].”
Button, K. S., Bal, L., Clark, A., & Shipley, T. (2016) · BMC Psychology · doi:10.1186/s40359-016-0167-7 · Cited by 58
“Mahoney [26] randomly assigned referees to review 1 of 5 versions of a manuscript, all with identical introduction and methods sections, but different results”
Methods for evaluating information sources: An annotated catalogue
Hjørland, B. (2012) · Journal of Information Science · doi:10.1177/0165551512439178 · Cited by 78
“An empirical study by Mahoney [15] found that reviewers are strongly biased against manuscripts that report results contrary to their theoretical perspective.”
BackgroundIncommensurable paradigms, cognitive bias and the politics of marketing theory
Tadajewski, M. (2008) · Marketing Theory · doi:10.1177/1470593108093557 · Cited by 82
“…scholars at the institution garner more publications in that outlet) and the nature of the findings will also be important to subsequent acceptance or rejection (Brown, 1998; Mahoney, 1977), especially with regard to the extent that they support the paradigmatic presuppositions of the reviewers.”
BackgroundMethodInfluentialReporting research results: a moral obligation for all researchers.
Moher, D. (2007) · Canadian Journal of Anesthesia/Journal canadien d anesthésie · doi:10.1007/bf03022653 · Cited by 60
Benda, W. G. G., & Engels, T. (2010) · International Journal of Forecasting · doi:10.1016/j.ijforecast.2010.03.003 · Cited by 66
“Several psychological studies confirm this tendency (Mahoney, 1977; Weller, 2001, p.223-224) .”
BackgroundGetting your paper to the right journal: a case study of an academic paper
van Teijlingen, E., & Hundley, V. A. (2002) · Journal of Advanced Nursing · doi:10.1046/j.1365-2648.2002.02135.x · Cited by 60
Adversarial Collaboration: The Next Science Reform
Clark, C. J., & Tetlock, P. E. (2023) · Springer eBooks · doi:10.1007/978-3-031-29148-7_32 · Cited by 72
Ortega, J. L. (2017) · Scientometrics · doi:10.1007/s11192-017-2399-6 · Cited by 59
“Mahoney (1977) found that reviewers were biased against manuscripts which showed results contrary to their theoretical perspective.”
BackgroundThe Nonuse of Psychological Research at Two Federal Agencies
Arkes, H. R. (2003) · Psychological Science · doi:10.1111/1467-9280.01410 · Cited by 59
“Mahoney (1977) found that the congruence of research findings with a reviewer’s point of view was a significant influence on a manuscript’s rating.”
BackgroundIntroduction to a special issue on the assessment of interdisciplinary research
Laudel, G., & Origgi, G. (2006) · Research Evaluation · doi:10.3152/147154406781776066 · Cited by 59
Transforming Conservation: A Practical Guide to Evidence and Decision Making
Amano, T., Bako, L. A., Best, M., Boenisch, N., Boersch‐Supan, P. H., Browne, D., Buckley, Y. M., Burgman, M. A., Cadotte, M. W., Canessa, S., et al. (2022) · Open Book Publishers · doi:10.11647/obp.0321 · Cited by 61
A d-statistic for single-case designs that is equivalent to the usual between-groups d-statistic
Shadish, W. R., Hedges, L. V., Pustejovsky, J. E., Boyajian, J. G., Sullivan, K. J., Andrade, A., & Barrientos, J. L. (2014) · Neuropsychological Rehabilitation · doi:10.1080/09602011.2013.819021 · Cited by 54
“In meta-analyses of betweengroups experiments, the mechanism is hypothesised to be prejudice against the null hypothesis, that is, that reviewers are less likely to accept a manuscript for publication if it reports nonsignificant results (Mahoney, 1977).”
BackgroundEmperor’s New Clothes: The Reinvention of Peer Review as Myth
Macdonald, S. (2014) · Journal of Management Inquiry · doi:10.1177/1056492614554773 · Cited by 58
“Referees are known to be inherently conservative and suspicious of anything new (Luukkonen, 2012), and to prefer positive findings (House of Commons, 2011; Mahoney, 1977).”
BackgroundBias in the Counseling Process: How to Recognize and Avoid It
Morrow, K., & DEIDAN, C. T. (1992) · Journal of Counseling & Development · doi:10.1002/j.1556-6676.1992.tb01663.x · Cited by 57
A physics editor comments on Peters and Ceci's peer-review study
Adair, R. K. (1982) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00011195 · Cited by 59
Lay Judgments of Judicial Decision Making
Simon, D., & Scurich, N. (2011) · Journal of Empirical Legal Studies · doi:10.1111/j.1740-1461.2011.01238.x · Cited by 67
Reliability, fairness, objectivity and other inappropriate goals in peer review
Bailar, J. C. (1991) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00065705 · Cited by 62
The insufficiencies of methodological inadequacy
Hogan, R. T. (1982) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00011420 · Cited by 57
Clinical outcomes of behavioral treatments for pica in children with developmental disabilities
Call, N. A., Simmons, C. A., Mevers, J. E. L., & Alvarez, J. P. (2015) · Journal of Autism and Developmental Disorders · doi:10.1007/s10803-015-2375-z · Cited by 55
“In contrast, case studies that fail to replicate such treatment effects have a lower probability of publication (Mahoney 1977; Sham and Smith 2014).”
ResultThe scientific reinvention of forensic science
Koehler, J. J., Mnookin, J. L., & Saks, M. J. (2023) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2301840120 · Cited by 67
Mesquida, C., Murphy, J., Lakens, D., & Warne, J. P. (2023) · Journal of Sports Sciences · doi:10.1080/02640414.2023.2269357 · Cited by 57
“This phenomenon includes editors and reviewers selectively publishing studies with significant findings (i.e., review bias; Mahoney, 1977) and researchers deciding not to submit studies with non-significant results (i.e., the file-drawer problem; Rosenthal, 1979).”
BackgroundEitan, O., Viganola, D., Inbar, Y., Dreber, A., Johannesson, M., Pfeiffer, T., Thau, S., & Uhlmann, E. L. (2018) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2018.06.004 · Cited by 50
“Future research should investigate via a randomized experiment (Peters & Ceci, 1980, 1982; Mahoney, 1977) whether important gatekeepers such as reviewers and editors continue to favor research consistent with their political values.”
BackgroundJennings, M. M., Pany, K. J., & Reckers, P. M. J. (2006) · Accounting Horizons · doi:10.2308/acch.2006.20.3.253 · Cited by 55
Aardvark et al.: quality journals and gamesmanship in management studies
Macdonald, S., & Kam, J. (2007) · Journal of Information Science · doi:10.1177/0165551507077419 · Cited by 56
“Referees very much prefer papers that confirm their own opinions [54].”
BackgroundPersistence of false paradigms in low-power sciences
Akerlof, G. A., & Michaillat, P. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1816454115 · Cited by 54
Walker, R. D., Barros, B., Conejo, R., Neumann, K., & Telefont, M. (2015) · F1000Research · doi:10.12688/f1000research.6012.2 · Cited by 57
Peer review in forensic science
Ballantyne, K. N., Edmond, G., & Found, B. J. (2017) · Forensic Science International · doi:10.1016/j.forsciint.2017.05.020 · Cited by 59
Open science and modified funding lotteries can impede the natural selection of bad science
Smaldino, P. E., Turner, M. A., & Kallens, P. C. (2019) · Royal Society Open Science · doi:10.1098/rsos.190194 · Cited by 66
The Ombudsman: Is Review By Peers As Fair As It Appears?
Armstrong, J. S. (1982) · INFORMS Journal on Applied Analytics · doi:10.1287/inte.12.5.62 · Cited by 54
Confirmatory Bias and Confirmatory Distortion
Martindale, D. A. (2005) · Journal of Child Custody · doi:10.1300/j190v02n01_03 · Cited by 54
“…1981; Borum, Otto, & Golding, 1993; Dailey, 1952; Davies, 2003; Garb, 1994; Greenwald & Pratkanis, 1988; Greenwald, Pratkanis, Leippe, & Baumgardner, 1986; Haverkamp, 1993; Hogarth, 1981; Klayman & Ha, 1987; Mahoney, 1977; Otto, 1989; Strohmer, Shivy, & Chiodo, 1990; Tversky & Kahneman, 1974).”
BackgroundCrossick, G. (2016) · Insights the UKSG journal · doi:10.1629/uksg.280 · Cited by 52
What warrants our claims? A methodological evaluation of argument structure
Ketokivi, M. A., & Mantere, S. (2021) · Journal of Operations Management · doi:10.1002/joom.1137 · Cited by 74
The Bloodless Coup: The Infiltration of Organization Science by Uncertainty and Values
Connell, A. F., & Nord, W. R. (1996) · The Journal of Applied Behavioral Science · doi:10.1177/0021886396324005 · Cited by 52
Publication System Biases Associated with the Statistical Testing Paradigm*
LINDSAY, R. M. (1994) · Contemporary Accounting Research · doi:10.1111/j.1911-3846.1994.tb00435.x · Cited by 51
Peer review: A philosophically faulty concept which is proving disastrous for science
Horrobin, D. F. (1982) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00011444 · Cited by 56
Publication, politics, and scientific progress
Mahoney, M. J. (1982) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00011481 · Cited by 52
Self-citationnegative results in social science
Lehrer, D., Leschke, J., Lhachimi, S. K., Vasiliu, A., & Weiffen, B. (2007) · European Political Science · doi:10.1057/palgrave.eps.2210114 · Cited by 50
The transformative power of values-enacted scholarship
Agate, N., Kennison, R., Konkiel, S., Long, C. P., Rhody, J., Sacchi, S., & Weber, P. (2020) · Humanities and Social Sciences Communications · doi:10.1057/s41599-020-00647-z · Cited by 52
“…Onbekend et al., 2016) or even outright citation malpractice (Davenport and Snyder, 1995), cite only those with whom they agree (Hojat et al., 2003; Mahoney, 1977), insist that their Ph.D. students cite them in every work (Hüppauf, 2018; Sugimoto, 2014), require undergraduates buy their $200…”
BackgroundManuscript evaluation by journal referees and editors: Randomness or bias?
Colman, A. M. (1982) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00011298 · Cited by 50
Anosmic peer review: A rose by another name is evidently not a rose
Scarr, S. W. (1982) · Behavioral and Brain Sciences · doi:10.1017/s0140525x0001164x · Cited by 51
Interreferee agreement and acceptance rates in physics
Lazarus, D. (1982) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00011468 · Cited by 52
Clinical attrition due to biased preclinical assessments of potential efficacy
Lindner, M. D. (2007) · Pharmacology & Therapeutics · doi:10.1016/j.pharmthera.2007.05.002 · Cited by 52
“Since some false positives are always produced, just by chance, the bias to publish significant, therapeutic effects means that false positives are overrepresented and may dominate the published literature (Mahoney, 1977; Begg & Berlin, 1989; for review, see Sterling, 1959; Mahoney, 1985,1987; Wilson et al., 1993).”
BackgroundSolomon, S. M., & Hackett, E. J. (1996) · Science Technology & Human Values · doi:10.1177/016224399602100201 · Cited by 49
On peer review: “We have met the enemy and he is us”
Cicchetti, D. V. (1982) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00011286 · Cited by 48
Interrater agreement for reviews for Developmental Review
Whitehurst, G. J. “. (1983) · Developmental Review · doi:10.1016/0273-2297(83)90009-6 · Cited by 47
What is the source of bias in peer review?
Over, R. (1982) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00011559 · Cited by 48
Peer Review: From “Sacred Ideals” to “Profane Realities”
Johnson, D. R., & Hermanowicz, J. C. (2017) · Higher education · doi:10.1007/978-3-319-48983-4_10 · Cited by 46
Toward a General Framework of Biased Reasoning: Coherence-Based Reasoning
Simon, D., & Read, S. J. (2023) · Perspectives on Psychological Science · doi:10.1177/17456916231204579 · Cited by 57
“Alternatively, coherence can be attained by altering the evidence to comport with the preconceived beliefs, as observed in the studies by Lord et al. (1979) and Mahoney (1977).”
Kovera, M. B., Russano, M. B., & McAuliff, B. D. (2002) · Psychology Public Policy and Law · doi:10.1037/1076-8971.8.2.180 · Cited by 46
Focal randomisation: an optimal mechanism for the evaluation of R&D projects
Brezis, E. S. (2007) · Science and Public Policy · doi:10.3152/030234207x265394 · Cited by 46
“The first is called the confirmatory bias, which has been analyzed by Mahoney (1977).”
BackgroundZelinsky, N. A. M., & Shadish, W. (2018) · Developmental Neurorehabilitation · doi:10.3109/17518423.2015.1100690 · Cited by 41
“A few studies have detected it in the SCD literature [49, 50]).”
BackgroundTeel, T. L., Bright, A. D., Manfredo, M. J., & Brooks, J. J. (2006) · Society & Natural Resources · doi:10.1080/08941920600561140 · Cited by 46
“Consistent with the social psychological literature on biased processing (e.g., see Biek et al. 1996; Lord et al. 1979; Mahoney 1977; Vallone et al. 1985), we hypothesized that if exposed to information on both sides of a contentious issue, subjects would rate the information consistent with their…”
BackgroundIllegitimate Values, Confirmation Bias, and Mandevillian Cognition in Science
Peters, U. (2021) · The British Journal for the Philosophy of Science · doi:10.1093/bjps/axy079 · Cited by 45
Citation gamesmanship: testing for evidence of ego bias in peer review
Sugimoto, C. R., & Cronin, B. (2012) · Scientometrics · doi:10.1007/s11192-012-0845-z · Cited by 44
“…publish articles from in-country authors (e.g., Daniel 1993; Link 1998; Ernst and Kienbacher 1991), women tend to apply more stringent review criteria than men (e.g., Wing et al. 2010), and reviewers tend to favor articles with epis-temological frameworks similar to their own (e.g., Mahoney 1977).”
BackgroundEpstein, W. M. (2004) · Research on Social Work Practice · doi:10.1177/1049731504265838 · Cited by 43
“The few direct tests of publication bias in the social sciences have usually been constrained by small samples, limited coverage, and ambiguous findings (Mahoney, 1977; Peters & Ceci, 1982).”
BackgroundHonest signaling in academic publishing
Tiokhin, L., Panchanathan, K., Lakens, D., Vazire, S., Morgan, T. J. H., & Zollman, K. (2021) · PLoS ONE · doi:10.1371/journal.pone.0246675 · Cited by 46
“Further, we focus on papers varying in only quality, whereas papers and their perceived value actually vary along many dimensions (e.g., novelty, type of result (positive or negative), whether conclusions support scientists’ preconceptions [6, 68, 90]).”
Pardeck, J. T. (1992) · Research on Social Work Practice · doi:10.1177/104973159200200405 · Cited by 44
A multi-disciplinary perspective on emergent and future innovations in peer review
Tennant, J. P., Dugan, J. M., Graziotin, D., Jacques, D. C., Waldner, F., Mietchen, D., Elkhatib, Y., B. Collister, L., Pikas, C. K., Crick, T., et al. (2017) · F1000Research · doi:10.12688/f1000research.12037.2 · Cited by 38
Pilot studies in family planning and reproductive health care
van Teijlingen, E., & Hundley, V. A. (2005) · Journal of Family Planning and Reproductive Health Care · doi:10.1783/1471189054483735 · Cited by 43
Young, S. N. (2009) · Journal of Psychiatry and Neuroscience · doi:10.1139/jpn.0952 · Cited by 38
Climate Control: Gender and Racial Bias in Engineering?Preprint
Williams, J. C., Li, S., Rincon, R., & Finn, P. (2016) · SSRN Electronic Journal · doi:10.2139/ssrn.4014946 · Cited by 64
Walking the talk: the need for a trial registry for development interventions
Rasmussen, O. D., Malchow‐Møller, N., & Andersen, T. B. (2011) · Journal of Development Effectiveness · doi:10.1080/19439342.2011.605160 · Cited by 42
“Mahoney (1977) implemented a randomised trial changing only the results section in a paper to 78 referees.”
MethodOpen up: a survey on open and non-anonymized peer reviewing
Besançon, L., Rönnberg, N., Löwgren, J., Tennant, J., & Cooper, M. D. (2020) · Research Integrity and Peer Review · doi:10.1186/s41073-020-00094-z · Cited by 47
Aubert, A. H., Esculier, F., & Lienert, J. (2020) · Operations Research Perspectives · doi:10.1016/j.orp.2020.100156 · Cited by 42
“tributes to (1) overcome publication and confirmation biases [62,63], which can lead to overrate treatments and distort results of meta-analyses, (2) establish good practice guidelines (or at least many recommendations), and (3) emphasize research opportunities.”
BackgroundPerhaps it was right to reject the resubmitted manuscripts
Thomas, G. J. (1982) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00011675 · Cited by 42
Breaking Groupthink: Why Scientific Identity and Norms Mitigate Ideological Epistemology
Bavel, J. J. V., Reinero, D. A., Harris, E., Robertson, C. E., & Pärnamets, P. (2020) · Psychological Inquiry · doi:10.1080/1047840x.2020.1722599 · Cited by 46
“Clark & Winegard cite experimental evidence that reviewer assessments of scientific articles and pamphlets were more positive if they were consistent with the prior perspective of the reviewer (Koehler, 1993; Mahoney, 1977).”
BackgroundMorris, E. K. (2009) · The Behavior Analyst · doi:10.1007/bf03392184 · Cited by 47
The impact of a paper’s new combinations and new components on its citation
Yan, Y., Tian, S., & Zhang, J. (2019) · Scientometrics · doi:10.1007/s11192-019-03314-6 · Cited by 41
“A clear understanding from reviewers is crucial because peer review is generally regarded as an essential step in the path to publication (Mahoney 1977).”
BackgroundPeer reviewing: Improve or be rejected
Howe, M. J. A. (1982) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00011456 · Cited by 41
Evidence-based advertising: An application to persuasion
Scott Armstrong, J. (2011) · International Journal of Advertising · doi:10.2501/ija-30-5-743-767 · Cited by 40
Feminism and Evolutionary Psychology: Moving Forward
Eagly, A. H., & Wood, W. (2013) · Sex Roles · doi:10.1007/s11199-013-0315-y · Cited by 43
Strang, D., & Siler, K. (2015) · Sociological Theory · doi:10.1177/0735275115572152 · Cited by 39
“In a classic demonstration of this effect, Mahoney (1977) sent a contrived manuscript to 75 reviewers, all of whom received the same introduction and methods section but where the direction of findings varied.”
BackgroundInfluentialImproving norms in research culture to incentivize transparency and rigor
Mellor, D. T. (2021) · Educational Psychologist · doi:10.1080/00461520.2021.1902329 · Cited by 41
“And that preference for such results does indeed exist in the form of publication bias, which has been well documented for decades (Atkinson et al., 1982; Franco et al., 2014; Mahoney, 1977; Sterling, 1959).”
BackgroundVuong, Q., Huyền, N. T. T., Pham, T.-H., Ho, M., & Nguyen, M. (2021) · Humanities and Social Sciences Communications · doi:10.1057/s41599-021-00788-9 · Cited by 42
“Thus, this dominance is more likely to make innovative or new ideas under-evaluated or suppressed (Atkeson and Taylor, 2019; Mahoney, 1977).”
Background
Metadata
- Authors
- Mahoney, M. J.
- Affiliations
- Pennsylvania State University (US)
- Corresponding author
- Mahoney, Michael J.
- ORCID
- Mahoney, Michael J.
- Venue
- Cognitive Therapy and Research
- Published
- 1977-06
- Volume
- 1
- Issue
- 2
- Pages
- 161-175
- ISSN
- 0147-5916 (ISSN-L)
- 1573-2819
- OpenAlex
- W2164136002
- Semantic Scholar
- d8cf5c798397b6a0be1b41f18f979f0988f1ece7
- Cited by
- 1,013
- References
- 29
- Citations by year
- 2026: 22 · 2025: 18 · 2024: 25 · 2023: 28 · 2022: 52 · 2021: 45 · 2020: 52 · 2019: 40 · 2018: 31 · 2017: 51 · 2016: 38 · 2015: 43 · 2014: 41 · 2013: 51 · 2012: 23
- FWCI
- 5.62
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- www.springer.com/tdm (tdm)
- Locations
- Cognitive Therapy and Research (publishedVersion)
- Topics
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Library and Information Sciences › Publishing and Scholarly Communication
- Keywords
- confirmation bias, peer review, publication bias, scientific journals, manuscript review, interobserver agreement, negative results, epistemology of science
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.