The DOI matches its Crossref record; 70 of the 70 references it cites were found in a registry or an index.
Science or Art? How Aesthetic Standards Grease the Way Through the Publication Bottleneck but Undermine Science
Abstract
The current crisis in psychological research involves issues of fraud, replication, publication bias, and false positive results. I argue that this crisis follows the failure of widely adopted solutions to psychology’s similar crisis of the 1970s. The untouched root cause is an information-economic one: Too many studies divided by too few publication outlets equals a bottleneck. Articles cannot pass through just by showing theoretical meaning and methodological rigor; their results must appear to support the hypothesis perfectly. Consequently, psychologists must master the art of presenting perfect-looking results just to survive in the profession. This favors aesthetic criteria of presentation in a way that harms science’s search for truth. Shallow standards of statistical perfection distort analyses and undermine the accuracy of cumulative data; narrative expectations encourage dishonesty about the relationship between results and hypotheses; criteria of novelty suppress replication attempts. Concerns about truth in research are emerging in other sciences and may eventually descend on our heads in the form of difficult and insensitive regulations. I suggest a more palatable solution: to open the bottleneck, putting structures in place to reward broader forms of information sharing beyond the exquisite art of present-day journal publication.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“researchers can confirm the absence of p -hacking in their research reports by (a) actively affirming the disclosure of their data collection stopping rule, data exclusions, measures, and manipulations (Simmons, Nelson, & Simonsohn, 2012), (b) providing logical and principled justifications for nonstandard data exclusions and analytical approaches (Giner-Sorolla, 2012, p. 568), (c) providing”
From Does preregistration improve the credibility of research findings? (2020) · cites this paper as Giner-Sorolla, 2012, p. 568
In the citing paper
“And, clean papers in which the evidence consistently supports the proposed conclusion are more likely to be published than papers with equivocal evidence, exceptions, or other uncertainties 16,17 .”
From Initial evidence of research quality of registered reports compared with the standard publishing model (2021) · cites this paper as [17]
Citations
Cites
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · PLoS Medicine · doi:10.1371/journal.pmed.0020124 · Cited by 10,505
The Elaboration Likelihood Model of Persuasion
Petty, R. E., & Cacioppo, J. T. (1986) · Advances in Experimental Social Psychology · doi:10.1016/s0065-2601(08)60214-2 · Cited by 8,822
Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011) · Psychological Science · doi:10.1177/0956797611417632 · Cited by 6,567
Chaiken, S. (1980) · Journal of Personality and Social Psychology · doi:10.1037//0022-3514.39.5.752 · Cited by 5,516
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · CHANCE · doi:10.1080/09332480.2005.10722754 · Cited by 4,702 · cited 3×
“It was in that field that Ioannidis (2005) declared that “most published research findings are false” because of incentives to publish only positive results.”
BackgroundCohen, J. (1994) · American Psychologist · doi:10.1037/0003-066x.49.12.997 · Cited by 4,003
The Handbook of Research Synthesis.
Howel, D., Cooper, H., & Hedges, L. V. (1995) · Biometrics · doi:10.2307/2533295 · Cited by 3,657
Processing Fluency and Aesthetic Pleasure: Is Beauty in the Perceiver's Processing Experience?
Reber, R., Schwarz, N., & Winkielman, P. (2004) · Personality and Social Psychology Review · doi:10.1207/s15327957pspr0804_3 · Cited by 3,149
Raise standards for preclinical cancer research
Begley, C. G., & Ellis, L. M. (2012) · Nature · doi:10.1038/483531a · Cited by 2,979 · cited 1×
“Confidence in the applied usefulness of the publication process has taken a hit, as clinical trials have demonstrated unusually low success rates when trying to reproduce academic results (Begley & Ellis, 2012; Prinz, Schlange, & Asadullah, 2011; Turner, Matthews, et al., 2008).”
ResultStatistical methods in psychology journals: Guidelines and explanations.
Wilkinson, L., & Task Force on Statistical Inference (1999) · American Psychologist · doi:10.1037/0003-066x.54.8.594 · Cited by 2,791
Selective publication of antidepressant trials and its influence on apparent efficacy
Turner, E. H., Matthews, A. M., Linardatos, E., Tell, R. A., & Rosenthal, R. (2008) · New England Journal of Medicine · doi:10.1056/nejmsa065779 · Cited by 2,435
Wicker, A. W. (1969) · Journal of Social Issues · doi:10.1111/j.1540-4560.1969.tb00619.x · Cited by 2,152
HARKing: Hypothesizing after the results are known
Kerr, N. L. (1998) · Personality and Social Psychology Review · doi:10.1207/s15327957pspr0203_4 · Cited by 1,989
Believe it or not: how much can we rely on published data on potential drug targets?
Prinz, F., Schlange, T., & Asadullah, K. (2011) · Nature Reviews Drug Discovery · doi:10.1038/nrd3439-c1 · Cited by 1,888
The Evolutionary Psychology of Facial Beauty
Rhodes, G. (2005) · Annual Review of Psychology · doi:10.1146/annurev.psych.57.102904.190208 · Cited by 1,821
Statistical significance in psychological research.
Lykken, D. T. (1968) · Psychological bulletin · doi:10.1037/h0026141 · Cited by 1,363
Changing Order: Replication and Induction in Scientific Practice
Star, S. L., & Collins, H. M. (1988) · Technology and Culture · doi:10.2307/3105095 · Cited by 1,283
Daston, L., & Galison, P. (1992) · Representations · doi:10.2307/2928741 · Cited by 1,269
Artifact in behavioral research
Rosenthal, R., & Rosnow, R. L. (1969) · Medical Entomology and Zoology · Cited by 1,186
Changing order: Replication and induction in scientific practice
(1987) · Journal of Economic Psychology · doi:10.1016/0167-4870(87)90014-6 · Cited by 1,133
The Handbook of Research Synthesis
Hanka, R. (1994) · BMJ · doi:10.1136/bmj.309.6952.488a · Cited by 1,065
Consequences of prejudice against the null hypothesis.
Greenwald, A. G. (1975) · Psychological Bulletin · doi:10.1037/h0076157 · Cited by 1,022 · cited 2×
“However, other issues concerned all areas of psychology: limitations of null-hypothesis significance testing, bias toward positive results in publication, and the resulting lack of credibility of the standard research article (Elms, 1975; Greenwald, 1975).”
BackgroundWagenmakers, E.-J., Wetzels, R., Borsboom, D., & van der Maas, H. L. J. (2011) · Journal of Personality and Social Psychology · doi:10.1037/a0022790 · Cited by 977
Bem, D. J. (2011) · Journal of Personality and Social Psychology · doi:10.1037/a0021524 · Cited by 893
Gigerenzer, G. (2004) · The Journal of Socio-Economics · doi:10.1016/j.socec.2004.09.033 · Cited by 890
The rules of the game called psychological science
Bakker, M., van Dijk, A., & Wicherts, J. M. (2012) · Perspectives on Psychological Science · doi:10.1177/1745691612459060 · Cited by 800
An exploratory test for an excess of significant findings
Ioannidis, J. P., & Trikalinos, T. A. (2007) · Clinical Trials · doi:10.1177/1740774507079441 · Cited by 751
The stability of behavior: II. Implications for psychological research.
Epstein, S. J. (1980) · American Psychologist · doi:10.1037/0003-066x.35.9.790 · Cited by 744
Publication Bias in Meta‐Analysis
Rothstein, H. R., Sutton, A. J., & Borenstein, M. (2005) · Publication Bias in Meta‐Analysis · doi:10.1002/0470870168.ch1 · Cited by 603
The Evolution of the Physicist's Picture of Nature
Dirac, P. A. M. (1963) · Scientific American · doi:10.1038/scientificamerican0563-45 · Cited by 540
Why Current Publication Practices May Distort Science
Young, N. S., Ioannidis, J. P. A., & Al‐Ubaydli, O. (2008) · PLoS Medicine · doi:10.1371/journal.pmed.0050201 · Cited by 533
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2019) · CHANCE · doi:10.1080/09332480.2019.1579573 · Cited by 474
Sterling, T. D., Rosenbaum, W. L., & Weinkam, J. J. (1995) · The American Statistician · doi:10.1080/00031305.1995.10476125 · Cited by 453
Scientific Utopia: I. Opening Scientific Communication
Nosek, B. A., & Bar‐Anan, Y. (2012) · Psychological Inquiry · doi:10.1080/1047840x.2012.692215 · Cited by 387
The crisis of confidence in social psychology.
Elms, A. C. (1975) · American Psychologist · doi:10.1037/0003-066x.30.10.967 · Cited by 330
Final Collapse of the Neyman-Pearson Decision Theoretic Framework and Rise of the neoFisherian
Hurlbert, S. H., & Lombardi, C. M. (2009) · Annales Zoologici Fennici · doi:10.5735/086.046.0501 · Cited by 305
Exploration in animals and humans
Archer, J. E., & Birke, L. I. A. (1983) · Medical Entomology and Zoology · Cited by 248
Too good to be true: Publication bias in two prominent studies from experimental psychology
Francis, G. (2012) · Psychonomic Bulletin & Review · doi:10.3758/s13423-012-0227-9 · Cited by 221
The Merits of Unconscious Thought in Creativity
Zhong, C., Dijksterhuis, A., & Galinsky, A. D. (2008) · Psychological Science · doi:10.1111/j.1467-9280.2008.02176.x · Cited by 206
Failure to replicate mood-dependent retrieval
Bower, G. H., & Mayer, J. D. (1985) · Bulletin of the Psychonomic Society · doi:10.3758/bf03329773 · Cited by 184
LeBel, E. P., & Peters, K. R. (2011) · Review of General Psychology · doi:10.1037/a0025172 · Cited by 163
Reforming Science: Methodological and Cultural Reforms
Casadevall, A., & Fang, F. C. (2012) · Infection and Immunity · doi:10.1128/iai.06183-11 · Cited by 156
The social psychology of science
Gergen, K. J. (1994) · Choice Reviews Online · doi:10.5860/choice.32-0910 · Cited by 119
Does Sunshine Prime Loyal? Affective Priming in the Naming Task
Klauer, K. C., & Musch, J. (2001) · The Quarterly Journal of Experimental Psychology Section A · doi:10.1080/713755986 · Cited by 108
Psychology’s Woes and a Partial Cure: The Value of Replication
Roediger, H. L., III (2012) · APS observer · Cited by 101
The great betrayal: fraud in science
(2005) · Choice Reviews Online · doi:10.5860/choice.42-4611 · Cited by 99
Exploring the Cold-to-Hot Empathy Gap in Smokers
Sayette, M. A., Loewenstein, G. F., Griffin, K. M., & Black, J. J. (2008) · Psychological Science · doi:10.1111/j.1467-9280.2008.02178.x · Cited by 96
Misprescription and misuse of one-tailed tests
LOMBARDI, C. M., & HURLBERT, S. H. (2009) · Austral Ecology · doi:10.1111/j.1442-9993.2009.01946.x · Cited by 93
J. B. Watson's Little Albert, Cyril Burt's twins, and the need for a critical science.
Samelson, F. (1980) · American Psychologist · doi:10.1037/0003-066x.35.7.619 · Cited by 92
The Nature of Social and Personality Psychology as Reflected in JPSP, 1965-2000.
Quiñones-Vidal, E., García, J. J. L., Ortega, M. P., & Tortosa-Gil, F. (2004) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.86.3.435 · Cited by 90
Publication Trends in JPSP: A Three-Decade Review
Reis, H. T., & Stiller, J. D. (1992) · Personality and Social Psychology Bulletin · doi:10.1177/0146167292184011 · Cited by 86
Mackintosh, N. J. (1995) · Oxford University Press eBooks · doi:10.1093/acprof:oso/9780198523369.001.0001 · Cited by 82
Beware the creeping cracks of bias
Sarewitz, D. (2012) · Nature · doi:10.1038/485149a · Cited by 80
Short, Sweet, and Problematic? The Rise of the Short Report in Psychological Science
Ledgerwood, A., & Sherman, J. W. (2012) · Perspectives on Psychological Science · doi:10.1177/1745691611427304 · Cited by 72
Twenty Years of PSPB: Trends in Content, Design, and Analysis
Sherman, R. C., Buddie, A. M., Dragan, K. L., End, C. M., & Finney, L. J. (1999) · Personality and Social Psychology Bulletin · doi:10.1177/0146167299025002004 · Cited by 65
Meta-analytic synthesis for theory development.
Miller, N., & Pollock, V. E. (1994) · Cited by 61
The A nna K arenina principle: A way of thinking about success in science
Bornmann, L., & Marx, W. (2012) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.22661 · Cited by 54
Crisis discussions in psychology—New historical and philosophical perspectives
Sturm, T., & Mülberger, A. (2011) · Studies in History and Philosophy of Science Part C Studies in History and Philosophy of Biological and Biomedical Sciences · doi:10.1016/j.shpsc.2011.11.001 · Cited by 53
Bones, A. K. (2012) · Perspectives on Psychological Science · doi:10.1177/1745691612441216 · Cited by 51
The Great Betrayal: Fraud in Science
Wilmshurst, P. T. (2004) · BMJ · doi:10.1136/bmj.329.7471.922 · Cited by 49
The Evolution of Sex Differences in Jealousy: Failure to Replicate Previous Results
Grice, J. W., & Seely, E. (2000) · Journal of Research in Personality · doi:10.1006/jrpe.2000.2284 · Cited by 35
Replication, replication, replication
Ritchie, S. J., Wiseman, R. J., & French, C. C. (2012) · University of Hertfordshire Research Archive (University of Hertfordshire) · Cited by 33
AESTHETICS IN SCIENCE AND IN ART
Engler, G. (1990) · The British Journal of Aesthetics · doi:10.1093/bjaesthetics/30.1.24 · Cited by 32
On the Use of Narrative as Argument
Voss, J. F., Wiley, J., & Sandak, R. (1999) · Routledge eBooks · doi:10.4324/9781410603135-17 · Cited by 21
Implicit editorial policies and the integrity of psychology as an empirical science.
Wong, P. T. P. (1981) · American Psychologist · doi:10.1037/0003-066x.36.6.690 · Cited by 14
Social Psychological Research: The Comparison of Four Journals
Witte, E. H., & Brandt, V. C. (2010) · Social Science Open Access Repository (GESIS – Leibniz Institute for the Social Sciences) · doi:10.23668/psycharchives.9080 · Cited by 9
Did Sir Cyril Burt Fake His Research on Heritability of Intelligence? Part I.
Gillie, O. (1977) · Phi Delta Kappan · Cited by 9
What to do about fraud charges in science; or, will the Burt affair ever end?
Samelson, F. (1997) · Genetica · doi:10.1007/bf02259518 · Cited by 9
An effort at studying Rorschach content symbolism: The frog response.
Sapolsky, A. (1964) · Journal of Consulting Psychology · doi:10.1037/h0046144 · Cited by 8
What’s new in the economics of arts and culture?
Potts, J. (2007) · Dialogue A Journal of Mormon Thought · Cited by 4
Research works printed but not linked: 10
- Carpenter D., 2007, Northwestern University Law Review, 101, 687
- 10.1037/0022-3514.39.5.752 · DOI · Paper page
- Chamberlin J., 2000, APA Monitor, 31, 36
- Eco U., 1985, Daedalus, 114, 161
- Gillie O., 1977, The Phi Delta Kappan, 58, 469
- ISI Web of Knowledge. (2012). Journal citation reports. Retrieved from http://admin-apps.webofknowledge.com/JCR/JCR?PointOfEntry=Home&SID=R2H17OMe9G5oN@26iil
- Kaiser C. R., 2012, Dialogue, 26, 8
- Pennington N., 1991, Cardozo Law Review, 13, 519
- Ritchie S. J., 2012, The Psychologist, 25, 346
- Voss J. F., 1999, Narrative comprehension, causality, and coherence: Essays in honor of Tom Trabasso, 235
Cited by
Sorted by citations
The preregistration revolution
Nosek, B. A., Ebersole, C. R., DeHaven, A. C., & Mellor, D. T. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1708274114 · Cited by 1,998
Pashler, H., & Wagenmakers, E. (2012) · Perspectives on Psychological Science · doi:10.1177/1745691612465253 · Cited by 1,321
Nosek (2012) · doi:10.1177/1745691612459058 · Cited by 1,301
Replicability, Robustness, and Reproducibility in Psychological Science
Nosek, B. A., Hardwicke, T. E., Moshontz, H., Allard, A., Corker, K. S., Dreber, A., Fidler, F., Hilgard, J., Kline Struhl, M., Nuijten, M. B., et al. (2022) · Annual Review of Psychology · doi:10.1146/annurev-psych-020821-114157 · Cited by 770
Registered Reports: A Method to Increase the Credibility of Published Results
Nosek, B. A., & Lakens, D. (2014) · Social Psychology · doi:10.1027/1864-9335/a000192 · Cited by 716
“This creates incentives to avoid or ignore replications and negative results, even at the expense of accuracy (Giner-Sorolla, 2012; Nosek, Spies, & Motyl, 2012).”
Background
Metadata
- Authors
- Giner-Sorolla, R.
- Affiliations
- University of Kent, United Kingdom
- Venue
- Perspectives on Psychological Science
- Publisher
- SAGE Publications
- Published
- 2012-11
- Volume
- 7
- Issue
- 6
- Pages
- 562-571
- ISSN
- 1745-6916 (ISSN-L)
- 1745-6924
- OpenAlex
- W2125515636
- Cited by
- 274
- References
- 70
- Citations by year
- 2026: 5 · 2025: 9 · 2024: 8 · 2023: 21 · 2022: 17 · 2021: 17 · 2020: 40 · 2019: 40 · 2018: 19 · 2017: 19 · 2016: 22 · 2015: 14 · 2014: 19 · 2013: 18 · 2012: 5
- FWCI
- 23.99
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Perspectives on Psychological Science (publishedVersion)
- PubMed (publishedVersion)
- Kent Academic Repository (University of Kent) (submittedVersion)
- Topics
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
- Social Sciences › Library and Information Sciences › Academic Publishing and Open Access
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Keywords
- publication bias, replication, scientific fraud, false positives, statistical significance thresholds, information economics, open science, research integrity
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
