The DOI matches its Crossref record; 193 of the 195 references it cites were found in a registry or an index.
Peer-review practices of psychological journals: The fate of published articles, submitted again
Abstract
A growing interest in and concern about the adequacy and fairness of modern peer-review practices in publication and funding are apparent across a wide range of scientific disciplines. Although questions about reliability, accountability, reviewer bias, and competence have been raised, there has been very little direct research on these variables. The present investigation was an attempt to study the peer-review process directly, in the natural setting of actual journal referee evaluations of submitted manuscripts. As test materials we selected 12 already published research articles by investigators from prestigious and highly productive American psychology departments, one article from each of 12 highly regarded and widely read American psychology journals with high rejection rates (80%) and nonblind refereeing practices. With fictitious names and institutions substituted for the original ones (e.g., Tri-Valley Center for Human Potential), the altered manuscripts were formally resubmitted to the journals that had originally refereed and published them 18 to 32 months earlier. Of the sample of 38 editors and reviewers, only three (8%) detected the resubmissions. This result allowed nine of the 12 articles to continue through the review process to receive an actual evaluation: eight of the nine were rejected. Sixteen of the 18 referees (89%) recommended against publication and the editors concurred. The grounds for rejection were in many cases described as “serious methodological flaws.” A number of possible interpretations of these data are reviewed and evaluated.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Holding the affiliation constant was important because previous research documented an impact of institutional affiliation on the outcome of the review process under single-anonymized evaluation (4–6).”
From Nobel and novice: Author prominence affects peer review (2022) · cites this paper as [5]
In the citing paper
“Double-anonymizing the peer-review process could also help level the playing field for academics from other marginalized groups, giving them a fairer chance to succeed, which, in turn, would promote diverse points of view in journal output and eventually expand the journal peer-reviewer pool to authors coming from such backgrounds (3–5, 19–23).”
From Nobel and novice: Author prominence affects peer review (2022) · cites this paper as [5]
In the citing paper
“Double-anonymizing the peer-review process could also help level the playing field for academics from other marginalized groups, giving them a fairer chance to succeed, which, in turn, would promote diverse points of view in journal output and eventually expand the journal peer-reviewer pool to authors coming from such backgrounds (3–5, 19–23).”
From Nobel and novice: Author prominence affects peer review (2022) · cites this paper as [5]
In the citing paper
“Double-anonymizing the peer-review process could also help level the playing field for academics from other marginalized groups, giving them a fairer chance to succeed, which, in turn, would promote diverse points of view in journal output and eventually expand the journal peer-reviewer pool to authors coming from such backgrounds (3–5, 19–23).”
From Nobel and novice: Author prominence affects peer review (2022) · cites this paper as [5]
In the citing paper
“Empirical work on the reliability of peer review has so far only examined the variability between reviewers, often by examining the agreement of independent reviewers examining the same papers [5,16–22].”
From Examining uncertainty in journal peer reviewers’ recommendations: a cross-sectional study (2024) · cites this paper as [16]
Citations
Cites
Statistical principles in experimental design.
Winer, B. J. (1962) · McGraw-Hill Book Company · doi:10.1037/11774-000 · Cited by 26,755
The Structure of Scientific Revolutions: 50th Anniversary Edition
Kuhn, T. S., & Hacking, I. (2012) · doi:10.7208/chicago/9780226458144.001.0001 · Cited by 21,170
Convergent and discriminant validation by the multitrait-multimethod matrix.
Campbell, D. T., & Fiske, D. W. (1959) · Psychological Bulletin · doi:10.1037/h0046016 · Cited by 17,116
(1979) · Medical Entomology and Zoology · Cited by 11,252
Toward an experimental ecology of human development.
Bronfenbrenner, U. (1977) · American Psychologist · doi:10.1037/0003-066x.32.7.513 · Cited by 10,853
Social Theory and Social Structure
Ross, E. J., & Merton, R. K. (1958) · The American Catholic Sociological Review · doi:10.2307/3709394 · Cited by 10,315
Weighted kappa: nominal scale agreement provision for scaled disagreement or partial credit
Cohen, J. (1968) · Psychological Bulletin · doi:10.1037/h0026256 · Cited by 8,736
The Matthew Effect in Science: The reward and communication systems of science are considered.
Merton, R. K. (1968) · Science · doi:10.1126/science.159.3810.56 · Cited by 6,720
Social Theory and Social Structure.
Bock, K. E., & Merton, R. K. (1969) · American Sociological Review · doi:10.2307/2091967 · Cited by 6,660
Neisser, U. (1976) · Cited by 6,101
Research works printed but not linked: 80
- Smigel, 1970, Factors in the editorial decision, American Sociologist, 5, 19
- Korten F. & Griffith B. (ca. 1970) Editorial review in psychological journals: A survey of authors, editors, and readers. Washington, D. C.: American Psychological Association. [BCG]
- Stinchcombe, 1969, On journal editing as a probabilistic process, American Sociologist, 4, 116
- Coe, 1967, Editorial policies of major economic journals, Quarterly Review of Economics and Business, 7, 37
- Zuckerman, 1973, The scoiology of science
- Manwell, 1981, Honesty in science: A partial test of a sociobiological model of the social structure of science, Search, 12, 151
- Crane, 1967, The gatekeepers of science: Some factors affecting the selection of articles for scientific journals, American Sociologist, 32, 195
- Cole, 1978, Peer review in the National Science Foundation: Phase 1 of a study
- 10.1086/260098 · DOI · Paper page
- 10.1037/0003-066X.30.11.1049 · doi:10.1037/0003-066x.30.11.1049 · Paper page
- Cone J. D. & Foster S. L. (in press) Direct observation in clinical psychology. In: Handbook of research methods in clinical psychology, ed. P. C. Kendall & J. N. Butcher. New York: John Wiley & Sons. [JDC]
- Bernard, 1980, CARS: Computer assisted referee selection, Journal of Research Communication Studies, 2, 149
- Glenn, 1976, The journal article review process: Some proposals for change, American Sociologist, 11, 179
- Editorial note. (1965) Helgolander Wissenschaftliche Meeresuntersuchen 12 (07): 218. [CM]
- Markle, 1977, Author's guide to journals in psychology, psychiatry, if social work
- Ruderfer, 1980, The fallacy of peer review – Judgment without science and a case history, Speculations in Science and Technology, 3, 533
- 10.2307/1128626 · doi:10.2307/1128626 · Paper page
- 1963, Reports of Project on Scientific Information Exchange in Psychology, 1–3
- Yalow, 1978, Les prix nobel en 1977, 243
- Snizek, 1981, The effect of theory group association on the evaluative content of book reviews in sociology, American Sociologist, 16, 185
- Bartley, 1962, The retreat to commitment
- 10.1037/0003-066X.33.10.920 · doi:10.1037/0003-066x.33.10.920 · Paper page
- 10.1037/0003-066X.34.2.127 · doi:10.1037/0003-066x.34.2.127 · Paper page
- 10.1037/0003-066X.32.12.1013 · doi:10.1037/0003-066x.32.12.1013
- Swindel, 1975, A previously unannounced form of the Gaussian distribution: The golden rule of arts and sciences, Journal of Irreproducible Results, 21, 8
- Scott W. A. (1977) Methodological consensus among journal referees: A follow-up study of JPSP manuscripts. Unpublished manuscript. [WAS]
- Honig, 1980, Concluding anti-relativity, Speculations in Science and Technology, 3, 361
- Honig, 1980, Editorials on our evolving policy: Opening statements; Further statements on speculation; “They laughed at Columbus” and other author syndromes; Our first year, Modesty and age “paradigms”; Einstein centennial issue – Alternates to special relativity; Mathematics in physical science, or why the tail wags the dog; Comment on submissions; Some additional thoughts on speculation; Anti-relativity, Speculations in Science and Technology, 3, 233
- Kosinski, 1968, Steps
- Greenberg, 1980, Scams and sleaze in science, Clinical Chemistry News, 6, 5
- 10.1037/0003-066X.33.12.1064 · doi:10.1037/0003-066x.33.12.1064 · Paper page
- NIH Grants Peer Review Study Team (1978) Grants peer review. Opinions on the NIH grants peer review system. Report to the Director, NIH. [RTL]
- Mahoney M. J. (in press) Clinical psychology and scientific inquiry. International Journal of Psychology. [MJM]
- 10.1037/0003-066X.35.3.300 · doi:10.1037/0003-066x.35.3.300 · Paper page
- Honig, 1980, The review process: Before, after and during, Speculations in Science and Technology, 3, 513
- Webb, 1979, Continuing education: Refereeing journal articles, Teaching Psychology, 6, 59
- 10.1037/0003-066X.32.10.793 · doi:10.1037/0003-066x.32.10.793 · Paper page
- 10.1037/0003-066X.34.9.798.a · DOI
- Editoral. (1979) Physical Review Letters 43. [RKA]
- Hensler D. R. (1976) Perceptions of the National Science Foundation peer review process: A report on a survey of NSF reviewers and applicants. NSF publication #77–33. [RTL, CM]
- Cole, 1981, Peer review in the National Science Foundation: Phase II of a study
- Cicchetti, 1979, Journal of Abnormal Psychology. 1979 Proceedings of the Social Statistics Section, 596
- Harnad S. (1982) Rational disagreement in peer review. (Submitted for publication.) [Ed.]
- Moyal, 1980, The Australian Academy of Science: The anatomy of a scientific elite: parts 1 and 2, Search, 11, 231
- Harnad S. (1982) Rational disagreement in peer review. (Submitted for publication.) [Ed.]
- 10.1037/0003-066X.32.12.1013 · doi:10.1037/0003-066x.32.12.1013
- Price, 1970, Communications among scientists and engineers
- Honig, 1980, Concluding anti-relativity, Speculations in Science and Technology, 3, 361
- Margulis, 1977, Peer review attacked (letter), The Sciences, 17, 5
- 10.1037/0003-066X.32.5.301 · doi:10.1037/0003-066x.32.5.301 · Paper page
- 1979, Reviews of Landsey's The scientific publication system in social science, Contemporary Sociology, 8, 814
- Carta, 1978, Forum for rejected papers, IEEE Spectrum, 15, 13
- 10.1002/j.2326-1951.1979.tb01767.x · DOI
- Ross, 1979, Rejected, New West, 4, 39
- Ross P. F. (1981) The sciences' self-management: Manuscript refereeing, peer review, and goals in science. Unpublished manuscript. [taDPP]
- 10.5962/bhl.title.54282 · DOI · Paper page
- 10.2307/2094272 · DOI · Paper page
- Wright, 1970, Truth and its keepers, New Scientist, 45, 402
- Sayre, 1975, Kosnlind Franklin and DNA
- Manwell, 1979, Peer review: A case history from the Australian Research Grants Committee, Search, 10, 81
- Hensler D. R. (1976) Perceptions of the National Science Foundation peer review process: A report on a survey of NSF reviewers and applicants. NSF publication #77–33. [RTL, CM]
- Colman, 1979, Editorial role in author-referee disagreements, Bulletin of the British Psychological Society, 32, 390
- Michie D. (1978) Peer review and the bureaucracy. Times Higher Education Supplement 08 4, p. 11. [CM]
- Yotopoulos, 1961, Institutional affiliation of the contributors to three professional journals, American Economic Review, 51, 665
- 1977, Guide and journal lists
- 10.1177/014616727600200301 · DOI
- Stnrk-Adamec C. & Adamec R. (in press) Breaking into the grant proposal market. International Journal of Women's Studies. [rDPP]
- Weick K. E. (in press) Systematic observational methods. In: The handbook of social psychology, ed. G. Lindzey & E. Aronson. 3d ed. [JDC]
- 10.1136/bmj.2.5912.216 · DOI
- Crane, 1972, Invisible colleges
- Honig, 1980, The review process: Before, after and during, Speculations in Science and Technology, 3, 513
- Martin, 1981, The scientific straightjacket: The power structure of science and the suppression of environmental scholarship, Ecologist, 11, 33
- Honig, 1980, Editorials on our evolving policy: Opening statements; Further statements on speculation; “They laughed at Columbus” and other author syndromes; Our first year, Modesty and age “paradigms”; Einstein centennial issue – Alternates to special relativity; Mathematics in physical science, or why the tail wags the dog; Comment on submissions; Some additional thoughts on speculation; Anti-relativity, Speculations in Science and Technology, 3, 233
- Robertson, 1976, Towards open refereeing, New Scientist, 71, 410
- Freese, 1979, On changing some role relationships in the editorial review process, American Sociologist, 14, 231
- 10.1177/014616727600300101 · DOI · Paper page
- Mahoney, 1976, Scientist as subject, 187
- 10.1126/science.11643505 · DOI
- Griffith, 1976, Proceedings, First International Conference on Social Studies of Science
- Tobach E. (1980) “… that ye be judged.” In An evaluation of the peer review system in psychological research, chairman J. Demarest, open forum presented at the American Psychological Convention, Montreal. [taDPP]
Cited by
Implicit social cognition: Attitudes, self-esteem, and stereotypes
Greenwald, A. G., & Banaji, M. R. (1995) · Psychological Review · doi:10.1037/0033-295x.102.1.4 · Cited by 6,682
“Another variant appeared in a controversial study by Peters and Ceci (1982), who submitted manuscripts to journals that had previously published them, using fictitious authors' names and deliberately unimpressive (fictitious) institutional affiliations.”
BackgroundComputer Support for Knowledge-Building Communities
Scardamalia, M., & Bereiter, C. (1994) · Journal of the Learning Sciences · doi:10.1207/s15327809jls0303_3 · Cited by 2,164
“Unreliability of judgment, bias, political maneuvering, conservatism, failure to detect gross errors-all are familiar (see Peters & Ceci, 1982, and the whole journal issue devoted to discussion of their experiment in which previously published articles were slightly disguised and resubmitted to the…”
BackgroundNosek (2012) · doi:10.1177/1745691612459058 · Cited by 1,301
Lee, C. J., Sugimoto, C. R., Zhang, G., & Cronin, B. (2013) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.22784 · Cited by 1,000
“As expressed by Peters and Ceci (1982b), universalism in the context of peer review requires that an author's research be " judged on the merit of [his/her] ideas, not on the basis of academic rank, sex, place of work, publication record, and so on " (p. 252).”
InfluentialChapter 10: Addressing reporting biases
Egger, M., & Moher, D. (2011) · Cited by 997
Metadata
- Authors
- Peters, D. P., & Ceci, S. J.
- Affiliations
- University of North Dakota (US)
- Cornell University (US)
- ORCID
- Ceci, Stephen J.
- Venue
- Behavioral and Brain Sciences
- Publisher
- Cambridge University Press (CUP)
- Published
- 1982-06
- Volume
- 5
- Issue
- 2
- Pages
- 187-195
- ISSN
- 0140-525X (ISSN-L)
- 1469-1825
- OpenAlex
- W2102073367
- Cited by
- 1,079
- References
- 195
- Citations by year
- 2026: 12 · 2025: 15 · 2024: 18 · 2023: 32 · 2022: 72 · 2021: 38 · 2020: 43 · 2019: 49 · 2018: 15 · 2017: 40 · 2016: 32 · 2015: 28 · 2014: 42 · 2013: 28 · 2012: 31
- FWCI
- 22.66
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Behavioral and Brain Sciences (publishedVersion)
- Topics
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
- Social Sciences › Library and Information Sciences › Publishing and Scholarly Communication
- Social Sciences › Library and Information Sciences › Academic Writing and Publishing
- Keywords
- peer review, rejection rate, psychology journals, reviewer bias, editorial decision-making, methodological errors, publication practices
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Peer-review practices of psychological journals: The fate of published articles, submitted again. https://app.cochat.ai/papers/10.1017/s0140525x00011183
- The paper
- Peters, D. P., & Ceci, S. J. (1982). Peer-review practices of psychological journals: The fate of published articles, submitted again. Behavioral and Brain Sciences. https://doi.org/10.1017/s0140525x00011183
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.