Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 193 of the 195 references it cites were found in a registry or an index.

Peer-review practices of psychological journals: The fate of published articles, submitted again

Peters, Douglas P., Ceci, Stephen J. · 1982 · Behavioral and Brain Sciences

Abstract

A growing interest in and concern about the adequacy and fairness of modern peer-review practices in publication and funding are apparent across a wide range of scientific disciplines. Although questions about reliability, accountability, reviewer bias, and competence have been raised, there has been very little direct research on these variables. The present investigation was an attempt to study the peer-review process directly, in the natural setting of actual journal referee evaluations of submitted manuscripts. As test materials we selected 12 already published research articles by investigators from prestigious and highly productive American psychology departments, one article from each of 12 highly regarded and widely read American psychology journals with high rejection rates (80%) and nonblind refereeing practices. With fictitious names and institutions substituted for the original ones (e.g., Tri-Valley Center for Human Potential), the altered manuscripts were formally resubmitted to the journals that had originally refereed and published them 18 to 32 months earlier. Of the sample of 38 editors and reviewers, only three (8%) detected the resubmissions. This result allowed nine of the 12 articles to continue through the review process to receive an actual evaluation: eight of the nine were rejected. Sixteen of the 18 referees (89%) recommended against publication and the editors concurred. The grounds for rejection were in many cases described as “serious methodological flaws.” A number of possible interpretations of these data are reviewed and evaluated.

Overview

Integrity signals

Registry

Crossref · 10.1017/s0140525x00011183

Cites

193 of 193 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Citations

Cites

Show all 195 references

Research works printed but not linked: 80

  1. Smigel, 1970, Factors in the editorial decision, American Sociologist, 5, 19
  2. Korten F. & Griffith B. (ca. 1970) Editorial review in psychological journals: A survey of authors, editors, and readers. Washington, D. C.: American Psychological Association. [BCG]
  3. Stinchcombe, 1969, On journal editing as a probabilistic process, American Sociologist, 4, 116
  4. Coe, 1967, Editorial policies of major economic journals, Quarterly Review of Economics and Business, 7, 37
  5. Zuckerman, 1973, The scoiology of science
  6. Manwell, 1981, Honesty in science: A partial test of a sociobiological model of the social structure of science, Search, 12, 151
  7. Crane, 1967, The gatekeepers of science: Some factors affecting the selection of articles for scientific journals, American Sociologist, 32, 195
  8. Cole, 1978, Peer review in the National Science Foundation: Phase 1 of a study
  9. 10.1086/260098 · DOI · Paper page
  10. 10.1037/0003-066X.30.11.1049 · doi:10.1037/0003-066x.30.11.1049 · Paper page
  11. Cone J. D. & Foster S. L. (in press) Direct observation in clinical psychology. In: Handbook of research methods in clinical psychology, ed. P. C. Kendall & J. N. Butcher. New York: John Wiley & Sons. [JDC]
  12. Bernard, 1980, CARS: Computer assisted referee selection, Journal of Research Communication Studies, 2, 149
  13. Glenn, 1976, The journal article review process: Some proposals for change, American Sociologist, 11, 179
  14. Editorial note. (1965) Helgolander Wissenschaftliche Meeresuntersuchen 12 (07): 218. [CM]
  15. Markle, 1977, Author's guide to journals in psychology, psychiatry, if social work
  16. Ruderfer, 1980, The fallacy of peer review – Judgment without science and a case history, Speculations in Science and Technology, 3, 533
  17. 10.2307/1128626 · doi:10.2307/1128626 · Paper page
  18. 1963, Reports of Project on Scientific Information Exchange in Psychology, 1–3
  19. Yalow, 1978, Les prix nobel en 1977, 243
  20. Snizek, 1981, The effect of theory group association on the evaluative content of book reviews in sociology, American Sociologist, 16, 185
  21. Bartley, 1962, The retreat to commitment
  22. 10.1037/0003-066X.33.10.920 · doi:10.1037/0003-066x.33.10.920 · Paper page
  23. 10.1037/0003-066X.34.2.127 · doi:10.1037/0003-066x.34.2.127 · Paper page
  24. 10.1037/0003-066X.32.12.1013 · doi:10.1037/0003-066x.32.12.1013
  25. Swindel, 1975, A previously unannounced form of the Gaussian distribution: The golden rule of arts and sciences, Journal of Irreproducible Results, 21, 8
  26. Scott W. A. (1977) Methodological consensus among journal referees: A follow-up study of JPSP manuscripts. Unpublished manuscript. [WAS]
  27. Honig, 1980, Concluding anti-relativity, Speculations in Science and Technology, 3, 361
  28. Honig, 1980, Editorials on our evolving policy: Opening statements; Further statements on speculation; “They laughed at Columbus” and other author syndromes; Our first year, Modesty and age “paradigms”; Einstein centennial issue – Alternates to special relativity; Mathematics in physical science, or why the tail wags the dog; Comment on submissions; Some additional thoughts on speculation; Anti-relativity, Speculations in Science and Technology, 3, 233
  29. Kosinski, 1968, Steps
  30. Greenberg, 1980, Scams and sleaze in science, Clinical Chemistry News, 6, 5
  31. 10.1037/0003-066X.33.12.1064 · doi:10.1037/0003-066x.33.12.1064 · Paper page
  32. NIH Grants Peer Review Study Team (1978) Grants peer review. Opinions on the NIH grants peer review system. Report to the Director, NIH. [RTL]
  33. Mahoney M. J. (in press) Clinical psychology and scientific inquiry. International Journal of Psychology. [MJM]
  34. 10.1037/0003-066X.35.3.300 · doi:10.1037/0003-066x.35.3.300 · Paper page
  35. Honig, 1980, The review process: Before, after and during, Speculations in Science and Technology, 3, 513
  36. Webb, 1979, Continuing education: Refereeing journal articles, Teaching Psychology, 6, 59
  37. 10.1037/0003-066X.32.10.793 · doi:10.1037/0003-066x.32.10.793 · Paper page
  38. 10.1037/0003-066X.34.9.798.a · DOI
  39. Editoral. (1979) Physical Review Letters 43. [RKA]
  40. Hensler D. R. (1976) Perceptions of the National Science Foundation peer review process: A report on a survey of NSF reviewers and applicants. NSF publication #77–33. [RTL, CM]
  41. Cole, 1981, Peer review in the National Science Foundation: Phase II of a study
  42. Cicchetti, 1979, Journal of Abnormal Psychology. 1979 Proceedings of the Social Statistics Section, 596
  43. Harnad S. (1982) Rational disagreement in peer review. (Submitted for publication.) [Ed.]
  44. Moyal, 1980, The Australian Academy of Science: The anatomy of a scientific elite: parts 1 and 2, Search, 11, 231
  45. Harnad S. (1982) Rational disagreement in peer review. (Submitted for publication.) [Ed.]
  46. 10.1037/0003-066X.32.12.1013 · doi:10.1037/0003-066x.32.12.1013
  47. Price, 1970, Communications among scientists and engineers
  48. Honig, 1980, Concluding anti-relativity, Speculations in Science and Technology, 3, 361
  49. Margulis, 1977, Peer review attacked (letter), The Sciences, 17, 5
  50. 10.1037/0003-066X.32.5.301 · doi:10.1037/0003-066x.32.5.301 · Paper page
  51. 1979, Reviews of Landsey's The scientific publication system in social science, Contemporary Sociology, 8, 814
  52. Carta, 1978, Forum for rejected papers, IEEE Spectrum, 15, 13
  53. 10.1002/j.2326-1951.1979.tb01767.x · DOI
  54. Ross, 1979, Rejected, New West, 4, 39
  55. Ross P. F. (1981) The sciences' self-management: Manuscript refereeing, peer review, and goals in science. Unpublished manuscript. [taDPP]
  56. 10.5962/bhl.title.54282 · DOI · Paper page
  57. 10.2307/2094272 · DOI · Paper page
  58. Wright, 1970, Truth and its keepers, New Scientist, 45, 402
  59. Sayre, 1975, Kosnlind Franklin and DNA
  60. Manwell, 1979, Peer review: A case history from the Australian Research Grants Committee, Search, 10, 81
  61. Hensler D. R. (1976) Perceptions of the National Science Foundation peer review process: A report on a survey of NSF reviewers and applicants. NSF publication #77–33. [RTL, CM]
  62. Colman, 1979, Editorial role in author-referee disagreements, Bulletin of the British Psychological Society, 32, 390
  63. Michie D. (1978) Peer review and the bureaucracy. Times Higher Education Supplement 08 4, p. 11. [CM]
  64. Yotopoulos, 1961, Institutional affiliation of the contributors to three professional journals, American Economic Review, 51, 665
  65. 1977, Guide and journal lists
  66. 10.1177/014616727600200301 · DOI
  67. Stnrk-Adamec C. & Adamec R. (in press) Breaking into the grant proposal market. International Journal of Women's Studies. [rDPP]
  68. Weick K. E. (in press) Systematic observational methods. In: The handbook of social psychology, ed. G. Lindzey & E. Aronson. 3d ed. [JDC]
  69. 10.1136/bmj.2.5912.216 · DOI
  70. Crane, 1972, Invisible colleges
  71. Honig, 1980, The review process: Before, after and during, Speculations in Science and Technology, 3, 513
  72. Martin, 1981, The scientific straightjacket: The power structure of science and the suppression of environmental scholarship, Ecologist, 11, 33
  73. Honig, 1980, Editorials on our evolving policy: Opening statements; Further statements on speculation; “They laughed at Columbus” and other author syndromes; Our first year, Modesty and age “paradigms”; Einstein centennial issue – Alternates to special relativity; Mathematics in physical science, or why the tail wags the dog; Comment on submissions; Some additional thoughts on speculation; Anti-relativity, Speculations in Science and Technology, 3, 233
  74. Robertson, 1976, Towards open refereeing, New Scientist, 71, 410
  75. Freese, 1979, On changing some role relationships in the editorial review process, American Sociologist, 14, 231
  76. 10.1177/014616727600300101 · DOI · Paper page
  77. Mahoney, 1976, Scientist as subject, 187
  78. 10.1126/science.11643505 · DOI
  79. Griffith, 1976, Proceedings, First International Conference on Social Studies of Science
  80. Tobach E. (1980) “… that ye be judged.” In An evaluation of the peer review system in psychological research, chairman J. Demarest, open forum presented at the American Psychological Convention, Montreal. [taDPP]

Cited by

  • Implicit social cognition: Attitudes, self-esteem, and stereotypes

    Greenwald, A. G., & Banaji, M. R. (1995) · Psychological Review · doi:10.1037/0033-295x.102.1.4 · Cited by 6,682

    “Another variant appeared in a controversial study by Peters and Ceci (1982), who submitted manuscripts to journals that had previously published them, using fictitious authors' names and deliberately unimpressive (fictitious) institutional affiliations.”
    Background
  • Computer Support for Knowledge-Building Communities

    Scardamalia, M., & Bereiter, C. (1994) · Journal of the Learning Sciences · doi:10.1207/s15327809jls0303_3 · Cited by 2,164

    “Unreliability of judgment, bias, political maneuvering, conservatism, failure to detect gross errors-all are familiar (see Peters & Ceci, 1982, and the whole journal issue devoted to discussion of their experiment in which previously published articles were slightly disguised and resubmitted to the…”
    Background
  • Untitled work

    Nosek (2012) · doi:10.1177/1745691612459058 · Cited by 1,301

  • Bias in peer review

    Lee, C. J., Sugimoto, C. R., Zhang, G., & Cronin, B. (2013) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.22784 · Cited by 1,000

    “As expressed by Peters and Ceci (1982b), universalism in the context of peer review requires that an author's research be " judged on the merit of [his/her] ideas, not on the basis of academic rank, sex, place of work, publication record, and so on " (p. 252).”
    Influential
  • Chapter 10: Addressing reporting biases

    Egger, M., & Moher, D. (2011) · Cited by 997

See all 1,079

Metadata

Authors
Peters, D. P., & Ceci, S. J.
Affiliations
  • University of North Dakota (US)
  • Cornell University (US)
Venue
Behavioral and Brain Sciences
Published
1982-06
Volume
5
Issue
2
Pages
187-195
ISSN
  • 0140-525X (ISSN-L)
  • 1469-1825
OpenAlex
W2102073367
Cited by
1,079
References
195
Citations by year
2026: 12 · 2025: 15 · 2024: 18 · 2023: 32 · 2022: 72 · 2021: 38 · 2020: 43 · 2019: 49 · 2018: 15 · 2017: 40 · 2016: 32 · 2015: 28 · 2014: 42 · 2013: 28 · 2012: 31
FWCI
22.66
Type
journal article
Language
en
Open access
closed
Locations
Behavioral and Brain Sciences (publishedVersion)
Topics
  • Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
  • Social Sciences › Library and Information Sciences › Publishing and Scholarly Communication
  • Social Sciences › Library and Information Sciences › Academic Writing and Publishing
Keywords
peer review, rejection rate, psychology journals, reviewer bias, editorial decision-making, methodological errors, publication practices

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Peer-review practices of psychological journals: The fate of published articles, submitted again. https://app.cochat.ai/papers/10.1017/s0140525x00011183
The paper
Peters, D. P., & Ceci, S. J. (1982). Peer-review practices of psychological journals: The fate of published articles, submitted again. Behavioral and Brain Sciences. https://doi.org/10.1017/s0140525x00011183

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.