The DOI matches its Crossref record; 57 of the 57 references it cites were found in a registry or an index.
The ups and downs of peer review
Abstract
This article traces the history of peer review of scientific publications, plotting the development of the process from its inception to its present-day application. We discuss the merits of peer review and its weaknesses, both perceived and real, as well as the practicalities of several major proposed changes to the system. It is our hope that readers will gain a better appreciation of the complexities of the process and, when serving as reviewers themselves, will do so in a manner that will enhance the utility of the exercise. We also propose the development of an international on-line training program for accreditation of potential referees.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The International Committee of Medical Journal Editors
van der Weyden, M. (2006) · Proceedings of the Workshop on Publishing for Biomedical Journal Editors and Reviewers · doi:10.2349/biij.2.4.e54-13 · Cited by 1,998
Internet encyclopaedias go head to head
Giles, J. (2005) · Nature · doi:10.1038/438900a · Cited by 1,978
The New Encyclopaedia Britannica
Goetz, P. W. (1991) · Cited by 1,592
MEDICINE AND SCIENCE IN SPORTS AND EXERCISE???1984
Tipton, C. M. (1984) · Medicine & Science in Sports & Exercise · doi:10.1249/00005768-198401000-00002 · Cited by 598
Munhoz Junior, E. (2006) · Epidemiologia e Serviços de Saúde · doi:10.5123/s1679-49742006000100002 · Cited by 416
van Rooyen, S., Godlee, F., Evans, S., Black, N., & Smith, R. (1999) · BMJ · doi:10.1136/bmj.318.7175.23 · Cited by 415
International Committee of Medical Journal Editors.
Riis, P. J. (1990) · PubMed · Cited by 415
Peer Review in 18th-Century Scientific Journalism
Kronick, D. A. (1990) · JAMA: The Journal of the American Medical Association · doi:10.1001/jama.1990.03440100021002 · Cited by 350
The Evolution of Editorial Peer Review
Burnham, J. C. (1990) · JAMA: The Journal of the American Medical Association · doi:10.1001/jama.1990.03440100023003 · Cited by 344
Godlee, F., Gale, C. R., & Martyn, C. N. (1998) · JAMA · doi:10.1001/jama.280.3.237 · Cited by 333 · cited 5×
“(26) modified a previously published article by intentionally introducing weaknesses and examined the effects of blinding reviewers or authors and the signing of reviews on the ability of a referee to detect errors.”
BackgroundInfluentialManuscript Quality before and after Peer Review and Editing at Annals of Internal Medicine
Goodman, S. N., Berlin, J., Fletcher, S. W., & Fletcher, R. H. (1994) · Annals of Internal Medicine · doi:10.7326/0003-4819-121-1-199407010-00003 · Cited by 324
Effect of Blinding and Unmasking on the Quality of Peer Review: A Randomized Trial
van Rooyen, S., Godlee, F., Evans, S., Smith, R., & Black, N. (1998) · JAMA · doi:10.1001/jama.280.3.234 · Cited by 280
Peer Review in Health Sciences
Godlee, F., & Jefferson, T. (1999) · DigitalGeorgetown (Georgetown University Library) · Cited by 268
Peer review for journals: Evidence on quality control, fairness, and innovation
Armstrong, J. S. (1997) · Science and Engineering Ethics · doi:10.1007/s11948-997-0017-3 · Cited by 262
The Effects of Blinding on the Quality of Peer Review: A Randomized Trial
McNutt, R. A. (1990) · JAMA · doi:10.1001/jama.1990.03440100079012 · Cited by 261
Effect of Blinded Peer Review on Abstract Acceptance
Ross, J. S., Gross, C. P., Desai, M. M., Hong, Y., Grant, A. O., Daniels, S. R., Hachinski, V. C., Gibbons, R. J., Gardner, T. J., & Krumholz, H. M. (2006) · JAMA · doi:10.1001/jama.295.14.1675 · Cited by 250
Peer review in biomedical publication
Ingelfinger, F. J. (1974) · The American Journal of Medicine · doi:10.1016/0002-9343(74)90635-4 · Cited by 241
Peer review. Crude and understudied, but indispensable
Kassirer, J. P. (1994) · JAMA · doi:10.1001/jama.272.2.96 · Cited by 235
The effects of blinding on the quality of peer review. A randomized trial
Mcnutt, R. (1990) · JAMA · doi:10.1001/jama.263.10.1371 · Cited by 216
Ethics and scientific publication
Benos, D. J., Fabres, J., Farmer, J., Gutierrez, J. P., Hennessy, K., Kosek, D., Lee, J. H., Olteanu, D., Russell, T., Shaikh, F., et al. (2005) · Advances in Physiology Education · doi:10.1152/advan.00056.2004 · Cited by 209
Self-citationHojat, M., Gonnella, J. S., & Caelleigh, A. S. (2003) · Advances in Health Sciences Education · doi:10.1023/a:1022670432373 · Cited by 199
Is There Gender Bias in JAMA's Peer Review Process?
Gilbert, J. R., Williams, E. S., & Lundberg, G. D. (1994) · JAMA · doi:10.1001/jama.1994.03520020065018 · Cited by 170
Barriers to scientific contributions: The author's formula
Armstrong, J. S. (1982) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00011201 · Cited by 150
The Messenger under Attack — Intimidation of Researchers by Special-Interest Groups
Deyo, R. A., Psaty, B. M., Simon, G., et al. (1997) · New England Journal of Medicine · doi:10.1056/nejm199704173361611 · Cited by 147
Benos, D. J., Kirk, K. L., & Hall, J. E. (2003) · AJP Advances in Physiology Education · doi:10.1152/advan.00057.2002 · Cited by 143
Self-citationIs there gender bias in JAMA's peer review process?
Gilbert, J. R. (1994) · JAMA · doi:10.1001/jama.272.2.139 · Cited by 135
Reliability of Editors' Subjective Quality Ratings of Peer Reviews of Manuscripts
Callaham, M. L. (1998) · JAMA · doi:10.1001/jama.280.3.229 · Cited by 122
Does declaration of competing interests affect readers' perceptions? A randomised trial
Chaudhry, S., Schroter, S., Smith, R., & Morris, J. (2002) · BMJ · doi:10.1136/bmj.325.7377.1391 · Cited by 120
A Citation Analysis of the Impact of Blinded Peer Review
Laband, D. N. (1994) · JAMA · doi:10.1001/jama.1994.03520020073020 · Cited by 114
Atkinson, D. R., Furlong, M. J., & Wampold, B. E. (1982) · Journal of Counseling Psychology · doi:10.1037/0022-0167.29.2.189 · Cited by 109
Masking Author Identity in Peer Review: What Factors Influence Masking Success?
Cho, M. K., Justice, A. C., Winker, M. A., Berlin, J. A., Waeckerle, J. F., Callaham, M. L., Rennie, D., & and the PEER Investigators (1998) · JAMA · doi:10.1001/jama.280.3.243 · Cited by 104
Effect of Institutional Prestige on Reviewers' Recommendations and Editorial Decisions
Garfunkel, J. M. (1994) · JAMA: The Journal of the American Medical Association · doi:10.1001/jama.1994.03520020063017 · Cited by 95
Variation in Journal Peer Review Systems
Hargens, L. L. (1990) · JAMA · doi:10.1001/jama.1990.03440100052008 · Cited by 92
Gender factors in reviewer recommendations for manuscript publication.
Lloyd, M. E. (1990) · Journal of Applied Behavior Analysis · doi:10.1901/jaba.1990.23-539 · Cited by 90
Financial Interest and Its Disclosure in Scientific Publications
Krimsky, S. (1998) · JAMA · doi:10.1001/jama.280.3.225 · Cited by 90
Second-Order Peer Review of the Medical Literature for Clinical Practitioners
Haynes, R. B. (2006) · JAMA · doi:10.1001/jama.295.15.1801 · Cited by 88
Peer Review: A Study of Reliability
Ceci, S. J., & Peters, D. P. (1982) · Change The Magazine of Higher Learning · doi:10.1080/00091383.1982.10569910 · Cited by 87
Variation in journal peer review systems. Possible causes and consequences.
Hargens, L. L. (1990) · PubMed · Cited by 81
Reviewers' perceptions of the peer review process for a medical education journal
Snell, L. S., & Spencer, J. (2004) · Medical Education · doi:10.1111/j.1365-2929.2004.02026.x · Cited by 79
Postpublication Criticism and the Shaping of Clinical Knowledge
Horton, R. C. (2002) · JAMA · doi:10.1001/jama.287.21.2843 · Cited by 68
Effect of blinding and unmasking on the quality of peer review
van Rooyen, S., Godlee, F., Evans, S. J. W., Smith, R. W., & Black, N. A. (1999) · Journal of General Internal Medicine · doi:10.1046/j.1525-1497.1999.09058.x · Cited by 63
Who are the peer reviewers and how much do they review?
Yankauer, A. (1990) · PubMed · Cited by 62
Lock, S. (1990) · JAMA · doi:10.1001/jama.1990.03440100045006 · Cited by 61
McCook, A. (2006) · The Scientist (Philadelphia, Pa. Online)/The Scientist · Cited by 58
Scientific Integrity: Text and Cases in Responsible Conduct of Research
Macrina, F. L. (2005) · Cited by 58
Changes to Manuscripts During the Editorial Process
Purcell, G. P., Donovan, S. L., & Davidoff, F. (1998) · JAMA · doi:10.1001/jama.280.3.227 · Cited by 58
... And How the Problems Eluded Peer Reviewers and Editors
Couzin, J. (2006) · Science · doi:10.1126/science.311.5757.23 · Cited by 45
Does Editorial Peer Review Work?
Lock, S. (1994) · Annals of Internal Medicine · doi:10.7326/0003-4819-121-1-199407010-00012 · Cited by 45
The Communities of Scientists and Journal Peer Review
Knoll, E. (1990) · JAMA · doi:10.1001/jama.1990.03440100030004 · Cited by 43
Editorial Peer Review in US Medical Journals
Weller, A. C. (1990) · JAMA · doi:10.1001/jama.1990.03440100048007 · Cited by 41
AMERICAN COLLEGE OF SPORTS MEDICINE
&NA; (1984) · Medicine & Science in Sports & Exercise · doi:10.1249/00005768-198401000-00001 · Cited by 37
Basic philosophy and concepts underlying scientific peer review
Stehbens, W. E. (1999) · Medical Hypotheses · doi:10.1054/mehy.1997.0628 · Cited by 33
Tyler, C. (2006) · Atmospheric Science Letters · doi:10.1002/asl.132 · Cited by 30
A comparison of authors publishing in two groups of U.S. medical journals.
Weller, A. C. (1996) · Europe PMC (PubMed Central) · Cited by 18
Peer review – still the well‐functioning quality control and enhancer in scientific research
Isohanni, M. K. (2005) · Acta Psychiatrica Scandinavica · doi:10.1111/j.1600-0447.2005.00601.x · Cited by 10
Kilwein, J. H. (1999) · Journal of Clinical Pharmacy and Therapeutics · doi:10.1046/j.1365-2710.1999.00243.x · Cited by 9
Peer review on the Internet: launching eMJA peer review study 2
Bingham, C. M., & Weyden, M. B. V. D. (1998) · The Medical Journal of Australia · doi:10.5694/j.1326-5377.1998.tb140244.x · Cited by 8
Cited by
The Academic, Economic and Societal Impacts of Open Access: An Evidence-Based Review
Tennant, J. P., Waldner, F., Jacques, D. C., Masuzzo, P., Collister, L. B., & Hartgerink, C. H. J. (2016) · F1000Research · doi:10.12688/f1000research.8460.3 · Cited by 560
Scientific Utopia: I. Opening Scientific Communication
Nosek, B. A., & Bar-Anan, Y. (2012) · Psychological Inquiry · doi:10.1080/1047840x.2012.692215 · Cited by 368
Is open peer review the fairest system? No
Khan, K. (2010) · BMJ · doi:10.1136/bmj.c6425 · Cited by 77
Challenges, experiments, and computational solutions in peer review
Shah, N. B. (2022) · Communications of the ACM · doi:10.1145/3528086 · Cited by 66
Doing peer review and receiving feedback: impact on scientific literacy and writing skills
Geithner, C. A., & Pollastro, A. N. (2016) · Advances in Physiology Education · doi:10.1152/advan.00071.2015 · Cited by 35
Metadata
- Authors
- Benos, D. J., Bashari, E., Chaves, J. M., Gaggar, A., Kapoor, N., LaFrance, M., Mans, R., Mayhew, D., McGowan, S., Polter, A., Qadri, Y., Sarfare, S., Schultz, K., Splittgerber, R., Stephenson, J., Tower, C., Walton, R. G., & Zotov, A.
- Venue
- Advances in Physiology Education
- Publisher
- American Physiological Society
- Published
- 2007-06
- Volume
- 31
- Issue
- 2
- Pages
- 145-152
- ISSN
- 1043-4046 (ISSN-L)
- 1522-1229
- OpenAlex
- W2158156550
- Cited by
- 294
- References
- 57
- FWCI
- 73.38
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.