The DOI matches its Crossref record.
Reviewing Peer Review
Abstract
Peer review, in which experts in the field scrutinize and critique scientific results prior to publication, is fundamental to scientific progress, and the achievements of science in the last century are an endorsement of its value. Peer review influences more than just science. The Intergovernmental Panel on Climate Change and other similar advisory groups base their judgments on peer-reviewed literature, and this is part of their success. Many legal decisions and regulations also depend on peer-reviewed science. Thus, thorough, expert review of research results--without compensation--is an obligation that scientists shoulder for both science and the general public.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“and the same papers being reviewed multiple times due to high rejection rates [1].”
From Can LLM feedback enhance review quality? A randomized study of 20K reviews at ICLR 2025 (2025) · cites this paper as [1]
Citations
Cites
Still being checked.
Cited by
Sorted by citations
Bornmann, L. (2011) · Annual Review of Information Science and Technology · doi:10.1002/aris.2011.1440450112 · Cited by 567
Duration and quality of the peer review process: the author’s perspective
Huisman, J., & Smits, J. (2017) · Scientometrics · doi:10.1007/s11192-017-2310-5 · Cited by 303
Kovanis, M., Porcher, R., Ravaud, P., & Trinquart, L. (2016) · PLoS ONE · doi:10.1371/journal.pone.0166387 · Cited by 277
Liang, W., Zhang, Y., Cao, H., Wang, B., Ding, D., Yang, X., Vodrahalli, K., He, S., Smith, D., Yin, Y., et al. (2023) · arXiv · doi:10.48550/arxiv.2310.01783 · Cited by 243
Harley, D., Acord, S. K., Earl‐Novell, S., Lawrence, S., & King, C. J. (2010) · RePEc: Research Papers in Economics · Cited by 213
The risks associated with Artificial General Intelligence: A systematic review
McLean, S. G., Read, G. J. M., Thompson, J., Baber, C., Stanton, N. A., & Salmon, P. M. (2021) · Journal of Experimental & Theoretical Artificial Intelligence · doi:10.1080/0952813x.2021.1964003 · Cited by 258
Does incentive provision increase the quality of peer review? An experimental study
Squazzoni, F., Bravo, G., & Takács, K. (2012) · Research Policy · doi:10.1016/j.respol.2012.04.014 · Cited by 102
Nine Criteria for a Measure of Scientific Output
Kreiman, G., & Maunsell, J. H. R. (2011) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2011.00048 · Cited by 96
Zhou, Y., Sun, Y., Wei, Y. D., & Lin, G. C. S. (2010) · Journal of Economic Geography · doi:10.1093/jeg/lbp065 · Cited by 96
Suchak, T., Aliu, A. E., Harrison, C., Zwiggelaar, R., Geifman, N., & Spick, M. (2025) · PLOS Biology · doi:10.1371/journal.pbio.3003152 · Cited by 95
Guraya, S. Y., Norman, R. I., Khoshhal, K. I., Guraya, S. S., & Forgione, A. (2016) · Pakistan Journal of Medical Sciences · doi:10.12669/pjms.326.10490 · Cited by 98
Overburdening of peer reviewers: a multi-stakeholder perspective on causes and effects
Severin, A., & Chataway, J. (2021) · Learned Publishing · doi:10.1002/leap.1392 · Cited by 73
The crisis of peer review: Part of the evolution of science
Horta, H., & Jung, J. (2024) · Higher Education Quarterly · doi:10.1111/hequ.12511 · Cited by 77
Squazzoni, F., & Gandelli, C. (2012) · Journal of Informetrics · doi:10.1016/j.joi.2011.12.005 · Cited by 65
Attitudes of referees in a multidisciplinary journal: An empirical analysis
Casnici, N., Grimaldo, F., Gilbert, N., & Squazzoni, F. (2016) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.23665 · Cited by 63
LINKING BIG: THE CONTINUING PROMISE OF EVOLUTIONARY SYNTHESIS
Sidlauskas, B. L., Ganapathy, G., Hazkani‐Covo, E., Jenkins, K. P., Lapp, H., McCall, L. W., Price, S. A., Scherle, R., Spaeth, P. A., & Kidd, D. M. (2009) · Evolution · doi:10.1111/j.1558-5646.2009.00892.x · Cited by 58
Meyer, P., Hoeng, J., Rice, J. J., Norel, R., Sprengel, J., Stolle, K., Bonk, T., Corthesy, S., Royyuru, A. K., Peitsch, M. C., et al. (2012) · Bioinformatics · doi:10.1093/bioinformatics/bts116 · Cited by 52
Walker, R. D., Barros, B., Conejo, R., Neumann, K., & Telefont, M. (2015) · F1000Research · doi:10.12688/f1000research.6012.2 · Cited by 54
Bornmann, L., Weymuth, C., & Daniel, H. (2009) · Scientometrics · doi:10.1007/s11192-009-0011-4 · Cited by 47
The next generation of action ecology : novel approaches towards global ecological research
White, R. L., Sutton, A. E., Salguero‐Gómez, R., Bray, T. C., Campbell, H., Cieraad, E., Geekiyanage, N., Gherardi, L., Hughes, A. C., Jørgensen, P. S., et al. (2015) · Ecosphere · doi:10.1890/es14-00485.1 · Cited by 38
Gallo, S. A., Carpenter, A. S., & Glisson, S. R. (2013) · PLoS ONE · doi:10.1371/journal.pone.0071693 · Cited by 37
Peer Review in Academic Promotion and Publishing: Its Meaning, Locus, and FuturePreprint
Harley, D., & Acord, S. K. (2011) · RePEc: Research Papers in Economics · Cited by 37
Kovanis, M., Porcher, R., Ravaud, P., & Trinquart, L. (2015) · Scientometrics · doi:10.1007/s11192-015-1800-6 · Cited by 37
Xue, Z., He, G., Liu, J., Jiang, Z., Zhao, S. X., & Lu, W. (2022) · Information Processing & Management · doi:10.1016/j.ipm.2022.103216 · Cited by 34
Is preprint the future of science? A thirty year journey of online preprint servicesPreprint
Xie, B., Shen, Z., & Wang, K. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.09066 · Cited by 32
Social Simulation That 'Peers into Peer Review'
Squazzoni, F., & Takács, K. (2011) · Journal of Artificial Societies and Social Simulation · doi:10.18564/jasss.1821 · Cited by 31
The miracle of peer review and development in science: an agent-based model
Righi, S., & Takács, K. (2017) · Scientometrics · doi:10.1007/s11192-017-2244-y · Cited by 38
Improving the Peer Review Process: A Proposed Market System
Frijters, P., & Torgler, B. (2016) · Scientometrics · doi:10.1007/s11192-019-03076-1 · Cited by 29
Forti, L. R., Solino, L., & Szabo, J. K. (2021) · Humanities and Social Sciences Communications · doi:10.1057/s41599-021-00920-9 · Cited by 32
A Three-Decade History of the Duration of Peer Review
Lyman, R. L. (2013) · Journal of Scholarly Publishing · doi:10.3138/jsp.44.3.001 · Cited by 26
Editorial behaviors in peer review
Wang, W. W., Kong, X., Zhang, J., Chen, Z., Xia, F., & Wang, X. (2016) · SpringerPlus · doi:10.1186/s40064-016-2601-y · Cited by 26
Bianchi, F., & Squazzoni, F. (2022) · Science and Public Policy · doi:10.1093/scipol/scac027 · Cited by 24
Juniors Seek an End to the Impact Factor Race
Brischoux, F., & Cook, T. R. (2009) · BioScience · doi:10.1525/bio.2009.59.8.2 · Cited by 19
A framework for assessing the peer review duration of journals: case study in computer science
Bilalli, B., Munir, R. F., & Abelló, A. (2020) · Scientometrics · doi:10.1007/s11192-020-03742-9 · Cited by 17
Public and expert collaborative evaluation model and algorithm for enterprise knowledge
Le, C., Xin-jian, G., Pan, K., Dai, F., & Guo-ning, Q. (2012) · Enterprise Information Systems · doi:10.1080/17517575.2011.644583 · Cited by 16
Bianchi, F., & Squazzoni, F. (2015) · Winter Simulation Conference · doi:10.1109/wsc.2015.7408561 · Cited by 16
Bias in peer review: a case study
Walker, R. D., Barros, B., Conejo, R., Neumann, K., & Telefont, M. (2015) · F1000Research · doi:10.12688/f1000research.6012.1 · Cited by 17
Barczyński, B. J., Graczyński, M. R., & Kalina, R. M. (2009) · Journal of Human Kinetics · doi:10.2478/v10078-009-0017-2 · Cited by 16
Peer review and over-competitive research funding fostering mainstream opinion to monopoly
Fang, H. (2010) · Scientometrics · doi:10.1007/s11192-010-0323-4 · Cited by 15
Frixione, E., Ruiz, L., & Hernández, G. (2016) · PLoS ONE · doi:10.1371/journal.pone.0155732 · Cited by 15
Preserving the Quality of Scientific Research: Peer Review of Research Articles
Silva, P. U. K. D., & Vance, C. K. (2017) · Fascinating life sciences · doi:10.1007/978-3-319-50627-2_6 · Cited by 17
Ending elitism in peer-review publication
Aarssen, L. W., & Lortie, C. J. (2009) · Ideas in Ecology and Evolution · doi:10.4033/iee.2009.2.4.e · Cited by 15
Archives, Access and Artificial Intelligence
(2022) · Bielefeld University Press eBooks · doi:10.1515/9783839455845 · Cited by 14
The academic status of reviewers predicts their language use
Sun, Z., Cao, C. C., Ma, C., & Li, Y. (2023) · Journal of Informetrics · doi:10.1016/j.joi.2023.101449 · Cited by 15
Understanding delays in publishing interdisciplinary research
Zhang, Y., & Wang, Y. (2024) · Information Processing & Management · doi:10.1016/j.ipm.2024.103826 · Cited by 14
Eve, M. P., Neylon, C., O’Donnell, D. P., Moore, S., Gadie, R., Odeniyi, V., & Parvin, S. (2020) · Cambridge University Press eBooks · doi:10.1017/9781108783521 · Cited by 16
The hurdles of academic publishing from the perspective of journal editors: a case study
Mrowinski, M. J., Fronczak, A., Fronczak, P., Nedić, O., & Dekanski, A. B. (2020) · Scientometrics · doi:10.1007/s11192-020-03619-x · Cited by 14
Squazzoni, F. (2010) · Archivio Istituzionale della Ricerca (Universita Degli Studi Di Milano) · doi:10.2383/33640 · Cited by 13
The AI Review Lottery: Widespread AI-Assisted Peer Reviews Boost Paper Scores and Acceptance Rates
Russo, G., Horta Ribeiro, M., Davidson, T. R., Veselovsky, V., & West, R. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3757667 · Cited by 12
A large-scale randomized study of large language model feedback in peer review
Thakkar, N., Yuksekgonul, M., Silberg, J., Garg, A., Peng, N., Sha, F., Yu, R., Vondrick, C., & Zou, J. (2026) · Nature Machine Intelligence · doi:10.1038/s42256-026-01188-x · Cited by 11
Archives, Access and Artificial Intelligence
Eve, M. P., Gadie, R., Odeniyi, V., & Parvin, S. (2022) · Digital humanities research · doi:10.14361/9783839455845 · Cited by 27
Syberg, K., Almroth, B. C., Fernandez, M. O., Baztan, J., Bergmann, M., Thompson, R. C., Gündoğdu, S., Knoblauch, D., Gomiero, A., Monclús, L., et al. (2024) · Microplastics and Nanoplastics · doi:10.1186/s43591-024-00091-9 · Cited by 18
Arous, I., Yang, J., Khayati, M., & Cudré-Mauroux, P. (2021) · International World Wide Web Conference (WWW) · doi:10.1145/3442381.3450088 · Cited by 11
Timmis, K. N., Williams, P. M., Karahan, Z. C., López‐García, P., Rainey, P. B., Chavarría, M., Greening, C., Steward, K. F., Hallsworth, J. E., Pereira, C. S., et al. (2025) · Microbial Biotechnology · doi:10.1111/1751-7915.70180 · Cited by 11
Neshaei, S. P., Rietsche, R., Su, X., & Wambsganss, T. (2024) · Proceedings of the 29th International Conference on Intelligent User Interfaces · doi:10.1145/3640543.3645169 · Cited by 9
Castelnuovo, G., Limonta, D., Sarmiento, L., & Molinari, E. (2010) · Clinical Practice and Epidemiology in Mental Health · doi:10.2174/1745017901006010109 · Cited by 9
Developmental Direction for Review System of the Journal of Korean Academy of Nursing
Kim, H. W., Chung, M.-S., Park, J., Suh, Y. O., Suk, M. H., Shin, H., Yang, J. H., Jang, H. J., & Jung, M. S. (2007) · Daehan ganho haghoeji · doi:10.4040/jkan.2007.37.3.422 · Cited by 7
Screening, sorting, and the feedback cycles that imperil peer review
Bergstrom, C. T., & Gross, K. (2026) · PLoS Biology · doi:10.1371/journal.pbio.3003650 · Cited by 13
Yu, H., Liang, Y., & Xie, Y. (2025) · Scientometrics · doi:10.1007/s11192-025-05264-8 · Cited by 6
Fuente-Ballesteros, A., & Zuin, V. G. (2026) · Green Analytical Chemistry · doi:10.1016/j.greeac.2026.100349 · Cited by 7
Can LLM feedback enhance review quality? A randomized study of 20K reviews at ICLR 2025
Thakkar, N., Yuksekgonul, M., Silberg, J., Garg, A., Peng, N., Sha, F., Yu, R., Vondrick, C., & Zou, J. (2025) · arXiv · doi:10.48550/arxiv.2504.09737 · Cited by 4
Let the crowd be my peers? How researchers assess the prospects of social peer review
Matt, C., Hoerndlein, C., & Heß, T. (2017) · Electronic Markets · doi:10.1007/s12525-017-0247-4 · Cited by 4
The unforeseen impact of meeting abstracts on cancer research
Krauskopf, E. (2011) · Annals of Oncology · doi:10.1093/annonc/mdr406 · Cited by 4
Yuan, P. H., MacLean, L. J., Li, E. A., Yin, S., & Micieli, J. A. (2021) · Journal of Neuro-Ophthalmology · doi:10.1097/wno.0000000000001158 · Cited by 4
Peer-review in the Internet agePreprint
Sobkowicz, P. (2008) · arXiv (Cornell University) · doi:10.48550/arxiv.0810.0486 · Cited by 4
Scientific conduct and misconduct: honesty is still the best policy
Dahanukar, N., & Molur, S. (2012) · Journal of Threatened Taxa · doi:10.11609/jott.04092012.2845-8 · Cited by 4
Editorial time management: Peer review dates and other key dates of Spanish Communication journals
Segarra‐Saavedra, J., Hidalgo‐Marí, T., & Tur‐Viñes, V. (2023) · Learned Publishing · doi:10.1002/leap.1569 · Cited by 4
Kang, Y., Zhang, C., Sun, Z., & Li, Y. (2025) · Journal of Informetrics · doi:10.1016/j.joi.2025.101709 · Cited by 5
Wanted: a ‘Reviewer Effectiveness Index‘
Baveye, P. C. (2012) · Learned Publishing · doi:10.1087/20120311 · Cited by 3
Quality Control through Peer Review Process in Scholarly Communication: Review of Related Literature
Gupta, V. K. (2017) · IRA-International Journal of Management & Social Sciences · doi:10.21013/jmss.v8.n3.p3 · Cited by 3
“Critique is not a verb”: is peer review stifling the dialogue in disaster scholarship?
Chmutina, K., Cheek, W., & von Meding, J. K. (2022) · Disaster Prevention and Management An International Journal · doi:10.1108/dpm-09-2021-0266 · Cited by 3
Acuna, D. E., Teplitskiy, M., Evans, J. A., & Kording, K. P. (2022) · PLoS ONE · doi:10.1371/journal.pone.0273994 · Cited by 3
Universalism and particularism in the recommendations of the nobel prize for science
Ko, B.-K., Jang, Y., & Yang, J. (2024) · Scientometrics · doi:10.1007/s11192-023-04921-0 · Cited by 3
Peer review and over-competitive research funding fostering mainstream opinion to monopoly. Part II
Liu, X. Z., & Fang, H. (2011) · Scientometrics · doi:10.1007/s11192-011-0526-3 · Cited by 3
Peer Review in Ophthalmology: A Collaborative Approach to Training the Next Generation of Reviewers
Chan, M. F. (2024) · Investigative Ophthalmology & Visual Science · doi:10.1167/iovs.65.10.1 · Cited by 3
Who gets the grant? A persona-based investigation into research funding panelist preferences
Santos, J. M. (2024) · Research Evaluation · doi:10.1093/reseval/rvae030 · Cited by 3
Quis judicabit ipsos judices? A case study on the dynamics of competitive funding panel evaluations
Santos, J. M. (2022) · Research Evaluation · doi:10.1093/reseval/rvac021 · Cited by 2
Time for Thymol – Let’s Flush out the Chronic Parasites in Peer-Review
Simha, A. (2024) · Group & Organization Management · doi:10.1177/10596011241280326 · Cited by 2
Clinical Research Authorships: Ethics and Problem-Solving
Katib, A. A. (2020) · Canadian Journal of Bioethics · doi:10.7202/1073787ar · Cited by 4
Wissenschaftliche Originalarbeiten in deutscher Sprache – Ein Anachronismus
Braus, D. F., & Baethge, C. (2009) · Psychiatrische Praxis · doi:10.1055/s-0028-1090152 · Cited by 2
Clinical Utility of the Adapted Biopsychosocial Model: An Initial Validation Through Peer Review
Gentry, K., Snyder, K., & Utley, J. J. (2021) · The Open Journal of Occupational Therapy · doi:10.15453/2168-6408.1750 · Cited by 2
How to get published in Palynology (or any other journal)
Riding, J. B. (2021) · Palynology · doi:10.1080/01916122.2021.1965666 · Cited by 2
Peer review under the microscope. An agent-based model of scientific collaboration
Squazzoni, F., & Gandelli, C. (2012) · Winter Simulation Conference · doi:10.1109/wsc.2012.6465283 · Cited by 2
OPENING PANDORA’S BOX: A CRITIQUE OF PEER REVIEW PROCESSES
Grosu-Rădulescu, L.-M. (2022) · SYNERGY · doi:10.24818/syn/2022/18/1.11 · Cited by 2
Groves, C. P., & Bahir, M. M. (2014) · Marine Mammal Science · doi:10.1111/mms.12147 · Cited by 2
Pautasso, M., & Jeger, M. J. (2015) · European Journal of Plant Pathology · doi:10.1007/s10658-015-0731-7 · Cited by 2
The Miracle of Peer Review and Development in Science: An Agent-Based ModelPreprint
Righi, S., & Takkcs, K. (2016) · SSRN Electronic Journal · doi:10.2139/ssrn.2807289 · Cited by 2
A formal study of co-opetition in scholarly publishingPreprint
García, J. A., Sánchez, R. M. R., & Fdez-Valdivia, J. (2021) · SSRN Electronic Journal · doi:10.2139/ssrn.3824655 · Cited by 2
Nurturing ideas, cultivating reviewers: The C.E.D.A.R. tree of generative peer review
Cristofaro, M., & Duplat, V. (2025) · European Management Journal · doi:10.1016/j.emj.2025.10.011 · Cited by 2
Alhoori, H., Fox, E. A., Frommholz, I., Liu, H., Coupette, C., Rieck, B., Ghosal, T., & Wu, J. F. (2023) · ACM/IEEE Joint Conference on Digital Libraries · doi:10.1109/jcdl57899.2023.00077 · Cited by 1
Modeling submission decision behavior: the effect of relevance and value
Xu, X., Xie, J., Lyu, D., Zhang, Z., Jiang, L., & Cheng, Y. (2026) · Scientometrics · doi:10.1007/s11192-026-05547-8 · Cited by 2
Is the problem developmental review, or the development of peer review?
Obenauer, W. G., & Griep, Y. (2026) · Industrial and Organizational Psychology · doi:10.1017/iop.2025.10050 · Cited by 1
Opening The Black-Box Of Referee Behaviour. An Agent-Based Model Of Peer Review
Squazzoni, F., & Gandelli, C. (2012) · European Conference on Modelling and Simulation (ECMS) · doi:10.7148/2012-0647-0653 · Cited by 1
A Discussion on Governmental Research Grants
Fang, H. (2014) · Science and Engineering Ethics · doi:10.1007/s11948-014-9582-4 · Cited by 1
Castelnuovo, G., Limonta, D., Sarmiento, L., & Molinari, E. (2010) · Clinical Practice and Epidemiology in Mental Health · doi:10.2174/17450179010060100109 · Cited by 1
The scientific importance of asking questions at meetings: Why virtual debate is not enough
O’Malley, M. A., & Leonelli, S. (2010) · BioEssays · doi:10.1002/bies.201000112 · Cited by 1
Genter, M. B., Brock, W. J., & Mattes, W. B. (2015) · Toxicological Sciences · doi:10.1093/toxsci/kfu284 · Cited by 1
Medina, L. R. (2020) · Tapuya Latin American Science Technology and Society · doi:10.1080/25729861.2020.1697575 · Cited by 1
All Hands on Deck: Nicotine and Tobacco Research Needs Reviewers More Than Ever
Hawk, L. W. (2023) · Nicotine & Tobacco Research · doi:10.1093/ntr/ntad091 · Cited by 1
Revisiting Peer Review: Is it Time for you to be a Peer Reviewer?
Generali, J. A. (2012) · Hospital Pharmacy · doi:10.1310/hpj4705-326
Metadata
- Authors
- Alberts, B., Hanson, B., & Kelner, K. L.
- Affiliations
- Bruce Alberts is the Editor-in-Chief of Science.
- Brooks Hanson is Deputy Editor for physical sciences at Science.
- Katrina L. Kelner is Deputy Editor for life sciences at Science.
- Venue
- Science
- Published
- 2008-07-04
- Volume
- 321
- Issue
- 5885
- Pages
- 15-15
- ISSN
- 0036-8075 (ISSN-L)
- 1095-9203
- OpenAlex
- W1968836033
- Cited by
- 119
- References
- 0
- Citations by year
- 2026: 10 · 2025: 7 · 2024: 13 · 2023: 7 · 2022: 10 · 2021: 10 · 2020: 6 · 2017: 5 · 2016: 7 · 2015: 7 · 2014: 3 · 2013: 2 · 2012: 9
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Science (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Library and Information Sciences › Publishing and Scholarly Communication
- Social Sciences › Library and Information Sciences › Academic Publishing and Open Access
- Social Sciences › Library and Information Sciences › Academic Writing and Publishing
- Keywords
- peer review, scholarly publishing, scientific integrity, science policy, IPCC, science and law, role of scientists
- MeSH terms
- Periodicals as Topic, Publishing, Peer Review, Research
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review