Cited by 623Research, Science, and Academiascientometrics and bibliometrics researchPsychology Research and Bibliometrics

The DOI matches its Crossref record; 3 of the 3 references it cites were found in a registry or an index.

Chance and Consensus in Peer Review

Cole, Stephen, Cole, Jonathan R., Simon, Gary A.·1981·Science

DOI 10.1126/science.7302566

Open this paper in CoChat — free account

Abstract

An experiment in which 150 proposals submitted to the National Science Foundation were evaluated independently by a new set of reviewers indicates that getting a research grant depends to a significant extent on chance. The degree of disagreement within the population of eligible reviewers is such that whether or not a proposal is funded depends in a large proportion of cases upon which reviewers happen to be selected for it. No evidence of systematic bias in the selection of NSF reviewers was found.

Integrity signals

Registry

Crossref · 10.1126/science.7302566

Cites

3 of 3 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Toward a Metric of Science: The Advent of Science Indicators.
Wilken, P. H., Elkana, Y., Lederberg, J., Merton, R. Κ., Thackray, A. W., & Zuckerman, H. (1979) · Social Forces · doi:10.2307/2577293 · Cited by 280
Peer Review and the Support of Science
Cole, S., Rubin, L., & Cole, J. R. (1977) · Scientific American · doi:10.1038/scientificamerican1077-34 · Cited by 140
Self-citation
Which researcher will get the grant?
Cole, J. R., & Cole, S. (1979) · Nature · doi:10.1038/279575a0 · Cited by 21
Self-citation

Flagged first, then by citations

Show all 3 references

Research works printed but not linked: 1

COLE, S, METRIC SCI ADVENT SC: 162 (1978).

Sorted by citations

Items 401–500 of 623

Book Review: Social studies of evaluation
Rosental, C. (2010) · Social Studies of Science · doi:10.1177/0306312709357645 · Cited by 4
“See for instance Cole et al. (1981) and Fournier et al. (1988).”
Method
Research on peer-review practices: Problems of interpretation, application, and propriety
Wilson, W. A. (1982) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00011705 · Cited by 4
Responsibility in reviewing and research
Tax, S., & Rubinstein, R. A. (1982) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00011663 · Cited by 4
Making the plausible implausible: A favorable review of Peters and Ceci's target article
Millman, J. (1982) · Behavioral and Brain Sciences · doi:10.1017/s0140525x0001150x · Cited by 4
On forecasting validity and finessing reliability
Gilmore, J. B. (1991) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00065821 · Cited by 4
Judging document content versus social functions of refereeing: Possible and impossible tasks
Griffith, B. C. (1982) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00011407 · Cited by 4
Toward openness and fairness in the review process
Rourke, B. P. (1991) · Behavioral and Brain Sciences · doi:10.1017/s0140525x0006595x · Cited by 4
Exploiting large language models in peer review: indirect prompt injection attacks and integrity probes
Torrielli, F., Locci, S., Rapp, A., & Di, L. (2026) · Scientometrics · doi:10.1007/s11192-026-05695-x · Cited by 6
Imaginary carrot or effective fertiliser? A rejoinder on funding and productivity
Heyard, R., Philipp, T., & Hottenrott, H. (2021) · Scientometrics · doi:10.1007/s11192-021-04130-7 · Cited by 3
Rituals of reason: Experimental evidence on the social acceptability of lotteries in allocation problems
Bouacida, E., & Foucart, R. (2025) · Games and Economic Behavior · doi:10.1016/j.geb.2025.04.001 · Cited by 3
Measures of Performance of Universities and Their Faculty in Japan
Shirabe, M. (2004) · Information Knowledge Systems Management · doi:10.3233/iks-2004-00071 · Cited by 2
Quality Control through Peer Review Process in Scholarly Communication: Review of Related Literature
Gupta, V. K. (2017) · IRA-International Journal of Management & Social Sciences · doi:10.21013/jmss.v8.n3.p3 · Cited by 3
The Impact of Data Source on the Ranking of Computer Scientists Based on Citation Indicators: A Comparison of Web of Science and Scopus.
Zhang, L. (2014) · Issues in Science and Technology Librarianship · doi:10.29173/istl1596 · Cited by 3
(Fehlende) Anwesenheit bei Begutachtungsprozessen: Zum Wandel der sozialen Interaktionen im virtuellen Format
Paruschke, L., Barlösius, E., & Philipps, A. (2023) · Zeitschrift für Soziologie · doi:10.1515/zfsoz-2023-2027 · Cited by 3
Universalism and particularism in the recommendations of the nobel prize for science
Ko, B.-K., Jang, Y., & Yang, J. (2024) · Scientometrics · doi:10.1007/s11192-023-04921-0 · Cited by 3
“Studies have revealed that peer review is not free from bias; this is true in the contexts of publications, grants, research fellowships, and even scientific policy (Bornmann & Daniel, 2005; Casnici et al., 2017; Cole et al., 1981; Huber et al., 2022; Langfeldt, 2006; Pier et al., 2018).”
The Necessity of Commensuration Bias in Grant Peer Review
Heesen, R. (2022) · Ergo an Open Access Journal of Philosophy · doi:10.3998/ergo.2240 · Cited by 3
Les étapes du financement de la recherche académique : de la soumission en partenariat jusqu’à l’attribution et aux publications
Banal‐Estañol, A., Macho‐Stadler, I., & Pérez‐Castrillo, D. (2019) · Revue économique · doi:10.3917/reco.705.0625 · Cited by 3
A Game Theoretic Approach to Peer Review of Grant Proposals
Bayindir, E. E., Gürdal, M. Y., & Sağlam, İ. (2019) · Journal of Informetrics · doi:10.1016/j.joi.2019.100981 · Cited by 3
La psicología social de la ciencia: una revisión bibliográfica de su estado actual
GESCIT, G. (2007) · Athenea Digital Revista de pensamiento e investigación social · doi:10.5565/rev/athenea.406 · Cited by 3
Blind review of research proposals in Korea: Its effectiveness and factors affecting applicant detection
Lee, M., Om, K., & Koh, J. (1999) · Scientometrics · doi:10.1007/bf02458466 · Cited by 3
Peer review: Explicit criteria and training can help
Delcomyn, F. (1991) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00065778 · Cited by 3
Behind Door #3: The Hard-to-Please Grant Reviewer.
Miner, J. T. (2011) · Research management review · Cited by 3
The State of Peer Review in Criminology: Literary Theory, Perceptions, and the Catch-22 Metaphor of Peer Review
Higgins, E. M. (2017) · Journal of Criminal Justice Education · doi:10.1080/10511253.2017.1420809 · Cited by 3
Identification of Core Journals and Analysis of Collaboration Networks Among Researchers in Ibero America
Rogel-Salazar, J., & Rogel-Salazar, R. (2014) · Journal of Librarianship and Scholarly Communication · doi:10.7710/2162-3309.1139 · Cited by 3
Entrepreneurial President: Richard Atkinson and the University of California, 1995-2003Preprint
Pelfrey, P. A. (2012) · RePEc: Research Papers in Economics · Cited by 3
Scientific communication: So where do we go from here?
Hartley, J. R. (1982) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00011419 · Cited by 3
Peer review: Prediction of the future or judgment of the past?
Louttit, R. T. (1982) · Behavioral and Brain Sciences · doi:10.1017/s0140525x0001147x · Cited by 3
Computer-assisted referee selection as a means of reducing potential editorial bias
Bernard, H. R. (1982) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00011249 · Cited by 3
Some indices of the reliability of peer review
Rosenthal, R. (1991) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00065948 · Cited by 3
SELECTIVITY IN FUNDING: EVALUATION OF RESEARCH IN AUSTRALIA
Franklin, J. J. (1988) · Prometheus · doi:10.1080/08109028808631838 · Cited by 3
Should the blinded lead the blinded?
Lock, S. (1991) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00065894 · Cited by 3
Reliability, bias, or quality: What is the issue?
Nelson, K. (1982) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00011547 · Cited by 3
Chairman's action: The importance of executive decisions in peer review
Tyrer, P. J. (1991) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00065997 · Cited by 3
What to do about peer review: Is the cure worse than the disease?
Zentall, T. R. (1991) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00066012 · Cited by 3
Justice, efficiency and epistemology in the peer review of scientific manuscripts
Mahoney, M. J. (1991) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00065900 · Cited by 3
Anomalies in the peer-review system: A case study of the journal of High Energy PhysicsPreprint
Sikdar, S., Matteo Marsili, A., Ganguly, N., et al. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1608.04875 · Cited by 3
“That the reviewers often fail to reach consensus ([5]) and that rejected papers are often cited more in the long run ([2]), have already been pointed out.”
Background
The Cultivation of Social Work Knowledge: Towards a More Robust System of Peer ReviewPreprint
Dunleavy, D. J. (2020) · doi:10.31235/osf.io/rvqze · Cited by 3
Development and Validation of a Scoring Rubric for Editorial Evaluation of Peer-review Quality: A Pilot Study.
Love, J. N., Messman, A., Ilgen, J. S., Merritt, C., Coates, W. C., Ander, D. S., & Way, D. P. (2024) · PubMed Central · doi:10.5811/westjem.18432 · Cited by 2
The interplay of author and editor gender in acceptance delays: evidence from accounting journals
Argilés-Bosch, J. M., Kasperskaya, Y., García-Blandón, J., & Ravenda, D. (2025) · Scientometrics · doi:10.1007/s11192-025-05259-5 · Cited by 2
“A final and important remark addresses the current crisis of peer review, often described as random (Cole et al., 1981; Peters & Ceci, 1982), arbitrary (Gans & Shepherd, 1994), inefficient (Reinhart & Schendzielorz, 2020; Roumbanis, 2019), lengthy and flawed or biased (Street & Ward, 2019).”
Prestige at Play: University Hierarchies and the Reproduction of Funding Inequalities
Larrègue, J., & Pavie, A. (2025) · Canadian Review of Sociology/Revue canadienne de sociologie · doi:10.1111/cars.70012 · Cited by 2
Measures for the Quality of Individual Articles, Authors and Journals
Rajesh, R., Rajeev, K., Ragesh, N. K., & Gopakumar, V. (2010) · International Journal of Computer Applications · doi:10.5120/1369-1844 · Cited by 2
“Lot of literature has appeared which favors and criticizes impact factor [6], [7], [9], [13], [14], [15], [16], [19], [20], [21], [22], [23].”
Background
The Many Dilemmas of Grant Peer Review
Langfeldt, L. (2024) · SpringerBriefs in political science · doi:10.1007/978-3-031-69580-3_5 · Cited by 3
Feyerabend and Physics
Svozil, K. (2007) · Veröffentlichungen des Instituts Wiener Kreis · doi:10.1007/978-3-211-68146-6_7 · Cited by 2
Symposium on peer reviewing in scholarly journals
Gebhardt, R. C., Berkenkotter, C., Arrington, P. K., Hesse, D. D., Fontaine, S. I., & Hunter, S. M. (1995) · Rhetoric Review · doi:10.1080/07350199509359186 · Cited by 2
“Censorship”, early childhood research quarterly and qualitative research: Not so much aced out as an own goal?
Chattoe‐Brown, E. (2014) · Early Childhood Research Quarterly · doi:10.1016/j.ecresq.2014.10.007 · Cited by 2
Limitations of the impact factor
Lenhard, M., & Johnson, T. R. C. (2007) · European Journal of Obstetrics & Gynecology and Reproductive Biology · doi:10.1016/j.ejogrb.2007.03.009 · Cited by 2
Peer review is not enough: Editors must work with librarians to ensure access to research
Fuller, S. (1991) · Behavioral and Brain Sciences · doi:10.1017/s0140525x0006581x · Cited by 2
The process of peer review: Unanswered questions
Nelson, L. D. (1991) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00065924 · Cited by 2
Polaridad de los comentarios y consistencia interna en los informes de arbitraje de artículos de investigación
Astudillo, C., Squadrito, K., Varas, G., González, C., & Sabaj, O. (2016) · Acta bioethica · doi:10.4067/s1726-569x2016000100013 · Cited by 2
Publication Patt ern of scientists of Physical Research Laboratory (PRL), Ahmedabad, India
Anilkumar, N. (2013) · Lincoln (University of Nebraska) · Cited by 2
Resolving disputes over science in natural resource agency decisionmaking
Ruell, E. W., Burkardt, N., & Clark, D. R. (2010) · Cited by 2
Wie zuverlässig ist das Peer-Review-Verfahren? Eine Untersuchung der Interrater-Reliabilität von Gutachter*innen auf DGPuK-Tagungen
Koch, T., & Geiß, S. (2019) · Studies in Communication and Media · doi:10.5771/2192-4007-2019-2-203 · Cited by 1
Research Grants and Agenda Shaping
Martin, B. G. (2020) · Springer eBooks · doi:10.1007/978-3-030-36822-7_7 · Cited by 2
Funding transformative research - rationale and impacts
Luukkonen, T. (2022) · Edward Elgar Publishing eBooks · doi:10.4337/9781788970815.00011 · Cited by 2
A critical approach to the evaluation of the quality of accounting research in the Spanish university systand its implications
Argilés-Bosch, J. M., García-Blandón, J., & Ravenda, D. (2023) · Revista de Contabilidad · doi:10.6018/rcsar.439921 · Cited by 2
Unreliable peer review: Causes and cures of human misery
Colman, A. M. (1991) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00065742 · Cited by 2
Design and development of an evaluation framework for the NHMRC: Review of international practice, overview of NHMRC's data and grant management systems, and an evaluation framework and metrics
Guthrie, S., Rodriguez-Rincon, D., McInroy, G. R., Ioppolo, B., & Gunashekar, S. (2019) · RAND Corporation eBooks · doi:10.7249/rr2984 · Cited by 2
The Reliability of Individual and Pooled Expert Judgments
Reaven, S. J. (1987) · Springer eBooks · doi:10.1007/978-1-4684-5317-1_16 · Cited by 2
Technology Transfer at the National Institutes of Health
Rohrbaugh, M. L., & Stanton, B. R. (2009) · Wiley eBooks · doi:10.1002/9783527627301.ch3 · Cited by 2
Optional published refereeing
Gordon, R. A. (1982) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00011390 · Cited by 2
NSF Peer Review (Continued)
Humphreys, L. G. (1982) · Science · doi:10.1126/science.215.4531.344-a · Cited by 2
The evolution of conceptual systems in science
Hull, D. L. (1992) · World Futures · doi:10.1080/02604027.1992.9972295 · Cited by 2
Peer review and the structure of knowledge
Blissett, M. (1982) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00011262 · Cited by 3
Biases, decisions and auctorial rebuttal in the peer-review process
Palermo, D. S. (1982) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00011560 · Cited by 2
Assessing Peer Review in the Quest for Improved Medical Services
Hershey, N., & Bontempo, L. C. (1989) · Quality Assurance and Utilization Review · doi:10.1177/0885713x8900400402 · Cited by 2
Cognitive relativism and peer-review bias
Gordon, M. D. (1982) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00011389 · Cited by 2
2004: A scenario of peer review in the future
Porter, A. L. (1982) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00011596 · Cited by 2
Evaluating scholarly works: How many reviewers? How much anonymity?
Cone, J. D. (1991) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00065754 · Cited by 2
Consensus and the reliability of peer-review evaluations
Cole, S. (1991) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00065730 · Cited by 2
Self-citation
Peer Review: The Continual Need for Reassessment
Levy, J. A. (1984) · Cancer Investigation · doi:10.3109/07357908409018445 · Cited by 2
Grant reviewer perceptions of the quality and effectiveness of panel discussionPreprint
Gallo, S. A., Schmaling, K. B., THOMPSON, L. A., & Glisson, S. R. (2019) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/586685 · Cited by 2
“This may be related to the natural rater variability in assessing research quality that is inherent in peer review [31,32], which likely exists independent of communication setting.”
Background
The Multidimensional Assessment of Scholarly Research ImpactPreprint
Moed, H. F., & Halevi, G. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1406.5520 · Cited by 2
The Peer Review Question
Singer, I. M. (1981) · Science · doi:10.1126/science.214.4527.1292-b · Cited by 2
The Peer Review Question
Atkinson, R. C. (1981) · Science · doi:10.1126/science.214.4527.1292-a · Cited by 2
Peer Review of Abstracts Submitted to the Graduate Student and New Evaluators Topical Interest Group for the 2006 American Evaluation Association Conference
Schröter, D. C., Coryn, C. L. S., & Montrosse‐Moorhead, B. (2008) · Journal of MultiDisciplinary Evaluation · doi:10.56645/jmde.v5i9.171 · Cited by 2
Can Large Language Models Evaluate Grant Proposal Quality? Revisiting the Wennerås and Wold Peer Review Data
Sandström, U., & Thelwall, M. (2026) · Journal of Data and Information Science · doi:10.1515/jdis-2026-0048 · Cited by 1
“Early quantitative studies already showed that agreement between reviewers is often low, with correlations between independent scores commonly ranging from weak to moderate (e.g., Cole et al, 1981; Cicchetti, 1991).”
Greenlighting Innovative Projects: How Evaluation Structure Shapes the Perceived Feasibility of Early-Stage Ideas
N. Lane, J., Friis, S., Cai, T., Menietti, M., Weber, G., & Guinan, E. (2024) · doi:10.2139/ssrn.4769282 · Cited by 3
The “Normative Structure” of Social Science: Merton’s Ideas as a Story of Success and Side Effects
Hillmert, S. (2022) · Serendipities Journal for the Sociology and History of the Social Sciences · doi:10.7146/serendipities.v6i2.130014 · Cited by 1
Is there gender bias in awarding cum laude for the PhD thesis?
van den Besselaar, P., & Mom, C. (2024) · Scientometrics · doi:10.1007/s11192-024-04985-6 · Cited by 2
“Finally, the peer review literature has shown the many uncertainties and biases in peer and panel review processes (Chubin & Hackett, 1990; Cole et al., 1977, 1981; Cole, 1992; Lamont, 2010; Van Arensbergen et al., 2014), making it unlikely that the committee-based evaluation of PhD theses would be…”
Publication Delay and Article Citation: Evidence from Accounting
Argilés-Bosch, J. M., Ravenda, D., & García-Blandón, J. (2025) · Serials Review · doi:10.1080/00987913.2025.2466999 · Cited by 1
“In this vein, Peters and Ceci (1982) and Cole et al. (1981) conclude that there are no objective (or predominantly objective) criteria in the review process to assess the quality of papers and make reliable and accountable decisions about their novelty, contribution, and interest to the academic…”
Merit, competition and gender: scientific promotion in public research organisations
Castro, L. C., Casado, C., & Menéndez, L. S. (2025) · Humanities and Social Sciences Communications · doi:10.1057/s41599-025-05102-5 · Cited by 1
“Instead, factors such as the “ luck of the reviewer draw ” (Cole et al., 1981), the “ most similar to me ” (Banal-Estañol et al., 2023), and having a colleague or co-author in the evaluation panel are highlighted.”
Peer review for funding decisions
Liu, Y., Li, S., & Rousseau, R. (2025) · Journal of Data and Information Science · doi:10.2478/jdis-2025-0050 · Cited by 1
“Nevertheless, to answer the public criticism of the peer review system raised in Congressional hearings and elsewhere, Cole et al. (1977, 1981), working as consultants of the National Academy of Science’s Committee on Science and Public Policy (COSPUP), tested the peer review system of the NSF.”
Peer-Review, Bibliometrie, Altmetrics – gibt es einen Königsweg bei Evaluierungen?
Mittermaier, B. (2026) · Ubiquity Press eBooks · doi:10.5334/bdh.h · Cited by 1
Distortions from Unbiased Noise in Sequential Review: Why the Research Excellence Framework Needs a Noise Audit
Ward, R. (2026) · Quarterly Journal of Experimental Psychology · doi:10.1177/17470218261446939 · Cited by 2
“When it is assessed, noise in all its forms is found to effect professional judgement in all sorts of domains (Allison et al, 2014; Cole et al., 1981; Dror & Cole, 2010; Pier et al., 2018; Simonsohn, 2007).”
Luck and Predictability in the Life Course
van de Rijt, A., Bernardi, F., Foley, W., & Sage, L. (2026) · Annual Review of Sociology · doi:10.1146/annurev-soc-031324-115112 · Cited by 2
Prediction of high-impact research: a historical review and research opportunities
Park, G., & Yi, S. (2026) · Scientometrics · doi:10.1007/s11192-026-05660-8 · Cited by 1
“…despite its overall effectiveness, the peer-review process can occasionally involve subjective elements and variability (Seglen, 1997; Ernst et al., 1993; Cole et al., 1981), potentially leading to instances where high-impact research might not be immediately recognized or fully appreciated.”
Incentivizing High-Quality Reviews in Peer-to-Peer Settings
Yue, J. Z., Böhm, K., & von Stackelberg, S. (2014) · International Journal of Virtual Communities and Social Networking · doi:10.4018/ijvcsn.2014010101 · Cited by 1
FROM AFFIRMATIVE ACTION TO DIVERSITY: RATIONALES FOR FACULTY DIVERSIFICATION EFFORTS IN A STATE ANTI-AFFIRMATIVE ACTION REGULATORY ENVIRONMENT
Muñiz, M. M. (2012) · Cited by 1
Whose Article Is It Anyway?: Student Editors and Law Reviews
Potuto, J. R. (2016) · Indiana Law Review · doi:10.18060/4806.0100 · Cited by 1
Perspectives from early career researchers on the publication process in ecology - a response to Statzner & Resh (2010)
Rlinghaus, R. O. A., & Ndrew, A. (2011) · Cited by 1
CGG toolkit: Software components for computational genomics
Vasileiou, D., Karapiperis, C., Baltsavia, I., Chasapi, A., Ahrén, D., Janssen, P., Iliopoulos, I., Promponas, V. J., Enright, A. J., & Ouzounis, C. A. (2023) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1011498 · Cited by 1
La recherche en biologie-santé à l’ANR
Damerval, T. (2024) · Bulletin de l Académie Nationale de Médecine · doi:10.1016/j.banm.2024.02.006 · Cited by 1
Open to talent? How scientists assess merit and diversity in hiring
Johnson, D. R., & Vaidyanathan, B. (2024) · Higher Education · doi:10.1007/s10734-024-01244-8 · Cited by 1
“…found that reviewers generally evaluate based on merit rather than particularistic characteristics such as institutional affiliation (e.g., Cole et al., 1981; Zucker-man & Merton, 1971), subsequent waves of research suggested that evidence of reviewer consideration of irrelevant criteria…”
Referee quality and article citations: a case study of a finance journal
Bonnett, J. M., & Siganos, A. (2024) · Studies in Higher Education · doi:10.1080/03075079.2024.2406389 · Cited by 1
“On the one hand, some studies (Cole, Cole, and Simon 1981; Dobele 2015; Gans and Shepherd 1994; Welch 2014) have shown that referees often reach different conclusions when reviewing the same articles, indicating that it is not easy to assess article quality.”
Influential
Introduction: Scientific Creativity
Simonton, D. K. (2004) · Cambridge University Press eBooks · doi:10.1017/cbo9781139165358.003 · Cited by 1
Die Gründung der Soziologischen Revue
Hartmann, H. (2017) · Springer eBooks · doi:10.1007/978-3-658-07614-6_50 · Cited by 1
Risking Groupthink in Impact Assessment
Derrick, G. (2018) · Springer eBooks · doi:10.1007/978-3-319-63627-6_6 · Cited by 1
La structure genrée du champ sociologique québécois
Larrègue, J., & Warren, J. (2024) · Recherches sociographiques · doi:10.7202/1113760ar · Cited by 1
Scientific Information in Continuous Characteristics Spaces
Scharnhorst, A., Wouters, P. F., & Ebeling, W. (2003) · Data Archiving and Networked Services (DANS) · Cited by 1
Peer review of submissions to the annual American Evaluation Association conference by the Graduate Student & New Evaluators Topical Interest Group
Schröter, D. C., Coryn, C. L. S., & Montrosse‐Moorhead, B. (2008) · Journal of MultiDisciplinary Evaluation · Cited by 1

Authors: Cole, S., Cole, J. R., & Simon, G. A.
Affiliations: Stony Brook University (US)
Columbia University (US)
ORCID: Cole, Stephen
Venue: Science
Publisher: American Association for the Advancement of Science (AAAS)
Published: 1981-11-20
Volume: 214
Issue: 4523
Pages: 881-886
ISSN: 0036-8075 (ISSN-L)
1095-9203
DOI: 10.1126/science.7302566
OpenAlex: W2091375917
Semantic Scholar: 20d3ad461253ead52398dcc0b194c5f5af1f6e08
Cited by: 623
References: 3
Citations by year: 2026: 15 · 2025: 16 · 2024: 16 · 2023: 16 · 2022: 21 · 2021: 17 · 2020: 27 · 2019: 24 · 2018: 18 · 2017: 17 · 2016: 21 · 2015: 23 · 2014: 22 · 2013: 15 · 2012: 18
FWCI: 279.85
Type: journal article
Language: en
Open access: closed
Locations: Science (publishedVersion)
PubMed (publishedVersion)
DigitalGeorgetown (Georgetown University Library) (submittedVersion)
Topics: Social Sciences › Business, Management and Accounting › Management of Technology and Innovation › Research, Science, and Academia
Social Sciences › Library and Information Sciences › scientometrics and bibliometrics research
Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
Keywords: peer review, research funding, National Science Foundation, reviewer disagreement, reviewer selection, scientific consensus, research evaluation
MeSH terms: Peer Review, Research Support as Topic, United States

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Chance and Consensus in Peer Review

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Citations

Cites

Cited by

Related

Metadata

Record

About these checks