Chance and Consensus in Peer Review

Cole, Stephen, Cole, Jonathan R., Simon, Gary A.·1981·Science
DOI 10.1126/science.7302566

Cited by (623)

Items 101–200 of 623

  1. Scientific quality in the eyes of the scientist. A questionnaire study
    Sven Hemlin · 1993 · Scientometrics
  2. Grants Peer Review in Theory and Practice
    Daryl E. Chubin · 1994 · Evaluation Review
  3. Peer review: A philosophically faulty concept which is proving disastrous for science
    David F. Horrobin · 1982 · Behavioral and Brain Sciences
  4. Publication, politics, and scientific progress
    Michael J. Mahoney · 1982 · Behavioral and Brain Sciences
  5. Scientometric indicators: peer-review, bibliometric methods and conflict of interests
    Primož Južnič, Stojan Pečlin, Matjaž Žaucer, Tilen Mandelj, Miro Pušnik, Franci Demšar · 2010 · Scientometrics
  6. THE VALIDITY AND RELIABILITY OF EVALUATION OF SCHOLARLY PERFORMANCE
    A. J. Nederhof · 1988 · Elsevier eBooks
  7. Manuscript evaluation by journal referees and editors: Randomness or bias?
    Andrew M. Colman · 1982 · Behavioral and Brain Sciences
  8. Interreferee agreement and acceptance rates in physics
    David Lazarus · 1982 · Behavioral and Brain Sciences
  9. The Validation of Peer Review through Research Impact Measures and the Implications for Funding Strategies
    Stephen A. Gallo, Afton S. Carpenter, David Irwin, Caitlin McPartland, Joseph Travis, Sofie Reynders, LISA A. THOMPSON, Scott R. Glisson · 2014 · PLoS ONE
  10. Methods for evaluating information sources: An annotated catalogue
    Birger Hjørland · 2012 · Journal of Information Science
  11. Anosmic peer review: A rose by another name is evidently not a rose
    Sandra W. Scarr · 1982 · Behavioral and Brain Sciences
  12. ‘Your comments are meaner than your score’: score calibration talk influences intra- and inter-panel variability during scientific grant peer review
    Elizabeth L. Pier, Joshua Raclaw, A. Kaatz, M. Brauer, M. Carnes, Mitchell J. Nathan, Cecilia E. Ford · 2017 · Research Evaluation
  13. How reliable is peer review? An examination of operating grant proposals simultaneously submitted to two similar peer review systems
    Corinne S. Hodgson · 1997 · Journal of Clinical Epidemiology
  14. Research on research evaluation
    Hemlin, Sven · 1996 · Social Epistemology
  15. An efficient system to fund science: from proposal review to peer-to-peer distributions
    Johan Bollen, David Crandall, Damion Junk, Ying Ding, Katy Börner · 2016 · Scientometrics
  16. Peer review and innovation
    Raymond E. Spier · 2002 · Science and Engineering Ethics
  17. What is the source of bias in peer review?
    Ray Over · 1982 · Behavioral and Brain Sciences
  18. On peer review: “We have met the enemy and he is us”
    Domenic V. Cicchetti · 1982 · Behavioral and Brain Sciences
  19. Peer Review: From “Sacred Ideals” to “Profane Realities”
    David Roland Johnson, Joseph C. Hermanowicz · 2017 · Higher education
  20. Peer review for the evaluation of academic research: lessons from the Italian experience
    Emanuela Reale, Barbara Anna, Antonio Costantini · 2007 · Research Evaluation
  21. A Survey of University Institutional Review Boards: Characteristics, Policies, and Procedures
    Gregory J. Hayes, Steven C. Hayes, Thane A. Dykstra · 1995 · IRB Ethics and Human Research
  22. The role of metrics in peer assessments
    Liv Langfeldt, Ingvild Reymert, Dag W. Aksnes · 2020 · Research Evaluation
  23. Federal research impact assessment: Axioms, approaches, applications
    Ronald Neil Kostoff · 1995 · Scientometrics
  24. Citation frequency and journal impact: valid indicators of scientific quality?
    Per O. Seglen · 1991 · Journal of Internal Medicine
  25. The Selection of Scientific Talent in the Allocation of Research Grants
    Pleun van Arensbergen, Peter van den Besselaar · 2012 · Higher Education Policy
  26. Strategyproof peer selection using randomization, partitioning, and apportionment
    Haris Aziz, Omer Lev, Nicholas Mattei, Jeffrey S. Rosenschein, Toby Walsh · 2019 · Artificial Intelligence
  27. Referee Recommendations
    Ivo Welch · 2014 · Review of Financial Studies
  28. Inter-rater reliability and validity of peer reviews in an interdisciplinary field
    Jens Jirschitzka, Aileen Oeberst, Richard Göllner, Ulrike Creß · 2017 · Scientometrics
  29. The role of peer review in Norwegian quality assurance: potential consequences for excellence and diversity
    Liv Langfeldt, Bjørn Stensaker, Lee Harvey · 2009 · Higher Education
  30. Of Asteroids and Dinosaurs: The Role of the Press in the Shaping of Scientific Debate
    Elisabeth S. Clemens · 1986 · Social Studies of Science
  31. A Kuhnian Critique of Psychometric Research on Peer Review
    Carole J. Lee · 2012 · Philosophy of Science
  32. Scientific research evaluation: a review of methods and various contexts of their application
    Terttu Luukkonen-Gronow · 1987 · R and D Management
  33. Menage a Quoi? Optimal Number of Peer Reviewers
    Richard R. Snell · 2015 · PLoS ONE
  34. Engaged, Embedded, Enjoined: Science and Technology Studies in the National Science Foundation
    Edward J. Hackett, Diana R. Rhoten · 2011 · Science and Engineering Ethics
  35. Perhaps it was right to reject the resubmitted manuscripts
    Garth J. Thomas · 1982 · Behavioral and Brain Sciences
  36. Peer reviewing: Improve or be rejected
    Michael J. A. Howe · 1982 · Behavioral and Brain Sciences
  37. How do journals of different rank instruct peer reviewers? Reviewer guidelines in the field of management
    Marco Seeber · 2020 · Scientometrics
  38. Research Policy: Problems with peer review and alternatives
    Judith Smith · 1988 · BMJ
  39. When will the editors start to edit?
    Leonard D. Goodstein · 1982 · Behavioral and Brain Sciences
  40. WILLFUL BLINDNESS: THE INEFFICIENT REWARD STRUCTURE IN ACADEMIC RESEARCH
    Stan J. Liebowitz · 2013 · Economic Inquiry
  41. Does the need for agreement among reviewers inhibit the publication controversial findings?
    J. Scott Armstrong, Raymond Hubbard · 1991 · Behavioral and Brain Sciences
  42. CO-CITATION BIBLIOMETRIC MODELING AS A TOOL FOR S&T POLICY AND R&D MANAGEMENT: ISSUES, APPLICATIONS, AND DEVELOPMENTS
    J.J. Franklin, Ron Johnston · 1988 · Elsevier eBooks
  43. The Vanishing Physician Scientist: A Critical Review and Analysis
    Gordon, Richard · 2012 · Accountability in Research
  44. What We Still Don't Know About Peer Review
    Omar Sabaj, Carlos González Vergara, Álvaro Piña-Stranger · 2016 · Journal of Scholarly Publishing
  45. Toward predicting research proposal success
    Kevin W. Boyack, Caleb Smith, Richard Klavans · 2017 · Scientometrics
  46. Peer review at the Deutsche Forschungsgemeinschaft
    Inhalt Hartmann, Friedhelm Neidhardt · 1990 · Scientometrics
  47. Technological communities and the diffusion of knowledge: a replication and validation
    Koenraad Debackere, Michael A. Rappa · 1994 · R and D Management
  48. Theoretical implications of failure to detect prepublished submissions
    Douglas Lee Eckberg · 1982 · Behavioral and Brain Sciences
  49. Exploring the determinants of research performance for early-career researchers: a literature review
    Danielle Hyunsook Lee · 2023 · Scientometrics
  50. Economics of science: survey and suggestions
    Esther‐Mirjam Sent · 1999 · Journal of Economic Methodology
  51. Rejection, rebuttal, revision: Some flexible features of peer review
    Donald B. Rubin · 1982 · Behavioral and Brain Sciences
  52. Financial Development and Shadow Economy: A Bibliometric Analysis Using the Scopus Database (1985–2021)
    Sami Ur Rahman, Faisal Faisal, Adnan Ali · 2022 · Journal of the Knowledge Economy
  53. Peer Review, Program Officers and Science Funding
    Paul J. Roebber, David M. Schultz · 2011 · PLoS ONE
  54. Measuring bias, burden and conservatism in research funding processes
    Susan Guthrie, Daniela Rodriguez-Rincon, Gordon R. McInroy, Becky Ioppolo, Salil Gunashekar · 2019 · F1000Research
  55. Evaluation and Decision Making in Higher Education
    Julie Renee Posselt, Theresa E. Hernandez, Cynthia Diana Villarreal, Aireale J. Rodgers, Lauren N. Irwin · 2019 · Higher education
  56. Frame Search and Re-search: How Quantitative Sociological Articles Change During Peer Review
    Misha Teplitskiy · 2015 · The American Sociologist
  57. Credibility and Relevance in Environmental Policy: Measuring Strategies and Performance among Science Assessment Organizations
    Ann Campbell Keller · 2009 · Journal of Public Administration Research and Theory
  58. Bibliometric analysis for science policy: An evaluation of the United Kingdom's research performance in ocean currents and protein crystallography
    David Crouch, J. Irvine, Ben R. Martin · 1986 · Scientometrics
  59. Are peer reviews of grant proposals reliable? An analysis of Economic and Social Research Council (ESRC) funding applications
    John Jerrim, R. D. de Vries · 2020 · The Social science journal (Fort Collins)
  60. Factors Predicting the Scientific Wealth of Nations
    Jüri Allïk, Kalmer Lauk, Anu Realo · 2020 · Cross-Cultural Research
  61. Interdisciplinarity can aid the spread of better methods between scientific communities
    Paul E. Smaldino, Cailin O’Connor · 2022 · Collective Intelligence
  62. Folk judgments of causation
    Joshua Knobe · 2009 · Studies in History and Philosophy of Science Part A
  63. Basic philosophy and concepts underlying scientific peer review
    W.E. Stehbens · 1999 · Medical Hypotheses
  64. Science peer review for the 21st century: Assessing scientific consensus for decision-making while managing conflict of interests, reviewer and process bias
    Christopher R. Kirman, Tage Simon, Sean M. Hays · 2019 · Regulatory Toxicology and Pharmacology
  65. Risk evaluation in peer review of grant applications
    Stephen A. Gallo, LISA A. THOMPSON, Karen B. Schmaling, Scott R. Glisson · 2018 · Environment Systems & Decisions
  66. Using simplified peer review processes to fund research: a prospective study
    Danielle L. Herbert, Nicholas Graves, Philip Clarke, Adrian Gerard Barnett · 2015 · BMJ Open
  67. Is Peer Review Censorship?
    Arturo Casadevall, Ferric C. Fang · 2009 · Infection and Immunity
  68. An intervention to improve the reliability of manuscript reviews for the Journal of the American Academy of Child and Adolescent Psychiatry
    1993 · American Journal of Psychiatry
  69. Rethinking the Funding Line at the Swiss National Science Foundation: Bayesian Ranking and Lottery
    Rachel Heyard, Manuela Ott, Georgia Salanti, Matthias Egger · 2022 · Statistics and Public Policy
  70. Peer review in funding-by-lottery: A systematic overview and expansion
    Jamie Shaw · 2022 · Research Evaluation
  71. How peer-review constrains cognition: on the frontline in the knowledge sector
    Stephen J. Cowley · 2015 · Frontiers in Psychology
  72. What should be rewarded? Gender and evaluation criteria for tenure and promotion
    Laura Cruz Castro, Luís Sanz Menéndez · 2021 · Journal of Informetrics
  73. Peer review in an Era of Evaluation
    Eva Forsberg, Lars Geschwind, Sara Levander, Wieland Wermke · 2022 · Springer eBooks
  74. When Zero May Not Be Zero: A Cautionary Note on the Use of Inter-Rater Reliability in Evaluating Grant Peer Review
    Erosheva, Elena A., Martinková, Patrícia, Lee, Carole J. · 2021 · Journal of the Royal Statistical Society Series A: Statistics in Society
  75. ‘Are you siding with a personality or the grant proposal?’: observations on how peer review panels function
    John Coveney, Danielle L. Herbert, Kathy Hill, Karen E Mow, Nicholas Graves, Adrian Gerard Barnett · 2017 · Research Integrity and Peer Review
  76. Concentration of Danish research funding on individual researchers and research topics: Patterns and potential drivers
    Emil Bargmann Madsen, Kaare Aagaard · 2020 · Quantitative Science Studies
  77. Analyzing sentiments in peer review reports: Evidence from two science funding agencies
    Junwen Luo, Thomas Feliciani, Martin Reinhart, Judith Hartstein, Vineeth Das, Olalere Alabi, Kalpana Shankar · 2021 · Quantitative Science Studies
  78. Student Engagement with Peer Assessment: A Review of Pedagogical Design and Technologies
    Karen Stepanyan, Richard Mather, Hamilton Jones, Carlo Lusuardi · 2009 · Lecture notes in computer science
  79. Experiments in Open Innovation at Harvard Medical School: What happens when an elite academic institution starts to rethink how research gets done?
    Eva C. Guinan, Kevin J. Boudreau, Karim R. Lakhani · 2013 · PubMed
  80. Peer-review research: Objections and obligations
    Douglas P. Peters, Stephen J. Ceci · 1982 · Behavioral and Brain Sciences
  81. Objectivity of the peer‐review process: Enduring myth, reality, and possible remedies
    Philippe C. Baveye · 2021 · Learned Publishing
  82. A retrospective analysis of the peer review of more than 75,000 Marie Curie proposals between 2007 and 2018
    David G. Pina, Ivan Buljan, Darko Hren, Ana Marušić · 2021 · eLife
  83. Peer review weighed in the balance.
    Stephen Lock · 1982 · BMJ
  84. Improving Scientists’ Judgments of Risk
    Kenneth R. Hammond, Barry F. Anderson, Jeffrey Sutherland, Barbara Marvin · 1985 · Springer eBooks
  85. Measuring metrics ‐ a 40‐year longitudinal cross‐validation of citations, downloads, and peer review in astrophysics
    Michael J. Kurtz, Edwin A. Henneken · 2016 · Journal of the Association for Information Science and Technology
  86. The relative success of private funders and government funders in funding important science
    Arthur M. Diamond · 2006 · European Journal of Law and Economics
  87. The predictive power of NSF reviewers and panels
    Samuel M. Scheiner, Lynette M. Bouchie · 2013 · Frontiers in Ecology and the Environment
  88. Publication and Rejection among Successful Ecologists
    Phillip Cassey, Tim M. Blackburn · 2004 · BioScience
  89. Drittmittel und Publikationen
    Richard Münch · 2006 · Soziologie
  90. Chance, concurrence, and clustering. Analysis of reviewers' recommendations on 1,000 submissions to the Journal of Clinical Investigation.
    Bruce F. Scharschmidt, A DeAmicis, Peter Bacchetti · 1994 · Journal of Clinical Investigation
  91. The principles and practices of peer review
    Ronald Neil Kostoff · 1997 · Science and Engineering Ethics
  92. Peer review reliability: The hierarchy of the sciences
    Charles H.G. Crothers · 1993 · Behavioral and Brain Sciences
  93. Insiders, outsiders, and efficiency in a National Science Foundation panel.
    David Klahr · 1985 · American Psychologist
  94. Correlates of Departmental Quality in Regional Colleges and Universities
    Clifton F. Conrad, Robert T. Blackburn · 1985 · American Educational Research Journal
  95. Understanding the selection processes of public research projects in agriculture: The role of scientific merit
    Valentina Cristiana Materia, Stefano Pascucci, Christos Kolympiris · 2015 · Food Policy
  96. Feyerabend’s well-ordered science: how an anarchist distributes funds
    Jamie Shaw · 2018 · Synthese
  97. Accounting Research: A Critical View Of The Present Situation And Prospects
    Josep María Argilés-Bosch, Josep García-Blandón · 2011 · Revista de Contabilidad
  98. Towards a doctoral thesis through published works
    Lars H. Breimer, Dimitri P. Mikhailidis · 1993 · Biomedicine & Pharmacotherapy
  99. The predictive validity of peer review: A neglected issue
    Robert F. Bornstein · 1991 · Behavioral and Brain Sciences
  100. Time substitution and network effects with an application to nanobiotechnology policy for US universities
    Hirofumi Fukuyama, William Weber, Yin Xia · 2015 · Omega