The policy challenges of peer review: managing bias, conflict of interests and interdisciplinary assessments

Langfeldt, Liv

doi:10.3152/147154406781776039

Cited by 180Publishing and Scholarly Communicationscientometrics and bibliometrics researchResearch, Science, and Academia

The policy challenges of peer review: managing bias, conflict of interests and interdisciplinary assessments

Langfeldt, Liv·2006·Research Evaluation

DOI 10.3152/147154406781776039

Open this paper in CoChat — free account

Abstract

Characteristics and challenges of peer review are elucidated from three different perspectives: a social dynamics perspective defining peer review as an important control mechanism in the research community; an uncertainty perspective focusing on the inherent uncertainty in judging research quality: and an organisational perspective focusing on the effects of different ways of organising peer review. Findings from a broad set of empirical studies are used. Peer review often has some conservative and risk-minimising aspects, which may disfavour interdisciplinary and non-conventional research. When the aim is to actively promote interdisciplinary or other kinds of non-conventional research, the involved peer-review system consequently needs to be adjusted to a more risk-taking mode.

Integrity signals

Registry

Crossref · 10.3152/147154406781776039

Cites

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Reviewers weigh criteria such as originality, feasibility, investigator track record, and potential impact differently, even when operating under shared guidelines.
“Reviewers weigh criteria such as originality, feasibility, investigator track record, and potential impact differently, even when operating under shared guidelines (Langfeldt, 2006; Fogelholm et al., 2012).”
From Can Large Language Models Evaluate Grant Proposal Quality? Revisiting the Wennerås and Wold Peer Review Data (2026) · cites this paper as Langfeldt, 2006
Low inter-reviewer agreement is not primarily a consequence of reviewer incompetence but reflects the inherently multidimensional and judgement-based nature of grant assessment.
“More recent evidence suggests that low inter-reviewer agreement is not primarily a consequence of reviewer incompetence but rather reflects the inherently multidimensional and judgement-based nature of grant assessment.”
From Can Large Language Models Evaluate Grant Proposal Quality? Revisiting the Wennerås and Wold Peer Review Data (2026) · cites this paper as Langfeldt, 2006

Still being checked.

Sorted by citations

Evaluation of Interdisciplinary and Transdisciplinary Research
Klein, J. T. (2008) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2008.05.010 · Cited by 692 · see citing works
Interdisciplinary research has consistently lower funding success
Bromham, L. D., Dinnage, R., & Hua, X. (2016) · Nature · doi:10.1038/nature18315 · Cited by 602 · see citing works
How journal rankings can suppress interdisciplinary research: A comparison between Innovation Studies and Business & Management
Ràfols, I., Loet Leydesdorff, A., O’Hare, A., Nightingale, P., & Stirling, A. (2012) · Research Policy · doi:10.1016/j.respol.2012.03.015 · Cited by 581 · see citing works
Looking Across and Looking Beyond the Knowledge Frontier: Intellectual Distance and Resource Allocation in Science
Boudreau, K. J., Guinan, E. C., Lakhani, K. R., & Riedl, C. (2014) · Management Science · doi:10.1287/mnsc.2015.2285 · Cited by 489 · see citing works
‘Interdisciplinary strategies’ in U.S. research universities
Sá, C. M. (2007) · Higher Education · doi:10.1007/s10734-007-9073-5 · Cited by 218 · see citing works
Conservatism and risk-taking in peer review: Emerging ERC practices
Luukkonen, T. (2012) · Research Evaluation · doi:10.1093/reseval/rvs001 · Cited by 185 · see citing works
Collaboration, interdisciplinarity, and the epistemology of contemporary science
Andersen, H. M. (2015) · Studies in History and Philosophy of Science Part A · doi:10.1016/j.shpsa.2015.10.006 · Cited by 181 · see citing works
The kaleidoscope of disciplinarity
Sugimoto, C. R., & Weingart, S. (2015) · Journal of Documentation · doi:10.1108/jd-06-2014-0082 · Cited by 125 · see citing works
What do we know about grant peer review in the health sciences?
Guthrie, S., Ghiga, I., & Wooding, S. (2018) · F1000Research · doi:10.12688/f1000research.11917.2 · Cited by 102 · see citing works
Societal impact evaluation: Exploring evaluator perceptions of the characterization of impact under the REF2014: Table 1.
Samuel, G. N., & Derrick, G. (2015) · Research Evaluation · doi:10.1093/reseval/rvv007 · Cited by 99 · see citing works
Peer Review or Lottery? A Critical Analysis of Two Different Forms of Decision-making Mechanisms for Allocation of Research Grants
Roumbanis, L. (2019) · Science Technology & Human Values · doi:10.1177/0162243918822744 · Cited by 79 · see citing works
Research excellence in Africa: Policies, perceptions, and performance
Tijssen, R., & Kraemer‐Mbula, E. (2017) · Science and Public Policy · doi:10.1093/scipol/scx074 · Cited by 78 · see citing works
Communicating and compromising on disciplinary expertise in the peer review of research proposals
Huutoniemi, K. I. (2012) · Social Studies of Science · doi:10.1177/0306312712458478 · Cited by 73 · see citing works
The Pros and Cons of the Use of Altmetrics in Research Assessment
Thelwall, M. A. (2020) · Scholarly Assessment Reports · doi:10.29024/sar.10 · Cited by 68 · see citing works
What do we know about grant peer review in the health sciences?
Guthrie, S., Ghiga, I., & Wooding, S. (2017) · F1000Research · doi:10.12688/f1000research.11917.1 · Cited by 69 · see citing works
Overcoming early career barriers to interdisciplinary climate change research
Hein, C. J., Hoeve, J. E. T., Gopalakrishnan, S., Livneh, B., Adams, H. D., Marino, E. K., & Weiler, C. S. (2018) · Wiley Interdisciplinary Reviews Climate Change · doi:10.1002/wcc.530 · Cited by 63 · see citing works
Genetic frontiers for conservation: an assessment of synthetic biology and biodiversity conservation: technical assessment
Alphey, L., Brooks, T. M., Delborne, J., & Adams, J. S. (2019) · doi:10.2305/iucn.ch.2019.05.en · Cited by 62 · see citing works
Academic judgments under uncertainty: A study of collective anchoring effects in Swedish Research Council panel groups
Roumbanis, L. (2016) · Social Studies of Science · doi:10.1177/0306312716659789 · Cited by 59 · see citing works
The predictive validity of peer review: A selective review of the judgmental forecasting qualities of peers, and implications for innovation in science
Benda, W. G. G., & Engels, T. (2010) · International Journal of Forecasting · doi:10.1016/j.ijforecast.2010.03.003 · Cited by 58 · see citing works
The Evaluation Scale: Exploring Decisions About Societal Impact in Peer Review Panels
Derrick, G., & Samuel, G. N. (2016) · Minerva · doi:10.1007/s11024-016-9290-0 · Cited by 59 · see citing works
Scholarly motivations to conduct interdisciplinary climate change research
Milman, A., Marston, J. M., Godsey, S. E., Bolson, J., Jones, H. P., & Weiler, C. S. (2015) · Journal of Environmental Studies and Sciences · doi:10.1007/s13412-015-0307-z · Cited by 53 · see citing works
Tourism research quality: Reviewing and assessing interdisciplinarity
Oviedo‐García, M. Á. (2015) · Tourism Management · doi:10.1016/j.tourman.2015.08.011 · Cited by 48 · see citing works
Peer review for the evaluation of academic research: lessons from the Italian experience
Reale, E., Anna, B., & Costantini, A. (2007) · Research Evaluation · doi:10.3152/095820207x227501 · Cited by 48 · see citing works
Starting to Publish Academic Research as a Doctoral Student
Stoilescu, D., & McDougall, D. E. (2010) · International journal of doctoral studies · doi:10.28945/1333 · Cited by 45 · see citing works
How to improve the use of metrics
Frey, B. S., & Osterloh, M. (2010) · Nature · doi:10.1038/465870a · Cited by 45 · see citing works
Solving multiple-criteria R&D project selection problems with a data-driven evidential reasoning rule
Liu, F., Chen, Y., Yang, J., Xu, D., & Liu, W. (2018) · International Journal of Project Management · doi:10.1016/j.ijproman.2018.10.006 · Cited by 43 · see citing works
Evaluation in research funding agencies: Are structurally diverse teams biased against?
Banal‐Estañol, A., Macho‐Stadler, I., & Pérez‐Castrillo, D. (2019) · Research Policy · doi:10.1016/j.respol.2019.04.008 · Cited by 45 · see citing works
Menage a Quoi? Optimal Number of Peer Reviewers
Snell, R. R. (2015) · PLoS ONE · doi:10.1371/journal.pone.0120838 · Cited by 41 · see citing works
Inconsistencies in the Governance of Interdisciplinarity: the Case of the Italian Higher Education System
Donina, D., Seeber, M., & Paleari, S. (2017) · Science and Public Policy · doi:10.1093/scipol/scx019 · Cited by 41 · see citing works
How do journals of different rank instruct peer reviewers? Reviewer guidelines in the field of management
Seeber, M. (2020) · Scientometrics · doi:10.1007/s11192-019-03343-1 · Cited by 39 · see citing works
Assessing the impact of JIBS as an interdisciplinary journal: A network approach
Cantwell, J., Piepenbrink, A., & Shukla, P. (2014) · Journal of International Business Studies · doi:10.1057/jibs.2014.40 · Cited by 39 · see citing works
From a Disciplinary to an Interdisciplinary Design Research: Developing an Integrative Approach for Design
Chou, W. H., & Wong, J. (2015) · International Journal of Art & Design Education · doi:10.1111/jade.12017 · Cited by 31 · see citing works
Capabilities for transdisciplinary research
O’Donovan, C., Michalec, O., & Moon, J. (2021) · Research Evaluation · doi:10.1093/reseval/rvab038 · Cited by 31 · see citing works
Defining the role of cognitive distance in the peer review process with an explorative study of a grant scheme in infection biology
Wang, Q., & Sandström, U. (2015) · Research Evaluation · doi:10.1093/reseval/rvv009 · Cited by 31 · see citing works
Peer review assessment of originality in tourism journals: critical perspective of key gatekeepers
Rodríguez, I., Makkonen, T., & Williams, A. M. (2019) · Annals of Tourism Research · doi:10.1016/j.annals.2019.04.003 · Cited by 30 · see citing works
Peer review in funding-by-lottery: A systematic overview and expansion
Shaw, J. (2022) · Research Evaluation · doi:10.1093/reseval/rvac022 · Cited by 30 · see citing works
Disagreement and Agonistic Chance in Peer Review
Roumbanis, L. (2022) · Science, Technology, & Human Values · doi:10.1177/01622439211026016 · Cited by 29 · see citing works
Evaluation for the allocation of university research project funding: Can rules improve the peer review?
Reale, E., & Zinilli, A. (2017) · Research Evaluation · doi:10.1093/reseval/rvx019 · Cited by 29 · see citing works
Conditions that do or do not disadvantage interdisciplinary research proposals in project evaluation
Seeber, M., Vlegels, J., & Cattaneo, M. (2022) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.24617 · Cited by 24 · see citing works
Assessing the quality of feedback in the peer-review process
Dobele, A. R. (2015) · Higher Education Research & Development · doi:10.1080/07294360.2015.1011086 · Cited by 23 · see citing works
Incorporation of Early Career Researchers within multidisciplinary research at academic institutions
Sobey, A. J., Townsend, N., Metcalf, C., Bruce, K. D., & Fazi, F. M. (2013) · Research Evaluation · doi:10.1093/reseval/rvt004 · Cited by 23 · see citing works
The impact a-gender: gendered orientations towards research Impact and its evaluation
Chubb, J., & Derrick, G. (2020) · Humanities and Social Sciences Communications · doi:10.1057/s41599-020-0438-z · Cited by 23 · see citing works
Expert panel reviews of research centers: The site visit process
Lawrenz, F. P., Thao, M., & Johnson, K. (2012) · Evaluation and Program Planning · doi:10.1016/j.evalprogplan.2012.01.003 · Cited by 22 · see citing works
Selbststeuerung der Wissenschaft: Peer Review
Neidhardt, F. (2010) · VS Verlag für Sozialwissenschaften eBooks · doi:10.1007/978-3-531-91993-5_19 · Cited by 22 · see citing works
How Competition for Funding Impacts Scientific Practice: Building Pre-fab Houses but no Cathedrals
Meirmans, S. (2024) · Science and Engineering Ethics · doi:10.1007/s11948-024-00465-5 · Cited by 21 · see citing works
Making the cut: How panel reviewers use evaluation devices to select applications at the European Research Council
Brunet, L., & Müller, R. (2022) · Research Evaluation · doi:10.1093/reseval/rvac040 · Cited by 22 · see citing works
Multi/inter/transdisciplinary assessment: A systemic framework proposal to evaluate graduate courses and research teams
de Oliveira, T. M., Amaral, L., & Pacheco, R. C. D. S. (2018) · Research Evaluation · doi:10.1093/reseval/rvy013 · Cited by 21 · see citing works
Agricultural data management and sharing: Best practices and case study
Moore, E. K., Kriesberg, A., Schroeder, S., Geil, K. L., Haugen, I., Barford, C. C., Johns, E., Arthur, D. K., Sheffield, M., Ritchie, S., et al. (2021) · Agronomy Journal · doi:10.1002/agj2.20639 · Cited by 20 · see citing works
How to identify peer-reviewed publications: Open-identity labels in scholarly book publishing
Kulczycki, E., Rozkosz, E. A., Engels, T., Guns, R., Hołowiecki, M., & Pölönen, J. (2019) · PLoS ONE · doi:10.1371/journal.pone.0214423 · Cited by 19 · see citing works
Editorial delay of food research papers is influenced by authors’ experience but not by country of origin of the manuscripts
Yegros, A., & Amat, C. B. (2009) · Scientometrics · doi:10.1007/s11192-008-2164-y · Cited by 18 · see citing works
Mapping the frontiers of evaluation of public-sector R&D programs
Feller, I. (2007) · Science and Public Policy · doi:10.3152/030234207x258996 · Cited by 18 · see citing works
Reviewer training for improving grant and journal peer review
Hesselberg, J.-O., Dalsbø, T. K., Strømme, H., Svege, I. C., & Fretheim, A. (2023) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.mr000056.pub2 · Cited by 17 · see citing works
The Novelty Paradox & Bias for Normal Science: Evidence from Randomized Medical Grant Proposal EvaluationsPreprint
Boudreau, K. J., Guinan, E. C., Lakhani, K. R., & Riedl, C. (2012) · SSRN Electronic Journal · doi:10.2139/ssrn.2184791 · Cited by 17 · see citing works
Interdisciplinary knowledge combinations and emerging technological topics: Implications for reducing uncertainties in research evaluation
Kwon, S., Youtie, J., & Porter, A. L. (2020) · Research Evaluation · doi:10.1093/reseval/rvaa029 · Cited by 16 · see citing works
Exploring the antecedents of interdisciplinarity at the European Research Council: a topic modeling approach
Bonaccorsi, A. J., Melluso, N., & Massucci, F. A. (2022) · Scientometrics · doi:10.1007/s11192-022-04368-9 · Cited by 17 · see citing works
Peer Review and Bibliometric: Potentials and Problems
Bornmann, L. (2011) · Springer eBooks · doi:10.1007/978-94-007-1116-7_8 · Cited by 16 · see citing works
Looking Across and Looking Beyond the Knowledge Frontier: Intellectual Distance and Resource Allocation in SciencePreprint
Boudreau, K. J., Guinan, E. C., Lakhani, K. R., & Riedl, C. (2014) · SSRN Electronic Journal · doi:10.2139/ssrn.2478627 · Cited by 16 · see citing works
Preserving the Quality of Scientific Research: Peer Review of Research Articles
Silva, P. U. K. D., & Vance, C. K. (2017) · Fascinating life sciences · doi:10.1007/978-3-319-50627-2_6 · Cited by 15 · see citing works
Tracking the evolution of waste recycling research using overlay maps of science
Garechana, G., Belver, R. M. R., Carrasco, E. C., & Gavilanes, J. (2012) · Waste Management · doi:10.1016/j.wasman.2012.01.017 · Cited by 15 · see citing works
Ejes de discusión en la evaluación de la ciencia: revisión por pares, bibliometría y pertinencia
Sarthou, N. F. (2016) · Revista de Estudios Sociales · doi:10.7440/res58.2016.06 · Cited by 15 · see citing works
Judicious Use of Bibliometrics to Supplement Peer Evaluations of Research in Kinesiology
Knudson, D. V. (2019) · Kinesiology Review · doi:10.1123/kr.2017-0046 · Cited by 17 · see citing works
Evaluation of research proposals by peer review panels: broader panels for broader assessments?
Abma-Schouten, R., Gijbels, J., Reijmerink, W., & Meijer, I. (2023) · Science and Public Policy · doi:10.1093/scipol/scad009 · Cited by 14 · see citing works
Evaluations by Peer Review in Science
Bornmann, L. (2013) · Springer Science Reviews · doi:10.1007/s40362-012-0002-3 · Cited by 14 · see citing works
The Responsible Conduct of Bioethics Research
Master, Z. (2011) · Accountability in Research · doi:10.1080/08989621.2011.557302 · Cited by 13 · see citing works
Responses to rejection: The experiences of six women early career researchers in the Education department of an English university
Mercer, J. (2013) · Women s Studies International Forum · doi:10.1016/j.wsif.2013.03.008 · Cited by 13 · see citing works
Effects and Interactions of Researcher’s Motivation and Personality in Promoting Interdisciplinary and Transdisciplinary Research
Katoh, S., Aalbers, R., & Sengoku, S. (2021) · Sustainability · doi:10.3390/su132212502 · Cited by 26 · see citing works
How to measure interdisciplinary research? A systemic design for the model of measurement
Cantone, G. G. (2024) · Scientometrics · doi:10.1007/s11192-024-05085-1 · Cited by 15 · see citing works
German Professors’ Motivation to Act as Peer Reviewers in Accreditation and Evaluation Procedures
Ohly, S., & Schneijderberg, C. (2021) · Minerva · doi:10.1007/s11024-020-09430-5 · Cited by 12 · see citing works
Peer reviewers’ dilemmas: a qualitative exploration of decisional conflict in the evaluation of grant applications in the medical humanities and social sciences
Vallée-Tourangeau, G., Wheelock, A., Vandrevala, T., & Harries, P. (2022) · Humanities and Social Sciences Communications · doi:10.1057/s41599-022-01050-6 · Cited by 12 · see citing works
Understanding relationship between topic selection and academic performance of scientific teams based on entity popularity trend
Zhang, T., Tan, F., Yu, C., Wu, J., & Xu, J. (2022) · Aslib Journal of Information Management · doi:10.1108/ajim-03-2022-0135 · Cited by 12 · see citing works
The Program Evaluation Standards Applied for Metaevaluation Purposes: Investigating Interrater Reliability and Implications for Use.
Wingate, L. A. (2009) · ProQuest LLC eBooks · Cited by 11 · see citing works
Cognitive Bias in Peer Review : a New Approach
Sandström, U. (2009) · Cited by 10 · see citing works
A systematic review on patient perceptions and clinician‐reported outcomes when comparing digital and analog workflows for complete dentures
Fouda, A., Tonogai, J., McDermott, P., Wang, D. X., & Dong, C. C. S. (2024) · Journal of Prosthodontics · doi:10.1111/jopr.13999 · Cited by 9 · see citing works
The effect of facilitating interdisciplinary cooperation on the research productivity of university research teams: The moderating role of government assistance
Ai-hua, C., & Wang, X. (2021) · Research Evaluation · doi:10.1093/reseval/rvab001 · Cited by 8 · see citing works
Let’s agree to agree: The situational academic quality of the UK REF as consensual public knowledge
Milyaeva, S., & Neyland, D. (2023) · Social Studies of Science · doi:10.1177/03063127231152915 · Cited by 10 · see citing works
Does the perceived quality of interdisciplinary research vary between fields?
Thelwall, M. A., Kousha, K., Stuart, E., Makita, M., Abdoli, M., Wilson, P., & Levitt, J. M. (2023) · Journal of Documentation · doi:10.1108/jd-01-2023-0012 · Cited by 9 · see citing works
Evaluation of Research in Context A Quick Scan of an Emerging Field
Merkx, F., van der Weijden, I., Oostveen, A., van den Besselaar, P., & Spaapen, J. B. (2007) · Cited by 9 · see citing works
Selbststeuerung der Wissenschaft durch Peer-Review-Verfahren
Neidhardt, F. (2016) · Springer eBooks · doi:10.1007/978-3-658-05455-7_22 · Cited by 9 · see citing works
Reviewer training for improving grant and journal peer review
Hesselberg, J., Dalsbø, T. K., Strømme, H., Svege, I. C., & Fretheim, A. (2020) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.mr000056 · Cited by 9 · see citing works
Expert elicitation on the uncertainties associated with chronic wasting disease
Tyshenko, M. G., Oraby, T. F., Darshan, S., Westphal, M., Croteau, M. C., Aspinall, W. P., ElSaadany, S., Krewski, D. R., & Cashman, N. R. (2016) · Journal of Toxicology and Environmental Health · doi:10.1080/15287394.2016.1174007 · Cited by 9 · see citing works
Interdisciplinary Accountability in the Evaluation of Research Proposals Prospects for academic quality control across disciplinary boundaries
Huutoniemi, K. I. (2012) · Työväentutkimus Vuosikirja · Cited by 9 · see citing works
A Nordic evaluation tradition? A look at the peer-reviewed evaluation literature
Nielsen, S. B., & Winther, D. M. (2014) · Evaluation · doi:10.1177/1356389014540703 · Cited by 8 · see citing works
Research cultures as an explanatory factor
Gläser, J., Bielick, J., Jungmann, R., Laudel, G., Lettkemann, E., Petschick, G., & Tschida, U. (2015) · Österreichische Zeitschrift für Soziologie · doi:10.1007/s11614-015-0177-3 · Cited by 8 · see citing works
The Power of Peer Review on Transdisciplinary Discovery
Mäkinen, E. I. (2019) · Science Technology & Human Values · doi:10.1177/0162243918822741 · Cited by 8 · see citing works
Selecting competent referees to assess research projects proposals: A study of referees' registers
Abramo, G., D’Angelo, C. A., & Viel, F. V. (2012) · Research Evaluation · doi:10.1093/reseval/rvs031 · Cited by 7 · see citing works
Research Programmes That Promote Novel, Ambitious, Unconventional and High-Risk Research: An Analysis
Prendergast, P. J., Brown, S., & Britton, J. R. (2008) · Industry and Higher Education · doi:10.5367/000000008785201793 · Cited by 7 · see citing works
Frequency and Type of Conflicts of Interest in the Peer Review of Basic Biomedical Research Funding Applications: Self-Reporting Versus Manual Detection
Gallo, S. A., LeMaster, M. P., & Glisson, S. R. (2015) · Science and Engineering Ethics · doi:10.1007/s11948-015-9631-7 · Cited by 7 · see citing works
Evaluation of Research(ers) and its Threat to Epistemic Pluralisms
Viola, M. (2018) · European journal of analytic philosophy · doi:10.31820/ejap.13.2.4 · Cited by 7 · see citing works
Efficiency and flexibility in research funding. A comparative study of funding instruments and review criteria.
Langfeldt, L., & Scordato, L. (2016) · BIBSYS Brage (BIBSYS (Norway)) · doi:10.13140/rg.2.1.4568.2165 · Cited by 7 · see citing works
Self-citation
Similar‐to‐me effects in the grant application process: Applicants, panellists, and the likelihood of obtaining funds
Banal‐Estañol, A., Liu, Q., Macho‐Stadler, I., & Pérez‐Castrillo, D. (2023) · R and D Management · doi:10.1111/radm.12601 · Cited by 6 · see citing works
Evaluating Transdisciplinary Research Quality
Borlaug, S. B., & Svartefoss, S. M. (2024) · SpringerBriefs in political science · doi:10.1007/978-3-031-69580-3_3 · Cited by 6 · see citing works
Competitive funding and academic-industry collaboration: policy trends and insights
Behfar, S. K., Shekhtman, L., & Crowcroft, J. (2024) · Data & Policy · doi:10.1017/dap.2024.81 · Cited by 6 · see citing works
More Than Euros: Exploring the Construction of Project Grants as Prizes and Consolations
Edlund, P. (2023) · Minerva · doi:10.1007/s11024-023-09501-3 · Cited by 4 · see citing works
Exploring the degree of delegated authority for the peer review of societal impact
Derrick, G., & Samuel, G. N. (2017) · Science and Public Policy · doi:10.1093/scipol/scx091 · Cited by 6 · see citing works
Neue Evaluationsregime? Von der Inquisition zur Evaluation
Hornbostel, S. (2008) · VS Verlag für Sozialwissenschaften eBooks · doi:10.1007/978-3-531-90863-2_5 · Cited by 6 · see citing works
The Long Shadow of Peer Review on the Preparation of Artist Educators
Walker, L. N. (2013) · Arts Education Policy Review · doi:10.1080/10632913.2013.826058 · Cited by 6 · see citing works
Reviewing applications by women - critical use of additive and reasoning evaluation methodsPreprint
Melin, G. (2007) · RePEc: Research Papers in Economics · Cited by 6 · see citing works
The European Project OpenUP: OPENing UP New Methods, Indicators and Tools for Peer Review, Impact Measurement and Dissemination of Research Results
Bardi, A., Casarosa, V., & Manghi, P. (2017) · Communications in computer and information science · doi:10.1007/978-3-319-73165-0_24 · Cited by 5 · see citing works
Selbststeuerung der Wissenschaft durch Peer-Review-Verfahren
Neidhardt, F. (2015) · Springer eBooks · doi:10.1007/978-3-658-05677-3_22-1 · Cited by 5 · see citing works
Smart Policies for Societies in Transition
Benner, M., Marklund, G., & Serger, S. S. (2022) · Edward Elgar Publishing eBooks · doi:10.4337/9781788970815 · Cited by 5 · see citing works

Possibilities for the peer review of interdisciplinary research: a small-scale qualitative study
Vesma, G. (2026) · Research Integrity and Peer Review · doi:10.1186/s41073-026-00204-3
Conclave in the Tower of Babel: how peers review interdisciplinary research proposals
Laudel, G. (2006) · Research Evaluation · doi:10.3152/147154406781776048 · Cited by 70 · see citing works
Improving the Peer Review Process: An Examination of Commonalities between Scholarly Societies and Knowledge Networks
Nousala, S. (2010) · RMIT Research Repository (RMIT University Library)
Evaluation of multi- and inter-disciplinary research – the no-peer problem
Elliott, J. A. W. (2011) · Cryobiology · doi:10.1016/j.cryobiol.2011.01.016 · Cited by 2 · see citing works
The Loss of Innovation: Peer Review in Multi- and Interdisciplinary Research
Perper, T., Bailis, S., Gottlieb, S. E., & Klein, J. T. (1989) · Cited by 5 · see citing works

Authors: Langfeldt, L.
Corresponding author: Langfeldt, Liv
ORCID: Langfeldt, Liv
Venue: Research Evaluation
Publisher: Oxford University Press (OUP)
Published: 2006-04-01
Volume: 15
Issue: 1
Pages: 31-41
ISSN: 0958-2029 (ISSN-L)
1471-5449
DOI: 10.3152/147154406781776039
OpenAlex: W2078257123
Cited by: 179
References: 0
Citations by year: 2026: 11 · 2025: 6 · 2024: 18 · 2023: 7 · 2022: 12 · 2021: 6 · 2020: 13 · 2019: 9 · 2018: 11 · 2017: 9 · 2016: 12 · 2015: 14 · 2014: 4 · 2013: 7 · 2012: 12
FWCI: 29.01
Type: journal article
Language: en
Open access: closed
Locations: Research Evaluation (publishedVersion)
RePEc: Research Papers in Economics (submittedVersion)
Topics: Social Sciences › Arts and Humanities › History and Philosophy of Science › Publishing and Scholarly Communication
Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › scientometrics and bibliometrics research
Social Sciences › Decision Sciences › Information Systems and Management › Research, Science, and Academia
Keywords: peer review, research evaluation policy, peer review bias, conflicts of interest, interdisciplinary research, research quality assessment

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

The policy challenges of peer review: managing bias, conflict of interests and interdisciplinary assessments

Abstract

Integrity signals

Claims the paper makes

Claims other papers make about this one

Cites

Cited by

Related

Record

About these checks