The DOI matches its Crossref record; 112 of the 112 references it cites were found in a registry or an index.
How journal rankings can suppress interdisciplinary research: A comparison between Innovation Studies and Business & Management
Open accessAbstract
This study provides quantitative evidence on how the use of journal rankings can disadvantage interdisciplinary research in research evaluations. Using publication and citation data, it compares the degree of interdisciplinarity and the research performance of a number of Innovation Studies units with that of leading Business & Management Schools (BMS) in the UK. On the basis of various mappings and metrics, this study shows that: (i) Innovation Studies units are consistently more interdisciplinary in their research than Business & Management Schools; (ii) the top journals in the Association of Business Schools’ rankings span a less diverse set of disciplines than lower-ranked journals; (iii) this results in a more favourable assessment of the performance of Business & Management Schools, which are more disciplinary-focused. This citation-based analysis challenges the journal ranking-based assessment. In short, the investigation illustrates how ostensibly ‘excellence-based’ journal rankings exhibit a systematic bias in favour of mono-disciplinary research. The paper concludes with a discussion of implications of these phenomena, in particular how the bias is likely to affect negatively the evaluation and associated financial resourcing of interdisciplinary research organisations, and may result in researchers becoming more compliant with disciplinary authority over time.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Software survey: VOSviewer, a computer program for bibliometric mapping
van Eck, N. J., & Waltman, L. (2010) · Scientometrics · doi:10.1007/s11192-009-0146-3 · Cited by 21,484
A Set of Measures of Centrality Based on Betweenness
Freeman, L. C. (1977) · Sociometry · doi:10.2307/3033543 · Cited by 10,295
The New production of knowledge: the dynamics of science and research in contemporary societies
(1995) · Choice Reviews Online · doi:10.5860/choice.32-4463 · Cited by 7,046
What is research collaboration?
Katz, J. S., & Martin, B. R. (1997) · Research Policy · doi:10.1016/s0048-7333(96)00917-1 · Cited by 2,972
Why the impact factor of journals should not be used for evaluating research
Seglen, P. O. (1997) · BMJ · doi:10.1136/bmj.314.7079.497 · Cited by 2,447 · cited 3×
“We should stress that Impact Factor-based measures ave been convincingly shown to be a worse indicator of quality han citations (Seglen, 1997).”
BackgroundInfluentialThe difference: how the power of diversity creates better groups, firms, schools, and societies
(2007) · Choice Reviews Online · doi:10.5860/choice.45-1534 · Cited by 2,280
Exploratory Social Network Analysis with Pajek: Revised and Expanded Edition for Updated Software
De Nooy, W., Mrvar, A., & Batagelj, V. (2018) · Cambridge University Press eBooks · doi:10.1017/9781108565691 · Cited by 2,052
Uncertainty and Quality in Science for Policy
Funtowicz, S. O., & Ravetz, J. R. (1990) · doi:10.1007/978-94-009-0621-1 · Cited by 1,575
Stirling, A. (2008) · Science, Technology, & Human Values · doi:10.1177/0162243907311265 · Cited by 1,567
Self-citationExploratory Social Network Analysis with Pajek
de Nooy, W., Mrvar, A., & Batagelj, V. (2005) · doi:10.1017/cbo9780511806452 · Cited by 1,244
A general framework for analysing diversity in science, technology and society
Stirling, A. (2007) · Journal of The Royal Society Interface · doi:10.1098/rsif.2007.0213 · Cited by 1,151
Self-citationThe Oxford handbook of interdisciplinarity
(2011) · Choice Reviews Online · doi:10.5860/choice.48-2406 · Cited by 1,016
(2005) · Springer eBooks · doi:10.1007/1-4020-2755-9 · Cited by 979
Is science becoming more interdisciplinary? Measuring and mapping six research fields over time
Porter, A. L., & Rafols, I. (2009) · Scientometrics · doi:10.1007/s11192-008-2197-2 · Cited by 785
Self-citationWagner, C. S., Roessner, J. D., Bobb, K., Klein, J. T., Boyack, K. W., Keyton, J., Rafols, I., & Börner, K. (2011) · Journal of Informetrics · doi:10.1016/j.joi.2010.06.004 · Cited by 769
Self-citationCollaborative Research Across Disciplinary and Organizational Boundaries
Cummings, J. N., & Kiesler, S. (2005) · Social Studies of Science · doi:10.1177/0306312705055535 · Cited by 759
Stokols, D., Hall, K. L., Taylor, B. K., & Moser, R. P. (2008) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2008.05.002 · Cited by 753
Stirling, A. (2010) · Nature · doi:10.1038/4681029a · Cited by 675
Self-citationDiversity and network coherence as indicators of interdisciplinarity: case studies in bionanoscience
Rafols, I., & Meyer, M. (2010) · Scientometrics · doi:10.1007/s11192-009-0041-y · Cited by 656
Self-citationBetweenness centrality as an indicator of the interdisciplinarity of scientific journals
Leydesdorff, L. (2007) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.20614 · Cited by 612
Self-citationImpact of bibliometrics upon the science system: Inadvertent consequences?
Weingart, P. (2005) · Scientometrics · doi:10.1007/s11192-005-0007-7 · Cited by 602
Analyzing interdisciplinarity: Typology and indicators
Huutoniemi, K., Klein, J. T., Bruun, H., & Hukkinen, J. (2010) · Research Policy · doi:10.1016/j.respol.2009.09.011 · Cited by 596
Interdisciplinarity: A Critical Assessment
Jacobs, J. A., & Frickel, S. (2009) · Annual Review of Sociology · doi:10.1146/annurev-soc-070308-115954 · Cited by 596
Barry, A., Born, G., & Weszkalnys, G. (2008) · Economy and Society · doi:10.1080/03085140701760841 · Cited by 586
Innovation studies—The emerging structure of a new scientific field
Fagerberg, J., & Verspagen, B. (2009) · Research Policy · doi:10.1016/j.respol.2008.12.006 · Cited by 565
Martin, B. R., & Irvine, J. (1983) · Research Policy · doi:10.1016/0048-7333(83)90005-7 · Cited by 527
A global map of science based on the ISI subject categories
Leydesdorff, L., & Rafols, I. (2009) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.20967 · Cited by 506
Self-citationHas Management Studies Lost Its Way? Ideas for More Imaginative and Innovative Research
Alvesson, M., & Sandberg, J. (2013) · Journal of Management Studies · doi:10.1111/j.1467-6486.2012.01070.x · Cited by 503
Factors associated with disciplinary and interdisciplinary research collaboration
van Rijnsoever, F. J., & Hessels, L. K. (2011) · Research Policy · doi:10.1016/j.respol.2010.11.001 · Cited by 481
Innovation: Exploring the knowledge base
Fagerberg, J., Fosaas, M., & Sapprasert, K. (2012) · Research Policy · doi:10.1016/j.respol.2012.03.008 · Cited by 459
Journal list fetishism and the perversion of scholarship: reactivity and the ABS list
Willmott, H. (2011) · Organization · doi:10.1177/1350508411403532 · Cited by 453
Science overlay maps: A new tool for research policy and library management
Rafols, I., Porter, A. L., & Leydesdorff, L. (2010) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.21368 · Cited by 430
Self-citationComparison of Methods: Different Methods of Converting Vibrations to Electricity
Roundy, S., Wright, P. K., & Rabaey, J. M. (2004) · Energy Scavenging for Wireless Sensor Networks · doi:10.1007/978-1-4615-0485-6_3 · Cited by 422
Coordination costs and project outcomes in multi-university collaborations
Cummings, J. N., & Kiesler, S. (2007) · Research Policy · doi:10.1016/j.respol.2007.09.001 · Cited by 410
Mintzberg, H. (2004) · Management today/Management Today · Cited by 406
Measuring researcher interdisciplinarity
Porter, A. L., Cohen, A. S., David Roessner, J., & Perreault, M. (2007) · Scientometrics · doi:10.1007/s11192-007-1700-5 · Cited by 394
Risks and Rewards of an Interdisciplinary Research Path
Rhoten, D., & Parker, A. (2004) · Science · doi:10.1126/science.1103628 · Cited by 367
Interdisciplinary integration in Europe: the case of the Fifth Framework programme
Bruce, A., Lyall, C., Tait, J., & Williams, R. (2004) · Futures · doi:10.1016/j.futures.2003.10.003 · Cited by 367
Not by Productivity Alone: How Visibility and Specialization Contribute to Academic Earnings
Leahey, E. (2007) · American Sociological Review · doi:10.1177/000312240707200403 · Cited by 356
The use of multiple indicators in the assessment of basic research
Martin, B. R. (1996) · Scientometrics · doi:10.1007/bf02129599 · Cited by 356
Organizational and institutional influences on creativity in scientific research
Heinze, T., Shapira, P., Rogers, J. D., & Senker, J. M. (2009) · Research Policy · doi:10.1016/j.respol.2009.01.014 · Cited by 355
A Multi-Level Systems Perspective for the Science of Team Science
Börner, K., Contractor, N., Falk-Krzesinski, H. J., Fiore, S. M., Hall, K. L., Keyton, J., Spring, B., Stokols, D., Trochim, W., & Uzzi, B. (2010) · Science Translational Medicine · doi:10.1126/scitranslmed.3001399 · Cited by 348
Women in interdisciplinary science: Exploring preferences and consequences
Rhoten, D., & Pfirman, S. (2007) · Research Policy · doi:10.1016/j.respol.2006.08.001 · Cited by 303
Indicators of the interdisciplinarity of journals: Diversity, centrality, and citations
Leydesdorff, L., & Rafols, I. (2011) · Journal of Informetrics · doi:10.1016/j.joi.2010.09.002 · Cited by 301
Self-citationOn the economics and analysis of diversity
Stirling, A. (1998) · Figshare · Cited by 290
Self-citationClustering thescience citation index ® using co-citations: I. A comparison of methods
Small, H., & Sweeney, E. (1985) · Scientometrics · doi:10.1007/bf02017157 · Cited by 282
Interdisciplinary research: meaning, metrics and nurture
Porter, A. L., Roessner, J. D., Cohen, A. S., & Perreault, M. (2006) · Research Evaluation · doi:10.3152/147154406781775841 · Cited by 265
New Light on Old Boys: Cognitive and Institutional Particularism in the Peer Review System
Travis, G. D. L., & Collins, H. M. (1991) · Science, Technology, & Human Values · doi:10.1177/016224399101600303 · Cited by 245
Interdisciplinary Research: From Belief to Reality
Metzger, N., & Zare, R. N. (1999) · Science · doi:10.1126/science.283.5402.642 · Cited by 233
Bibliometric performance measures
Narin, F., & Hamilton, K. S. (1996) · Scientometrics · doi:10.1007/bf02129596 · Cited by 232
On the relationship between interdisciplinarity and scientific impact
Larivière, V., & Gingras, Y. (2010) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.21226 · Cited by 223
The Changing Governance of the Sciences: The Advent of Research Evaluation Systems
(2007) · Sociology of the Sciences Yearbook · doi:10.1007/978-1-4020-6746-4 · Cited by 222
OSWALD, A. J. (2007) · Economica · doi:10.1111/j.1468-0335.2006.00575.x · Cited by 214
Caveats for the use of citation indicators in research and journal evaluations
Leydesdorff, L. (2008) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.20743 · Cited by 209
Self-citationAssessing the nature of nanotechnology: can we uncover an emerging general purpose technology?
Youtie, J., Iacopetta, M., & Graham, S. (2008) · The Journal of Technology Transfer · doi:10.1007/s10961-007-9030-6 · Cited by 203
Creative Knowledge Environments
Hemlin, S., Allwood, C. M., & Martin, B. R. (2008) · Creativity Research Journal · doi:10.1080/10400410802060018 · Cited by 202
Practising Interdisciplinarity
(2000) · University of Toronto Press eBooks · doi:10.3138/9781442678729 · Cited by 197
de Solla Price, D. (1993) · Research Policy · doi:10.1016/0048-7333(93)90061-l · Cited by 194
deS. Price, D. (1984) · Research Policy · doi:10.1016/0048-7333(84)90003-9 · Cited by 189
Langfeldt, L. (2006) · Research Evaluation · doi:10.3152/147154406781776039 · Cited by 179
Modifying the journal impact factor by fractional citation weighting: The audience factor
Zitt, M., & Small, H. (2008) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.20880 · Cited by 179
Assessing expert interdisciplinary work at the frontier: an empirical exploration
Mansilla, V. B. (2006) · Research Evaluation · doi:10.3152/147154406781776075 · Cited by 175
Zitt, M., Ramanana-Rahary, S., & Bassecoulard, E. (2005) · Scientometrics · doi:10.1007/s11192-005-0218-y · Cited by 171
Interactive overlays: A new method for generating global journal maps from Web-of-Science data
Leydesdorff, L., & Rafols, I. (2012) · Journal of Informetrics · doi:10.1016/j.joi.2011.11.003 · Cited by 169
Self-citationIs multidisciplinary research more highly cited? A macrolevel study
Levitt, J. M., & Thelwall, M. (2008) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.20914 · Cited by 169
Criteria for scientific choice
Weinberg, A. M. (1963) · Minerva · doi:10.1007/bf01096248 · Cited by 168
Bhupatiraju, S., Nomaler, Ö., Triulzi, G., & Verspagen, B. (2012) · Research Policy · doi:10.1016/j.respol.2012.03.011 · Cited by 165
Leydesdorff, L., & Bornmann, L. (2011) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.21609 · Cited by 163
Self-citationRàfols, I., & Loet Leydesdorff, A. (2009) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.21086 · Cited by 150
Lee, F. S. (2006) · Cambridge Journal of Economics · doi:10.1093/cje/bel021 · Cited by 142
Science and technology studies: Exploring the knowledge base
Martin, B. R., Nightingale, P., & Yegros, A. (2012) · Research Policy · doi:10.1016/j.respol.2012.03.010 · Cited by 137
Self-citationThe Assessment of Research Quality in UK Universities: Peer Review or Metrics?
Taylor, J. (2010) · British Journal of Management · doi:10.1111/j.1467-8551.2010.00722.x · Cited by 130
Fairness as Appropriateness: Negotiating Epistemological Differences in Peer Review
Mallard, G., Lamont, M., & Guetzkow, J. (2009) · Science Technology & Human Values · doi:10.1177/0162243908329381 · Cited by 130
Scale-independent indicators and research evaluation
Katz, J. S. (2000) · Science and Public Policy · doi:10.3152/147154300781782156 · Cited by 129
Goodall, A. H. (2008) · Journal of Management Inquiry · doi:10.1177/1056492607311930 · Cited by 123
Peer review and the relevance gap: ten suggestions for policy-makers
Nightingale, P., & Scott, A. (2007) · Science and Public Policy · doi:10.3152/030234207x254396 · Cited by 120
Self-citationLowe, P. D., & Phillipson, J. (2006) · Journal of Agricultural Economics · doi:10.1111/j.1477-9552.2006.00045.x · Cited by 116
Influence of interdisciplinarity on peer-review and bibliometric evaluations in physics research
Rinia, E. J., van Leeuwen, T. N., van Vuren, H. G., & van Raan, A. F. J. (2001) · Research Policy · doi:10.1016/s0048-7333(00)00082-2 · Cited by 111
Harley, S., & Lee, F. S. (1997) · Human Relations · doi:10.1177/001872679705001105 · Cited by 111
The Social Construction Of Bibliometric Evaluations
Gläser, J., & Laudel, G. (2007) · Sociology of the sciences/Sociology of the sciences yearbook · doi:10.1007/978-1-4020-6746-4_5 · Cited by 110
Peer Review of Interdisciplinary Research Proposals
Porter, A. L., & Rossini, F. A. (1985) · Science Technology & Human Values · doi:10.1177/016224398501000304 · Cited by 107
Normalization at the field level: Fractional counting of citations
Loet Leydesdorff, A., & Opthof, T. (2010) · Journal of Informetrics · doi:10.1016/j.joi.2010.05.003 · Cited by 99
Loet Leydesdorff, A., & Ràfols, I. (2011) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.21509 · Cited by 97
Analysis of Cross-Disciplinary Research Through Bibliometric Tools
Bordons, M., Morillo, F., & Gómez, I. (2006) · Kluwer Academic Publishers eBooks · doi:10.1007/1-4020-2755-9_20 · Cited by 92
A quantitative view on the coming of age of interdisciplinarity in the sciences 1980-1999
Braun, T., & Schubert, A. P. (2003) · Scientometrics · doi:10.1023/a:1025439910278 · Cited by 85
Interdisciplinarity as a multidimensional concept: its measure in three different research areas
Menéndez, L. S., Bordons, M., & Zulueta, M. A. (2001) · Research Evaluation · doi:10.3152/147154401781777123 · Cited by 77
Loet Leydesdorff, A. (2007) · Scientometrics · doi:10.1007/s11192-007-1694-z · Cited by 76
Peer Review, the Research Assessment Exercise and the Demise of Non-Mainstream Economics
Lee, F. S., & Harley, S. (1998) · Capital & Class · doi:10.1177/030981689806600103 · Cited by 71
The UK Research Assessment Exercise
Martin, B. R., & Whitley, R. D. (2010) · Oxford University Press eBooks · doi:10.1093/acprof:oso/9780199590193.003.0002 · Cited by 69
Impact measures of interdisciplinary research in physics
Rinia, E. J., van Leeuwen, T. N., & van Raan, A. F. J. (2002) · Scientometrics · doi:10.1023/a:1014856625623 · Cited by 68
Rhoten, D. R., O'Connor, E., & Hackett, E. J. (2009) · Thesis Eleven · doi:10.1177/0725513608099121 · Cited by 67
The UK Research Assessment Exercise: A Case of Regulatory Capture?
Whitley, R. D., Martin, B. R., Gläser, J., & Engwall, L. (2010) · Figshare · Cited by 66
Tourish, D. (2011) · Leadership · doi:10.1177/1742715011407385 · Cited by 64
Lowe, P. D., & Phillipson, J. (2009) · Environment and Planning A Economy and Space · doi:10.1068/a4175 · Cited by 63
Introduction to a special issue on the assessment of interdisciplinary research
Laudel, G., & Origgi, G. (2006) · Research Evaluation · doi:10.3152/147154406781776066 · Cited by 59
Remaining problems with the “New Crown Indicator” (MNCS) of the CWTS
Loet Leydesdorff, A., & Tobias Opthof, A. (2010) · Journal of Informetrics · doi:10.1016/j.joi.2010.10.003 · Cited by 55
Quantitative and qualitative methods and measures in the evaluation of research
Roessner, D. (2000) · Research Evaluation · doi:10.3152/147154400781777296 · Cited by 54
Climate change diffusion: While the world tips, business schools lag
Patenaude, G. (2010) · Global Environmental Change · doi:10.1016/j.gloenvcha.2010.07.010 · Cited by 53
Ràfols, I. (2007) · Innovation The European Journal of Social Science Research · doi:10.1080/13511610701760770 · Cited by 49
Donovan, C. (2007) · Science and Public Policy · doi:10.3152/030234207x256529 · Cited by 45
Citation Counts: Are They Good Predictors of Rae Scores? A Bibliometric Analysis of RAE 2001Preprint
Mahdi, S., D’Este, P., & Neely, A. D. (2008) · SSRN Electronic Journal · doi:10.2139/ssrn.1154053 · Cited by 40
Critical Decision-making: A New Theory of Social Choice
Collingridge, D. (1982) · Medical Entomology and Zoology · Cited by 38
Ping, Z., & Loet Leydesdorff, A. (2011) · Journal of Informetrics · doi:10.1016/j.joi.2011.01.010 · Cited by 37
Llerena, P., & Meyer–Krahmer, F. (2003) · Edward Elgar Publishing eBooks · doi:10.4337/9781781950241.00013 · Cited by 37
Calibrating the zoom — a test of Zitt’s hypothesis
Adams, J. R., Gurney, K., & Jackson, L. (2008) · Scientometrics · doi:10.1007/s11192-007-1832-7 · Cited by 33
Behind citing-side normalization of citations: some properties of the journal impact factor
Zitt, M. (2011) · Scientometrics · doi:10.1007/s11192-011-0441-7 · Cited by 31
Mobilizing for change: A study of research units in emerging scientific fields
Clausen, T. H., Fagerberg, J., & Gulbrandsen, M. (2012) · Research Policy · doi:10.1016/j.respol.2012.03.014 · Cited by 30
Willmott, H. C. (2011) · Organization · doi:10.1177/1350508411403536 · Cited by 28
Citation Counts: Are They Good Predictors of RAE Scores?
Mahdi, S., & Neely, A. D. (2008) · Cited by 22
Kelly, A., Morris, H., & Harvey, C. E. (2009) · e-space (Manchester Metropolitan University) · Cited by 19
The UK Research Assessment Exercise: a Case of Regulatory Capture?
(4460437), B. M., & Whitley, R. D. (2010) · Figshare · Cited by 17
Peer review of interdisciplinary scientific papers
Lee, C. T. (2006) · Nature · doi:10.1038/nature05034 · Cited by 11
Cited by
Performance-based university research funding systems
Hicks, D. (2012) · Research Policy · doi:10.1016/j.respol.2011.09.007 · Cited by 1,011
A Review of Theory and Practice in Scientometrics
Mingers, J., & Leydesdorff, L. (2015) · European Journal of Operational Research · doi:10.1016/j.ejor.2015.04.002 · Cited by 1,002
A review of transdisciplinary research in sustainability science
Brandt, P., Ernst, A., Gralla, F., Luederitz, C., Lang, D. J., Newig, J., Reinert, F., Abson, D. J., & von Wehrden, H. (2013) · Ecological Economics · doi:10.1016/j.ecolecon.2013.04.008 · Cited by 819
Wagner, C. S., Roessner, J. D., Bobb, K., Klein, J. T., Boyack, K. W., Keyton, J., Rafols, I., & Börner, K. (2011) · Journal of Informetrics · doi:10.1016/j.joi.2010.06.004 · Cited by 769
Self-citationEvaluation Practices and Effects of Indicator Use—A Literature Review
Rijcke, S. D., Wouters, P. F., Rushforth, A. D., Franssen, T. P., & Hammarfelt, B. (2016) · Research Evaluation · doi:10.1093/reseval/rvv038 · Cited by 467
Metadata
- Authors
- Rafols, I., Leydesdorff, L., O’Hare, A., Nightingale, P., & Stirling, A.
- Venue
- Research Policy
- Publisher
- Elsevier BV
- Published
- 2012-09
- Volume
- 41
- Issue
- 7
- Pages
- 1262-1282
- ISSN
- 0048-7333 (ISSN-L)
- OpenAlex
- W2119392031
- Cited by
- 575
- References
- 112
- FWCI
- 43.87
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.