The DOI matches its Crossref record; 55 of the 55 references it cites were found in a registry or an index.
Bias against novelty in science: A cautionary tale for users of bibliometric indicators
Open accessAbstract
Research which explores unchartered waters has a high potential for major impact but also carries a high uncertainty of having minimal impact. Such explorative research is often described as taking a novel approach. This study examines the complex relationship between pursuing a novel approach and impact. We measure novelty by examining the extent to which a published paper makes first time ever combinations of referenced journals, taking into account the difficulty of making such combinations. We apply this newly developed measure of novelty to a set of one million research articles across all scientific disciplines. We find that highly novel papers, defined to be those that make more (distinct) new combinations, have more than a triple probability of being a top 1% highly cited paper when using a sufficiently long citation time window to assess impact. Moreover, follow-on papers that cite highly novel research are themselves more likely to be highly cited. However, novel research is also risky as it has a higher variance in the citation performance. These findings are consistent with the “high risk/high gain” characteristic of novel research. We also find that novel papers are typically published in journals with a lower than expected Impact Factor and are less cited when using a short time window. Our findings suggest that science policy, in particular funding decisions which are over reliant on traditional bibliometric indicators based on short-term direct citation counts and Journal Impact Factors, may be biased against novelty.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The Structure of Scientific Revolutions
Kuhn, T. S., & Schlegel, R. (1963) · Physics Today · doi:10.1063/1.3050879 · Cited by 35,538
Exploration and Exploitation in Organizational Learning
March, J. G. (1991) · Organization Science · doi:10.1287/orsc.2.1.71 · Cited by 21,942
An Evolutionary Theory of Economic Change.
Loasby, B. J., Nelson, R. R., & Winter, S. G. (1983) · The Economic Journal · doi:10.2307/2232409 · Cited by 21,366
Economic Welfare and the Allocation of Resources for Invention
Arrow, K. J. (1962) · The Rate and Direction of Inventive Activity · doi:10.1515/9781400879762-024 · Cited by 8,635
THE STRUCTURE OF SCIENTIFIC REVOLUTIONS
MCGUIRE, J. E. (1963) · Philosophical Books · doi:10.1111/j.1468-0149.1963.tb00813.x · Cited by 8,424
Structural Holes and Good Ideas
Burt, R. S. (2004) · American Journal of Sociology · doi:10.1086/421787 · Cited by 5,642
The associative basis of the creative process.
Mednick, S. (1962) · Psychological Review · doi:10.1037/h0048850 · Cited by 4,140
The Origins of Endogenous Growth
Romer, P. M. (1994) · Journal of Economic Perspectives · doi:10.1257/jep.8.1.3 · Cited by 3,783
The New Production of Knowledge
Gibbons, M. (1994) · Medical Entomology and Zoology · Cited by 3,405
Recombinant Uncertainty in Technological Search
Fleming, L. (2001) · Management Science · doi:10.1287/mnsc.47.1.117.10671 · Cited by 2,873
Bibliometrics: The Leiden Manifesto for Research Metrics
Hicks, D., Wouters, P., Waltman, L., de Rijcke, S., & Rafols, I. (2015) · Nature · doi:10.1038/520429a · Cited by 2,828
Phiel, C. J., Zhang, F., Huang, E. Y., Guenther, M. G., Lazar, M. A., & Klein, P. S. (2001) · Journal of Biological Chemistry · doi:10.1074/jbc.m101287200 · Cited by 1,758
Serial Block-Face Scanning Electron Microscopy to Reconstruct Three-Dimensional Tissue Nanostructure
Denk, W., & Horstmann, H. (2004) · PLoS Biology · doi:10.1371/journal.pbio.0020329 · Cited by 1,644
(1973) · Cited by 1,638
Atypical Combinations and Scientific Impact
Uzzi, B., Mukherjee, S., Stringer, M., & Jones, B. (2013) · Science · doi:10.1126/science.1240474 · Cited by 1,457
Performance-based university research funding systems
Hicks, D. (2012) · Research Policy · doi:10.1016/j.respol.2011.09.007 · Cited by 1,011
(2005) · Springer eBooks · doi:10.1007/1-4020-2755-9 · Cited by 979
Weitzman, M. L. (1998) · The Quarterly Journal of Economics · doi:10.1162/003355398555595 · Cited by 979
Facilitating Interdisciplinary Research
(2004) · National Academies Press eBooks · doi:10.17226/11153 · Cited by 962
Incentives and creativity: evidence from the academic life sciences
Azoulay, P., Graff Zivin, J. S., & Manso, G. (2011) · The RAND Journal of Economics · doi:10.1111/j.1756-2171.2011.00140.x · Cited by 648
Tradition and Innovation in Scientists’ Research Strategies
Foster, J. G., Rzhetsky, A., & Evans, J. A. (2015) · American Sociological Review · doi:10.1177/0003122415601618 · Cited by 613
van Raan, A. F. J. (2004) · Scientometrics · doi:10.1023/b:scie.0000018543.82441.f1 · Cited by 579
Scientific teams and institutional collaborations: Evidence from U.S. universities, 1981–1999
Adams, J. D., Black, G. C., Clemmons, J. R., & Stephan, P. E. (2005) · Research Policy · doi:10.1016/j.respol.2005.01.014 · Cited by 549
Self-citationBoudreau, K. J., Guinan, E. C., Lakhani, K. R., & Riedl, C. (2016) · Management Science · doi:10.1287/mnsc.2015.2285 · Cited by 481
Measuring technological novelty with patent-based indicators
Verhoeven, D., Bakker, J., & Veugelers, R. (2016) · Research Policy · doi:10.1016/j.respol.2015.11.010 · Cited by 453
Self-citationCreativity in science: chance, logic, genius, and zeitgeist
(2005) · Choice Reviews Online · doi:10.5860/choice.42-2773 · Cited by 448
Butler, L. (2003) · Research Policy · doi:10.1016/s0048-7333(02)00007-0 · Cited by 447
The Philosophical Basis of Peer Review and the Suppression of Innovation
Horrobin, D. F. (1990) · JAMA: The Journal of the American Medical Association · doi:10.1001/jama.1990.03440100162024 · Cited by 436
Towards an explanatory and computational theory of scientific discovery
Chen, C., Chen, Y., Horowitz, M., Hou, H., Liu, Z., & Pellegrino, D. (2009) · Journal of Informetrics · doi:10.1016/j.joi.2009.03.004 · Cited by 426
Citation time window choice for research impact evaluation
Wang, J. (2013) · Scientometrics · doi:10.1007/s11192-012-0775-9 · Cited by 416
Self-citationHow much is a collaboration worth? A calibrated bibliometric model
Katz, J. S., & Hicks, D. (1997) · Scientometrics · doi:10.1007/bf02459299 · Cited by 402
Business Cycles: A Theoretical, Historical, and Statistical Analysis of the Capitalist Process.
Morgenstern, O., & Schumpeter, J. A. (1940) · Journal of the American Statistical Association · doi:10.2307/2278731 · Cited by 392
Creativity in scientific teams: Unpacking novelty and impact
Lee, Y.-N., Walsh, J. P., & Wang, J. (2015) · Research Policy · doi:10.1016/j.respol.2014.10.007 · Cited by 360
Self-citationStudies of scientific discovery: Complementary approaches and convergent findings.
Klahr, D., & Simon, H. A. (1999) · Psychological Bulletin · doi:10.1037/0033-2909.125.5.524 · Cited by 352
Yegros-Yegros, A., Rafols, I., & D’Este, P. (2015) · PLOS ONE · doi:10.1371/journal.pone.0135095 · Cited by 342
The Four Literatures of Social Science
Hicks, D. (2006) · Handbook of Quantitative Science and Technology Research · doi:10.1007/1-4020-2755-9_22 · Cited by 332
Glänzel, W., & Schubert, A. (2003) · Scientometrics · doi:10.1023/a:1022378804087 · Cited by 329
Prematurity and Uniqueness in Scientific Discovery
Stent, G. S. (1972) · Scientific American · doi:10.1038/scientificamerican1272-84 · Cited by 289
Choosing experiments to accelerate collective discovery
Rzhetsky, A., Foster, J. G., Foster, I. T., & Evans, J. A. (2015) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1509757112 · Cited by 249
The number that's devouring science
Monastersky, R. (2005) · The chronicle of higher education · Cited by 240
Interdisciplinarity and Impact: Distinct Effects of Variety, Balance, and Disparity
Wang, J., Thijs, B., & Glänzel, W. (2015) · PLOS ONE · doi:10.1371/journal.pone.0127298 · Cited by 210
Self-citationLong-Distance Interdisciplinarity Leads to Higher Scientific Impact
Larivière, V., Haustein, S., & Börner, K. (2015) · PLOS ONE · doi:10.1371/journal.pone.0122565 · Cited by 208
Scientific Discovery and the Psychology of Problem Solving
Simon, H. A. (1977) · Boston Studies in the Philosophy of Science · doi:10.1007/978-94-010-9521-1_16 · Cited by 185
Scientific discovery as problem solving
Simon, H. A., Langley, P. W., & Bradshaw, G. L. (1981) · Synthese · doi:10.1007/bf01064262 · Cited by 167
Weaving the fabric of science: Dynamic network models of science's unfolding structure
Shi, F., Foster, J. G., & Evans, J. A. (2015) · Social Networks · doi:10.1016/j.socnet.2015.02.006 · Cited by 159
Drawing Theories Apart: The Dispersion of Feynman Diagrams in Postwar Physics
Merzbacher, E. (2006) · Physics Today · doi:10.1063/1.2435652 · Cited by 158
Editors’ JIF-boosting stratagems – Which are appropriate and which not?
Martin, B. R. (2016) · Research Policy · doi:10.1016/j.respol.2015.09.001 · Cited by 128
Measuring knowledge transfer between fields of science
Rinia, E. J., van Leeuwen, T. N., Bruins, E. E. W., et al. (2002) · Scientometrics · doi:10.1023/a:1016078331752 · Cited by 126
Bornmann, L., Loet Leydesdorff, A., & Wang, J. (2013) · Journal of Informetrics · doi:10.1016/j.joi.2013.11.005 · Cited by 108
Self-citation"Scientific Autobiography" and Other Papers
Nielsen, J. R., Planck, M., & Gaynor, F. (1950) · Books Abroad · doi:10.2307/40089223 · Cited by 100
Costas, R., van Leeuwen, T. N., & van Raan, A. F. J. (2009) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.21244 · Cited by 95
Overbuilding Research Capacity
Alberts, B. (2010) · Science · doi:10.1126/science.1197077 · Cited by 41
Costas, R., van Leeuwen, T. N., & van Raan, A. F. J. (2012) · Research Policy · doi:10.1016/j.respol.2012.11.006 · Cited by 14
Search for evergreens in science: A functional data analysis
Zhang, R., Wang, J., & Mei, Y. (2017) · Journal of Informetrics · doi:10.1016/j.joi.2017.05.007 · Cited by 6
Self-citation(2009)
Cited by
Guidelines for advancing theory and practice through bibliometric research
Mukherjee, D., Lim, W. M., Kumar, S., & Donthu, N. (2022) · Journal of Business Research · doi:10.1016/j.jbusres.2022.04.042 · Cited by 1,135
Circular economy business models: The state of research and avenues ahead
Ferasso, M., Beliaeva, T., Kraus, S., Clauss, T., & Ribeiro‐Soriano, D. (2020) · Business Strategy and the Environment · doi:10.1002/bse.2554 · Cited by 588
The citation advantage of linking publications to research data
Colavizza, G., Hrynaszkiewicz, I., Staden, I., Whitaker, K., & McGillivray, B. (2020) · PLOS ONE · doi:10.1371/journal.pone.0230416 · Cited by 335
Swedberg, R. (2020) · The Production of Knowledge · doi:10.1017/9781108762519.002 · Cited by 293
Reliance on science: Worldwide front‐page patent citations to scientific articles
Marx, M., & Fuegi, A. (2020) · Strategic Management Journal · doi:10.1002/smj.3145 · Cited by 266
Metadata
- Authors
- Wang, J., Veugelers, R., & Stephan, P.
- Venue
- Research Policy
- Publisher
- Elsevier BV
- Published
- 2017-10
- Volume
- 46
- Issue
- 8
- Pages
- 1416-1436
- ISSN
- 0048-7333 (ISSN-L)
- OpenAlex
- W3123050710
- Cited by
- 540
- References
- 55
- FWCI
- 33.46
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.