The DOI matches its Crossref record; 56 of the 56 references it cites were found in a registry or an index.
What is societal impact of research and how can it be assessed? a literature survey
Abstract
Since the 1990s, the scope of research evaluations becomes broader as the societal products (outputs), societal use (societal references), and societal benefits (changes in society) of research come into scope. Society can reap the benefits of successful research studies only if the results are converted into marketable and consumable products (e.g., medicaments, diagnostic tools, machines, and devices) or services. A series of different names have been introduced which refer to the societal impact of research: third stream activities, societal benefits, societal quality, usefulness, public values, knowledge transfer, and societal relevance. What most of these names are concerned with is the assessment of social, cultural, environmental, and economic returns (impact and effects) from results (research output) or products (research outcome) of publicly funded research. This review intends to present existing research on and practices employed in the assessment of societal impact in the form of a literature survey. The objective is for this review to serve as a basis for the development of robust and reliable methods of societal impact measurement.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The New production of knowledge: the dynamics of science and research in contemporary societies
(1995) · Choice Reviews Online · doi:10.5860/choice.32-4463 · Cited by 7,046
The New Production of Knowledge: The Dynamics of Science and Research in Contemporary Societies.
Baber, Z., Gibbons, M., Limoges, C., Nowotny, H., Schwartzman, S., Scott, P., & Trow, M. (1995) · Contemporary Sociology · doi:10.2307/2076669 · Cited by 4,406
The economic benefits of publicly funded basic research: a critical review
Salter, A. J., & Martin, B. R. (2001) · Research Policy · doi:10.1016/s0048-7333(00)00091-3 · Cited by 1,358
The increasing linkage between U.S. technology and public science
Narin, F., Hamilton, K. S., & Olivastro, D. (1997) · Research Policy · doi:10.1016/s0048-7333(97)00013-9 · Cited by 1,281
Academic research and industrial innovation
Mansfield, E. (1991) · Research Policy · doi:10.1016/0048-7333(91)90080-a · Cited by 1,262
Stephan, P. (2012) · Harvard University Press eBooks · doi:10.4159/harvard.9780674062757 · Cited by 934
Expert Opinions in Forecasting: The Role of the Delphi Technique
Rowe, G., & Wright, G. (2001) · International Series in Operations Research & Management Science · doi:10.1007/978-0-306-47630-3_7 · Cited by 777
Real Science: What it Is and What it Means
Ziman, J. (2000) · doi:10.1017/cbo9780511541391 · Cited by 760
Academic research and industrial innovation: An update of empirical findings
Mansfield, E. (1998) · Research Policy · doi:10.1016/s0048-7333(97)00043-7 · Cited by 643
Bornmann, L. (2011) · Annual Review of Information Science and Technology · doi:10.1002/aris.2011.1440450112 · Cited by 567
Self-citation
Cited by
Bornmann, L., & Mutz, R. (2015) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.23329 · Cited by 1,576
Self-citationFortunato, S., Bergstrom, C. T., Börner, K., Evans, J. A., Helbing, D., Milojević, S., Petersen, A. M., Radicchi, F., Sinatra, R., Uzzi, B., et al. (2018) · Science · doi:10.1126/science.aao0185 · Cited by 1,497
“The development of alternative metrics covering web (88) and social media (89) activity and societal impact (90) is critical in this regard.”
Conducting Educational Design Research
McKenney, S., & Reeves, T. C. (2018) · Routledge eBooks · doi:10.4324/9781315105642 · Cited by 1,354
Citations, Citation Indicators, and Research Quality: An Overview of Basic Concepts and Theories
Aksnes, D. W., Langfeldt, L., & Wouters, P. (2019) · Sage Open · doi:10.1177/2158244019829575 · Cited by 1,257
Do Altmetrics Work? Twitter and Ten Other Social Web Services
Thelwall, M., Haustein, S., Larivière, V., & Sugimoto, C. R. (2013) · PLoS ONE · doi:10.1371/journal.pone.0064841 · Cited by 907
Bornmann, L. (2014) · Journal of Informetrics · doi:10.1016/j.joi.2014.09.005 · Cited by 578
Self-citationAcademic engagement: A review of the literature 2011-2019
Perkmann, M., Salandra, R., Tartari, V., McKelvey, M., & Hughes, A. (2021) · Research Policy · doi:10.1016/j.respol.2020.104114 · Cited by 556
Ethical framework for Artificial Intelligence and Digital technologies
Ashok, M., Madan, R., Joha, A., & Sivarajah, U. (2022) · International Journal of Information Management · doi:10.1016/j.ijinfomgt.2021.102433 · Cited by 469
Luederitz, C., Schäpke, N., Wiek, A., Lang, D. J., Bergmann, M., Bos, J. J., Burch, S., Davies, A., Evans, J., König, A., et al. (2017) · Journal of Cleaner Production · doi:10.1016/j.jclepro.2016.09.005 · Cited by 367
Defining and assessing research quality in a transdisciplinary context
Belcher, B. M., Rasmussen, K. E., Kemshaw, M. R., & Zornes, D. A. (2016) · Research Evaluation · doi:10.1093/reseval/rvv025 · Cited by 273
Reale, E., Avramov, D., Canhial, K., Donovan, C., Flecha, R., Holm, P., Larkin, C., Lepori, B., Mosoni-Fried, J., Oliver, E., et al. (2018) · Research Evaluation · doi:10.1093/reseval/rvx025 · Cited by 258
Bainbridge, R., Tsey, K., McCalman, J., Kinchin, I., Saunders, V., Watkin Lui, F., Cadet-James, Y., Miller, A., & Lawson, K. (2015) · BMC Public Health · doi:10.1186/s12889-015-2052-3 · Cited by 259
The Open Innovation in Science research field: a collaborative conceptualisation approach
Beck, S., Bergenholtz, C., Bogers, M., Brasseur, T.-M., Conradsen, M. L., Di Marco, D., Distel, A. P., Dobusch, L., Dörler, D., Effert, A., et al. (2022) · Industry and Innovation · doi:10.1080/13662716.2020.1792274 · Cited by 254
Alternative metrics in scientometrics: a meta-analysis of research into three altmetrics
Bornmann, L. (2015) · Scientometrics · doi:10.1007/s11192-015-1565-y · Cited by 202
Self-citationMüller, A. M., Maher, C. A., Vandelanotte, C., Hingle, M., Middelweerd, A., Lopez, M. L., DeSmet, A., Short, C. E., Nathan, N., Hutchesson, M. J., et al. (2018) · Journal of Medical Internet Research · doi:10.2196/jmir.8954 · Cited by 180
Social impact in social media: A new method to evaluate the social impact of research
Pulido, C. M., Redondo-Sama, G., Sordé-Martí, T., & Flecha, R. (2018) · PLOS ONE · doi:10.1371/journal.pone.0203117 · Cited by 158
Public use and public funding of science
Yin, Y., Dong, Y., Wang, K., Wang, D., & Jones, B. F. (2022) · Nature Human Behaviour · doi:10.1038/s41562-022-01397-5 · Cited by 123
Bornmann, L. (2017) · Higher Education · doi:10.1007/s10734-016-9995-x · Cited by 117
Self-citationThe sociology of scientific validity: How professional networks shape judgement in peer review
Teplitskiy, M., Acuna, D., Elamrani-Raoult, A., Körding, K., & Evans, J. (2018) · Research Policy · doi:10.1016/j.respol.2018.06.014 · Cited by 113
Revisiting the Scientometric Conceptualization of Impact and Its Measurement
Abramo, G. (2018) · Journal of Informetrics · doi:10.1016/j.joi.2018.05.001 · Cited by 80
Carlson, J., & Harris, K. (2020) · PLoS Biology · doi:10.1371/journal.pbio.3000860 · Cited by 61
Hill, R., Yin, Y., Stein, C., Wang, X., Wang, D., & Jones, B. F. (2025) · Nature · doi:10.1038/s41586-025-09048-1 · Cited by 41
Bornmann, L., & Mutz, R. (2014) · arXiv · doi:10.48550/arxiv.1402.4578 · Cited by 11
Self-citationTerza Missione, Comunicazione, e Biblioteche accademiche
Sabba, F. (2020) · Bibliothecae.it · doi:10.6092/issn.2283-9364/10368 · Cited by 6
Marama, T., Neme, D., Gebrekirstos, L., Gido, R., Messele, H., Nega, A., Hailemariam, D., Derbew, M., & Aragaw, A. (2026) · Health Science Reports · doi:10.1002/hsr2.71937
Metadata
- Authors
- Bornmann, L.
- Venue
- Journal of the American Society for Information Science and Technology
- Publisher
- Wiley
- Published
- 2012-12-17
- Volume
- 64
- Issue
- 2
- Pages
- 217-233
- ISSN
- 1532-2882 (ISSN-L)
- 1532-2890
- OpenAlex
- W1979116104
- Cited by
- 731
- References
- 56
- FWCI
- 22.64
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.