Comparing Institutional-Level Bibliometric Research Performance Indicator Values Based on Different Affiliation Disambiguation Systems
Open accessAbstract
The present study is an evaluation of three frequently used institution name disambiguation systems. The Web of Science normalized institution names and Organization Enhanced system and the Scopus Affiliation ID system are tested against a complete, independent institution disambiguation system for a sample of German public sector research organizations. The independent system is used as the gold standard in the evaluations that we perform. We study the coverage of the disambiguation systems and, in particular, the differences in a number of commonly used bibliometric indicators. The key finding is that for the sample institutions, the studied systems provide bibliometric indicator values that have only a limited accuracy. Our conclusion is that for any use with policy implications, additional data cleaning for disambiguating affiliation data is recommended.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Both WoS OE and Scopus AFID provide adequate precision (0.95 and 0.96).
“both of the systems provide adequate levels of precision (0.95 for WoS OE and 0.96 for Scopus AFID)”
From Web of Science (WoS) and Scopus: The Titans of Bibliographic Information in Today’s Academic World (2021) · cites this paper as [72]
The recall of WoS OE and Scopus AFID is lower (0.93 and 0.86).
“The recall of both systems was determined to be considerably lower (0.93 for WoS and 0.86 for Scopus)”
From Web of Science (WoS) and Scopus: The Titans of Bibliographic Information in Today’s Academic World (2021) · cites this paper as [72]
Only 20% of investigated institutions were located in WoS OE, while 85% were located in Scopus AFID.
“only 20% of investigated institutions (445) were located in WoS OE, which was mainly attributed to the aforementioned fact that WoS OE does not cover non-university research institutes. Meanwhile, in the Scopus AFID system, 85% of institutions were located [72].”
From Web of Science (WoS) and Scopus: The Titans of Bibliographic Information in Today’s Academic World (2021) · cites this paper as [72]
Incomplete coverage and insufficient precision and recall of WoS and Scopus institutional disambiguation systems mean both systems cannot currently be fully relied on in evaluations of institution's research performance.
“an incomplete coverage and insufficient precision and recall of WoS and Scopus institutional disambiguation systems imply that both of these systems cannot currently be fully relied on in evaluations of institution's research performance [72].”
From Web of Science (WoS) and Scopus: The Titans of Bibliographic Information in Today’s Academic World (2021) · cites this paper as [72]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Moed, H. F., Bruin, R. E. D., & van Leeuwen, T. N. (1995) · Scientometrics · doi:10.1007/bf02017338 · Cited by 502 · see citing works
The Leiden ranking 2011/2012: Data collection, indicators, and interpretation
Waltman, L., Calero‐Medina, C., Kosten, J., Noyons, E., Tijssen, R., van Eck, N. J., van Leeuwen, T. N., van Raan, A. F. J., Visser, M. S., & Wouters, P. F. (2012) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.22708 · Cited by 452 · see citing works
NEMO: Extraction and normalization of organization names from PubMed affiliations
Jonnalagadda, S., & Topham, P. (2010) · Journal of Biomedical Discovery and Collaboration · doi:10.5210/disco.v5i0.3047 · Cited by 45 · see citing works
Using clustering strategies for creating authority files
French, J. C., Powell, A. L., & Schulman, E. (2000) · Journal of the American Society for Information Science · doi:10.1002/(sici)1097-4571(2000)51:8 · Cited by 39 · see citing works
Institution name disambiguation for research assessment
Huang, S., Yang, B., Yan, S., et al. (2013) · Scientometrics · doi:10.1007/s11192-013-1214-2 · Cited by 38 · see citing works
The unification of institutional addresses applying parametrized finite-state graphs (P-FSG)
Gálvez, C., & Anegón, F. D. M. (2006) · Scientometrics · doi:10.1007/s11192-006-0156-3 · Cited by 29 · see citing works
Efficient supervised and semi-supervised approaches for affiliations disambiguation
Cuxac, P., Lamirel, J.-C., & Bonvallot, V. (2013) · Scientometrics · doi:10.1007/s11192-013-1025-5 · Cited by 25 · see citing works
Standardizing formats of corporate source data
Gálvez, C., & Anegón, F. D. M. (2007) · Scientometrics · doi:10.1007/s11192-007-0101-0 · Cited by 25 · see citing works
Affiliation disambiguation for constructing semantic digital libraries
Jiang, Y., Zheng, H.-T., Wang, X., et al. (2011) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.21538 · Cited by 21 · see citing works
NEMO: Extraction and normalization of organization names from PubMed affiliation stringsPreprint
Jonnalagadda, S., & Topham, P. (2011) · arXiv (Cornell University) · doi:10.48550/arxiv.1107.5743 · Cited by 16 · see citing works
Disambiguation of author addresses in bibliometric databases - technical report.
Rimmert, C., Schwechheimer, H., & Winterhager, M. (2017) · PUB – Publications at Bielefeld University (Bielefeld University) · Cited by 12 · see citing works
Self-citation
Flagged first, then by citations
Research works printed but not linked: 1
- De Bruin, R. E., & Moed, H. F. (1990). The unification of addresses in specific publications. In L. Egghe & R. Rousseau (Eds.), Informetrics 1989/90 (pp. 65-78). Amsterdam: Elsevier Science Publishers.
Cited by
Sorted by citations
Web of Science (WoS) and Scopus: The Titans of Bibliographic Information in Today’s Academic World
Pranckutė, R. (2021) · Publications · doi:10.3390/publications9010012 · Cited by 2,696 · see citing works
- Authors
- Donner, P., Rimmert, C., & van Eck, N. J.
- Affiliations
- German Centre for Higher Education Research and Science Studies (DE)
- Bielefeld University (DE)
- Centre for Science and Technology Studies, Leiden University, Netherlands
- Venue
- Quantitative Science Studies
- Publisher
- MIT Press
- Published
- 2020-02
- Volume
- 1
- Issue
- 1
- Pages
- 150-170
- ISSN
- 2641-3337 (ISSN-L)
- OpenAlex
- W2996732749
- Cited by
- 54
- References
- 11
- Citations by year
- 2026: 10 · 2025: 8 · 2024: 6 · 2023: 8 · 2022: 9 · 2021: 8 · 2020: 5
- FWCI
- 3.68
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- Quantitative Science Studies (publishedVersion, cc-by)
- Leiden Repository (Leiden University) (submittedVersion, cc-by)
- Data Archiving and Networked Services (DANS) (submittedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- PUB – Publications at Bielefeld University (Bielefeld University) (submittedVersion)
- Funders
- Bundesministerium für Bildung und Forschung (01PQ13001, 01PQ17001)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › scientometrics and bibliometrics research
- Physical Sciences › Computer Science › Information Systems › Web visibility and informetrics
- Physical Sciences › Computer Science › Information Systems › Library Science and Information Systems
- Keywords
- research performance indicators, Web of Science, data cleaning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
