A Review of Microsoft Academic Services for Science of Science Studies
Open accessThe DOI matches its Crossref record; 8 of the 8 references it cites were found in a registry or an index.
Abstract
Since the relaunch of Microsoft Academic Services (MAS) 4 years ago, scholarly communications have undergone dramatic changes: more ideas are being exchanged online, more authors are sharing their data, and more software tools used to make discoveries and reproduce the results are being distributed openly. The sheer amount of information available is overwhelming for individual humans to keep up and digest. In the meantime, artificial intelligence (AI) technologies have made great strides and the cost of computing has plummeted to the extent that it has become practical to employ intelligent agents to comprehensively collect and analyze scholarly communications. MAS is one such effort and this paper describes its recent progresses since the last disclosure. As there are plenty of independent studies affirming the effectiveness of MAS, this paper focuses on the use of three key AI technologies that underlies its prowess in capturing scholarly communications with adequate quality and broad coverage: (1) natural language understanding in extracting factoids from individual articles at the web scale, (2) knowledge assisted inference and reasoning in assembling the factoids into a knowledge graph, and (3) a reinforcement learning approach to assessing scholarly importance for entities participating in scholarly communications, called the saliency, that serves both as an analytic and a predictive metric in MAS. These elements enhance the capabilities of MAS in supporting the studies of science of science based on the GOTO principle, i.e., good and open data with transparent and objective methodologies. The current direction of development and how to access the regularly updated data and tools from MAS, including the knowledge graph, a REST API and a website, are also described.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Harris, Z. S. (1954) · WORD · doi:10.1080/00437956.1954.11659520 · Cited by 4,807
Bibliometrics: The Leiden Manifesto for Research Metrics
Hicks, D., Wouters, P., Waltman, L., de Rijcke, S., & Rafols, I. (2015) · Nature · doi:10.1038/520429a · Cited by 2,828 · cited 1×
“However, a closer investigation shows that these two journals have influential articles in the field, such as the Leiden Manifesto (Hicks et al., 2015) in Nature and the coercive citation studies (Wilhite and Fong, 2012) in Science.”
BackgroundGarfield, E. (1972) · Science · doi:10.1126/science.178.4060.471 · Cited by 2,788
Citation Indexes for Science: A New Dimension in Documentation through Association of Ideas
Garfield, E. (1955) · Science · doi:10.1126/science.122.3159.108 · Cited by 2,334
Coercive Citation in Academic Publishing
Wilhite, A. W., & Fong, E. A. (2012) · Science · doi:10.1126/science.1212540 · Cited by 433 · cited 2×
“The design to unshackle the reliance on the overly reductive citation counts may also lead the saliency to be less susceptible to manipulations, ranging from citation coercions (Wilhite and Fong, 2012) to malicious cheating (López-Cózar et al., 2014) targeting metrics like the h-index.”
BackgroundThe Google scholar experiment: How to index false papers and manipulate bibliometric indicators
Delgado López‐Cózar, E., Robinson‐García, N., & Torres‐Salinas, D. (2014) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.23056 · Cited by 272
The Coverage of Microsoft Academic: Analyzing the Publication Output of a University
Hug, S. E., & Brändle, M. P. (2017) · Scientometrics · doi:10.1007/s11192-017-2535-3 · Cited by 120 · cited 1×
“…of self-improving, many independent studies have suggested that MAG data are in many aspects as accurate, if not more, than manually curated data (Herrmannova and Knoth, 2016; Harzing and Alakangas, 2017; Hug and Brändle, 2017; Hug et al., 2017; Thelwall, 2017, 2018a,b,c; Kousha et al., 2018).”
BackgroundSystematic Analysis of Agreement between Metrics and Peer Review in the UK REF
Traag, V. A., & Waltman, L. (2019) · Palgrave Communications · doi:10.1057/s41599-019-0233-x · Cited by 65 · cited 1×
“…is found to be too reductive in contrast to peer reviews for article-level assessments (Wilsdon, 2015), although there is evidence suggesting noises in such a simplified model may be “canceled out” through aggregations at a level higher than individual articles (Traag and Waltman, 2019).”
Cited by
Visser, M., van Eck, N. J., & Waltman, L. (2021) · Quantitative Science Studies · doi:10.1162/qss_a_00112 · Cited by 865
Microsoft Academic Graph: When experts are not enough
Wang, K., Shen, Z., Huang, C., Wu, C.-H., Dong, Y., & Kanakia, A. (2020) · Quantitative Science Studies · doi:10.1162/qss_a_00021 · Cited by 509
Self-citationGender-diverse teams produce more novel and higher-impact scientific ideas
Yang, Y., Tian, T. Y., Woodruff, T. K., Jones, B. F., & Uzzi, B. (2022) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2200841119 · Cited by 360
Emerging opportunities and challenges for the future of reservoir computing
Yan, M., Huang, C., Bienstman, P., Tino, P., Lin, W., & Sun, J. (2024) · Nature Communications · doi:10.1038/s41467-024-45187-1 · Cited by 346
Self-citationChen, C. (2020) · Frontiers in Research Metrics and Analytics · doi:10.3389/frma.2020.607286 · Cited by 231
Public use and public funding of science
Yin, Y., Dong, Y., Wang, K., Wang, D., & Jones, B. F. (2022) · Nature Human Behaviour · doi:10.1038/s41562-022-01397-5 · Cited by 123
Self-citationAlShebli, B., Makovi, K., & Rahwan, T. (2020) · Nature Communications · doi:10.1038/s41467-020-20617-y · Cited by 23
RetractedWang, L. L., Cachola, I., Bragg, J., Cheng, E. Y.-Y., Haupt, C., Latzke, M., Kuehl, B., van Zuylen, M., Wagner, L., & Weld, D. S. (2021) · arXiv · doi:10.48550/arxiv.2105.00076 · Cited by 22
“We construct a dataset of papers by sampling PDFs published in the years of 2010–2019 stratified across the 19 top level fields of study defined by Microsoft Academic Graph [42, 47].”
The gender gap in scholarly self-promotion on social media
Peng, H., Teplitskiy, M., Romero, D. M., & Horvát, E.-Á. (2025) · Nature Communications · doi:10.1038/s41467-025-60590-y · Cited by 17
Evaluating institutional open access performance: Methodology, challenges and assessment
Huang, C.-K., Neylon, C., Hosking, R., Montgomery, L., Wilson, K., Ozaygen, A., & Brookes-Kenworthy, C. (2020) · doi:10.1101/2020.03.19.998336 · Cited by 6
“Briefly, we gather output metadata from searches in Microsoft Academic3 (Sinha et al., 2015; Wang et al., 2019), Web of Science and Scopus, for each university.”
MethodSelf-citationCORD-19: The COVID-19 Open Research Dataset
Wang, L. L., Lo, K., Chandrasekhar, Y., Reas, R., Yang, J., Burdick, D., Eide, D., Funk, K., Katsis, Y., Kinney, R., et al. (2020) · arXiv · doi:10.48550/arxiv.2004.10706
Self-citation
Metadata
- Authors
- Wang, K., Shen, Z., Huang, C., Wu, C.-H., Eide, D., Dong, Y., Qian, J., Kanakia, A., Chen, A., & Rogahn, R.
- Venue
- Frontiers in Big Data
- Publisher
- Frontiers Media SA
- Published
- 2019-12-03
- Volume
- 2
- Pages
- 45-45
- ISSN
- 2624-909X (ISSN-L)
- OpenAlex
- W2993670251
- Cited by
- 144
- References
- 8
- FWCI
- 14.86
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.