AI Research Funding Portfolios and Extreme Growth
Open accessAbstract
Our work analyzes the artificial intelligence and machine learning (AI/ML) research portfolios of six large research funding organizations from the United States [National Institutes of Health (NIH) and National Science Foundation (NSF)]; Europe [European Commission (EC) and European Research Council (ERC)]; China [National Natural Science Foundation of China (NNSFC)]; and Japan [Japan Society for the Promotion of Science (JSPS)]. The data for this analysis is based on 127,000 research clusters (RCs) that are derived from 1.4 billion citation links between 104.8 million documents from four databases (Dimensions, Microsoft Academic Graph, Web of Science, and the Chinese National Knowledge Infrastructure). Of these RCs, 600 large clusters are associated with AI/ML topics, and 161 of these AI/ML RCs are expected to experience extreme growth between May 2020 and May 2023. Funding acknowledgments (in the corpus of the 104.9 million documents) are used to characterize the overall AI/ML research portfolios of each organization. NNSFC is the largest funder of AI/ML research and disproportionately funds computer vision. The EC, RC, and JSPS focus more efforts on natural language processing and robotics. The NSF and ERC are more focused on fundamental advancement of AI/ML rather than on applications. They are more likely to participate in the RCs that are expected to have extreme growth. NIH funds the largest relative share of general AI/ML research papers (meaning in areas other than computer vision, natural language processing, and robotics). We briefly describe how insights such as these could be applied to portfolio management decision-making.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
There are significant gaps in gender, race, and ethnicity among AI researchers.
“There are significant gaps in gender, race, and ethnicity [71, 72]”
From Use of artificial intelligence in critical care: opportunities and obstacles (2024) · cites this paper as [72]
The Results and Discussion describe funding portfolios, AI research fields, and authors’ geographic affiliations, but not gaps in researchers’ gender, race, or ethnicity. Searches for those demographic disparities found no passage addressing the claim; the paper’s Bias/Fairness research cluster is a topic of research, not evidence about researcher demographics. (LLM, conf=0.76)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 48,293 · see citing works
Expert political judgment: how good is it? how can we know?
(2006) · Choice Reviews Online · doi:10.5860/choice.43-3634 · Cited by 1,260 · cited 1× · see citing works
“Assumptions, opinions, and informal “gut instincts” from subject matter experts and non-experts alike can substitute for welldefined forecasts when they are absent, which may reduce the utility and accountability of these judgments (Tetlock, 2017).”
Background
Flagged first, then by citations
Research works printed but not linked: 23
- Boyack K. W. Klavans R. (2019). “Creation and analysis of large-scale bibliometric networks,” in Springer Handbook of Science and Technology Indicators, Springer Handbooks, Chapter 8, eds Glanzel W. Moed H. F. Schmoch U. Thelwall M. (Cham: Springer International Publishing), 187– 212. 10.1007/978-3-030-02511-3_8 · DOI
- Charikar M. Indyk P. Panigrahy R. (2002). “New algorithms for subset query, partial match, orthogonal range searching, and related problems,” in International Colloquium on Automata, Languages, and Programming (Berlin: Springer), 451– 462. 10.1007/3-540-45465-9_39 · DOI
- Dunham J. Melot J. Murdick D. (2020). Identifying the development and application of artificial intelligence in scientific text. arXiv [Preprints]. arXiv: 2002.07143. · arXiv
- Flagg M. Harris P. (2020). System re-engineering: a new policy framework for the American R&D system in a changed world. CSET Policy Brief. 1– 18. 10.51593/20200050 · DOI
- Gerstner L. V. Jr. (1972). The practice of business: can strategic planning pay off? Bus. Horiz. 15, 5– 16. 10.1016/0007-6813(72)90056-0 · DOI
- Klavans R. Boyack K. W. (2017a). Research portfolio analysis and topic prominence. J. Informetr. 11, 1158– 1174. 10.1016/j.joi.2017.10.002 · DOI
- Klavans R. Boyack K. W. (2017b). Which type of citation analysis generates the most accurate taxonomy of scientific and technical knowledge? J. Assoc. Inform. Sci. Technol. 68, 984– 998. 10.1002/asi.23734 · DOI
- Klavans R. Boyack K. W. Murdick D. A. (2020). A novel approach to predicting exceptional growth in research. PLoS ONE 15: e0239177. 10.1371/journal.pone.0239177 32931500 PMC7491740 · DOI
- Kuhn T. S. (1970). The Structure of Scientific Revolutions, 2nd Edn. Chicago, IL; London: University of Chicago Press.
- Liu W. Tang L. Hu G. (2020). Funding information in web of science: an updated overview. Scientometrics 122, 1509– 1524. 10.1007/s11192-020-03362-3 · DOI · Paper page
- Manku G. S. Jain A. Das Sarma A. (2007). “Detecting near-duplicates for web crawling,” in Proceedings of the 16th International Conference on World Wide Web (New York, NY: Association for Computing Machinery), 141– 150. 10.1145/1242572.1242592 · DOI · Paper page
- Martin S. Brown W. M. Klavans R. Boyack K. W. (2011). “Openord: an open-source toolbox for large graph layout,” in Visualization and Data Analysis 2011, Vol. 7868 (International Society for Optics and Photonics), 786806. 10.1117/12.871402 · DOI
- Reichardt J. Bornholdt S. (2006). Statistical mechanics of community detection. Phys. Rev. E 74: 016110. 10.1103/PhysRevE.74.016110 16907154 · DOI · Paper page
- Sjögårde P. Ahlgren P. (2018). Granularity of algorithmically constructed publication-level classifications of research publications: identification of topics. J. Informetr. 12, 133– 152. 10.1016/j.joi.2017.12.006 · DOI
- Small H. Boyack K. W. Klavans R. (2014). Identifying emerging topics in science and technology. Res. Policy 43, 1450– 1467. 10.1016/j.respol.2014.02.005 · DOI
- Smit H. T. J. Trigeorgis L. (2006). Strategic planning: valuing and managing portfolios of real options. R&D Manage. 36, 403– 419. 10.1111/j.1467-9310.2006.00440.x · DOI
- Tanioka T. (2019). Nursing and rehabilitative care of the elderly using humanoid robots. J. Med. Invest. 66, 19– 23. 10.2152/jmi.66.19 31064938 · DOI
- Traag V. A. Waltman L. van Eck N. J. (2019). From Louvain to Leiden: guaranteeing well-connected communities. Sci. Rep. 9: 5233. 10.1038/s41598-019-41695-z 30914743 PMC6435756 · DOI · Paper page
- Visser M. van Eck N. J. Waltman L. (2020). Large-scale comparison of bibliographic data sources: scopus, web of science, dimensions, crossref, and microsoft academic. arXiv 2005.10732. 10.1162/qss_a_00112 · DOI · Paper page
- Vonortas N. S. Ràfols I. (2018). “The use of research portfolios in science policy,” in Paper Presented at Government Decision-Making to Allocate Scientific Resources of the National Academies of Sciences (Washington, DC). 10.22163/fteval.2019.338 · DOI
- Wallace M. Ràfols I. (2015). Research portfolio analysis in science policy: moving from financial returns to societal benefits. Minerva 53, 89– 115. 10.1007/s11024-015-9271-8 · DOI
- Waltman L. Van Eck N. J. (2012). A new methodology for constructing a publication-level classification system of science. J. Am. Soc. Inform. Sci. Technol. 63, 2378– 2392. 10.1002/asi.22748 · DOI
- Waltman L. Van Eck N. J. (2013). A smart local moving algorithm for large-scale modularity-based community detection. Eur. Phys. J. B 86: 471. 10.1140/epjb/e2013-40829-0 · DOI · Paper page
Cited by
Sorted by citations
Use of artificial intelligence in critical care: opportunities and obstacles
Pinsky, M. R., Bedoya, A., Bihorac, A., Celi, L., Churpek, M., Economou-Zavlanos, N. J., Elbers, P., Saria, S., Liu, V., Lyons, P. G., et al. (2024) · Critical Care · doi:10.1186/s13054-024-04860-z · Cited by 160 · see citing works
“There are significant gaps in gender, race, and ethnicity [71, 72] Lack of diverse perspectives can negatively impact resulting products, as has plagued the AI field for years [73, 74].”
Background
- Authors
- Rahkovsky, I., Toney, A., Boyack, K. W., Klavans, R., & Murdick, D. A.
- Affiliations
- Georgetown University (US)
- SciTech Strategies (United States) (US)
- Corresponding author
- Rahkovsky, Ilya
- Venue
- Frontiers in Research Metrics and Analytics
- Publisher
- Frontiers Media SA
- Published
- 2021-04-06
- Volume
- 6
- Pages
- 630124-630124
- ISSN
- 2504-0537 (ISSN-L)
- PMC
- PMC8028401
- OpenAlex
- W3145247846
- Cited by
- 24
- References
- 2
- Citations by year
- 2026: 2 · 2025: 7 · 2024: 8 · 2023: 5 · 2022: 1 · 2021: 1
- FWCI
- 2.17
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Frontiers in Research Metrics and Analytics (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- Open Philanthropy Project (May 20, 2019)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › scientometrics and bibliometrics research
- Social Sciences › Decision Sciences › Information Systems and Management › Research, Science, and Academia
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
- Keywords
- research funding agencies, research clusters, citation networks, research portfolio management, National Institutes of Health, National Science Foundation, European Research Council, National Natural Science Foundation of China, Japan Society for the Promotion of Science, computer vision, natural language processing, robotics, research funding allocation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
