Database of NIH grants using machine-learned categories and graphical clustering
Open accessThe DOI matches its Crossref record.
Abstract
framework that is based on scientific research rather than NIH administrative and categorical designations. We found that topic-based categories are not strictly associated with the missions of individual Institutes but instead cut across the NIH, albeit in varying proportions consistent with each Institute’s distinct mission (Supplementary Table 1). The graphical map layout (Fig. 1) shows a global research structure that is logically coherent but only loosely related to Institute organization (Supplementary Table 1). We describe four example use cases (Supplementary Data). First, we show a query using an algorithm-derived category relevant to angiogenesis (Supplementary Fig. 1). Unlike standard keywordbased searches, this type of query allows retrieval of grants that are truly focused on a particular research area. In addition, the resulting graphical clusters reveal clear patterns in the relationships between the retrieved grants and the multiple Institutes funding this research. Second, we examine an NIH peer review study section. The database categories and clusters clarify the complex relationship between the NIH Institutes and the centralized NIH peer review system, which is distinct and independent from the Institutes. Third, we show an analysis of the NIH RCDC category ‘sleep research’ in conjunction with the database topics, the latter Database of NIH grants using machine-learned categories and graphical clustering
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cited by
A Data-Driven Analysis of Workers' Earnings on Amazon Mechanical Turk
Hara, K., Adams, A., Milland, K., Savage, S., Callison-Burch, C., & Bigham, J. P. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3174023 · Cited by 497
“methods are often used to categorize a set of unlabeled documents into a finite number of coherent topic clusters [7,27,61], in turn helping researchers to navigate a large corpus (HIT documents in our case) [17,18,24,52].”
MethodTermite: visualization techniques for assessing textual topic models
Chuang, J., Manning, C. D., & Heer, J. (2012) · Proceedings of the International Working Conference on Advanced Visual Interfaces · doi:10.1145/2254556.2254572 · Cited by 420
Hu, Y., Boyd-Graber, J., Satinoff, B., & Smith, A. (2014) · Machine Learning · doi:10.1007/s10994-013-5413-0 · Cited by 338
Computational psychotherapy research: scaling up the evaluation of patient–provider interactions
Imel, Z. E., Steyvers, M., & Atkins, D. C. (2015) · Psychotherapy · doi:10.1037/a0036841 · Cited by 146
“There is recent work using these models to explore the structure of National Institute of Health grant applications (Talley et al., 2011), publications from the Proceedings of the National Academy of Sciences (Griffiths & Steyvers, 2004), articles from the New York Times (Rubin, Chambers, Smyth, &…”
MethodSmall decisions with big impact on data analytics
Diesner, J. (2015) · Big Data & Society · doi:10.1177/2053951715617185 · Cited by 32
“…to enhance methods common in their fields, e.g. close reading and text coding, as well as advanced social data analytics and visualization techniques, such as topic modeling, sentiment analysis, and clustering (Abello et al., 2012; Talley et al., 2011; Underwood et al., 2013; Wang et al., 2007).”
Method
Metadata
- Authors
- Talley, E. M., Newman, D., Mimno, D., Herr, B. W., Wallach, H. M., Burns, G. A. P. C., Leenders, A. G. M., & McCallum, A.
- Venue
- Nature Methods
- Published
- 2011-05-27
- Volume
- 8
- Issue
- 6
- Pages
- 443-444
- ISSN
- 1548-7091 (ISSN-L)
- 1548-7105
- OpenAlex
- W2024722304
- Cited by
- 85
- References
- 0
- FWCI
- 8.44
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Database of NIH grants using machine-learned categories and graphical clustering. https://app.cochat.ai/papers/10.1038/nmeth.1619
- The paper
- Talley, E. M., Newman, D., Mimno, D., Herr, B. W., Wallach, H. M., Burns, G. A. P. C., Leenders, A. G. M., & McCallum, A. (2011). Database of NIH grants using machine-learned categories and graphical clustering. Nature Methods. https://doi.org/10.1038/nmeth.1619
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.