CoChat
Sign Up
Cited by 6Research, Science, and Academiascientometrics and bibliometrics researchEthics and Social Impacts of AI

The rise of large language models and the direction and impact of US federal research funding

Open access
Qian, Yifan, Wen, Zhe, Furnas, Alexander C., Bai, Yue, Shao, Erzhuo, Wang, Dashun·2026·Proceedings of the National Academy of Sciences
DOI 10.1073/pnas.2601439123PMC PMC13486522
Open this paper in CoChat — free account

Abstract

Federal research funding shapes the direction, diversity, and impact of the US scientific enterprise. Large language models (LLMs) are rapidly diffusing into scientific practice, holding substantial promise while raising widespread concerns. Despite growing attention to AI use in scientific writing and evaluation, little is known about how the rise of LLMs is reshaping the public funding landscape. Here, we examine LLM involvement at key stages of the federal funding pipeline by combining two complementary data sources: confidential NSF and NIH proposal submissions from two large US R1 universities, including funded, unfunded, and pending proposals, and the full population of publicly released NSF and NIH awards. We find that LLM use rises sharply beginning in 2023 and exhibits a bimodal distribution, indicating a clear split between minimal and substantive use. Across both private submissions and public awards, higher LLM involvement is consistently associated with lower semantic distinctiveness, positioning projects closer to recently funded work within the same agency. The consequences of this shift are agency-dependent. LLM use is positively associated with proposal success and higher early-stage publication output at NIH, whereas no comparable associations are observed at NSF. Notably, the productivity gains at NIH are concentrated in nonhit papers rather than the most highly cited work. Together, these findings provide large-scale evidence that the rise of LLMs is reshaping how scientific ideas are positioned, selected, and translated into publicly funded research, with implications for portfolio governance, research diversity, and the long-run impact of science.

AI snapshot

Written from the paper’s full text.

TL;DR

Using confidential NSF/NIH proposal submissions from two large US R1 universities plus the full population of public NSF/NIH awards, this paper finds that LLM use rose sharply after late 2022 and is associated with proposals positioned closer to recently funded work. At NIH, higher LLM involvement is linked to greater funding success and more publications, while at NSF no such relationships are observed.

Key findings

  • LLM involvement (α) was stable before late 2022 and rose sharply beginning in 2023 across private NSF/NIH submissions and public NSF/NIH awards; individual-grant α from 2023–2025 is bimodal, with one mode near zero and another around 10–15%.
  • Higher α is consistently associated with lower semantic distinctiveness across all four datasets: moving from the 25th to 75th percentile of LLM involvement corresponds to a ∼5-point decrease in distinctiveness percentile for NSF awards and ∼4 points for NIH awards.
  • For NIH submissions, moving from low to high LLM involvement corresponds to a ∼4-percentage-point increase in funding probability; for NSF proposals, no statistically significant association is found.
  • For NIH awards, higher α predicts ∼5% more publications, but the relationship attenuates and is not statistically significant for high-impact outputs (top 5% or top 1% papers); for NSF awards, no significant relationship with publication output is found.

Methods

  • The study combines four datasets: confidential NSF (D1, 1.6K proposals) and NIH (D2, 4.1K proposals) submissions from two large US R1 universities with request start dates 2021–2025, including funded, unfunded, and pending proposals; and public NSF (D3, 57K awards) and NIH (D4, 74K awards) award records from Dimensions over the same period, linked to resulting publications.
  • LLM involvement is estimated from textual traces in abstracts using an established detection method, with corpus-level and grant-level α estimates; semantic distinctiveness is measured as each abstract’s average cosine distance using SPECTER2 embeddings to abstracts funded in the previous year within the same agency, converted to within-year percentiles.
  • Regressions include grant start year, field, and investigator fixed effects, with controls for requested or awarded funding amount; proposal success and publication outputs are analyzed separately for NSF and NIH, with extensive robustness checks across model families, preprocessing choices, field/subagency subsets, and alternative specifications.

Limitations

  • The paper states that the within-investigator design cannot eliminate time-varying strategic choices in when and how investigators use LLMs, so results are conditional correlations, not causal effects; confidential proposal data come from only two large US R1 universities; the measurement strategy captures only textual traces in abstracts, not the full scope of LLM use; publication counts capture a larger share of NIH outputs than NSF outputs; and grants starting in 2023–2024 capture only the first one to two years of postaward output, leaving long-run high-impact outcomes open.
  • The paper also states that institutional accounts for agency differences are speculative, that the analysis documents agency-dependent associations but does not test mechanisms, and that the agency contrast should not be read as evidence of specific channels.
  • Evident from the text: the detection method relies on word-frequency statistics and abstracts alone, so it may not capture LLM use in other proposal sections or non-textual research development activities.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1073/pnas.2601439123

Cites

47 of 47 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

31 of 60 cited sources checked · 13 supported · 0 flagged · accuracy not yet measured

  • Premisecitation

    Recent large-scale evidence shows that the diffusion of LLMs has already been accompanied by sharp increases in scientific production across major preprint platforms.

    “Recent large-scale evidence shows that the diffusion of LLMs has already been accompanied by sharp increases in scientific production across major preprint platforms, underscoring their potential to substantially raise output at scale (16).”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites [16] K. Kusumegi, Scientific production in the era of large language models. Science 390, 1240– 1243 (2025). 41411417 source
  • Premisecitation

    Discovery is constrained by multiple frictions, from data collection and experimental throughput to coordination, implementation, and more, many of which LLM involvement does not directly resolve.

    “Discovery is constrained by multiple frictions, from data collection and experimental throughput to coordination, implementation, and more, many of which LLM involvement does not directly resolve.”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites [38] B. Jones, Artificial intelligence in research and development. National Bureau of Economic Research [Preprint] (2025). https://www.nber.org/papers/w34312 (Accessed 31 October 2025). source

      The paper lists data access, analysis, and laboratory procedures as distinct research tasks (Section 2, page unavailable), and says that gains in a narrow set of tasks need not overcome broader R&D bottlenecks (Section 5.3, page unavailable). It does not specifically assess LLM involvement or establish that LLMs leave the listed coordination and implementation frictions unresolved. (LLM, conf=0.83)

      Experimental — accuracy not yet measured
  • Premisecitation

    LLMs represent a potentially transformative development for scientific work.

    “LLMs represent a potentially transformative development for scientific work (15–21).”
    Quote in the paperNot a substantial claim5 of 7 sources checked
    • cites [15] W. Liang, Quantifying large language model usage in scientific papers. Nat. Hum. Behav. 9, 2599– 2609 (2025). 40760036 source
    • cites [16] K. Kusumegi, Scientific production in the era of large language models. Science 390, 1240– 1243 (2025). 41411417 source
    • cites [17] W. Liang et al., “Monitoring AI-modified content at scale: A case study on the impact of ChatGPT on AI conference peer reviews” in International Conference on Machine Learning (ICML), R. Salakhutdinov, Eds. (Proceedings of Machine Learning Research, 2024).

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [18] W. Liang, The widespread adoption of large language model-assisted writing across society. Patterns 6, 101366 (2025). 41472824 PMC12745980 source

      The Discussion section, “Cross-sector adoption of LLM-assisted writing” (page unavailable), describes LLMs as part of a broader transformation in writing and notes prior findings of widespread adoption among academic researchers. That supports a potentially transformative role in scientific writing, but this paper does not address scientific work more broadly. (LLM, conf=0.78)

      Experimental — accuracy not yet measured
    • cites [19] D. Kobak, R. González-Márquez, E. Á. Horvát, J. Lause, Delving into LLM-assisted writing in biomedical publications through excess vocabulary. Sci. Adv. 11, eadt3813 (2025). 40601754 PMC12219543 source
    • cites [20] J. Liu, Y. He, Z. Zheng, Y. Bu, C. Ni, AI-assisted writing is growing fastest among non-English-speaking and less established scientists. arXiv [Preprint] (2025). https://arxiv.org/abs/2511.15872 (Accessed 19 November 2025). source
    • cites [21] H. Bao, M. Sun, M. Teplitskiy, Where there’s a will there’s a way: ChatGPT is used more for science in countries where it is prohibited. Quant. Sci. Stud. 6, 716– 731 (2025). source

      The Conclusion (page not specified) reports ChatGPT use in scientific writing and suggests it may help scientists produce more outputs. It does not establish a transformative effect on scientific work more broadly; it reports only modest attention benefits and no statistically significant citation or journal-placement benefits. (LLM, conf=0.78)

      Experimental — accuracy not yet measured
  • Premisecitation

    The federal funding process plays a central role in shaping the direction, diversity, and long-run impact of the US scientific enterprise and the global research landscape.

    “Thus, the federal funding process plays a central role in shaping the direction, diversity, and long-run impact of the US scientific enterprise and—given America’s scientific leadership—the global research landscape as well (3–10).”
    Quote in the paperSubstantial claim4 of 8 sources checked
    • cites [3] D. Li, P. Azoulay, B. N. Sampat, The applied value of public investments in biomedical research. Science 356, 78– 81 (2017). 28360137 source
    • cites [4] E. Galkina Cleary, J. M. Beierlein, N. S. Khanuja, L. M. McNamee, F. D. Ledley, Contribution of NIH funding to new drug approvals 2010–2016. Proc. Natl. Acad. Sci. U.S.A. 115, 2329– 2334 (2018). 29440428 PMC5878010 source

      The Discussion reports that NIH-funded basic research contributed to all FDA-approved new drugs studied from 2010–2016 and warns that funding cuts could delay future drugs. That supports a central, long-run role for federal funding in biomedical innovation, but the paper does not establish effects on the diversity of US science or on the global research landscape. (LLM, conf=0.85)

      Experimental — accuracy not yet measured
    • cites [5] P. Azoulay, J. S. Graff Zivin, D. Li, B. N. Sampat, Public R&D investments and private-sector patenting: Evidence from NIH funding rules. Rev. Econ. Stud. 86, 117– 152 (2019). 31662587 PMC6818650 source

      The Introduction and Conclusion (pages not provided) show that NIH funding affects the direction and output of US biomedical innovation: funding in a research area increases subsequent private-sector patenting, including patents across disease areas. The paper does not establish that federal funding shapes the diversity or long-run impact of the US scientific enterprise as a whole, or the global research landscape. (LLM, conf=0.86)

      Experimental — accuracy not yet measured
    • cites [6] L. Fleming, H. Greene, G. Li, M. Marx, D. Yao, Government-funded research increasingly fuels innovation. Science 364, 1139– 1141 (2019). 31221848 source
    • cites [7] Y. Yin, Y. Dong, K. Wang, D. Wang, B. F. Jones, Public use and public funding of science. Nat. Hum. Behav. 6, 1344– 1350 (2022). 35798885 source

      The Abstract calls public investment a central feature of the scientific ecosystem, and the Results and Discussion describe funding decisions and substantial differences in funding across research fields (pages not available in our copy). These passages support the importance of funding to the US scientific enterprise, but they do not establish that the federal funding process shapes scientific diversity or long-run impact, or that its effects extend to the global research landscape. (LLM, conf=0.84)

      Experimental — accuracy not yet measured
    • cites [8] P. Azoulay, M. Clancy, D. Li, B. N. Sampat, What if NIH had been 40% smaller? Science 389, 1303– 1305 (2025). 40997163 source
    • cites [9] A. C. Furnas, N. Fishman, L. Rosenstiel, D. Wang, Partisan disparities in the funding of science in the United States. Science 389, 1195– 1200 (2025). 40966352 source
    • cites [10] Y. Wang et al., Funding the frontier: Visualizing the broad impact of science and science funding. arXiv [Preprint] (2025). https://arxiv.org/abs/2509.16323 (Accessed 19 September 2025). source

      The Introduction (p. 2) links science funding to scientific and societal impact, and the Discussion (p. 19) says funding models can shape the trajectory of science and innovation. The paper discusses federal funding portfolios and global comparisons, but it does not establish that the US federal funding process shapes the diversity of US science or, through US scientific leadership, the global research landscape. (LLM, conf=0.83)

      Experimental — accuracy not yet measured
  • Premisecitation

    Recent large-scale evidence shows that LLM diffusion has been accompanied by sharp increases in manuscript production.

    “recent large-scale evidence that LLM diffusion has been accompanied by sharp increases in manuscript production (16).”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites [16] K. Kusumegi, Scientific production in the era of large language models. Science 390, 1240– 1243 (2025). 41411417 source
  • May combine two claims. 42
  • Classified as probably not a substantial claim. 21

  • May combine two claims.

    Premisecitation

    Decisions about allocation of taxpayer dollars influence which scientific ideas receive support, which fields advance, and which investigators are able to initiate and sustain research programs.

    “Through competitive grant programs at agencies such as the NSF and the NIH, decisions about the allocation of taxpayer dollars influence which scientific ideas receive support, which fields advance, and which investigators are able to initiate and sustain research programs (3–10).”
    Quote in the paperSubstantial claim4 of 8 sources checked
    • cites [3] D. Li, P. Azoulay, B. N. Sampat, The applied value of public investments in biomedical research. Science 356, 78– 81 (2017). 28360137 source
    • cites [4] E. Galkina Cleary, J. M. Beierlein, N. S. Khanuja, L. M. McNamee, F. D. Ledley, Contribution of NIH funding to new drug approvals 2010–2016. Proc. Natl. Acad. Sci. U.S.A. 115, 2329– 2334 (2018). 29440428 PMC5878010 source

      The Discussion (page unavailable) says NIH funding supports investigator-initiated basic research and that cuts could slow research and delay new drugs. This supports the general importance of public funding, but the paper does not establish how allocation decisions determine which investigators can sustain programs or which fields advance; it also says NSF funding was outside its analysis. (LLM, conf=0.83)

      Experimental — accuracy not yet measured
    • cites [5] P. Azoulay, J. S. Graff Zivin, D. Li, B. N. Sampat, Public R&D investments and private-sector patenting: Evidence from NIH funding rules. Rev. Econ. Stud. 86, 117– 152 (2019). 31662587 PMC6818650 source

      The Empirical strategy section (page unavailable) explains that NIH funding rankings determine which applications and research areas receive support. The Grant-level Patent Match section (page unavailable) describes grants for new projects and renewals, but the paper does not establish the broader claims about NSF allocations, the advancement of scientific fields, or investigators’ ability to sustain research programs. (LLM, conf=0.82)

      Experimental — accuracy not yet measured
    • cites [6] L. Fleming, H. Greene, G. Li, M. Marx, D. Yao, Government-funded research increasingly fuels innovation. Science 364, 1139– 1141 (2019). 31221848 source
    • cites [7] Y. Yin, Y. Dong, K. Wang, D. Wang, B. F. Jones, Public use and public funding of science. Nat. Hum. Behav. 6, 1344– 1350 (2022). 35798885 source

      The Discussion (page unavailable) says that funding allocation involves decisions by legislators and funding agencies that affect what is funded. The Results (page unavailable) examine how public investment varies across research fields. Our copy does not establish that these decisions determine which fields advance or which investigators can initiate and sustain research programs. (LLM, conf=0.84)

      Experimental — accuracy not yet measured
    • cites [8] P. Azoulay, M. Clancy, D. Li, B. N. Sampat, What if NIH had been 40% smaller? Science 389, 1303– 1305 (2025). 40997163 source
    • cites [9] A. C. Furnas, N. Fishman, L. Rosenstiel, D. Wang, Partisan disparities in the funding of science in the United States. Science 389, 1195– 1200 (2025). 40966352 source
    • cites [10] Y. Wang et al., Funding the frontier: Visualizing the broad impact of science and science funding. arXiv [Preprint] (2025). https://arxiv.org/abs/2509.16323 (Accessed 19 September 2025). source

      The funding-agency discussion (p. 16) describes investment decisions as choices about ideas or people, and the scientific-investment section (p. 9) discusses selecting topics and principal investigators for future grants. These passages support the claim that funding decisions affect which ideas and investigators receive support. The paper does not establish the more specific claims that those decisions determine which fields advance or which investigators can initiate and sustain research programs. (LLM, conf=0.82)

      Experimental — accuracy not yet measured

  • May combine two claims.

    Interpretationcitation

    The productivity patterns align with and qualify recent evidence of large output gains following LLM diffusion in manuscript production.

    “These productivity patterns both align with and qualify recent evidence of large output gains following LLM diffusion in manuscript production (16).”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites [16] K. Kusumegi, Scientific production in the era of large language models. Science 390, 1240– 1243 (2025). 41411417 source

  • May combine two claims.

    Premisecitation

    LLMs are trained on large corpora of existing scientific text, including published papers and public records of funded projects, and may tend to produce high-probability, discipline-typical language.

    “Because LLMs are trained on large corpora of existing scientific text, including published papers and public records of funded projects, they may tend to produce high-probability, discipline-typical language.”
    Quote in the paperSubstantial claim1 of 3 sources checked
    • cites [23] J. Gao, D. Wang, Quantifying the use and potential benefits of artificial intelligence in scientific research. Nat. Hum. Behav. 8, 2281– 2292 (2024). 39394445 source
    • cites [26] Q. Hao, F. Xu, Y. Li, J. Evans, Artificial intelligence tools expand scientists’ impact but contract science’s focus. Nature 649, 1237– 1243 (2026). 41535462 source
    • cites [36] A. R. Doshi, O. P. Hauser, Generative AI enhances individual creativity but reduces the collective diversity of novel content. Sci. Adv. 10, eadn5290 (2024). 38996021 PMC11244532 source

      The Abstract and Discussion report that AI-assisted short stories were more similar to one another and that AI reduced variation in creative outputs, lending limited support to a tendency toward less varied writing. The paper does not establish that LLMs were trained on published scientific papers or funded-project records, or that they produce high-probability, discipline-typical language. (LLM, conf=0.76)

      Experimental — accuracy not yet measured

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    Higher LLM involvement is positively associated with greater writing complexity (Flesch Reading Ease score) in both proposals and awards.

    “Consistent with prior work (16), we find that higher LLM involvement is positively associated with greater writing complexity in both proposals and awards”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites [16] K. Kusumegi, Scientific production in the era of large language models. Science 390, 1240– 1243 (2025). 41411417 source

  • May combine two claims.

    Premisecitation

    When used in proposal preparation, LLMs may enhance fluency and alignment with prevailing norms while reducing rhetorical and topical variance, making proposed projects more closely resemble those that have recently succeeded.

    “When used in proposal preparation, such tools may enhance fluency and alignment with prevailing norms while reducing rhetorical and topical variance, making proposed projects more closely resemble those that have recently succeeded (23, 26, 36).”
    Quote in the paperSubstantial claim1 of 3 sources checked
    • cites [23] J. Gao, D. Wang, Quantifying the use and potential benefits of artificial intelligence in scientific research. Nat. Hum. Behav. 8, 2281– 2292 (2024). 39394445 source
    • cites [26] Q. Hao, F. Xu, Y. Li, J. Evans, Artificial intelligence tools expand scientists’ impact but contract science’s focus. Nature 649, 1237– 1243 (2026). 41535462 source
    • cites [36] A. R. Doshi, O. P. Hauser, Generative AI enhances individual creativity but reduces the collective diversity of novel content. Sci. Adv. 10, eadn5290 (2024). 38996021 PMC11244532 source

      The Abstract and Discussion report that AI-assisted short stories were better written but more similar to one another, supporting the claim’s general fluency-and-reduced-variance point. The Discussion explicitly limits generalization beyond short stories; the paper does not establish effects in proposal preparation, alignment with prevailing norms, or resemblance to recently successful projects (page numbers unavailable). (LLM, conf=0.87)

      Experimental — accuracy not yet measured

  • May combine two claims.

    Findingown data

    Decomposition of aggregate divergence into per-word contribution scores reveals that the LLM signal is driven by a concentrated set of evaluative and elaborative terms, while human-written text is characterized by simpler, more functional vocabulary.

    “This decomposition reveals that the LLM signal is driven by a concentrated set of evaluative and elaborative terms, while human-written text is characterized by simpler, more functional vocabulary”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites [41] R. J. Gallagher, Generalized word shift graphs: A method for visualizing and explaining pairwise comparisons between texts. EPJ Data Sci. 10, 4 (2021). source

      Split into 2 atoms — 0 confirmed, 2 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.

      Experimental — accuracy not yet measured

  • Classified as probably not a substantial claim.

    Premisecitation

    Prior work has documented large productivity increases at the manuscript level.

    “recent large-scale evidence that LLM diffusion has been accompanied by sharp increases in manuscript production (16).”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites [16] K. Kusumegi, Scientific production in the era of large language models. Science 390, 1240– 1243 (2025). 41411417 source

  • May combine two claims.

    Premisereasoning

    LLM use could shift the balance between exploration and exploitation in scientific funding, pulling proposals toward the center of existing funding patterns and potentially narrowing the diversity of ideas that are submitted for or receive public support.

    “In this way, LLM use could shift the balance between exploration and exploitation in scientific funding, pulling proposals toward the center of existing funding patterns and potentially narrowing the diversity of ideas that are submitted for or receive public support, even as it may lower the barriers to producing competitive applications (30, 37).”
    Quote in the paperSubstantial claim1 of 2 sources checked
    • cites [30] L. Liu, N. Dehmamy, J. Chown, C. L. Giles, D. Wang, Understanding the onset of hot streaks across artistic, cultural, and scientific careers. Nat. Commun. 12, 5392 (2021). 34518529 PMC8438033 source

      Split into 2 atoms — 0 confirmed, 2 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.

      Experimental — accuracy not yet measured
    • cites [37] J. G. March, Exploration and exploitation in organizational learning. Organ. Sci. 2, 71– 87 (1991). source

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing). May combine two claims.

    Premisecitation

    By dramatically reducing the cost of drafting, synthesizing, and revising complex text, LLMs may help scientists articulate ideas more clearly, navigate the growing burden of knowledge, and explore unfamiliar domains more efficiently.

    “By dramatically reducing the cost of drafting, synthesizing, and revising complex text, LLMs may help scientists articulate ideas more clearly, navigate the growing burden of knowledge, and explore unfamiliar domains more efficiently (28, 29).”
    Quote in the paperNot a substantial claim1 of 2 sources checked
    • cites [28] B. F. Jones, The burden of knowledge and the “death of the renaissance man’’: Is innovation getting harder? Rev. Econ. Stud. 76, 283– 317 (2009). source
    • cites [29] R. Hill, The pivot penalty in research. Nature 642, 999– 1006 (2025). 40437087 PMC12222024 source

      The Discussion (page unavailable) describes a growing difficulty in moving beyond established expertise, but does not address whether LLMs help scientists overcome it. Searches for LLMs and for help with drafting, synthesis, and revision found no passage supporting the attributed claim. (LLM, conf=0.76)

      Experimental — accuracy not yet measured

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Existing work has documented that LLM adoption is associated with changes in productivity and referencing behavior in manuscripts and published papers, and more broadly that AI-augmented research can contract the collective focus of science.

    “Existing work has documented that LLM adoption is associated with changes in productivity and referencing behavior in manuscripts and published papers, and more broadly that AI-augmented research can contract the collective focus of science (16, 26).”
    Quote in the paperNot a substantial claim1 of 2 sources checked
    • cites [16] K. Kusumegi, Scientific production in the era of large language models. Science 390, 1240– 1243 (2025). 41411417 source

      Split into 2 atoms — 0 confirmed, 1 partially supported, 0 disputed, 1 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.

      Experimental — accuracy not yet measured
    • cites [26] Q. Hao, F. Xu, Y. Li, J. Evans, Artificial intelligence tools expand scientists’ impact but contract science’s focus. Nature 649, 1237– 1243 (2026). 41535462 source

  • May combine two claims.

    Premisecitation

    Dimensions is a large-scale, integrated research information platform that uniquely enables systematic linking of research grants to their resulting publications through funding acknowledgment data, making it particularly well suited for studying downstream scientific outputs of funded grants.

    “Dimensions (https://www.dimensions.ai/) is a large-scale, integrated research information platform that uniquely enables systematic linking of research grants to their resulting publications through funding acknowledgment data, making it particularly well suited for studying downstream scientific outputs of funded grants.”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites [42] C. Herzog, D. Hook, S. Konkiel, Dimensions: Bringing down barriers between scientometricians and data. Quant. Sci. Stud. 1, 387– 395 (2020). source

      Split into 2 atoms — 0 confirmed, 2 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.

      Experimental — accuracy not yet measured

  • May combine two claims.

    Findingown data

    For NSF awards, moving from 25th to 75th percentile LLM involvement corresponds to a ∼5-point decrease in distinctiveness percentile; for NIH awards, ∼4 points.

    “For example, for NSF awards, moving from low LLM involvement (25th percentile) to high LLM involvement (75th percentile) corresponds to a ∼5-point decrease in the distinctiveness percentile; for NIH awards, the decrease is ∼4 points.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    NIH exhibits the same temporal pattern of LLM involvement as NSF but with overall lower average levels.

    “Over this period, NIH exhibits the same temporal pattern as NSF but with overall lower average levels of LLM involvement.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: Public grant abstracts with start dates in 2021 preceded widespread LLM use and thus represent human-written text without LLM involvement.
    • Rests on an assumption: Prompting GPT-3.5-turbo-0125 to rewrite abstracts simulates LLM involvement in proposal language.
    • Rests on an assumption: Textual traces in grant abstracts can estimate LLM use in grant proposals and awards.

  • May combine two claims.

    Findingown data

    LLM use (α) in NSF and NIH grant proposals and awards was relatively stable before late 2022 and rose sharply beginning in 2023, coinciding with the widespread availability of ChatGPT, across all four datasets.

    “Across all four datasets, α is relatively stable prior to late 2022, and then rises sharply beginning in 2023, coinciding with the widespread availability of ChatGPT.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: Public grant abstracts with start dates in 2021 preceded widespread LLM use and thus represent human-written text without LLM involvement.
    • Rests on an assumption: Prompting GPT-3.5-turbo-0125 to rewrite abstracts simulates LLM involvement in proposal language.
    • Rests on an assumption: Textual traces in grant abstracts can estimate LLM use in grant proposals and awards.

  • May combine two claims.

    Findingcitation

    Descriptive analyses of older cohorts suggest that early windows appear to capture a meaningful share of a grant's eventual output, and that early output is overall predictive of longer-run output.

    “Descriptive analyses of older cohorts suggest that such early windows appear to capture a meaningful share of a grant’s eventual output, and that early output is overall predictive of longer-run output”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    In the confidential submission data, LLM involvement appears unrelated to proposal success at NSF but positively associated with funding at NIH.

    “In the confidential submission data, LLM involvement appears unrelated to proposal success at NSF but positively associated with funding at NIH.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    In the public award data, there is no relationship between LLM involvement and publication output at NSF, whereas at NIH, higher LLM involvement predicts greater publication output, concentrated in nonhit papers rather than the most highly cited work.

    “In the public award data, we similarly detect no relationship between LLM involvement and publication output at NSF, whereas at NIH, higher LLM involvement predicts greater publication output, though concentrated in nonhit papers rather than the most highly cited work.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: Publication counts are a valid measure of downstream scientific output.

  • May combine two claims.

    Findingown data

    Varying tokenization scheme, stopword handling, and punctuation normalization across nine alternative pipelines yields α estimates highly correlated with the baseline, and main results broadly consistent under alternative pipelines.

    “we systematically evaluate its sensitivity to preprocessing choices by varying the tokenization scheme, stopword handling, and punctuation normalization across nine alternative pipelines. The resulting α estimates remain highly correlated with the baseline, and the main results are broadly consistent under alternative pipelines”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    LLM adoption is associated with how scientific ideas are positioned relative to recently funded work, whether proposals clear the funding threshold, and the publication output that funded projects go on to produce.

    “we provide systematic evidence that LLM adoption is associated with i) how scientific ideas are positioned relative to recently funded work, ii) whether proposals clear the funding threshold, and iii) the publication output that funded projects go on to produce.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Main results are robust to varying text-length thresholds, single- versus multi-investigator, and standardizing predictors.

    “Our results are further robust to varying text-length thresholds, single- versus multi-investigator, and standardizing predictors”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    The publication output gains observed are modest and context-dependent, absent at NSF and concentrated in nonhit papers at NIH.

    “The publication output gains we observe are modest and context-dependent, absent at NSF and concentrated in nonhit papers at NIH”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Removing all AI-related terms identified in prior work at every stage of the estimation pipeline yields measures highly correlated with original estimates, and all main regression results are robust to this exclusion.

    “we remove all AI-related terms identified in prior work (23) at every stage of the estimation pipeline and re-estimate α. The resulting measures remain highly correlated with the original estimates, and all main regression results are robust to this exclusion”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [23] J. Gao, D. Wang, Quantifying the use and potential benefits of artificial intelligence in scientific research. Nat. Hum. Behav. 8, 2281– 2292 (2024). 39394445 source

  • May combine two claims.

    Interpretationown data

    In the federal funding context, LLM adoption appears to translate into additional publications primarily at NIH, and predominantly among nonhit papers, while showing no detectable relationship with publication output at NSF.

    “In the federal funding context, where projects are selected through peer review and outputs reflect execution constraints and portfolio objectives rather than writing speed alone, LLM adoption appears to translate into additional publications primarily at NIH, and predominantly among nonhit papers, while showing no detectable relationship with publication output at NSF.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Premisecitation

    Recent work shows that AI can expand individual scientists' productivity and impact while simultaneously contracting the collective focus of science.

    “Recent work highlights this tension more broadly, showing that AI can expand individual scientists’ productivity and impact while simultaneously contracting the collective focus of science, underscoring the possibility that gains at the individual level may coexist with reduced diversity at the system level (26).”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [26] Q. Hao, F. Xu, Y. Li, J. Evans, Artificial intelligence tools expand scientists’ impact but contract science’s focus. Nature 649, 1237– 1243 (2026). 41535462 source

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    NSF has emphasized investigator responsibility for accuracy and originality and encouraged disclosure of generative AI use.

    “NSF has emphasized investigator responsibility for accuracy and originality and encouraged disclosure of generative AI use (see Notice to Research Community).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Notice to Research Community

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Study design and conductown data

    The analysis exploits within-investigator variation in LLM involvement, reporting conditional correlations within investigators, not causal estimates.

    “Note that this analysis exploits within-investigator variation in LLM involvement, reporting conditional correlations within investigators, not causal estimates.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationown data

    The consequences of LLM adoption for proposal selection are agency-dependent: LLM use does not systematically advantage proposals at NSF, but is associated with a higher likelihood of being funded at NIH, even for the same investigators.

    “Overall, these results indicate that the consequences of LLM adoption for proposal selection are agency-dependent: While LLM use does not systematically advantage proposals at NSF, it is associated with a higher likelihood of being funded at NIH, even for the same investigators.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationown data

    As LLM use rises, the proposal and award portfolio shifts toward the center of recently funded work, providing empirical evidence of a consistent reduction in semantic distinctiveness in federal research funding across both agencies.

    “These findings suggest that as LLM use rises, the proposal and award portfolio shifts toward the center of recently funded work, providing empirical evidence of a consistent reduction in semantic distinctiveness in federal research funding that is already occurring across both agencies.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Study design and conductown data

    Estimates are based on grants starting in 2023-2024 and therefore capture only the first one to two years of postaward publication output, reflecting early-stage rather than long-run productivity.

    “We emphasize that these estimates are based on grants starting in 2023–2024 and therefore capture only the first one to two years of postaward publication output, reflecting early-stage rather than long-run productivity.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationown data

    The downstream consequences of LLM adoption are agency-dependent; at NIH, higher α is associated with more follow-on publications, but the additional output is concentrated in nonhit papers rather than the most highly cited work.

    “Taken together, these results suggest that the downstream consequences of LLM adoption are also agency-dependent. At NIH, higher α is associated with more follow-on publications, but the additional output is concentrated in nonhit papers rather than the most highly cited work.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Study design and conductown data

    The regressions include grant start year, field, and investigator (PI and co-PI) fixed effects, along with controls for requested or awarded funding amount.

    “estimating regressions that include grant start year, field, and investigator (PI and co-PI) fixed effects, along with controls for requested or awarded funding amount”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Premisecitation

    The detection method used is an established method proposed by Liang et al. (15).

    “we leverage an established detection method proposed by Liang et al. (15).”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Liang et al. [15] W. Liang, Quantifying large language model usage in scientific papers. Nat. Hum. Behav. 9, 2599– 2609 (2025). 40760036 source

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data

    Results are broadly consistent by subagency.

    “We also conduct analogous robustness checks by subagency using the public award data and again find broadly consistent patterns”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationown data

    LLM adoption has rapidly become a detectable feature of federal research proposals and awards but remains highly uneven across them.

    “These results indicate that LLM adoption has rapidly become a detectable feature of federal research proposals and awards, but it remains highly uneven across them.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationown data

    The distinctiveness measure is not simply driven by LLM-mediated linguistic changes, and the evidence is consistent with convergence in substantive positioning rather than surface language alone.

    “This finding suggests that our distinctiveness measure is not simply driven by LLM-mediated linguistic changes, and that the evidence is consistent with convergence in substantive positioning rather than surface language alone.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductreasoning

    These fixed effects absorb time-invariant heterogeneity at the field and investigator levels while accounting for common shocks across grant start years.

    “These fixed effects absorb time-invariant heterogeneity at the field and investigator levels while accounting for common shocks across grant start years.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationreasoning

    The early effects of generative AI in science may arise less from accelerating the execution of research than from reshaping how ideas are articulated, packaged, and aligned with prevailing norms, while the harder bottlenecks of project design, coordination, and implementation remain.

    “the early effects of generative AI in science may arise less from accelerating the execution of research than from reshaping how ideas are articulated, packaged, and aligned with prevailing norms, while the harder bottlenecks of project design, coordination, and implementation remain”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationreasoning

    The benefits of generative AI may accrue first and most strongly where writing is the binding constraint, whereas their effects on the composition and ultimate impact of publicly funded research also hinge on institutional incentives and execution constraints.

    “The benefits of generative AI may accrue first and most strongly where writing is the binding constraint, whereas their effects on the composition and ultimate impact of publicly funded research also hinge on institutional incentives and execution constraints.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Study design and conductreasoning

    Within-investigator comparisons hold constant time-invariant differences across investigators, but cannot eliminate time-varying strategic choices in when and how investigators use LLMs.

    “While within-investigator comparisons hold constant time-invariant differences across investigators, they cannot eliminate time-varying strategic choices in when and how investigators use LLMs.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Study design and conductnone given

    Publication counts capture a larger share of NIH outputs than NSF outputs, which often include software, datasets, instrumentation, and training.

    “publication counts capture a larger share of NIH outputs than NSF outputs, which often include software, datasets, instrumentation, and training.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: Publication counts are a valid measure of downstream scientific output.

  • May combine two claims.

    Interpretationreasoning

    A portfolio that is closer to recent funding patterns implies reduced exploration in the idea landscape, which matters for public funders explicitly tasked with sustaining high-variance discovery, with implications for long-run impact and sustainability of science.

    “But it also implies reduced exploration in the idea landscape, which matters for public funders explicitly tasked with sustaining high-variance discovery, with implications for long-run impact and sustainability of science.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: Semantic distinctiveness relative to recently funded work measures exploration or novelty in the idea landscape.

  • May combine two claims.

    Interpretationreasoning

    A meaningful share of the productivity gains associated with LLM adoption may operate through accelerating communication-the speed and ease with which researchers articulate, package, and iterate on ideas-rather than accelerating execution, which remains constrained by project design, coordination, data collection, and other bottlenecks that LLM-enabled writing does not directly resolve.

    “A meaningful share of the productivity gains associated with LLM adoption may operate through accelerating communication—the speed and ease with which researchers articulate, package, and iterate on ideas—rather than accelerating execution, which remains constrained by project design, coordination, data collection, and other bottlenecks that LLM-enabled writing does not directly resolve.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductreasoning

    Our within-investigator, across-grant design leverages variation in LLM involvement across a given investigator's own proposals and awards, allowing us to examine whether more intensive LLM use within the same investigator is associated with differences in semantic positioning, proposal success, and downstream output.

    “our design leverages variation in LLM involvement across a given investigator’s own proposals and awards, allowing us to examine whether more intensive LLM use within the same investigator is associated with differences in semantic positioning, proposal success, and downstream output.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Premisereasoning

    We lack systematic evidence on how LLM adoption enters the funding pipeline, how it relates to proposal evaluation, and how it shapes the research that ultimately receives public support.

    “As a result, we lack systematic evidence on how LLM adoption enters the funding pipeline, how it relates to proposal evaluation, and how it shapes the research that ultimately receives public support.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductown data

    The analysis documents agency-dependent associations but does not test the mechanisms behind them.

    “Our analysis documents agency-dependent associations but does not test the mechanisms behind them”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Study design and conductreasoning

    Our measurement strategy-textual traces of LLM involvement in proposal abstracts-captures only one dimension of how LLMs may be used in proposal preparation.

    “our measurement strategy—textual traces of LLM involvement in proposal abstracts—captures only one dimension of how LLMs may be used in proposal preparation.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: Textual traces of LLM involvement in proposal abstracts can be detected from word-frequency statistics and reflect actual LLM involvement in proposal preparation.
    • Rests on an assumption: LLM involvement in proposal abstracts reflects LLM involvement in full proposal preparation.

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationreasoning

    Because federal grants deploy public funds, the rise of LLMs in proposals has implications not only for scientific efficiency but also for accountability and trust.

    “because federal grants deploy public funds, the rise of LLMs in proposals has implications not only for scientific efficiency but also for accountability and trust.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Study design and conductreasoning

    Our estimates should be interpreted as measuring how LLMs shape the articulation and positioning of ideas in proposal narratives, not the full scope of LLM use in research development.

    “our estimates should be interpreted as measuring how LLMs shape the articulation and positioning of ideas in proposal narratives, not the full scope of LLM use in research development.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: Textual traces of LLM involvement in proposal abstracts can be detected from word-frequency statistics and reflect actual LLM involvement in proposal preparation.
    • Rests on an assumption: LLM involvement in proposal abstracts reflects LLM involvement in full proposal preparation.

  • May combine two claims.

    Study design and conductown data

    By directly observing confidential submissions alongside the population of funded awards, we can separate proposal preparation from postaward editing and examine selection outcomes, not just the characteristics of funded projects.

    “By directly observing confidential submissions alongside the population of funded awards, we can separate proposal preparation from postaward editing and examine selection outcomes, not just the characteristics of funded projects.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Study design and conductnone given

    The data collection protocol on private NSF and NIH proposals from two US universities was reviewed and approved by the Institutional Review Board of Northwestern University (IRB No. STU00222200).

    “The data collection protocol on private NSF and NIH proposals from two US universities was reviewed and approved by the Institutional Review Board of Northwestern University (IRB No. STU00222200).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Study design and conductown data

    The confidential proposal data come from two large US R1 universities, and two institutions cannot capture the full diversity of US research settings.

    “the confidential proposal data come from two large US R1 universities. Although these data provide rare access to funded, unfunded, and pending proposals at the point of submission, two institutions cannot capture the full diversity of US research settings.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Study design and conductnone given

    When a paper has both a preprint and a published version, retaining only the published version avoids double counting.

    “When a paper has both a preprint and a published version, we retain only the published version to avoid double counting.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Study design and conductnone given

    The inclusion of unfunded proposals enables analyses that are not possible using public data alone.

    “The inclusion of unfunded proposals enables analyses that are not possible using public data alone.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Study design and conductreasoning

    The relationships reported should be interpreted as conditional correlations within investigators, not causal effects.

    “The relationships we report should therefore be interpreted as conditional correlations within investigators, not causal effects.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductnone given

    The private data capture idea articulation and selection at submission, including proposals that were never funded, while the public data capture postselection outcomes, including funded project descriptions and subsequent publications.

    “The private data capture idea articulation and selection at submission, including proposals that were never funded, while the public data capture postselection outcomes, including funded project descriptions and subsequent publications.”
    Quote in the paperSubstantial claim
55 of 55

  • The paper relies on this without support.

    Proposal texts, especially unfunded submissions, are typically confidential, making this stage of the funding pipeline difficult to study systematically at scale.

    “Yet proposal texts, especially unfunded submissions, are typically confidential, making this stage of the funding pipeline difficult to study systematically at scale.”

  • The paper relies on this without support.

    Public grant abstracts with start dates in 2021 preceded widespread LLM use and thus represent human-written text without LLM involvement.

    “Using public grant abstracts with start dates in 2021, which preceded the widespread use of LLMs, we estimate the word distribution of human-written text.”

  • The paper relies on this without support.

    Prompting GPT-3.5-turbo-0125 to rewrite abstracts simulates LLM involvement in proposal language.

    “We also estimate a corresponding word distribution of LLM-generated text by prompting OpenAI’s GPT-3.5-turbo-0125 model to rewrite the same abstracts, simulating LLM involvement in proposal language.”

  • The paper relies on this without support.

    Textual traces in grant abstracts can estimate LLM use in grant proposals and awards.

    “To estimate LLM use in federal grant proposals and awards, we analyze textual traces in their abstracts.”

  • The paper relies on this without support.

    Textual traces of LLM involvement in proposal abstracts can be detected from word-frequency statistics and reflect actual LLM involvement in proposal preparation.

    “because the detection method relies on word-frequency statistics”
    “our measurement strategy—textual traces of LLM involvement in proposal abstracts—captures only one dimension of how LLMs may be used in proposal preparation.”

  • The paper relies on this without support.

    Publication counts are a valid measure of downstream scientific output.

    “we construct measures of total publication output and high-impact output based on field- and year-normalized citation percentiles”
    “publication counts capture a larger share of NIH outputs than NSF outputs, which often include software, datasets, instrumentation, and training.”

  • The paper relies on this without support.

    Semantic distinctiveness relative to recently funded work measures exploration or novelty in the idea landscape.

    “higher LLM involvement is consistently associated with lower semantic distinctiveness: Proposals with greater LLM traces are positioned closer to recently funded work within the same agency.”
    “it also implies reduced exploration in the idea landscape”

  • The paper relies on this without support.

    Prior work has largely studied the finished outputs of science.

    “whereas prior work has largely studied the finished outputs of science”

  • The paper relies on this without support.

    Unfunded proposals are rarely observable at scale.

    “Because unfunded proposals are rarely observable at scale, most prior work has had to infer behavior from public records of funded projects or published papers.”

  • The paper relies on this without support.

    Prior analyses largely rely on comparisons across researchers, across papers, or across AI versus non-AI work.

    “But these analyses largely rely on comparisons across researchers, across papers, or across AI versus non-AI work.”

  • The paper relies on this without support.

    LLM involvement in proposal abstracts reflects LLM involvement in full proposal preparation.

    “our measurement strategy—textual traces of LLM involvement in proposal abstracts—captures only one dimension of how LLMs may be used in proposal preparation.”
11 of 11

Cites

Flagged first, then by citations

Research works printed but not linked: 1

  1. W. Liang et al. “Monitoring AI-modified content at scale: A case study on the impact of ChatGPT on AI conference peer reviews” in International Conference on Machine Learning (ICML) R. Salakhutdinov Eds. (Proceedings of Machine Learning Research 2024).

Cited by

Sorted by citations

Authors
Qian, Y., Wen, Z., Furnas, A. C., Bai, Y., Shao, E., & Wang, D.
Affiliations
Northwestern University (US)
Corresponding author
Wang, Dashun
ORCID
Venue
Proceedings of the National Academy of Sciences
Publisher
National Academy of Sciences
Published
2026-08-11
Volume
123
Issue
33
Pages
e2601439123-e2601439123
ISSN
  • 0027-8424 (ISSN-L)
  • 1091-6490
DOI
10.1073/pnas.2601439123
PMC
PMC13486522
OpenAlex
W7202196341
Semantic Scholar
7c5e8d9204022ff4c43922861b57b0b03412e24c
Cited by
5
References
39
Citations by year
2026: 5
FWCI
37.61
Type
journal article
Language
en
Open access
Licence
Locations
  • Proceedings of the National Academy of Sciences (publishedVersion, cc-by-nc-nd)
  • PubMed (publishedVersion)
  • PubMed Central (submittedVersion, cc-by-nc-nd)
  • arXiv (Cornell University)
Funders
National Science Foundation (2404035)
Topics
  • Social Sciences › Decision Sciences › Information Systems and Management › Research, Science, and Academia
  • Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › scientometrics and bibliometrics research
  • Social Sciences › Safety Research › Ethics and Social Impacts of AI
Keywords
large language models, federal research funding, NSF, National Institutes of Health, research proposal submission, diversity in research, AI in scientific writing, publication output
MeSH terms
Large Language Models, Financing, Government, Humans, Research Support as Topic, United States

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-23.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-23.

Cited sources are checked by gpt-6-sol.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review