Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat
Cited by 52

The DOI matches its Crossref record; 15 of the 15 references it cites were found in a registry or an index.

Large language models reduce public knowledge sharing on online Q&A platforms

del Rio-Chanona, R Maria, Laurentsyeva, Nadzeya, Wachs, Johannes · 2024 · PNAS Nexus

Abstract

Large language models (LLMs) are a potential substitute for human-generated data and knowledge resources. This substitution, however, can present a significant problem for the training data needed to develop future models if it leads to a reduction of human-generated content. In this work, we document a reduction in activity on Stack Overflow coinciding with the release of ChatGPT, a popular LLM. To test whether this reduction in activity is specific to the introduction of this LLM, we use counterfactuals involving similar human-generated knowledge resources that should not be affected by the introduction of ChatGPT to such extent. Within 6 months of ChatGPT’s release, activity on Stack Overflow decreased by 25% relative to its Russian and Chinese counterparts, where access to ChatGPT is limited, and to similar forums for mathematics, where ChatGPT is less capable. We interpret this estimate as a lower bound of the true impact of ChatGPT on Stack Overflow. The decline is larger for posts related to the most widely used programming languages. We find no significant change in post quality, measured by peer feedback, and observe similar decreases in content creation by more and less experienced users alike. Thus, LLMs are not only displacing duplicate, low-quality, or beginner-level content. Our findings suggest that the rapid adoption of LLMs reduces the production of public data needed to train them, with significant consequences.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1093/pnasnexus/pgae400

Cites

15 of 15 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 15 references

Research works printed but not linked: 31

  1. Henzinger, 2004, Extracting knowledge from the world wide web, Proc Natl Acad Sci U S A, 101, 5186, 10.1073/pnas.0307528100 · DOI
  2. Hess, 2003, Ideas, artifacts, and facilities: information as a common-pool resource, Law Contemp Probl, 66, 111
  3. Naveed, 2023, 10.48550/arXiv.2307.06435 · DOI · Paper page
  4. Smith, 2023, Astronomia ex machina: a history, primer and outlook on neural networks in astronomy, R Soc Open Sci, 10, 221454, 10.1098/rsos.221454 · DOI
  5. Kanjee, 2023, Accuracy of a generative artificial intelligence model in a complex diagnostic challenge, JAMA, 330, 78, 10.1001/jama.2023.8288 · DOI · Paper page
  6. Guo, 2023, 10.48550/arXiv.2305.18365 · DOI · Paper page
  7. Delile, 2023, 10.48550/arXiv.2306.11174 · arXiv · doi:10.48550/arxiv.2306.11174
  8. Widjojo, 2023, 10.48550/arXiv.2307.10793 · DOI
  9. Chiang, 2023, ChatGPT is a blurry JPEG of the web
  10. Arthur, 1989, Competing technologies, increasing returns, and lock-in by historical events, Econ J, 99, 116, 10.2307/2234208 · DOI · Paper page
  11. David, 1985, Clio and the economics of QWERTY, Am Econ Rev, 75, 332
  12. Stojkoski, 2024, Estimating digital product trade through corporate revenue data, Nat Commun, 15, 5262, 10.1038/s41467-024-49141-z · DOI
  13. Kreitmeir, 2023, 10.48550/arXiv.2304.09339 · DOI
  14. Bilinski, 2018, 10.48550/arXiv.1805.03273 · DOI · Paper page
  15. Eloundou, 2024, GPTs are GPTs: labor market impact potential of LLMs, Science, 384, 1306, 10.1126/science.adj0998 · doi:10.1126/science.adj0998 · Paper page
  16. Schumpeter, 1942, Capitalism, socialism, and democracy
  17. Gaessler, 2023, Training with AI: evidence from chess computers, Strat Manag J, 44, 2724, 10.1002/smj.3512 · DOI
  18. Taleb, 2012, Antifragile: how to live in a world we don’t understand
  19. Wyllie, 2024, 10.48550/arXiv.2403.07857 · arXiv · doi:10.48550/arxiv.2403.07857
  20. Tian-Zheng Wei, 2024, 10.48550/arXiv.2402.10892 · DOI
  21. Page, 2006, Path dependence, Quart J Polit Sci, 1, 87, 10.1561/100.00000006 · DOI · Paper page
  22. Vincent, 2021, A deeper investigation of the importance of Wikipedia links to search engine results, Proc ACM Hum-Comput Inter, 5, 1
  23. Lerner, 2002, Some simple economics of open source, J Ind Econ, 50, 197, 10.1111/1467-6451.00174 · DOI · Paper page
  24. Greenstein, 2014, Digital dark matter and the economic contribution of Apache, Res Policy, 43, 623, 10.1016/j.respol.2014.01.003 · DOI
  25. Xu, 2020, What makes geeks tick? A study of stack overflow careers, Manage Sci, 66, 587, 10.1287/mnsc.2018.3264 · DOI
  26. Evans, 2008, Electronic publication and the narrowing of science and scholarship, Science, 321, 395, 10.1126/science.1150473 · DOI · Paper page
  27. Sandrini, 2023, Generative ai and deceptive news consumption, Econ Lett, 232, 111317, 10.1016/j.econlet.2023.111317 · DOI
  28. Johnson, 2023, Power and progress: our thousand-year struggle over technology and prosperity
  29. Lehdonvirta, 2022, Cloud empires: how digital platforms are overtaking the state and how we can regain control, 10.7551/mitpress/14219.001.0001 · DOI
  30. Kummer, 2020, Unemployment and digital public goods contribution, Inform Syst Res, 31, 801, 10.1287/isre.2019.0916 · DOI
  31. Héder, 2022, The past, present and future of the ELKH cloud, Inform Társadalom, 22, 128, 10.22503/inftars.XXII.2022.2.8 · DOI

Cited by

See all 52

Metadata

Authors
del Rio-Chanona, R. M., Laurentsyeva, N., & Wachs, J.
Affiliations
  • Department of Computer Science, University College London , London WC1E 6EA ,
  • Faculty of Economics, LMU Munich , Munich 80539 ,
  • Department of Network Science, Corvinus University of Budapest , Budapest 1093 ,
Venue
PNAS Nexus
Published
2024-09
Volume
3
Issue
9
ISSN
2752-6542
Cited by
52
References
15
Language
en
Funders
Hungarian National Scientific Fund (OTKA FK 145960)

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Large language models reduce public knowledge sharing on online Q&A platforms. https://app.cochat.ai/papers/10.1093/pnasnexus/pgae400
The paper
del Rio-Chanona, R. M., Laurentsyeva, N., & Wachs, J. (2024). Large language models reduce public knowledge sharing on online Q&A platforms. PNAS Nexus. https://doi.org/10.1093/pnasnexus/pgae400

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.