The DOI matches its Crossref record; 15 of the 15 references it cites were found in a registry or an index.
Large language models reduce public knowledge sharing on online Q&A platforms
Abstract
Large language models (LLMs) are a potential substitute for human-generated data and knowledge resources. This substitution, however, can present a significant problem for the training data needed to develop future models if it leads to a reduction of human-generated content. In this work, we document a reduction in activity on Stack Overflow coinciding with the release of ChatGPT, a popular LLM. To test whether this reduction in activity is specific to the introduction of this LLM, we use counterfactuals involving similar human-generated knowledge resources that should not be affected by the introduction of ChatGPT to such extent. Within 6 months of ChatGPT’s release, activity on Stack Overflow decreased by 25% relative to its Russian and Chinese counterparts, where access to ChatGPT is limited, and to similar forums for mathematics, where ChatGPT is less capable. We interpret this estimate as a lower bound of the true impact of ChatGPT on Stack Overflow. The decline is larger for posts related to the most widely used programming languages. We find no significant change in post quality, measured by peer feedback, and observe similar decreases in content creation by more and less experienced users alike. Thus, LLMs are not only displacing duplicate, low-quality, or beginner-level content. Our findings suggest that the rapid adoption of LLMs reduces the production of public data needed to train them, with significant consequences.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“In programming, the most affected clusters are related to JavaScript, HTML, and CSS, which are also among the most affected programming languages in collaborative programming platforms such as Stack Overflow (del Rio-Chanona et al., 2024).”
From Winners and losers of generative AI: Early Evidence of Shifts in Freelancer Demand (2025) · cites this paper as del Rio-Chanona et al., 2024
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
“A Model of Growth: Through Creative Destruction”
Aghion, P., & Howitt, P. (1992) · Econometrica · doi:10.2307/2951599 · Cited by 7,529
Automation and New Tasks: How Technology Displaces and Reinstates Labor
Acemoglu, D., & Restrepo, P. (2019) · Journal of Economic Perspectives · doi:10.1257/jep.33.2.3 · Cited by 2,612
Experimental evidence on the productivity effects of generative artificial intelligence
Noy, S., & Zhang, W. (2023) · Science · doi:10.1126/science.adh2586 · Cited by 2,150 · cited 2×
“While early evidence shows that these models enhance productivity especially among new and inexperienced workers (Noy and Zhang, 2023; Brynjolfsson et al., 2023), there are other ways in which they may contribute to inequality between people and firms (Rock, 2019), for instance via potential negative side effects of a…”
A Comprehensive Overview of Large Language Models
Naveed, H., Khan, A. U., Qiu, S., Saqib, M., Anwar, S., Usman, M., Akhtar, N., Barnes, N., & Mian, A. (2025) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3744746 · Cited by 1,257
AI models collapse when trained on recursively generated data
Shumailov, I., Shumaylov, Z., Zhao, Y., Papernot, N., Anderson, R., & Gal, Y. (2024) · Nature · doi:10.1038/s41586-024-07566-y · Cited by 793
Welcome to the Era of ChatGPT et al.: The Prospects of Large Language Models
Teubner, T., Flath, C. M., Weinhardt, C., van der Aalst, W., & Hinz, O. (2023) · Business & Information Systems Engineering · doi:10.1007/s12599-023-00795-x · Cited by 384 · cited 2×
“The remarkable effectiveness of ChatGPT is reflected in its quick adoption (Teubner et al., 2023) and application across diverse fields including auditing (Gu et al.”
The False Promise of Imitating Proprietary LLMs
Gudibande, A., Wallace, E., Snell, C., Geng, X., Liu, H., Abbeel, P., Levine, S., & Song, D. (2023) · arXiv · doi:10.48550/arxiv.2305.15717 · Cited by 377 · cited 1×
“Given the growing evidence that data generated by LLMs are un - likely to effectively train new LLMs (22, 23), modelers face the real problem of running out of useful data.”
AI and the transformation of social science research
Grossmann, I., Feinberg, M., Parker, D. C., Christakis, N. A., Tetlock, P. E., & Cunningham, W. A. (2023) · Science · doi:10.1126/science.adi1778 · Cited by 377 · cited 1×
“poised to change how we do research (Grossmann et al., 2023) and present a strong competitor to search engines (Xu et al.”
The Impact of AI on Developer Productivity: Evidence from GitHub Copilot
Peng, S., Kalliamvakou, E., Cihon, P., & Demirer, M. (2023) · arXiv · doi:10.48550/arxiv.2302.06590 · Cited by 287 · cited 1×
“Because ChatGPT performs relatively well on software programming tasks (15), we study Stack Overflow, the largest online Q&A platform for software development and programming.”
OpenAssistant Conversations - Democratizing Large Language Model Alignment
Köpf, A., Kilcher, Y., von Rütte, D., Anagnostidis, S., Tam, Z.-R., Stevens, K., Barhoum, A., Duc, N. M., Stanley, O., Nagyfi, R., et al. (2023) · arXiv · doi:10.48550/arxiv.2304.07327 · Cited by 218
Research works printed but not linked: 31
- Henzinger, 2004, Extracting knowledge from the world wide web, Proc Natl Acad Sci U S A, 101, 5186, 10.1073/pnas.0307528100 · DOI
- Hess, 2003, Ideas, artifacts, and facilities: information as a common-pool resource, Law Contemp Probl, 66, 111
- Naveed, 2023, 10.48550/arXiv.2307.06435 · DOI · Paper page
- Smith, 2023, Astronomia ex machina: a history, primer and outlook on neural networks in astronomy, R Soc Open Sci, 10, 221454, 10.1098/rsos.221454 · DOI
- Kanjee, 2023, Accuracy of a generative artificial intelligence model in a complex diagnostic challenge, JAMA, 330, 78, 10.1001/jama.2023.8288 · DOI · Paper page
- Guo, 2023, 10.48550/arXiv.2305.18365 · DOI · Paper page
- Delile, 2023, 10.48550/arXiv.2306.11174 · arXiv · doi:10.48550/arxiv.2306.11174
- Widjojo, 2023, 10.48550/arXiv.2307.10793 · DOI
- Chiang, 2023, ChatGPT is a blurry JPEG of the web
- Arthur, 1989, Competing technologies, increasing returns, and lock-in by historical events, Econ J, 99, 116, 10.2307/2234208 · DOI · Paper page
- David, 1985, Clio and the economics of QWERTY, Am Econ Rev, 75, 332
- Stojkoski, 2024, Estimating digital product trade through corporate revenue data, Nat Commun, 15, 5262, 10.1038/s41467-024-49141-z · DOI
- Kreitmeir, 2023, 10.48550/arXiv.2304.09339 · DOI
- Bilinski, 2018, 10.48550/arXiv.1805.03273 · DOI · Paper page
- Eloundou, 2024, GPTs are GPTs: labor market impact potential of LLMs, Science, 384, 1306, 10.1126/science.adj0998 · doi:10.1126/science.adj0998 · Paper page
- Schumpeter, 1942, Capitalism, socialism, and democracy
- Gaessler, 2023, Training with AI: evidence from chess computers, Strat Manag J, 44, 2724, 10.1002/smj.3512 · DOI
- Taleb, 2012, Antifragile: how to live in a world we don’t understand
- Wyllie, 2024, 10.48550/arXiv.2403.07857 · arXiv · doi:10.48550/arxiv.2403.07857
- Tian-Zheng Wei, 2024, 10.48550/arXiv.2402.10892 · DOI
- Page, 2006, Path dependence, Quart J Polit Sci, 1, 87, 10.1561/100.00000006 · DOI · Paper page
- Vincent, 2021, A deeper investigation of the importance of Wikipedia links to search engine results, Proc ACM Hum-Comput Inter, 5, 1
- Lerner, 2002, Some simple economics of open source, J Ind Econ, 50, 197, 10.1111/1467-6451.00174 · DOI · Paper page
- Greenstein, 2014, Digital dark matter and the economic contribution of Apache, Res Policy, 43, 623, 10.1016/j.respol.2014.01.003 · DOI
- Xu, 2020, What makes geeks tick? A study of stack overflow careers, Manage Sci, 66, 587, 10.1287/mnsc.2018.3264 · DOI
- Evans, 2008, Electronic publication and the narrowing of science and scholarship, Science, 321, 395, 10.1126/science.1150473 · DOI · Paper page
- Sandrini, 2023, Generative ai and deceptive news consumption, Econ Lett, 232, 111317, 10.1016/j.econlet.2023.111317 · DOI
- Johnson, 2023, Power and progress: our thousand-year struggle over technology and prosperity
- Lehdonvirta, 2022, Cloud empires: how digital platforms are overtaking the state and how we can regain control, 10.7551/mitpress/14219.001.0001 · DOI
- Kummer, 2020, Unemployment and digital public goods contribution, Inform Syst Res, 31, 801, 10.1287/isre.2019.0916 · DOI
- Héder, 2022, The past, present and future of the ELKH cloud, Inform Társadalom, 22, 128, 10.22503/inftars.XXII.2022.2.8 · DOI
Cited by
Large language models for life cycle assessments: Opportunities, challenges, and risks
Preuss, N., Alshehri, A. S., & You, F. (2024) · Journal of Cleaner Production · doi:10.1016/j.jclepro.2024.142824 · Cited by 71
Winners and losers of generative AI: Early Evidence of Shifts in Freelancer Demand
Teutloff, O., Einsiedler, J., Kässi, O., Braesemann, F., Mishkin, P., & del Rio-Chanona, R. M. (2025) · Journal of Economic Behavior & Organization · doi:10.1016/j.jebo.2024.106845 · Cited by 39
Metadata
- Authors
- del Rio-Chanona, R. M., Laurentsyeva, N., & Wachs, J.
- Affiliations
- Department of Computer Science, University College London , London WC1E 6EA ,
- Faculty of Economics, LMU Munich , Munich 80539 ,
- Department of Network Science, Corvinus University of Budapest , Budapest 1093 ,
- Venue
- PNAS Nexus
- Publisher
- Oxford University Press (OUP)
- Published
- 2024-09
- Volume
- 3
- Issue
- 9
- ISSN
- 2752-6542
- Cited by
- 52
- References
- 15
- Language
- en
- Funders
- Hungarian National Scientific Fund (OTKA FK 145960)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Large language models reduce public knowledge sharing on online Q&A platforms. https://app.cochat.ai/papers/10.1093/pnasnexus/pgae400
- The paper
- del Rio-Chanona, R. M., Laurentsyeva, N., & Wachs, J. (2024). Large language models reduce public knowledge sharing on online Q&A platforms. PNAS Nexus. https://doi.org/10.1093/pnasnexus/pgae400
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.