The consequences of generative AI for online knowledge communities
Open accessAbstract
Generative artificial intelligence technologies, especially large language models (LLMs) like ChatGPT, are revolutionizing information acquisition and content production across a variety of domains. These technologies have a significant potential to impact participation and content production in online knowledge communities. We provide initial evidence of this, analyzing data from Stack Overflow and Reddit developer communities between October 2021 and March 2023, documenting ChatGPT’s influence on user activity in the former. We observe significant declines in both website visits and question volumes at Stack Overflow, particularly around topics where ChatGPT excels. By contrast, activity in Reddit communities shows no evidence of decline, suggesting the importance of social fabric as a buffer against the community-degrading effects of LLMs. Finally, the decline in participation on Stack Overflow is found to be concentrated among newer users, indicating that more junior, less socially embedded users are particularly likely to exit.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Evidence from Q&A online communities suggests that generative AI may reduce voluntary participation by diminishing opportunities for contributors to gain recognition and visibility for their expertise.
“evidence from Q&A online communities suggests that generative AI may reduce voluntary participation by diminishing opportunities for contributors to gain recognition and visibility for their expertise (Burtch et al. 2024).”
From The Impact of Generative AI on Collaborative Open-Source Software Development: Evidence from GitHub Copilot  (2024) · cites this paper as Burtch et al. 2024
The Abstract and Discussion report declining visits and question posting on Stack Overflow following ChatGPT’s release, supporting the claim that generative AI may reduce participation in a Q&A community. They do not attribute that decline to contributors losing opportunities for recognition or visibility of their expertise; the Discussion instead considers reliance on LLMs for knowledge acquisition in place of human peers. (LLM, conf=0.85)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Mostly Harmless Econometrics: An Empiricist's Companion
Angrist, J. D., & Pischke, J.-S. (2009) · doi:10.1515/9781400829828 · Cited by 9,467 · see citing works
Experimental Evidence on the Productivity Effects of Generative Artificial Intelligence
Noy, S., & Zhang, W. (2023) · doi:10.2139/ssrn.4375283 · Cited by 2,157 · see citing works
Experimental evidence on the productivity effects of generative artificial intelligence
Noy, S., & Zhang, W. (2023) · Science · doi:10.1126/science.adh2586 · Cited by 2,150 · see citing works
Dell'Acqua, F., McFowland III, E., Mollick, E. R., Lifshitz-Assaf, H., Kellogg, K., Rajendran, S., Krayer, L., Candelon, F., & Lakhani, K. R. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4573321 · Cited by 794 · see citing works
Matrix Completion Methods for Causal Panel Data Models
Athey, S., Bayati, M., Doudchenko, N., Imbens, G., & Khosravi, K. (2018) · doi:10.3386/w25132 · Cited by 497 · see citing works
Flagged first, then by citations
Research works printed but not linked: 17
- [2]Peng, S., Kalliamvakou, E., Cihon, P., Demirer, M. The impact of AI on developer productivity: Evidence from Github copilot. Preprint at https://arXiv.org/2302.06590 (2023).
- [4]Hwang EH Singh PV Argote L Knowledge sharing in online communities: Learning to cross geographic and hierarchical boundaries Organ. Sci. 2015 26 6 1593 1611 10.1287/orsc.2015.1009 · DOI
- [5]Hwang EH Krackhardt D Online knowledge communities: Breaking or sustaining knowledge silos? Prod. Oper. Manag. 2020 29 1 138 155 10.1111/poms.13098 · DOI
- [6]Bang, Y. et al. A multitask, multilingual, multimodal evaluation of ChatGPT on reasoning, hallucination, and interactivity. In Proc. of the 13th International Joint Conference on Natural Language Processing and the 3rd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 675–718 (2023).
- [7]Saxenian A Regional Advantage: Culture and Competition in Silicon Valley and Route 128 1996 Harvard University Press
- [10]Tubiana M Miguelez E Moreno R In knowledge we trust: Learning-by-interacting and the productivity of inventors Res. Policy 2022 51 1 104388 10.1016/j.respol.2021.104388 · DOI
- [12]Allen TJ Managing the Flow of Technology: Technology Transfer and the Dissemination of Technological Information within the R&D Organization 1984 MIT Press Books
- [13]Abadie A Using synthetic controls: Feasibility, data requirements, and methodological aspects J. Econ. Lit. 2021 59 2 391 425 10.1257/jel.20191450 · DOI · Paper page
- [14]Hollingsworth, A., Wing, C. Tactics for design and inference in synthetic control studies: An applied example using high-dimensional data. Available at SSRN, Paper no. 3592088 (2020).
- [15]Tibshirani R Regression shrinkage and selection via the lasso J. R. Stat. Soc. Ser. B Stat. Methodol. 1996 58 1 267 288 10.1111/j.2517-6161.1996.tb02080.x · DOI · Paper page
- [16]Goldberg S Johnson G Shriver S Regulating privacy online: An economic evaluation of the GDPR Am. Econ. J. Econ. Policy 2024 16 1 325 358 10.1257/pol.20210309 · DOI
- [17]Eichenbaum M Godinho de Matos M Lima F Rebelo S Trabandt M Expectations, infections, and economic activity J. Polit. Econ. 2023 10.1086/729449 · DOI
- [20]Antelmi, A., Cordasco, G., De Vinco, D., Spagnuolo, C.The age of snippet programming: Toward understanding developer communities in stack overflow and reddit. In Companion Proceedings of the ACM Web Conference, pp. 1218–1224 (2023).
- [21]Sengupta, S. ‘Learning to code in a virtual world’ A preliminary comparative analysis of discourse and learning in two online programming communities. In Conference Companion Publication of the 2020 on Computer Supported Cooperative Work and Social Computing, pp. 389–394 (2020).
- [23]Wu L Kane GC Network-biased technical change: How modern digital collaboration tools overcome some biases but exacerbate others Organ. Sci. 2021 32 2 273 292 10.1287/orsc.2020.1368 · DOI
- [24]Kabir, S., Udo-Imeh, D. N., Kou, B., Zhang, T. Who answers it better? An in-depth analysis of ChatGPT and stack overflow answers to software engineering questions. Preprint at https://arXiv.org/2308.02312 (2023).
- [25]Villalobos, P., Sevilla, J., Heim, L., Besiroglu, T., Hobbhahn, M., Ho, A. Will we run out of data? An analysis of the limits of scaling datasets in machine learning. Preprint at https://arXiv.org/2211.04325 (2022).
Cited by
Sorted by citations
Are Large Language Models a Threat to Digital Public Goods? Evidence from Activity on Stack Overflow
del Rio-Chanona, M., Laurentsyeva, N., & Wachs, J. (2023) · arXiv · doi:10.48550/arxiv.2307.07367 · Cited by 62 · see citing works
Song, F., Agarwal, A., & Wen, W. (2024) · SSRN Electronic Journal (preprint) · doi:10.2139/ssrn.4856935 · Cited by 30 · see citing works
- Authors
- Burtch, G., Lee, D., & Chen, Z.
- Affiliations
- Boston University (US)
- Corresponding author
- Burtch, Gordon
- Venue
- Scientific Reports
- Published
- 2024-05-06
- Volume
- 14
- Issue
- 1
- Pages
- 10413-10413
- ISSN
- 2045-2322 (ISSN-L)
- PMC
- PMC11074245
- OpenAlex
- W4396674231
- Cited by
- 83
- References
- 5
- Citations by year
- 2026: 39 · 2025: 28 · 2024: 12 · 2023: 4
- FWCI
- 29.70
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- Scientific Reports (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Topics
- Physical Sciences › Computer Science › Information Systems › Expert finding and Q&A systems
- Social Sciences › Communication › Knowledge Management and Sharing
- Social Sciences › Communication › Wikis in Education and Collaboration
- Keywords
- generative AI, large language models, ChatGPT, online knowledge communities, Stack Overflow, Reddit, developer communities, user participation, community decline, social fabric, new users
- MeSH terms
- Artificial Intelligence, Humans, Language, Knowledge, Internet
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
