The DOI matches its DataCite record.
The Impact of Large Language Models on Open-source Innovation: Evidence from GitHub Copilot
Open accessAbstract
Large Language Models (LLMs) are reshaping knowledge work, yet their impact on voluntary, self-guided open innovation forums (contributors choose tasks without managerial direction) may differ fundamentally from effects observed in organizational settings. We study this question in open-source software development, where individuals' contributions collectively drive innovation at a community level. Unlike product innovation, where typologies for classifying innovation are well established, knowledge work in open-source settings calls for a distinction grounded in the cognitive demand a task places on the contributor. Burgeoning literature distinguishes substantive contributions, which require creative problem formulation to introduce new functionality, from incremental contributions, which draw on comprehension of existing code to maintain and refine it. We exploit a natural experiment around GitHub Copilot's launch in October 2021, where Copilot supported languages like Python while not supporting R for business reasons, creating an exogenous partition between otherwise comparable ecosystems. Using three complementary identification strategies and two classification approaches, we find that Copilot availability increases open-source contributions by 28 to 40 percent. The increase in incremental contributions is significantly larger than the increase in substantive contributions across all specifications. This disparity is more pronounced in projects with higher activity levels and widens following a model upgrade: LLMs function more effectively when existing context helps define the problem and constrain solutions, tilting collaborative innovation toward exploitation of established codebases rather than exploration of new functionality. This paper provides a rare instance of causal field evidence on LLM effects, given the speed at which GenAI has exploded across the knowledge economy.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“They find a significant increase in overall code contributions, accompanied by a shift in the nature of innovation toward more routine and incremental changes.”
From AI-Assisted Programming Decreases the Productivity of Experienced Developers by Increasing the Technical Debt and Maintenance Burden (2025) · cites this paper as Yeverechyahu et al. 2024
The cited source says what the claim attributes to it.
In the citing paper
“Our identification strategy, which contrasts programming languages across treatment and control groups, is consistent with prior studies (Yeverechyahu et al. 2024).”
From AI-Assisted Programming Decreases the Productivity of Experienced Developers by Increasing the Technical Debt and Maintenance Burden (2025) · cites this paper as Yeverechyahu et al. 2024
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Still being checked.
Research works printed but not linked: 53
- Abadie A (2021) Using Synthetic Controls: Feasibility, Data Requirements, and Methodological Aspects. Journal of Economic Literature, 59(2), 391-425.
- Abadie A, Diamond A, Hainmueller J (2010) Synthetic control methods for comparative case studies: Estimating the effect of California's tobacco control program. Journal of the American Statistical Association. 105(490):493-505.
- Agrawal A, Gans JS, Goldfarb A (2018) Prediction machines: The simple economics of artificial intelligence. Harvard Business Review Press.
- Agrawal A, Gans JS, Goldfarb A (2023) Do we want less automation? Science , 381 (6654), pp.155-158.
- Arkhangelsky D, Athey S, Hirshberg DA, Imbens GW, Wager S (2021) Synthetic difference-indifferences. American Economic Review , 111 (12), 4088-4118.
- Autor DH (2003) Outsourcing at will: The contribution of unjust dismissal doctrine to the growth of employment outsourcing. Journal of Labor Economics , 21 (1), pp.1-42.
- Angrist JD, Pischke JS (2009) Mostly Harmless Econometrics: An Empiricist's Companion. Princeton University Press, Princeton, NJ.
- Benner MJ, Tushman ML (2003) Exploitation, exploration, and process management: The productivity dilemma revisited. Academy of Management Review . 28(2):238-56
- Berman R, Israeli A (2022) The value of descriptive analytics: Evidence from online retailers. Marketing Science , 41 (6), 1074-1096.
- Boussioux L, Lane JN, Zhang M, Jacimovic V, Lakhani KR (2024) The crowdless future? Generative AI and creative problem-solving. Organization Science . 35(5):1589-607
- Brown T, Mann B, Ryder N, Subbiah M, Kaplan JD, Dhariwal P, Neelakantan A, Shyam P, Sastry G, Askell A, Agarwal S (2020) Language models are few-shot learners. Advances in neural information processing systems . 33:1877-901.
- Brynjolfsson E, Mitchell T, Rock D (2018) What can machines learn and what does it mean for occupations and the economy?. In AEA papers and proceedings (Vol. 108, pp. 43-47). 2014 Broadway, Suite 305, Nashville, TN 37203: American Economic Association.
- Brynjolfsson E, Li D, Raymond LR (2025) Generative AI at work. The Quarterly Journal of Economics. 140(2): 889-942.
- Burtch G, Lee D, Chen Z (2024) The consequences of generative AI for online knowledge communities. Scientific Reports 14(1):10413.
- Chen W, Jin F, Xue L (2022) Flourish or perish? The impact of technological acquisitions on contributions to open-source software. Information Systems Research. 33(3):867-886.
- Chen Z, Chan J (2024) Large language model in creative work: The role of collaboration modality and user expertise. Management Science . 70(12):9101-17.
- Cheng X, Mayya R, Sedoc J (2024) To Err is Human; To Annotate, SILICON? Toward Robust Reproducibility in LLM Annotation arXiv preprint arXiv:2412.14461 · arXiv
- Cockburn A, Williams L (2000) The costs and benefits of pair programming. Extreme programming examined, 8, pp.223-247.
- Cohen J (1960) A coefficient of agreement for nominal scales. Educational and psychological measurement . 20(1):37-46.
- Dahlander L, Magnusson M (2008) How do firms make use of open source communities? Long range planning. 41(6):629-49.
- Dehejia RH, Wahba S (2002) Propensity score-matching methods for nonexperimental causal studies. Review of Economics and statistics. 84(1):151-61.
- Eloundou, T, Manning, S, Mishkin, P, & Rock, D (2024) GPTs are GPTs: Labor market impact potential of LLMs. Science, 384(6702), 1306-1308.
- Felin T, Zenger TR (2014) Closed or open innovation? Problem solving and the governance choice. Research policy ; 43(5):914-25.
- Gatignon H, Tushman ML, Smith W, Anderson P (2002) A structural approach to assessing innovation: Construct development of innovation locus, type, and characteristics . Management Science . 48(9):1103-22
- Gopal A, Chen P-Y, Oh W, Xu SX, Sarker S (2024) On Crafting Effective Theoretical Contributions for Empirical Papers in Economics of Information Systems: Some Editorial Reflections. Information Systems Research, 35(3), 917-935.
- Hattori LP, Lanza M (2008) On the nature of commits. In 2008 23rd IEEE/ACM international conference on automated software engineering-workshops (pp. 63-71).
- Henderson RM, Clark KB (1990) Architectural innovation: The reconfiguration of existing product technologies and the failure of established firms. Administrative science quarterly , 35(1990) 9-30.
- Hindle A, German DM, Holt R (2008) What do large commits tell us? a taxonomical study of large commits. In Proceedings of the 2008 international working conference on Mining software repositories (pp. 99-108).
- Lakhani KR, Von Hippel E (2003) How open source software works: 'free' user-to-user assistance. Research Policy 32(6):923-943.
- Lambrecht A, Tucker C, Zhang X (2023) TV advertising and online sales: A case study of intertemporal substitution effects for an online travel platform. Journal of Marketing Research . 61(2):248-270.
- Lerner J, Tirole J (2002) Some simple economics of open source. The Journal of Industrial Economics , 50(2), 197-234.
- Liaukonytė J, Tuchman A, Zhu X (2023) Spilling the Beans on Political Consumerism: Do Social Media Boycotts and Buycotts Translate to Real Sales Impact? Marketing Science, 42(1): 11-25.
- Lin YK, Maruping LM (2022) Open source collaboration in digital entrepreneurship. Organization Science . 33(1):212-230.
- Lin YK, Maruping LM (2025) Organizing for AI innovation: Insights from an empirical exploration of U.S. patents. MIS Quarterly. 49(3):1095-1122.
- Liu Y, Lou B, Lee D (2025) Generative AI, Open Source, and Application-Layer Product Development. SSRN Working Paper 5577891
- Lou B, Sun H, Sun T (2023) GPTs and Labor Markets in the Developing Economy: Evidence from China SSRN Working Paper 4426461
- March JG (1991) Exploration and exploitation in organizational learning. Organization Science . 2(1):7187.
- Maruping LM, Daniel SL, Cataldo M (2019) Developer centrality and the impact of value congruence and incongruence on commitment and code contribution activity in open source software communities. MIS Quarterly, 43 (3), 951-976.
- Mayya R, Viswanathan S (2025) Delaying Informed Consent: An empirical investigation of mobile apps' upgrade decisions. Management Science 71(8): 7113-7135.
- McHugh ML (2012). Interrater reliability: the kappa statistic. Biochemia Medica . 15;22(3):276-82.
- Medappa PK, Srivastava SC (2019) Does superposition influence the success of FLOSS projects? An examination of open-source software development by organizations and individuals. Information Systems Research . 30(3):764-86.
- Meyer BD (1995) Natural and Quasi-experiments in Economics. Journal of Business and Economic Statististics 13(2): 151-161
- Miric M, Jeppesen LB (2023) How does Competition Influence Innovative Effort within a Platform-based Ecosystem? Contrasting Paid and Unpaid Contributors. Research Policy , 52(7), 104790
- Peng S, Kalliamvakou E, Cihon P, Demirer M (2023) The impact of AI on developer productivity: Evidence from github copilot. arXiv. https://doi.org/10.48550/arXiv.2302.06590 · DOI · Paper page
- Piezunka H, Dahlander L (2019) Idea rejected, tie formed: Organizations' feedback on crowdsourced ideas. Academy of Management Journal . 62(2):503-530.
- Quinn M, Gutt D (2025) Heterogeneous Effects of Generative AI on Knowledge Seeking in Online Communities. Journal of Management Information Systems, 42(2):370-399.
- Rogers EM (2003) Diffusion of innovations. 5 th Edition. Simon and Schuster .
- Shah SK (2006) Motivation, governance, and the viability of hybrid forms in open-source software development. Management Science . 52(7):1000-14.
- Song F, Agarwal A, Wen W (2024) The impact of generative AI on collaborative open-source software development: Evidence from GitHub Copilot. arXiv preprint arXiv:2410.02091. · arXiv
- Tushman ML, Anderson P (1986) Technological Discontinuities and Organizational Environments. Administrative Science Quarterly , 31 (3) 439-465.
- von Hippel E, von Krogh G (2003) Open-source software and the 'private-collective' innovation model: Issues for organization science. Organization Science . 14(2):209-23.
- von Krogh G, Haefliger S, Spaeth S, Wallin MW (2012) Carrots and rainbows: Motivation and social practice in open source software development. MIS Quarterly , 36 (2), 649-676.
- Zhu K, Shi Q, Banerjee S (2025) Monetizing Platforms: An Empirical Analysis of Supply and Demand Responses to Entry Costs in Two-Sided Markets. Management Science, 72(3): 2380-2397.
Cited by
Sorted by citations
Song, F., Agarwal, A., & Wen, W. (2024) · SSRN Electronic Journal (preprint) · doi:10.2139/ssrn.4856935 · Cited by 30
How Much Does AI Impact Development Speed? an Enterprise-Based Randomized Controlled Trial
Paradis, E., Grey, K., Madison, Q., Nam, D., Macvean, A., Meimand, V., Zhang, N., Ferrari-Church, B., & Chandra, S. (2025) · 2025 IEEE/ACM 47th International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP) · doi:10.1109/icse-seip66354.2025.00060 · Cited by 20
Xu, F., Medappa, P. K., Tunc, M. M., Vroegindeweij, M., & Fransoo, J. C. (2025) · arXiv · doi:10.48550/arxiv.2510.10165 · Cited by 27
Metadata
- Authors
- Yeverechyahu, D., Mayya, R., & Oestreicher-Singer, G.
- Affiliations
- Tel Aviv University (IL)
- New York University (US)
- Corresponding author
- Mayya, Raveesh
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2024
- ISSN
- 1556-5068 (ISSN-L)
- arXiv
- 2409.08379
- OpenAlex
- W4393307211
- Cited by
- 20
- References
- 0
- Citations by year
- 2026: 10 · 2025: 8 · 2024: 2
- Type
- posted content
- Language
- en
- Open access
- green
- doi.org
- Locations
- SSRN Electronic Journal (acceptedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by)
- Journal of the Association for Information Systems (submittedVersion)
- arXiv (Cornell University) (cc-by)
- Funders
- European Commission
- Leonard N. Stern School of Business, New York University
- Topics
- Physical Sciences › Computer Science › Computer Science Applications › Open Source Software Innovations
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Physical Sciences › Computer Science › Information Systems › Software Engineering Techniques and Practices
- Keywords
- large language models, GitHub Copilot, open source software development, open innovation, knowledge work, generative AI, R programming language, innovation incentives, exploration versus exploitation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: The Impact of Large Language Models on Open-source Innovation: Evidence from GitHub Copilot. https://app.cochat.ai/papers/10.48550/arxiv.2409.08379
- The paper
- Yeverechyahu, D., Mayya, R., & Oestreicher-Singer, G. (2024). The Impact of Large Language Models on Open-source Innovation: Evidence from GitHub Copilot. arXiv. https://doi.org/10.48550/arxiv.2409.08379