PreprintCited by 20Open Source Software InnovationsSoftware Engineering ResearchSoftware Engineering Techniques and Practices

The DOI matches its DataCite record.

The Impact of Large Language Models on Open-source Innovation: Evidence from GitHub Copilot

Open access

Abstract

Large Language Models (LLMs) are reshaping knowledge work, yet their impact on voluntary, self-guided open innovation forums (contributors choose tasks without managerial direction) may differ fundamentally from effects observed in organizational settings. We study this question in open-source software development, where individuals' contributions collectively drive innovation at a community level. Unlike product innovation, where typologies for classifying innovation are well established, knowledge work in open-source settings calls for a distinction grounded in the cognitive demand a task places on the contributor. Burgeoning literature distinguishes substantive contributions, which require creative problem formulation to introduce new functionality, from incremental contributions, which draw on comprehension of existing code to maintain and refine it. We exploit a natural experiment around GitHub Copilot's launch in October 2021, where Copilot supported languages like Python while not supporting R for business reasons, creating an exogenous partition between otherwise comparable ecosystems. Using three complementary identification strategies and two classification approaches, we find that Copilot availability increases open-source contributions by 28 to 40 percent. The increase in incremental contributions is significantly larger than the increase in substantive contributions across all specifications. This disparity is more pronounced in projects with higher activity levels and widens following a model upgrade: LLMs function more effectively when existing context helps define the problem and constrain solutions, tilting collaborative innovation toward exploitation of established codebases rather than exploration of new functionality. This paper provides a rare instance of causal field evidence on LLM effects, given the speed at which GenAI has exploded across the knowledge economy.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2409.08379

Cites

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Still being checked.

Research works printed but not linked: 53

  1. Abadie A (2021) Using Synthetic Controls: Feasibility, Data Requirements, and Methodological Aspects. Journal of Economic Literature, 59(2), 391-425.
  2. Abadie A, Diamond A, Hainmueller J (2010) Synthetic control methods for comparative case studies: Estimating the effect of California's tobacco control program. Journal of the American Statistical Association. 105(490):493-505.
  3. Agrawal A, Gans JS, Goldfarb A (2018) Prediction machines: The simple economics of artificial intelligence. Harvard Business Review Press.
  4. Agrawal A, Gans JS, Goldfarb A (2023) Do we want less automation? Science , 381 (6654), pp.155-158.
  5. Arkhangelsky D, Athey S, Hirshberg DA, Imbens GW, Wager S (2021) Synthetic difference-indifferences. American Economic Review , 111 (12), 4088-4118.
  6. Autor DH (2003) Outsourcing at will: The contribution of unjust dismissal doctrine to the growth of employment outsourcing. Journal of Labor Economics , 21 (1), pp.1-42.
  7. Angrist JD, Pischke JS (2009) Mostly Harmless Econometrics: An Empiricist's Companion. Princeton University Press, Princeton, NJ.
  8. Benner MJ, Tushman ML (2003) Exploitation, exploration, and process management: The productivity dilemma revisited. Academy of Management Review . 28(2):238-56
  9. Berman R, Israeli A (2022) The value of descriptive analytics: Evidence from online retailers. Marketing Science , 41 (6), 1074-1096.
  10. Boussioux L, Lane JN, Zhang M, Jacimovic V, Lakhani KR (2024) The crowdless future? Generative AI and creative problem-solving. Organization Science . 35(5):1589-607
  11. Brown T, Mann B, Ryder N, Subbiah M, Kaplan JD, Dhariwal P, Neelakantan A, Shyam P, Sastry G, Askell A, Agarwal S (2020) Language models are few-shot learners. Advances in neural information processing systems . 33:1877-901.
  12. Brynjolfsson E, Mitchell T, Rock D (2018) What can machines learn and what does it mean for occupations and the economy?. In AEA papers and proceedings (Vol. 108, pp. 43-47). 2014 Broadway, Suite 305, Nashville, TN 37203: American Economic Association.
  13. Brynjolfsson E, Li D, Raymond LR (2025) Generative AI at work. The Quarterly Journal of Economics. 140(2): 889-942.
  14. Burtch G, Lee D, Chen Z (2024) The consequences of generative AI for online knowledge communities. Scientific Reports 14(1):10413.
  15. Chen W, Jin F, Xue L (2022) Flourish or perish? The impact of technological acquisitions on contributions to open-source software. Information Systems Research. 33(3):867-886.
  16. Chen Z, Chan J (2024) Large language model in creative work: The role of collaboration modality and user expertise. Management Science . 70(12):9101-17.
  17. Cheng X, Mayya R, Sedoc J (2024) To Err is Human; To Annotate, SILICON? Toward Robust Reproducibility in LLM Annotation arXiv preprint arXiv:2412.14461 · arXiv
  18. Cockburn A, Williams L (2000) The costs and benefits of pair programming. Extreme programming examined, 8, pp.223-247.
  19. Cohen J (1960) A coefficient of agreement for nominal scales. Educational and psychological measurement . 20(1):37-46.
  20. Dahlander L, Magnusson M (2008) How do firms make use of open source communities? Long range planning. 41(6):629-49.
  21. Dehejia RH, Wahba S (2002) Propensity score-matching methods for nonexperimental causal studies. Review of Economics and statistics. 84(1):151-61.
  22. Eloundou, T, Manning, S, Mishkin, P, & Rock, D (2024) GPTs are GPTs: Labor market impact potential of LLMs. Science, 384(6702), 1306-1308.
  23. Felin T, Zenger TR (2014) Closed or open innovation? Problem solving and the governance choice. Research policy ; 43(5):914-25.
  24. Gatignon H, Tushman ML, Smith W, Anderson P (2002) A structural approach to assessing innovation: Construct development of innovation locus, type, and characteristics . Management Science . 48(9):1103-22
  25. Gopal A, Chen P-Y, Oh W, Xu SX, Sarker S (2024) On Crafting Effective Theoretical Contributions for Empirical Papers in Economics of Information Systems: Some Editorial Reflections. Information Systems Research, 35(3), 917-935.
  26. Hattori LP, Lanza M (2008) On the nature of commits. In 2008 23rd IEEE/ACM international conference on automated software engineering-workshops (pp. 63-71).
  27. Henderson RM, Clark KB (1990) Architectural innovation: The reconfiguration of existing product technologies and the failure of established firms. Administrative science quarterly , 35(1990) 9-30.
  28. Hindle A, German DM, Holt R (2008) What do large commits tell us? a taxonomical study of large commits. In Proceedings of the 2008 international working conference on Mining software repositories (pp. 99-108).
  29. Lakhani KR, Von Hippel E (2003) How open source software works: 'free' user-to-user assistance. Research Policy 32(6):923-943.
  30. Lambrecht A, Tucker C, Zhang X (2023) TV advertising and online sales: A case study of intertemporal substitution effects for an online travel platform. Journal of Marketing Research . 61(2):248-270.
  31. Lerner J, Tirole J (2002) Some simple economics of open source. The Journal of Industrial Economics , 50(2), 197-234.
  32. Liaukonytė J, Tuchman A, Zhu X (2023) Spilling the Beans on Political Consumerism: Do Social Media Boycotts and Buycotts Translate to Real Sales Impact? Marketing Science, 42(1): 11-25.
  33. Lin YK, Maruping LM (2022) Open source collaboration in digital entrepreneurship. Organization Science . 33(1):212-230.
  34. Lin YK, Maruping LM (2025) Organizing for AI innovation: Insights from an empirical exploration of U.S. patents. MIS Quarterly. 49(3):1095-1122.
  35. Liu Y, Lou B, Lee D (2025) Generative AI, Open Source, and Application-Layer Product Development. SSRN Working Paper 5577891
  36. Lou B, Sun H, Sun T (2023) GPTs and Labor Markets in the Developing Economy: Evidence from China SSRN Working Paper 4426461
  37. March JG (1991) Exploration and exploitation in organizational learning. Organization Science . 2(1):7187.
  38. Maruping LM, Daniel SL, Cataldo M (2019) Developer centrality and the impact of value congruence and incongruence on commitment and code contribution activity in open source software communities. MIS Quarterly, 43 (3), 951-976.
  39. Mayya R, Viswanathan S (2025) Delaying Informed Consent: An empirical investigation of mobile apps' upgrade decisions. Management Science 71(8): 7113-7135.
  40. McHugh ML (2012). Interrater reliability: the kappa statistic. Biochemia Medica . 15;22(3):276-82.
  41. Medappa PK, Srivastava SC (2019) Does superposition influence the success of FLOSS projects? An examination of open-source software development by organizations and individuals. Information Systems Research . 30(3):764-86.
  42. Meyer BD (1995) Natural and Quasi-experiments in Economics. Journal of Business and Economic Statististics 13(2): 151-161
  43. Miric M, Jeppesen LB (2023) How does Competition Influence Innovative Effort within a Platform-based Ecosystem? Contrasting Paid and Unpaid Contributors. Research Policy , 52(7), 104790
  44. Peng S, Kalliamvakou E, Cihon P, Demirer M (2023) The impact of AI on developer productivity: Evidence from github copilot. arXiv. https://doi.org/10.48550/arXiv.2302.06590 · DOI · Paper page
  45. Piezunka H, Dahlander L (2019) Idea rejected, tie formed: Organizations' feedback on crowdsourced ideas. Academy of Management Journal . 62(2):503-530.
  46. Quinn M, Gutt D (2025) Heterogeneous Effects of Generative AI on Knowledge Seeking in Online Communities. Journal of Management Information Systems, 42(2):370-399.
  47. Rogers EM (2003) Diffusion of innovations. 5 th Edition. Simon and Schuster .
  48. Shah SK (2006) Motivation, governance, and the viability of hybrid forms in open-source software development. Management Science . 52(7):1000-14.
  49. Song F, Agarwal A, Wen W (2024) The impact of generative AI on collaborative open-source software development: Evidence from GitHub Copilot. arXiv preprint arXiv:2410.02091. · arXiv
  50. Tushman ML, Anderson P (1986) Technological Discontinuities and Organizational Environments. Administrative Science Quarterly , 31 (3) 439-465.
  51. von Hippel E, von Krogh G (2003) Open-source software and the 'private-collective' innovation model: Issues for organization science. Organization Science . 14(2):209-23.
  52. von Krogh G, Haefliger S, Spaeth S, Wallin MW (2012) Carrots and rainbows: Motivation and social practice in open source software development. MIS Quarterly , 36 (2), 649-676.
  53. Zhu K, Shi Q, Banerjee S (2025) Monetizing Platforms: An Empirical Analysis of Supply and Demand Responses to Entry Costs in Two-Sided Markets. Management Science, 72(3): 2380-2397.

Cited by

Metadata

Authors
Yeverechyahu, D., Mayya, R., & Oestreicher-Singer, G.
Affiliations
  • Tel Aviv University (IL)
  • New York University (US)
Corresponding author
Mayya, Raveesh
Venue
arXiv
Publisher
arXiv
Published
2024
ISSN
1556-5068 (ISSN-L)
OpenAlex
W4393307211
Cited by
20
References
0
Citations by year
2026: 10 · 2025: 8 · 2024: 2
Type
posted content
Language
en
Open access
Locations
  • SSRN Electronic Journal (acceptedVersion)
  • RePEc: Research Papers in Economics (submittedVersion)
  • arXiv (Cornell University) (submittedVersion, cc-by)
  • Journal of the Association for Information Systems (submittedVersion)
  • arXiv (Cornell University) (cc-by)
Funders
  • European Commission
  • Leonard N. Stern School of Business, New York University
Topics
  • Physical Sciences › Computer Science › Computer Science Applications › Open Source Software Innovations
  • Physical Sciences › Computer Science › Information Systems › Software Engineering Research
  • Physical Sciences › Computer Science › Information Systems › Software Engineering Techniques and Practices
Keywords
large language models, GitHub Copilot, open source software development, open innovation, knowledge work, generative AI, R programming language, innovation incentives, exploration versus exploitation

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review