Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 56 of the 56 references it cites were found in a registry or an index; none of the 63 sources its claims cite is checked for support yet.

Writing Code vs. Shipping Code: Productivity Effects Across Generations of AI Coding Tools

Open access
Demirer, Mert, Musolff, Leon, Yang, Liyuan · 2026 · National Bureau of Economic Research

Abstract

How do the productivity effects of AI evolve across successive generations of tools, and to what extent do task-level gains ultimately translate into final output?We study these questions in the context of software development, using data on more than 500,000 GitHub developers combined with their AI usage telemetry.In a matched event study design, we find that autocomplete, interactive coding agents, and autonomous coding agents each significantly increase coding activity ("commits"), with respective cumulative effects of 30%, 180%, and 240%.These gains, however, attenuate sharply across the production hierarchy: the 240% cumulative effect falls to 80% for the number of projects, and to 30% for actual releases.This pattern is consistent with the weaklink hypothesis: the strong productivity gains from AI are attenuated by human bottlenecks in the production chain, with an estimated elasticity of substitution of 0.23 between AI and human effort, which indicates strong complementarities.We further confirm these results across four major software marketplaces, finding a sharp increase in the number of new apps but no increase in total usage.Large task-level AI productivity gains have therefore translated only partially into shipped and used software thus far.

Overview

CoChat Summary

TL;DR

Using matched event studies on more than 500,000 GitHub developers, the paper finds each generation of AI coding tools raises task-level coding activity—autocomplete by about 30%, sync agents cumulatively by about 180%, and async agents cumulatively by about 240%—but gains attenuate sharply up the software production hierarchy, with sync agents raising releases by only 19.8%. A hierarchical CES model calibrated to the attenuation implies an elasticity of substitution of 0.23 between AI and human effort, and marketplace data show more new apps but no increase in total usage.

Key findings

  • Task-level commit effects grow across tool generations: autocomplete raises weekly commits by about 30%; adding sync agents brings the cumulative effect to about 180%; adding async agents brings it to about 240% (abstract). In weeks 21–30, Table 5 reports autocomplete +30.2%, pooled sync +153.2%, and async agent-authored commits +60.7%.
  • Effects attenuate across the production hierarchy. For sync agents, Table 6 reports +957.5% lines changed, +265.0% files touched, +153.2% commits, +86.0% pull requests, +47.3% repositories touched, and +19.8% releases. For autocomplete: +234.3% lines, +30.2% commits, and +9.0% releases. For async agents, pull requests rise +68.5%, slightly more than commits (+60.7%).
  • The calibrated model yields an elasticity of substitution of 0.23 and an output elasticity with respect to upstream input of 0.76, indicating strong complementarity between AI and human effort. Alternative models fit worse: the Cobb-Douglas loss is more than 100 times the baseline’s, and the decreasing-returns CES matches only with implausible parameters.
  • Marketplace and aggregate evidence: new app entry roughly doubled monthly on average across four marketplaces since early 2025, but total usage did not increase in any marketplace; the share of new apps failing to reach a modest audience rose from about 78% to 87% on iOS, 19% to 33% on Chrome, and 22% to 26% on Android between January 2025 and April 2026. The share of repositories abandoned after their first month rose from 60% in 2021 to 69% in 2026; sync adopters deleted 12.2 times as many lines as before versus 7.0 times as many added, and created 79% more branches.

Methods

  • Matched event-study / generalized difference-in-differences design using more than 500,000 GitHub developers, 388 million commits, and 46.9 million pull requests; 489,396 matched treated-control pairs across seven tools. Controls are drawn from one year earlier, matched on pre-period commits, with pair fixed effects, outcomes normalized by each developer’s pre-period mean, and standard errors clustered at the developer level.
  • Adoption events come from Microsoft telemetry for GitHub Autocomplete, GitHub Sync, and GitHub Pro, and from public GitHub traces for Claude Code (CLAUDE.md commits), GitHub Agent (pull requests by copilot-swe-agent[bot]), OpenAI Codex (codex/ branches), and Docker (Dockerfile commits). Async agent-authored activity is separated from human-authored activity.
  • A hierarchical CES production model maps software production from lines of code through files, commits, pull requests, projects, and releases; the model is calibrated to the autocomplete attenuation pattern to estimate the elasticity of substitution. Marketplace panels cover the Apple App Store, Google Play Store, Chrome Web Store, and SourceForge.

Limitations

  • Stated in the paper: data cover only open-source software and four major software marketplaces, with little enterprise-software coverage; generative AI has advanced furthest in software engineering, so quantitative estimates may not extrapolate to other domains; estimates reflect tools available through mid-2026; marketplace usage cannot distinguish a supply-side bottleneck from demand-side discovery/adoption congestion; per-tool differences may reflect adopter selection rather than tool effects; public-repository adoption measures undercount private use (identifying 24.2% of async agent users and capturing 72.9% of those adopters’ usage activity).
  • Stated in the paper: the observational design relies on parallel trends, and the time-shifted control strategy trades robustness to aggregate shocks after the last adoption for robustness to contamination and selection of non-adopters; the pre-trend test cannot detect shocks occurring after all adoptions or shocks coinciding with adoption.
  • Evident from the text: releases are sparse and measured at the repository level, so async release effects cannot be split into agent- and human-authored components and are not reported; branch outcomes are available only from March 2023, so autocomplete branch estimates are not reported; marketplace usage measures (ratings/downloads) are imperfect proxies for consumer surplus.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.3386/w35275

Cites

56 of 56 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Verification

Claims the paper makes

0 Valid · 8 Not valid · 43 Not checked · 443 Own results and interpretations

198 more claims

Create a free account to see all 494 claims and to verify the 29 references still open.

See all claims — free account
  • It may state two separate claims as one. 219
  • The quote only partly states this claim. 5
  • Probably not a substantial claim. 136

Create a free account to see these claims.

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

3 of 3

Citations

Cites

Research works printed but not linked: 15

  1. Aghion, P., B. F. Jones, and C. I. Jones (2019). Artificial Intelligence and Economic Growth. In A. Agrawal, J. Gans, and A. Goldfarb (Eds.), The Economics of Artificial Intelligence: An Agenda, pp. 237-290. University of Chicago Press.
  2. Eloundou, T., S. Manning, P. Mishkin, and D. Rock (2024). GPTs Are GPTs: Labor Market Impact Potential of LLMs. Science 384(6702), 1306-1308.
  3. Felten, E. W., M. Raj, and R. Seamans (2023). Occupational Heterogeneity in Exposure to Generative AI. SSRN Working Paper 4414065.
  4. Fruits, E. and K. Stout (2026). AI, Productivity, and Labor Markets: A Review of the Empirical Evidence. International Center for Law & Economics Issue Brief .
  5. Goldberg, S. and H. T. Lam (2026). Generative AI & Creative Goods: Market Expansion, Crowd-Out, and Copyright. SSRN Working Paper, No. 5152649.
  6. Hoffmann, M., S. Boysel, F. Nagle, S. Peng, and K. Xu (2024). Generative AI and the Nature of Work. SSRN Working Paper, No. 5007084.
  7. Hoffmann, M., F. Nagle, and Y. Zhou (2024). The Value of Open Source Software. SSRN Working Paper, No. 4693148.
  8. Jabarian, B. and L. Henkel (2026). Voice AI in Firms: A Natural Field Experiment on Automated Job Interviews. SSRN Working Paper, No. 5395709.
  9. Karger, E., O. Kuusela, J. Abaluck, K. A. Bryan, B. Halperin, T. R. Jones, C. Murphy, P. Trammell, M. Reynolds, D. Mayland, R. Viswanathan, A. Mittal, R. Ceppas de Castro, J. Rosenberg, and P. Tetlock (2026). Forecasting the Economic Effects of AI. NBER Working Paper, No. 35046.
  10. Kim, H., D. Kim, and R. Koning (2026). Mapping AI into Production: A Field Experiment on Firm Performance. SSRN Working Paper, No. 6513481.
  11. Kim, H. and R. Koning (2026). AI-Native Firms. SSRN Working Paper, No. 6905079.
  12. Nagle, F. (2019). Open Source Software and Firm Productivity. Management Science 65(3), 1191-1215.
  13. Sarkar, S. K. and L. Melas-Kyriazi (2026). Returns to Intelligence. SSRN Working Paper, No. References for Online Appendix
  14. Jones, C. I. (2026). AI and Our Economic Future. Journal of Economic Perspectives 40(3), 3-22.
  15. Samuelson, P. A. (1947). Foundations of Economic Analysis. Harvard University Press.

Cited by

See all 22

Metadata

Authors
Demirer, M., Musolff, L., & Yang, L.
Affiliations
  • National Bureau of Economic Research (US)
  • Massachusetts Institute of Technology (US)
Venue
National Bureau of Economic Research
Published
2026-05
OpenAlex
W7163081473
Cited by
22
References
56
Citations by year
2026: 22
Type
report
Language
en
Open access
Locations
National Bureau of Economic Research (publishedVersion)
Funders
Mack Institute for Innovation Management, Wharton School, University of Pennsylvania
Topics
  • Physical Sciences › Computer Science › Information Systems › Software Engineering Research
  • Physical Sciences › Computer Science › Information Systems › Software Engineering Techniques and Practices
  • Physical Sciences › Computer Science › Computer Science Applications › Open Source Software Innovations
Keywords
AI coding tools, software productivity, autocomplete, autonomous coding agents, human-AI complementarity, elasticity of substitution, software marketplace

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

A language model drew the claims from the paper on 2026-11-17, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-11-17 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.