The DOI matches its DataCite record; 16 of the 16 references it cites were found in a registry or an index; 2 of the 5 sources its claims cite were checked for support.
The Impact of AI on Developer Productivity: Evidence from GitHub Copilot
Open accessAbstract
Generative AI tools hold promise to increase human productivity. This paper presents results from a controlled experiment with GitHub Copilot, an AI pair programmer. Recruited software developers were asked to implement an HTTP server in JavaScript as quickly as possible. The treatment group, with access to the AI pair programmer, completed the task 55.8% faster than the control group. Observed heterogenous effects show promise for AI pair programmers to help people transition into software development careers.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
2 of 5 cited sources checked · 0 supported · 0 flagged · accuracy not yet measured
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“there has been little research on productivity impacts of AI-powered tools in professional contexts, cf.[Mozannar et al., 2022, Vaithilingam et al., 2022, Ziegler et al., 2022].”
§ The Impact of AI on Developer Productivity: Evidence from GitHub Copilot > Introduction
What the sources say
- Partly supportedcites Mozannar et al., 2022 sourceExperimental — accuracy not yet measured
The Introduction (no page supplied) says existing evaluations of AI code completion leave insufficient detail about productivity gains and possible inefficiencies. That supports a narrower research gap in AI-assisted programming, but not the broader claim that little research has examined productivity impacts of AI tools across professional contexts. The Background and Related Work section, in fact, describes several productivity studies. (LLM, conf=0.78)
- Not checked — no full text of the sourcecites Vaithilingam et al., 2022 source
- Partly supportedcites Ziegler et al., 2022 sourceExperimental — accuracy not yet measured
The Background section (page unavailable) describes this as the first study, to the authors’ knowledge, linking code-suggestion-tool usage measurements with developer productivity or happiness, and identifies two earlier studies. That supports limited prior research on AI code tools, but not the broader claim about AI-powered tools across professional contexts. (LLM, conf=0.84)
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes- Partly supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The treatment group, with access to the AI pair programmer, completed the task 55.8% faster than the control group.”
- Also stated in: § Introduction · plain
“the treated group completed the task 55.8% faster (95% confidence interval: 21-89%).”
- Also stated in: § Results · plain
“This represents a 55.8% reduction in completion time.”
§ The Impact of AI on Developer Productivity: Evidence from GitHub Copilot > Abstract
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes- Also stated in: § Introduction · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The p-value for the t-test is 0.0017, and a 95% confidence interval for the improvement is between [21%, 89%].”
§ The Impact of AI on Developer Productivity: Evidence from GitHub Copilot > Results
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesA conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This result suggests that Copilot increases average productivity significantly in our experiment population.”
§ The Impact of AI on Developer Productivity: Evidence from GitHub Copilot > Results
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesWhat the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“In this way, we accurately measured the timing and completion for each participant.”
§ The Impact of AI on Developer Productivity: Evidence from GitHub Copilot > Study Design
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesA conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This result provides indirect evidence that treated group benefited from Copilot during their task as their willingness to pay is significantly higher than the control group.”
§ The Impact of AI on Developer Productivity: Evidence from GitHub Copilot > Results
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesA conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Observed heterogenous effects show promise for AI pair programmers to help people transition into software development careers.”
- Also stated in: § Introduction · hedged
“These heterogeneous effects point towards promise for AI-pair programmers in support of expanding access to careers in software development.”
- Also stated in: § Discussion · hedged
“the productivity benefits for novice programmers and programmers of older age point to important possibilities for skill initiatives that support job transitions into software development.”
§ The Impact of AI on Developer Productivity: Evidence from GitHub Copilot > Abstract
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes- The authors state this as an assumption: The treatment effect is a linear function of the covariates of interest.
- Also stated in: § Introduction · hedged
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“We verify from telemetry after the experiment that all participants from the treated group have configured GitHub Copilot and accepted recommendations other than five who did not finish the sign up and thus started the experiment without the GitHub Copilot.”
§ The Impact of AI on Developer Productivity: Evidence from GitHub Copilot > Study Design
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes
- It may state two separate claims as one. 6
- The quote only partly states this claim. 1
- Probably not a substantial claim. 7
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“On average, participants in both treated and control groups estimated a 35% increase in productivity, which is an underestimation compared with the 55.8% increase in their revealed productivity.”
§ The Impact of AI on Developer Productivity: Evidence from GitHub Copilot > Results
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Conditioning on completing the task, the average completion time from the treated group is 71.17 minutes and 160.89 minutes for the control group.”
§ The Impact of AI on Developer Productivity: Evidence from GitHub Copilot > Results
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one. The quote only partly states this claim.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The results show that less experienced developers (years of professional coding), developers with heavy coding load (hours of coding per day), and older developers (developers aged between 25 and 44) benefit more from Copilot.”
- Also stated in: § Introduction · plain
“Developers with less programming experience, older programmers, and those who program more hours per day benefited the most.”
§ The Impact of AI on Developer Productivity: Evidence from GitHub Copilot > Results
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
- The authors state this as an assumption: The treatment effect is a linear function of the covariates of interest.
- Also stated in: § Introduction · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We also find that the treated group's success rate is 7 percentage points higher than the control group, but the estimate is not statistically significant, with a 95% confidence interval of [-0.11, 0.25].”
§ The Impact of AI on Developer Productivity: Evidence from GitHub Copilot > Results
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“If this result persists in further studies, the productivity benefits for novice programmers and programmers of older age point to important possibilities for skill initiatives that support job transitions into software development.”
§ The Impact of AI on Developer Productivity: Evidence from GitHub Copilot > Discussion
Strength:hedged· not yet calibratedCausal:conditional· not yet calibratedTests✓ Asserted: passes✗ Checkable: fails✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it.
- The authors state this as an assumption: The treatment effect is a linear function of the covariates of interest.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Once participants signed the contract, they were randomly split into control and treatment groups.”
- Also stated in: § Results · plain
“The 95 developers were randomly assigned into control and treated groups, with 45 in the treated group and 50 in control.”
§ The Impact of AI on Developer Productivity: Evidence from GitHub Copilot > Study Design
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
- Also stated in: § Results · plain
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“If the results of this study were to be extrapolated to the population level, a 55.8% increase in productivity would imply a significant amount of cost savings in the economy and have a notable impact on GDP growth.”
§ The Impact of AI on Developer Productivity: Evidence from GitHub Copilot > Discussion
Strength:hedged· not yet calibratedCausal:conditional· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Participants could see the tests but were unable to alter them.”
§ The Impact of AI on Developer Productivity: Evidence from GitHub Copilot > Study Design
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“The potential productivity impacts of AI have major implications for the labor market and firms, including changes in employment, skills, and firm organization [Raj andSeamans, 2018, Agrawal et al., 2019].”
§ The Impact of AI on Developer Productivity: Evidence from GitHub Copilot > Introduction
What the sources say
- Not matchedcites Raj andSeamans, 2018
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Not checked — no full text of the sourcecites Agrawal et al., 2019 source
Strength:plain· not yet calibratedCausal:conditional· not yet calibratedTests✓ Asserted: passes✗ Checkable: fails✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it.
- Not matched
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Thirty-five developers from both the treated and control groups completed the task and survey.”
§ The Impact of AI on Developer Productivity: Evidence from GitHub Copilot > Results
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Productivity benefits may vary across specific tasks and programming languages, so more research is needed to understand how our results generalizes to other tasks.”
§ The Impact of AI on Developer Productivity: Evidence from GitHub Copilot > Discussion
Strength:hedged· not yet calibratedCausal:conditional· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Using a standardized task provides us with precise measures of performance as it is difficult to measure productivity of software developers.”
§ The Impact of AI on Developer Productivity: Evidence from GitHub Copilot > Study Design
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
The authors state this as an assumption.
The treatment effect is a linear function of the covariates of interest.
“We assume the treatment effect is a linear function of the covariates of interest.”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source doesn’t support the claim as written.
In the citing paper
“While developers consistently report significant productivity gains from these tools, empirical measurements often show more modest improvements.”
From Towards Decoding Developer Cognition in the Age of AI Assistants (2025) · cites this paper as [9]
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe cited source doesn’t support the claim as written.
In the citing paper
“human-AI collaboration consistently improves measurable output quality in professional writing contexts”
From How Creative Is AI Writing? Generative and Collaborative AI in Japanese Fiction (2026) · cites this paper as Peng et al. (2023)
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“GitHub performed such an experiment where 95 participants were split into two groups and asked to write a web server.The study concluded by fnding that task completion was reduced by 55.8% in the Copilot condition [28].”
From Reading Between the Lines: Modeling User Behavior and Costs in AI-Assisted Programming (2024) · cites this paper as [28]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Evidence from existing (laboratory) experiments generally suggests significant productivity effects of generative AI tools for software developers.”
From The Productivity Effects of Generative AI: Evidence from a Field Experiment with GitHub Copilot (2024) · cites this paper as Peng et al. 2023
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Peng et al. (2023) hired 95 professional programmers to develop an HTTP server in JavaScript.The group that used GitHub Copilot completed the task 55.8% faster, yet there was no effect on whether the task was completed.”
From The Productivity Effects of Generative AI: Evidence from a Field Experiment with GitHub Copilot (2024) · cites this paper as Peng et al. (2023)
Experimental — accuracy not yet measured
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“While our participants reported, and prior work suggests [14,33,38,40], AI tools demonstrably increase short-term productivity through faster task completion and reduced blocked time”
From The Evolution of Information Seeking in Software Development: Understanding the Role and Impact of AI Assistants (2025) · cites this paper as [33]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“However, both Dakhel et al. (2023) and Peng et al. (2023) caution that novice programmers may place undue trust in AI-generated code without fully understanding its broader design or integration implications, increasing the likelihood of downstream quality and integration challenges.”
From AI-Assisted Programming Decreases the Productivity of Experienced Developers by Increasing the Technical Debt and Maintenance Burden (2025) · cites this paper as Peng et al. 2023
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“First, the introduction of GitHub Copilot leads to higher development activity at both the repository and individual levels, measured by commits and PR”
From AI-Assisted Programming Decreases the Productivity of Experienced Developers by Increasing the Technical Debt and Maintenance Burden (2025) · cites this paper as Peng et al. 2023
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Prior works have presented mixed results on whether AI helps or hinders coding productivity [Peng et al., 2023, Becker et al., 2025]; our study differs from prior results in that it is designed to study how AI affects skill formation while performing a task requiring new knowledge.”
From How AI Impacts Skill Formation (2026) · cites this paper as Peng et al., 2023
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Software engineering, in particular, has been identified as a profession in which AI tools can be readily applied and AI assistance significantly improves productivity in daily tasks [Peng et al., 2023, Cui et al., 2024].”
From How AI Impacts Skill Formation (2026) · cites this paper as Peng et al., 2023
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“AI assistance produces significant productivity gains across professional domains, particularly for novice workers.”
From How AI Impacts Skill Formation (2026) · cites this paper as Peng et al., 2023
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Cui et al. (2024) estimate a 26% productivity gain from GitHub Copilot among software developers, while Peng et al. (2023) report a 58% increase.”
From Still Waters, Rapid Currents: Early Labor Market Transformation under Generative AI (2025) · cites this paper as Peng et al. (2023)
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Existing empirical work on the micro-level effects of generative AI tools tends to find that access to these tools benefits less experienced workers more, compressing performance distributions [22; 23; 10; 24].”
From Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity (2025) · cites this paper as [10]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Prior work suggests that less experienced developers may rely more heavily on AI assistance [9]”
From Towards Decoding Developer Cognition in the Age of AI Assistants (2025) · cites this paper as [9]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“LLM-based code generation shows significant productivity improvements (e.g., GitHub Copilot improves productivity 55.8% (Peng et al., 2023))”
From CodeCRDT: Observation-Driven Coordination for Multi-Agent LLM Code Generation (2025) · cites this paper as Peng et al., 2023
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Our findings offer only partial support for the 'productivity gain' hypothesis previously reported in technical writing contexts (Noy and Zhang, 2023; Peng et al. 2023): while human-AI collaboration produces consistently acceptable prose with low variance, it does not demonstrably outperform original human writing in perceived quality”
From How Creative Is AI Writing? Generative and Collaborative AI in Japanese Fiction (2026) · cites this paper as Peng et al. 2023
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Peng et al. (2023) found comparable effects among software developers using GitHub Copilot, who completed entry-level tasks 55% faster than the control group.”
From How Creative Is AI Writing? Generative and Collaborative AI in Japanese Fiction (2026) · cites this paper as Peng et al. (2023)
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Dell’Acqua et al. [1] and Peng et al. [2] demonstrated that these tools can streamline research processes, enhance productivity, and allow experts to focus on higher-level analytical tasks.”
From Integrating large language models in biostatistical workflows for clinical and translational research (2025) · cites this paper as [2]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Prior studies show that generative AI can enhance individual developer productivity, measured as the amount of code generated per developer (i.e., intensive margin)”
From The Impact of Generative AI on Collaborative Open-Source Software Development: Evidence from GitHub Copilot (2024) · cites this paper as Peng et al. 2023
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“while reducing expected costs by lowering the time and effort required to complete development tasks.”
From The Impact of Generative AI on Collaborative Open-Source Software Development: Evidence from GitHub Copilot (2024) · cites this paper as Peng et al. 2023
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Prior studies on generative AI generally show that lower skilled individuals benefit more from AI tools because AI reduces task completion costs (e.g., Peng et al. 2023, Cui et al. 2026).”
From The Impact of Generative AI on Collaborative Open-Source Software Development: Evidence from GitHub Copilot (2024) · cites this paper as Peng et al. 2023
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“In fact, a recent study from GitHub, found that Copilot could potentially reduce task completion time by a factor of two [28].”
From Reading Between the Lines: Modeling User Behavior and Costs in AI-Assisted Programming (2024) · cites this paper as [28]
The cited source says what the claim attributes to it.
In the citing paper
“For Copilot users, our findings suggest a relatively small increase in average weekly activity; however, there is no statistically significant difference after adopting Copilot, which differs from prior studies (Al-Ahmad et al., 2024; Cui et al., 2024; Peng et al., 2023; Rajbhoj et al., 2024).”
From Developer Productivity With and Without GitHub Copilot: A Longitudinal Mixed-Methods Case Study (2026) · cites this paper as Peng et al., 2023
The cited source says what the claim attributes to it.
In the citing paper
“Instead, they focus on short-term gains in controlled or semi-controlled settings, either by measuring task completion time (Peng et al., 2023), or by simulating software projects to illustrate how GenAI tools can assist developers during different phases of the development process (Al-Ahmad et al., 2024; Rajbhoj et al., 2024).”
From Developer Productivity With and Without GitHub Copilot: A Longitudinal Mixed-Methods Case Study (2026) · cites this paper as Peng et al., 2023
The cited source says what the claim attributes to it.
In the citing paper
“For example, Peng et al. (2023) found that participants with access to GitHub Copilot completed standardized programming tasks 55.8% faster.”
From Developer Productivity With and Without GitHub Copilot: A Longitudinal Mixed-Methods Case Study (2026) · cites this paper as Peng et al. (2023)
The cited source says what the claim attributes to it.
In the citing paper
“GenAI solutions like GitHub Copilot have been shown to increase the productivity of software developers.”
From AI-Assisted Programming Decreases the Productivity of Experienced Developers by Increasing the Technical Debt and Maintenance Burden (2025) · cites this paper as Peng et al. 2023
The cited source says what the claim attributes to it.
In the citing paper
“Because AI tools can lower the skill barrier for writing code (Dakhel et al. 2023)”
From AI-Assisted Programming Decreases the Productivity of Experienced Developers by Increasing the Technical Debt and Maintenance Burden (2025) · cites this paper as Peng et al. 2023
The cited source says what the claim attributes to it.
In the citing paper
“These maintenance-related concerns are amplified by evidence that less experienced programmers appear to benefit more from the use of Copilot.”
From AI-Assisted Programming Decreases the Productivity of Experienced Developers by Increasing the Technical Debt and Maintenance Burden (2025) · cites this paper as Peng et al. 2023
The cited source says what the claim attributes to it.
In the citing paper
“Controlled experiments measure productivity gains of roughly 56% on isolated coding tasks (Peng et al., 2023)”
From Indie at Scale: How Large Language Models Make Community-Centered Game Development Economically Viable (2026) · cites this paper as Peng et al., 2023
The cited source says what the claim attributes to it.
In the citing paper
“GenAI coding assistants have demonstrated considerable potential to enhance developer productivity.”
From The Effects of GitHub Copilot on Computing Students' Programming Effectiveness, Efficiency, and Processes in Brownfield Coding Tasks (2025) · cites this paper as [28]
The cited source says what the claim attributes to it.
In the citing paper
“Cui et al. (2025); Gambacorta, Qiu, Shan, and Rees (2024); Peng, Kalliamvakou, Cihon, and Demirer (2023) document strong productivity increases for software development tasks, again with larger gains among less experienced developers.”
From Labor Market Consequences of Generative AI: Early Evidence from Norway (2026) · cites this paper as Peng, Kalliamvakou, Cihon, and Demirer (2023)
The cited source says what the claim attributes to it.
In the citing paper
“Contrary to previous work finding significant uplift or speedup of AI assistance for coding [Peng et al., 2023, Cui et al., 2024], our results do not show a significant improvement in productivity if we only look at the total completion time across the treatment and control groups.”
From How AI Impacts Skill Formation (2026) · cites this paper as Peng et al., 2023
The cited source says what the claim attributes to it.
In the citing paper
“significant productivity gains-often exceeding 15%-documented by RCTs in our study occupations”
From Still Waters, Rapid Currents: Early Labor Market Transformation under Generative AI (2025) · cites this paper as Peng et al., 2023
The cited source says what the claim attributes to it.
In the citing paper
“First, our reported time savings are highest precisely in the occupations covered by RCTs, such as customer and IT support (Brynjolfsson, Li and Raymond, 2025), marketing and human resources (Noy and Zhang, 2023), and software development (Peng et al., 2023).”
From Still Waters, Rapid Currents: Early Labor Market Transformation under Generative AI (2025) · cites this paper as Peng et al., 2023
The cited source says what the claim attributes to it.
In the citing paper
“Peng et al. [10] and Paradis et al. [11] find 56% and 21% speedups on coding tasks when using AI assistance”
From Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity (2025) · cites this paper as [10]
The cited source says what the claim attributes to it.
In the citing paper
“Frontier AI systems demonstrate impressive capabilities on a wide range of software benchmarks [3; 4; 5; 6; 7; 8; 9] and in experiments measuring AI's impact on developer productivity when completing synthetic tasks [10; 11].”
From Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity (2025) · cites this paper as [10]
The cited source says what the claim attributes to it.
In the citing paper
“For example, Peng et al. [10] asks developers to implement a very basic HTTP server in JavaScript to satisfy several automatic test cases that are shown to the developers-this task is a) unrepresentative of most software development work, and b) likely to be similar to a large amount of LLM training data, which may unfairly advantage AI systems relative to humans.”
From Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity (2025) · cites this paper as [10]
The cited source says what the claim attributes to it.
In the citing paper
“Peng et al. [10] | ↑ 56% faster”
From Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity (2025) · cites this paper as Peng et al. [10]
The cited source says what the claim attributes to it.
In the citing paper
“We hypothesize that analogously, AI helps our developers less compared to existing literature [10; 11] because our developers have substantially more experience on their respective repositories (5 years and 1,500 commits on average).”
From Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity (2025) · cites this paper as [10]
The cited source says what the claim attributes to it.
In the citing paper
“We note that limited token spend would not explain the difference between our results and other work that find positive speedup for programming tasks [10; 17].”
From Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity (2025) · cites this paper as [10]
The cited source says what the claim attributes to it.
In the citing paper
“We hypothesize that analogously to these size and complexity effects within our study, AI broadly helps our developers less compared to existing randomized controlled trials (RCTs) measuring speedup from AI tools because of the overall size and complexity of the repositories included”
From Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity (2025) · cites this paper as [10]
The cited source says what the claim attributes to it.
In the citing paper
“In a controlled experiment with 95 professional developers, Peng et.al. [9] found that access to GitHub Copilot led to a 55.8% reduction in task completion time for implementing an HTTP server, with less experienced and older developers benefiting the most.”
From Towards Decoding Developer Cognition in the Age of AI Assistants (2025) · cites this paper as [9]
The cited source says what the claim attributes to it.
In the citing paper
“The 34.9% reduction in task completion time and 26.2% improvement in test pass rates align with previous studies of the impact of GenAI programming assistants on both professional programmers [26, 28] and undergraduate students [13, 32] in greenfield programming scenarios.”
From The Effects of GitHub Copilot on Computing Students' Programming Effectiveness, Efficiency, and Processes in Brownfield Coding Tasks (2025) · cites this paper as [28]
The cited source says what the claim attributes to it.
In the citing paper
“The authors report that the treatment group completed the task 55.8% faster on average.”
From Echoes of AI: Investigating the downstream effects of AI assistants on software maintainability (2026) · cites this paper as Peng et al. (2023)
The cited source says what the claim attributes to it.
In the citing paper
“GitHub's research (Peng et al. 2023), which reported a 55% improvement.”
From Echoes of AI: Investigating the downstream effects of AI assistants on software maintainability (2026) · cites this paper as Peng et al. 2023
The cited source says what the claim attributes to it.
In the citing paper
“For example, Peng et al. (2023) reported a 55.8% speedup when implementing an HTTP server in JavaScript;”
From Echoes of AI: Investigating the downstream effects of AI assistants on software maintainability (2026) · cites this paper as Peng et al. (2023)
The cited source says what the claim attributes to it.
In the citing paper
“The early work by Peng et al. (2023) reported that junior developers benefited the most from GitHub Copilot,”
From Echoes of AI: Investigating the downstream effects of AI assistants on software maintainability (2026) · cites this paper as Peng et al. (2023)
The cited source says what the claim attributes to it.
In the citing paper
“Developers who used Copilot completed a JavaScript-based task 55.8% faster than those who did not have access to the tool”
From How do Copilot Suggestions Impact Developers' Frustration and Productivity? (2025) · cites this paper as Peng et al. [12]
The cited source says what the claim attributes to it.
In the citing paper
“For example, a controlled experiment by Peng et al. (2023) found that GitHub Copilot, a generative AI programming aid, significantly boosted productivity: the group using Copilot completed programming tasks approximately 56% faster than the control group without access to the tool.”
From Large language models at work in China’s labor market (2025) · cites this paper as Peng et al. (2023)
The cited source says what the claim attributes to it.
In the citing paper
“Other shortcomings include missing nuance or understanding of the impact of AI tools by developer- or task-level contexts (e.g., [13]), limited tool and task complexity in experimental settings (e.g., using GitHub Classroom rather than a more naturalistic developer environment [12]), and small sample sizes (e.g., only 32 participants in [15], 24 participants in [16], or 21 participants in [17]).”
From How Much Does AI Impact Development Speed? an Enterprise-Based Randomized Controlled Trial (2025) · cites this paper as [16]
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“their operational and environmental costs pose significant challenges [6, 30].”
From A Transformer-Based Approach for Smart Invocation of Automatic Code Completion (2024) · cites this paper as [30]
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Our results are consistent with those of Peng et al. (2023)”
From Echoes of AI: Investigating the downstream effects of AI assistants on software maintainability (2026) · cites this paper as Peng et al. 2023
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Peng et al. [12] showed that Copilot's suggestions are perceived as invasive by experienced developer.”
From How do Copilot Suggestions Impact Developers' Frustration and Productivity? (2025) · cites this paper as [12]
Experimental — accuracy not yet measured
Citations
Cites
The publisher hasn’t deposited this paper’s reference list.
Research works printed but not linked: 2
- [4]4. Athey, S. and Imbens, G. W. (2015). Machine learning methods for estimating heterogeneous causal effects. stat, 1050(5):1-26.
- [24]24. Raj, M. and Seamans, R. (2018). Artificial intelligence, labor, pro- ductivity, and the need for firm-level data. In The economics of artificial intelligence: An agenda, pages 553-565. University of Chicago Press.
Cited by
Sorted by citations
Items 101–200 of 340
An Experiment on Leveraging ChatGPT for Online Teaching and Assessment of Database Students
Carr, N., Shawon, F. R., & Jamil, H. M. (2023) · IEEE International Conference on Teaching, Assessment, and Learning for Engineering · doi:10.1109/tale56641.2023.10398239 · Cited by 16
“Although scant and too early, recent research on a LLM-based AI code generator GitHub Copilot indicate learning gains [32, 39] and productivity improvement [15, 31], more formal evaluations are slowly trickling in [13, 27, 43].”
ResultGenerative AI Uses and Risks for Knowledge Workers in a Science Organization
Wagman, K. B., Dearing, M. T., & Chetty, M. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713827 · Cited by 23
“Multiple studies have found that generative AI tools increase productivity more for less skilled workers [2, 32], including for workers in consulting [9] and programming [6, 22, 36].”
Hettrich, B., Krings, N., & Kock, A. (2025) · Creativity and Innovation Management · doi:10.1111/caim.70002 · Cited by 21
Exploring the Impacts of ChatGPT on Future Scientific WorkPreprint
Irons, J. L., Mason, C., Cooper, P. S., Sidra, S., Reeson, A. F., & Paris, C. L. (2023) · SocArXiv (OSF Preprints) · doi:10.31235/osf.io/j2u9x · Cited by 16
Cahill, C., & McCabe, K. T. (2024) · PS Political Science & Politics · doi:10.1017/s1049096524000155 · Cited by 15
Early Formalization of AI-tools Usage in Software Engineering in Europe: Study of 2023
Software engineering domain, A., & Pashchenko, D. S. (2023) · International Journal of Information Technology and Computer Science · doi:10.5815/ijitcs.2023.06.03 · Cited by 14
“Despite of accepting the fact of that AI impact is a significant at the moment [6], current study’s topic is in the early stage of formalization and all assumptions are preliminary.”
BackgroundArtificial Intelligence & Creativity: A manifesto for collaborationPreprint
Vinchon, F., Lubart, T., Bartolotta, S., et al. (2023) · doi:10.31234/osf.io/ukqc9 · Cited by 14
Don’t settle for the first! How many GitHub Copilot solutions should you check?
Oertel, J., Klünder, J., & Hebig, R. (2025) · Information and Software Technology · doi:10.1016/j.infsof.2025.107737 · Cited by 17
Ecosystem Graphs: The Social Footprint of Foundation Models
Bommasani, R., Soylu, D., Liao, T., Creel, K., & Liang, P. (2023) · doi:10.21203/rs.3.rs-2961271/v1 · Cited by 14
How Much Does AI Impact Development Speed? an Enterprise-Based Randomized Controlled Trial
Paradis, E., Grey, K., Madison, Q., Nam, D., Macvean, A., Meimand, V., Zhang, N., Ferrari-Church, B., & Chandra, S. (2025) · 2025 IEEE/ACM 47th International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP) · doi:10.1109/icse-seip66354.2025.00060 · Cited by 20
The Impact of Generative Artificial Intelligence on Socioeconomic Inequalities and Policy MakingPreprint
Capraro, V., Lentsch, A., Acemoğlu, D., Akgün, S., Akhmedova, A., Bilancini, E., Bonnefon, J., Brañas‐Garza, P., Butera, L., Douglas, K. M., et al. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4666103 · Cited by 14
Gong, X., Xu, W., & Qiao, A. (2025) · International Journal of Educational Technology in Higher Education · doi:10.1186/s41239-025-00552-y · Cited by 18
“However, Hartley et al. (2024) and Peng et al. (2023) indicated that the effectiveness of GAI feedback varied according to learners’ proficiency levels.”
AI and Jobs: Has the Inflection Point Arrived? Evidence from an Online Labor PlatformPreprint
Qiao, D., Rui, H., & Xiong, Q. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4656716 · Cited by 13
Generative AI and the future of equality norms
Danaher, J. (2024) · Cognition · doi:10.1016/j.cognition.2024.105906 · Cited by 12
“Peng et al. (2023) studied the impact of an AI assistant on programming speed and found that less skilled, older programmers benefitted more from integrating it into their workflow.”
A vision of human–AI collaboration for enhanced biological collection curation and research
Stenhouse, A., Fisher, N., Lepschi, B. J., Schmidt‐Lebuhn, A. N., Rodríguez, J., Turco, F., Reeson, A. F., Paris, C. L., & Thrall, P. H. (2025) · BioScience · doi:10.1093/biosci/biaf021 · Cited by 12
Encuesta de usos de ChatGPT en Argentina
Zukerfeld, M., Rabosto, A. N., Fredes, M., & DeMarco, C. (2023) · Hipertextos · doi:10.24215/23143924e075 · Cited by 21
Pirzado, F. A., Ahmed, A., Alipour, M. A., & Terashima‐Marín, H. (2025) · Journal of Computers in Education · doi:10.1007/s40692-025-00375-w · Cited by 19
“Similar findings were made by Peng et al. (2023), who discovered that participants who used GitHub Copilot as an AI partner programmer completed tasks 55 .”
Basha, M., & Rodríguez-Pérez, G. (2025) · Information and Software Technology · doi:10.1016/j.infsof.2025.107804 · Cited by 15
When humans and large language models collaborate, problem-finding illuminates
Sabbah, J., & Li, F. (2025) · Innovation · doi:10.1080/14479338.2025.2504428 · Cited by 15
“However, other studies have questioned the extent of their creative abilities (Noy & Zhang, 2023; Peng et al., 2023), raising concerns that they may lead to more homogeneous ideas rather than fostering true creative diversity (Dell’acqua et al., 2023; Doshi & Hauser, 2023).”
Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity
Becker, J., Rush, N., Barnes, E., & Rein, D. (2025) · arXiv · doi:10.48550/arxiv.2507.09089 · Cited by 16
“We note that limited token spend would not explain the difference between our results and other work that find positive speedup for programming tasks [10; 17].”
InfluentialSîrbu, A.-G., & Czibula, G. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.125821 · Cited by 11
Statically Contextualizing Large Language Models with Typed Holes
Blinn, A., Li, X., Kim, J. H., & Omar, C. (2024) · Proceedings of the ACM on Programming Languages · doi:10.1145/3689728 · Cited by 10
“As a result, AI assistants have become capable enough to substantially impact developer productivity [9, 61, 74, 88].”
BackgroundAi, Task Changes in Jobs, and Worker Reallocation
Gathmann, C., Grimm, F., & Winkler, E. (2024) · doi:10.2139/ssrn.5069982 · Cited by 10
FhGenie: A Custom, Confidentiality-Preserving Chat AI for Corporate and Scientific Use
Weber, I. M., Linka, H., Mertens, D., Muryshkin, T., Opgenoorth, H., & Langer, S. (2024) · IEEE International Conference on Software Architecture Companion (ICSA-C) · doi:10.1109/icsa-c63560.2024.00011 · Cited by 17
A Showdown of ChatGPT vs DeepSeek in Solving Programming Tasks
Shakya, R., Vadiee, F., & Khalil, M. (2025) · International Conference on New Trends in Computing Sciences (ICTCS) · doi:10.1109/ictcs65341.2025.10989361 · Cited by 10
Algorithmic Collusion or Competition: the Role of Platforms' Recommender SystemsPreprint
Xu, X., Lee, S., & Tan, Y. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4579458 · Cited by 11
NoCodeGPT: A No‐Code Interface for Building Web Apps With Language Models
Monteiro, M., Branco, B. C., Silvestre, S., Avelino, G. A., & Valente, M. T. (2025) · Software Practice and Experience · doi:10.1002/spe.3432 · Cited by 9
“One exception is a paper by Peng et al. [12] in which the authors use GitHub Copilot to implement a web server starting from a high-level textual description.”
Cinquin, O. (2024) · Briefings in Bioinformatics · doi:10.1093/bib/bbaf045 · Cited by 10
HALoGEN: Fantastic LLM Hallucinations and Where to Find Them
Ravichander, A., Ghela, S., Wadden, D., & Choi, Y. (2025) · Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2025.acl-long.71 · Cited by 13
Al Haque, E., Brown, C., LaToza, T. D., & Johnson, B. (2025) · Proceedings of the 33rd ACM International Conference on the Foundations of Software Engineering · doi:10.1145/3696630.3731665 · Cited by 9
Use of Generative AIs in the Digital Communication and Marketing Sector in Spain
Martínez-Rolán, X., Valencia, J. M. C., & Piñeiro-Otero, T. (2023) · Management and industrial engineering · doi:10.1007/978-3-031-42060-3_5 · Cited by 13
PwnPilot: Reflections on Trusting Trust in the Age of Large Language Models and AI Code Assistants
Horne, D. (2023) · Congress in Computer Science, Computer Engineering, and Applied Computing (CSCE) · doi:10.1109/csce60160.2023.00396 · Cited by 10
Assessment of ChatGPT's Proficiency in Software Development
Kim, D., Chen, J., Ming, H., & Lu, L. (2023) · Congress in Computer Science, Computer Engineering, and Applied Computing (CSCE) · doi:10.1109/csce60160.2023.00421 · Cited by 8
University Students' Perception and Expectations of Generative AI Tools for Software Engineering
Yabaku, M., & Ouhbi, S. (2024) · IEEE International Conference on Software Engineering Education and Training (CSEE&T) · doi:10.1109/cseet62301.2024.10663035 · Cited by 14
Kim, D., & Ming, H. (2025) · Information and Software Technology · doi:10.1016/j.infsof.2025.107787 · Cited by 10
Use of LLM for SMEs, opportunities and challenges
Szilágyi, R., & Tóth, M. (2024) · Journal of Agricultural Informatics · doi:10.17700/jai.2023.14.2.703 · Cited by 8
ChatGPT Hallucinates Nonexistent Citations: Evidence from Economics
Buchanan, J., Hill, S., & Shapoval, O. (2023) · doi:10.2139/ssrn.4632842 · Cited by 8
Generative AI, Platform Stances, and Content Creator BehaviorPreprint
Huang, H., Fu, R., & Ghose, A. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4670714 · Cited by 8
Voice AI in Firms: A Natural Field Experiment on Automated Job Interviews
Jabarian, B., & Henkel, L. (2025) · doi:10.2139/ssrn.5395709 · Cited by 9
A Transformer-Based Approach for Smart Invocation of Automatic Code Completion
de Moor, A., van Deursen, A., & Izadi, M. (2024) · Proceedings of the 1st ACM International Conference on AI-Powered Software · doi:10.1145/3664646.3664760 · Cited by 7
Grambow, S. C., Desai, M., Weinfurt, K. P., Lindsell, C. J., Pencina, M. J., Rende, L., & Pomann, G.-M. (2025) · Journal of Clinical and Translational Science · doi:10.1017/cts.2025.10064 · Cited by 7
AI Can Help Workers Gain New Technical Capabilities Without Gaining Knowledge
Wiles, E., Krayer, L., Abbadi, M., Awasthi, U., Kennedy, R., Mishkin, P., Sack, D., & Candelon, F. (2024) · doi:10.2139/ssrn.4944588 · Cited by 7
(2025) · doi:10.1109/dcoss-iot65416.2025.00162 · Cited by 7
Chatterjee, S., Liu, C. L., Rowland, G., & Hogarth, T. (2024) · doi:10.5121/csit.2024.140702 · Cited by 7
Defining our future with generative AI
Suri, S. (2024) · Nature Computational Science · doi:10.1038/s43588-024-00694-5 · Cited by 9
Artificial intelligence and wage inequality
Georgieff, A. (2024) · OECD artificial intelligence papers · doi:10.1787/bf98a45c-en · Cited by 18
Developer Productivity With and Without GitHub Copilot: A Longitudinal Mixed-Methods Case Study
Stray, V., Brandtzæg, E. G., Wivestad, V., Barbala, A., & Moe, N. B. (2026) · Proceedings of the Annual Hawaii International Conference on System Sciences · doi:10.24251/hicss.2026.880 · Cited by 9
“In the study by Peng et al. (2023), developers were asked to implement an HTTP server in JavaScript as quickly as possible.”
InfluentialRittikulsittichai, S., & Siriborvornratanakul, T. (2024) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2024.3521873 · Cited by 8
Wong, N., Cheng, N.-Y., Oewel, B., Genuario, K. E., Stoeckl, S. E., Schueller, S. M., Ahmed, I., van der Hoek, A., & Reddy, M. C. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713250 · Cited by 10
Mukhopadhyay, A., & Luther, K. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713283 · Cited by 10
Angrisani, M., Casanova, M., Fast, N. J., Narang, J., & Schroeder, J. (2026) · PNAS Nexus · doi:10.1093/pnasnexus/pgag007 · Cited by 7
AI and Job Satisfaction: Evidence from Glassdoor Employees' ReviewsPreprint
Chiong, K. X., & Xie, Y. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4976340 · Cited by 7
The Cybernetic Teammate: A Field Experiment on Generative AI Reshaping Teamwork and ExpertisePreprint
Dell’Acqua, F., Ayoubi, C., Lifshitz‐Assaf, H., Sadun, R., Mollick, E., Mollick, L., Han, Y., Goldman, J., Nair, H., Taub, S., et al. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5207588 · Cited by 9
Bridging Expertise Gaps: The Role of LLMs in Human-AI Collaboration for CybersecurityPreprint
Tariq, S., Singh, R., Chhetri, M. B., Nepal, ., & Paris, C. L. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5462737 · Cited by 6
The Widening Gap: The Benefits and Harms of Generative AI for Novice Programmers
Prather, J., Reeves, B., Leinonen, J., MacNeil, S., Randrianasolo, A. S., Becker, B., Kimmel, B., Wright, J., & Briggs, B. (2024) · arXiv · Cited by 196
“• Social and professional topics → Computing education ; CS1 ; • Human-centered computing → Human computer interaction (HCI) ; Empirical studies in HCI ; User studies ; Natural language interfaces ; • Computing methodologies → Artificial intelligence .”
Alam, L., Mamun, T. I., & Mueller, S. T. (2024) · Journal of Cognitive Engineering and Decision Making · doi:10.1177/15553434241310500 · Cited by 5
Neamţu, D.-M., Bejinaru, R., Anichiti, A., Butnaru, G. I., & Hapenciuc, C. V. (2025) · Kybernetes · doi:10.1108/k-10-2024-2918 · Cited by 7
What makes users contribute more in knowledge exchange? Self-benefit vs other-benefit appeals
Qi, T., Zhou, X., Tang, J., & Wang, T. (2025) · Journal of Knowledge Management · doi:10.1108/jkm-09-2024-1139 · Cited by 5
Gao, Y., Arava, S. K., Li, Y., & WSnyder, J. (2024) · International Journal on Cybernetics & Informatics · doi:10.5121/ijci.2024.130202 · Cited by 9
Maximizing Generative AI Benefits with Task Creativity and Human Validation
Sinha, C., Vracheva, V. P., & Nistor, C. (2024) · Journal of Behavioral and Applied Management · doi:10.21818/001c.122147 · Cited by 6
Perspectives de l'emploi de l'OCDE
OECD (2009) · Perspectives de l'emploi de l'OCDE/OCDE perspectives de l'emploi · doi:10.1787/19991274 · Cited by 5
Generative AI as a New Platform for Applications DevelopmentPreprint
Cusumano, M. A., Farias, V. F., & Ramakrishnan, R. (2024) · doi:10.21428/e4baedd9.f189351f · Cited by 7
Early Impacts of M365 CopilotPreprint
Dillon, E. W., Jaffe, S., Peng, S., & Cambon, A. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5216968 · Cited by 5
Self-citationThe Economics of Bicycles for the Mind
Agrawal, A., Gans, J. S., & Avi Goldfarb, B. (2025) · National Bureau of Economic Research · doi:10.3386/w34034 · Cited by 6
Co-intelligent Design of Catalysis Research with Large Language Models: Hype or Reality?
Balcells, D. (2025) · ACS Catalysis · doi:10.1021/acscatal.5c04238 · Cited by 5
How AI Impacts Skill Formation
Shen, J. H., & Tamkin, A. (2026) · arXiv · doi:10.48550/arxiv.2601.20245 · Cited by 120
“Prior works have presented mixed results on whether AI helps or hinders coding productivity [Peng et al., 2023, Becker et al., 2025]; our study differs from prior results in that it is designed to study how AI affects skill formation while performing a task requiring new knowledge.”
InfluentialAI-driven income inequality and preferences for redistribution
Veselý, Š., & Amaris, G. (2025) · Economic Analysis and Policy · doi:10.1016/j.eap.2025.06.019 · Cited by 4
Yılmaz, F. G. K., Olpak, Y. Z., & Yılmaz, R. (2026) · Journal of Research on Technology in Education · doi:10.1080/15391523.2026.2683793 · Cited by 4
From challenges to opportunities: navigating the human response to automated agents in the workplace
Ðula, I., Berberena, T., Keplinger, K., & Wirzberger, M. (2024) · Humanities and Social Sciences Communications · doi:10.1057/s41599-024-03962-x · Cited by 4
Whitaker, A. C. (2024) · Artivate A Journal of Entrepreneurship in the Arts · doi:10.34053/artivate.13.1.220 · Cited by 4
Creative Products and Artificial Intelligence
Cropley, D. H., Marrone, R. L., Medeiros, K. E., & van Broekhoven, K. (2025) · Springer eBooks · doi:10.1007/978-3-031-82415-9_3 · Cited by 4
Heinrich, T., Baily, S., Chen, K.-W., DeOliveira, J., Park, S., & Wang, N. (2025) · PLoS ONE · doi:10.1371/journal.pone.0328041 · Cited by 6
Refining the Scrum Paradigm: A Comprehensive Research of Software Development Practices (2020–2023)
Pashchenko, D. S. (2024) · Computing&AI Connect · doi:10.69709/caic.2024.103102 · Cited by 5
GenAI and Prompt Engineering: A Progressive Framework for Empowering the Workforce
Schuckart, A. (2024) · European Conference on Pattern Languages of Programs · doi:10.1145/3698322.3698348 · Cited by 5
Freeman, B. S., Arriola, K., Cottell, D., Lawlor, E., Erdman, M., Sutherland, T., & Wells, B. (2024) · American Journal of Artificial Intelligence · doi:10.11648/j.ajai.20240802.16 · Cited by 6
Bahmani, A. H., Cha, K., Alavi, A., Dixit, A., Ross, A., Park, R. M., Goncalves, F., Ma, S., Saxman, P., Nair, R. V., et al. (2025) · Communications Medicine · doi:10.1038/s43856-025-01034-y · Cited by 8
ChatGPT: how to use it and the pitfalls/cautions in academia
Lee, J. (2025) · Annals of Pediatric Endocrinology & Metabolism · doi:10.6065/apem.2550028.014 · Cited by 5
Qi, J., Zhao, H., Xiang, R., Zhang, Y., Zhang, X., Wang, C., Jian, L., & Ji, F. (2026) · Journal of Research on Technology in Education · doi:10.1080/15391523.2025.2601067 · Cited by 7
Infusing Artificial Intelligence Into Software Engineering and the DevSecOps Continuum
Bannon, T. (2024) · Computer · doi:10.1109/mc.2024.3423108 · Cited by 3
Calyam, P. P., Clemm, A., Pandey, A., Roy, U., Keller, A., Das, S. K., & Calvert, K. L. (2024) · IEEE Internet Computing · doi:10.1109/mic.2024.3485954 · Cited by 3
Preethi, P., Ragavan, V., Abinandhana, C., Umamaheswari, G., & Suvethaa, D. R. (2024) · International Conference on Emerging Research in Computational Science (ICERCS) · doi:10.1109/icercs63125.2024.10895857 · Cited by 3
Pandey, D. L., Risal, N., & Basnet, B. J. (2023) · Asian Journal of Economics Business and Accounting · doi:10.9734/ajeba/2023/v23i241186 · Cited by 6
A Prompt-Based Approach for Software Development
Hamdi, M., & Kim, L. D. (2023) · International Conference on Computational Science and Computational Intelligence (CSCI) · doi:10.1109/csci62032.2023.00267 · Cited by 3
Fleming, M. (2024) · Review of Income and Wealth · doi:10.1111/roiw.12711 · Cited by 3
Multi-source Analysis of AI Vulnerabilities: Methodology and Algorithms of Data Collection
Neretin, O., Kharchenko, V. S., & Fesenko, H. V. (2023) · IEEE International Conference on Intelligent Data Acquisition and Advanced Computing Systems: Technology and Applications (IDAACS) · doi:10.1109/idaacs58523.2023.10348671 · Cited by 3
Information Foraging in the Era of AI: Exploring the Effect of ChatGPT on Digital Q&A PlatformsPreprint
Sanatizadeh, A., Lu, Y., Zhao, K., & Hu, Y. (2023) · Social Science Research Network · doi:10.2139/ssrn.4459729 · Cited by 3
How Generative AI Impacts Content Engagement: Evidence from an Online Learning PlatformPreprint
Narang, U., Ahlbom, C.-P., & Banerjee, S. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4558719 · Cited by 4
Will Artificial Intelligence Get in the Way of Achieving Gender Equality?Preprint
Isaksson, S., Buitrago, C. F., & Carvajal, D. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4758197 · Cited by 3
Generative AI and the Nature of WorkPreprint
Hoffmann, M., Boysel, S., Nagle, F., Peng, S., & Xu, K. S. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.5007084 · Cited by 3
Self-citationAI, Investment Decisions, and InequalityPreprint
Kim, A., Kim, D. S., Muhn, M., Nikolaev, V. V., & So, E. C. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.5075727 · Cited by 3
How Experience Moderates the Impact of Generative AI Ideas on the Research ProcessPreprint
Doshi, A. R., Chai, S., & Troebinger, M. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5013086 · Cited by 5
Automation, AI, and the Intergenerational Transmission of KnowledgePreprint
Ide, E. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5360803 · Cited by 3
Proactively Developing and Assisting the Workforce in the Age of AIPreprint
Lee, Y. S., Soroushian, J., Bullock, J. B., Carroll, M., Dokko, J. K., Dominski, J., Hinds, R., Holzer, H. J., Horrigan, M., Munyikwa, Z., et al. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5366876 · Cited by 3
Tracking Employment Changes in AI-Exposed JobsPreprint
Chandar, B. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5384519 · Cited by 5
Organizational Ohm's Law in Practice: Measuring Engineering Productivity
Liu, F., & McKinnon, T. (2024) · IEEE Transactions on Engineering Management · doi:10.1109/tem.2024.3421892 · Cited by 3
Future-proofing the EU: ordoliberal governance and algorithmic regulation
Küsters, A. (2025) · Constitutional Political Economy · doi:10.1007/s10602-025-09467-2 · Cited by 3
Artificial intelligence: Redefining the retirement pattern
Aísa, R., & Cabeza, J. (2025) · Research in Economics · doi:10.1016/j.rie.2025.101062 · Cited by 2
Generative KI in der Wissensarbeit: Wahrnehmung, Nutzen und Akzeptanz von Microsoft 365 Copilot
Schmidt, C., Petzolt, S., Beinhauer, W., Weber, I., & Langer, S. (2026) · Zeitschrift für Ernährungswissenschaft · doi:10.1007/s41449-026-00517-5
An efficient probabilistic hardware architecture for diffusion-like models
Jelinčič, A., Lockwood, O., Garlapati, A., Schillinger, P., Chuang, I. L., Verdon, G., & McCourt, T. (2026) · npj Unconventional Computing · doi:10.1038/s44335-026-00075-3 · Cited by 2
Impacto de ChatGPT en la enseñanza: Un enfoque de aula invertida para fundamentos de programación
Escuela Profesional de Ingeniería de Sistemas, U. N. D. S. A., Calsin, M. A., López, M. A., Escuela Profesional de Ingeniería de Sistemas, U. N. D. S. A., Castro-Gutiérrez, E., & Escuela Profesional de Ingeniería de Sistemas, U. N. D. S. A. (2023) · RISTI - Revista Ibérica de Sistemas e Tecnologias de Informação · doi:10.17013/risti.52.97112 · Cited by 2
Metadata
- Authors
- Peng, S., Kalliamvakou, E., Cihon, P., & Demirer, M.
- Affiliations
- Microsoft (United States) (US)
- Massachusetts Institute of Technology (US)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2023
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2302.06590
- OpenAlex
- W4320854981
- Semantic Scholar
- 15abedb29536d50afeeec739a25358255cbda3e8
- Cited by
- 287
- References
- 0
- Citations by year
- 2026: 52 · 2025: 88 · 2024: 96 · 2023: 50
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Physical Sciences › Computer Science › Information Systems › Software Engineering Techniques and Practices
- Physical Sciences › Computer Science › Computer Science Applications › Open Source Software Innovations
- Keywords
- GitHub Copilot, generative AI, developer productivity, software development, JavaScript, treatment effect heterogeneity
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-11, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-11 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
