CoChat
Sign Up
Cited by 6Software Engineering Techniques and PracticesSoftware Engineering ResearchArtificial Intelligence Applications

The role of firm AI capabilities in generative AI-pair coding

Open access
Bughin, Jacques·2024·Journal of Decision Systems
DOI 10.1080/12460125.2024.2428187
Open this paper in CoChat — free account

Abstract

Generative Artificial Intelligence (genAI) is the latest evidence of the transformative value of AI in organisations. One promising avenue is in software engineering, where genAI can contribute to coding by pairing with developers. Based on a sample of global companies, two key findings emerge from an analysis of the productivity impact of genAI pair coding. Coding quality is negatively correlated with productivity throughput gains, while quality-adjusted productivity gains depend on the extent to which firms have deployed AI capabilities in the form of data, skills upgrading and AI governance. As observed with other digital technologies, the success of using genAI is closely linked to complementary technical skills and organisational resources.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1080/12460125.2024.2428187

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

The citation lists haven’t been fetched yet

References 66 · Cited by 6

Research works printed but not linked: 41

  1. Agrawal, K. (2023). Towards adoption of Generative AI in organizational settings. Journal of Computer Information Systems , 1-16.
  2. Chrysovalantou, M., & Petrakis, E. (2011). Timing of technology adoption and product market competition. International Journal of Industrial Organization , 29(5), 513-523.
  3. Chowdhury, S., Dey, P., Joel-Edgar, S., Bhattacharya, S., Rodriguez-Espindola, O., Abadie, A., & Truong, L. (2023). Unlocking the value of artificial intelligence in human resource management through AI capability framework. Human Resource Management Review , 33(1), 100899.
  4. Coutinho, M., Marques, L., Santos, A., Dahia, M., França, C., & de Souza Santos, R. (2024, July). The role of generative AI in software development productivity: A pilot case study. In Proceedings of the 1st ACM International Conference on AI-Powered Software (pp. 131-138).
  5. Czarnitzki, D., Fernández, G. P. and Rammer, C. (2023), 'Artificial intelligence and firm-level productivity', Journal of Economic Behavior & Organization, Vol. 211, pp. 188-205
  6. Dakhel, A. M., Nikanjam, A., Khomh, F., Desmarais, M., & Jiang, Z. (2023). Github Co-Pilot AI pair programmer: Asset or liability? Journal of Systems and Software , 203, 111734.
  7. Damioli, G., Van Roy, V., Vertesy, D., & Vivarelli, M. (2022). AI technologies and employment: Micro evidence from the supply side. Applied Economics Letters , 1-6
  8. Desouza, K. C., Dawson, G. S., & Chenok, D. (2020). Designing, developing and deploying artificial intelligence systems: Lessons from and for the public sector. Business Horizons , 63(2), 205-213 Dakhel, A. M., Nikanjam, A., Khomh, F., Desmarais, M., & Jiang, Z. (2023). Github Co-Pilot AI pair programmer: Asset or liability? Journal of Systems and Software , 203, 111734.
  9. Damioli, G., Van Roy, V., Vertesy, D., & Vivarelli, M. (2022). AI technologies and employment: Micro evidence from the supply side. Applied Economics Letters , 1-6
  10. Fountaine, T., McCarthy, B. and Saleh, T. (2019), Building the AI-powered organization, Harvard Business Review, Vol. 97, No. 4, pp. 62-73. 11.
  11. Gao, X., & Feng, H. (2023). AI-driven productivity gains: Artificial intelligence and firm productivity. Sustainability , 15 (11), 8934.
  12. Haque, M., & Li, S. (2022). I think this is the most disruptive technology: Exploring sentiments of Chat-GPT early adopters using Twitter data. arXiv preprint arXiv:2212.05856 . · arXiv
  13. Haque, M., & Li, S. (2023). The potential use of Chat-GPT for debugging and bug fixing. EAI Endorsed Transactions on AI and Robotics , 2(1).
  14. Hardin, J., & Hilbe, J. (2007). Generalized linear models and extensions (2nd ed.). Stata Press Idrisov, B., & Schlippe, T. (2024). Program code generation with generative AIs. Algorithms , 17(2), 62.
  15. Hartmann, P., & Henkel, J. (2020). The rise of corporate science in AI: Data as a strategic resource. Academy of Management Discoveries , 6 (3), 359-381.
  16. Huang, K., Huang, J., & Catteddu, D. (2024). GenAI Data Security. In Generative AI Security: Theories and Practices (pp. 133-162). Cham: Springer Nature Switzerland Imai, S. (2022, May). Is github copilot a substitute for human pair-programming? An empirical study.
  17. Jo, A. (2023). The promise and peril of generative AI. Nature , 614(1), 214-216.
  18. Katz, D. M., Bommarito, M. J., Gao, S., & Arredondo, P. (2023). GPT-4 passes the bar exam. Available at SSRN 4389233.
  19. Kazemitabaar, M., Chow, J., Ma, C. K. T., Ericson, B. J., Weintrop, D., & Grossman, T. (2023). Studying the effect of AI code generators on supporting novice learners in introductory programming. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems , 1-23.
  20. Lazzeri, F. (2023). Adopting Machine Learning and AI for Data-driven Decisions. In Impact of Artificial Intelligence in Business and Society (pp. 33-57). Routledge.
  21. Lee, Yong Suk, et al. "When does AI pay off? AI-adoption intensity, complementary investments, and R&D strategy." Technovation 118 (2022): 102590
  22. Li, M. M., Dickhaut, E., Bruhin, O., Wache, H., & Weritz, P. (2024). More Than Just Efficiency: Impact of Generative AI on Developer Productivity. AMCIS 2024 Proceedings. https://aisel.aisnet.org/amcis2024/incl\_sustain/incl\_sustain/2
  23. Liu, Y ., Thanh Le-Cong, Ratnadira Widyasari, Chakkrit Tantithamthavorn, Li Li, Xuan-Bach D. Le, & Lo, D. (2024). Refining ChatGPT-generated code: Characterizing and mitigating code quality issues. ACM Transactions on Software Engineering and Methodology, 33 (5), Article 116. https://doi.org/10.1145/364367 · DOI
  24. Liang, J., Yang, C., & Myers, B. (2023). Understanding the usability of AI programming assistants. arXiv preprint arXiv:2303.17125 . · arXiv
  25. Lindell, M. K., & Whitney, D. J. (2001). Accounting for common method variance in cross-sectional research designs. Journal of Applied Psychology, 86 (1), 114-121.
  26. Lui, A.K.H., Lee, M.C.M. & Ngai, E.(2022). Impact of artificial intelligence investment on firm value. Ann Oper Res 308 , 373-388. https://doi.org/10.1007/s10479-020-03862-8 Ma, Q, T Wu, & Koedinger, K (2023). Is AI the better programming partner? Human-Human pair programming vs. Human-AI pAIr programming." arXiv preprint arXiv:2306.05153 . · DOI
  27. Maita, A., Martins, L. C., Lopez Paz, C. R., Peres, S. M., & Fantinato, M. (2015). Process mining through artificial neural networks and support vector machines. Business Process Management Journal, 21 (6), 1391-1415.
  28. Memeti, S., Pllana, S., Binotto, A., Kolodziej, J., & Brandic, I. (2019). Using meta-heuristics and machine learning for software optimization of parallel computing systems: A systematic literature review. Computing, 2019 , 893-936.
  29. Mikalef, P., & Gupta, M. (2021). Artificial intelligence capability: Conceptualization, measurement calibration and empirical study of its impact on organizational creativity and company performance. Information & Management, 58 (3), 103434.
  30. Mozannar, H., Bansal, G., Fourney, A., & Horvitz, E. (2024, May). Reading between the lines: Modeling user behavior and costs in AI-assisted programming. In Proceedings of the CHI Conference on Human Factors in Computing Systems (pp. 1-16).
  31. Murphy-Hill, E., et al. (2021). What predicts software developers' productivity? IEEE Transactions on Software Engineering, 47 (3), 582-594. https://doi.org/10.1109/TSE.2019.2900308 · DOI · Paper page
  32. Patel, V , and Manan S (2022). Artificial intelligence and machine learning in drug discovery and development. Intelligent Medicine 2.3 (2022): 134-140
  33. Peng, S., Kalliamvakou, E., Cihon, P., & Demirer, M. (2023). The impact of AI on developer productivity: Evidence from GitHub Co-Pilot. arXiv preprint arXiv:2302.06590 . · arXiv
  34. Perry, N., et al. (2022). Do users write more insecure code with AI assistants? arXiv preprint arXiv:2211.03622 · arXiv
  35. Shinkyu, A., & Sueishi, N. (2022). Small tuning parameter selection for the debiased Lasso. arXiv preprint arXiv:2208.08679 . · arXiv
  36. Sinha, P., Shastri, A., & Loimer, S. (2023). How generative AI will change sales. Harvard Business Review .
  37. Shahid, N., Rappon, T., & Berta, W. (2019). Applications of artificial neural networks in healthcare organizational decision: A scoping review. PLOS One, 14 (2).
  38. Schmidt, P., Biessmann, F., & Teubner, T. (2020). Transparency and trust in artificial intelligence systems. Journal of Decision Systems, 29 (4), 260-278.
  39. Shin, Y., Meneely, A., Williams, L., & Osborne, J. A. (2011). Evaluating complexity, code churn, and developer activity metrics as indicators of software vulnerabilities. IEEE Transactions on Software Engineering, 37 (6), 772-787.
  40. Spinellis, D, (2024). Pair Programming With Generative AI. IEEE Software , vol. 41, no. 3, pp. 16-18. Steck, H., Baltrunas, L., Elahi, E., Liang, D., Raimond, Y ., & Basilico, J. (2021). Deep learning for recommender systems: A Netflix case study. AI Magazine , 42 (3), 7-18.
  41. Teece, G., Pisano, R., & Shuen, A. (1997). Dynamic capabilities and strategic management. Strategic Management Journal, 18(7), 509-533.

Authors
Bughin, J.
Affiliations
Université Libre de Bruxelles (BE)
Corresponding author
Bughin, Jacques
ORCID
Bughin, Jacques
Venue
Journal of Decision Systems
Publisher
Informa UK Limited
Published
2024-11-27
Pages
1-22
ISSN
  • 1246-0125 (ISSN-L)
  • 2116-7052
DOI
10.1080/12460125.2024.2428187
OpenAlex
W4404765197
Cited by
6
References
66
Citations by year
2026: 5 · 2025: 1
FWCI
2.26
Type
journal article
Language
en
Open access
Licence
other-oa
Locations
  • Journal of Decision System (publishedVersion)
  • RePEc: Research Papers in Economics (submittedVersion)
  • Dépôt institutionnel de l'Université libre de Bruxelles (Université Libre de Bruxelles) (submittedVersion, other-oa)
Topics
  • Physical Sciences › Computer Science › Information Systems › Software Engineering Techniques and Practices
  • Physical Sciences › Computer Science › Information Systems › Software Engineering Research
  • Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence Applications
Keywords
generative AI, pair programming, AI capabilities, software productivity, code quality, AI governance, skill development, complementary skills, organizational resources, digital technology adoption

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review