Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 33 of the 33 references it cites were found in a registry or an index; 9 of the 20 sources its claims cite were checked for support.

Human-AI Collaboration in Software Development: A Mixed-Methods Study of Developers' Use of GitHub Copilot and ChatGPT

Open access
Stray, Viktoria, Barbala, Astri, Wivestad, Viggo Tellefsen · 2025 · Proceedings of the 33rd ACM International Conference on the Foundations of Software Engineering

Abstract

The integration of AI-powered coding assistants into software development presents both opportunities and challenges. These tools promise enhanced productivity and code quality while introducing significant concerns related to workflow integration, accuracy, reliability, and ethical considerations. This study investigates how software developers interact with Generative AI (GenAI) tools in a large-scale public sector organization. Using a mixed-methods approach, we analyze 13 interviews and a survey with 114 respondents to explore the impact of GitHub Copilot (IDE-integrated AI) and ChatGPT (conversational AI assistant) on software engineering workflows. The Human-AI Collaboration and Adaptation Framework (HACAF) is central to our analysis, which integrates insights from several technology acceptance models and theories. Our findings reveal that compatibility within existing workflows plays a crucial role in effectively utilizing GenAI tools. We explore individual, technological, and social factors that influence the adoption of these tools, highlighting the importance of organizational support and collective user patterns within teams. By studying both users and non-users, we provide insights into the factors driving GenAI adoption and the barriers preventing full integration. Our findings contribute to discussions on AI-enhanced IDEs, human-AI collaboration in coding, and the evolving role of LLM-based assistants in modern software development.

Overview

CoChat Summary

## TL;DR

This mixed-methods case study examines how developers in a large Norwegian public-sector organization used GitHub Copilot and ChatGPT, using 13 interviews and a survey of 114 respondents. The paper finds that adoption depends strongly on workflow compatibility, personal innovativeness, organizational support, and social/team norms, with clear differences between users and non-users.

## Key findings

  • Among the 114 survey respondents, 67 were Copilot users and 47 were non-users; among the 13 interviewees, 4 were non-users and 1 had tried Copilot for a week and quit because it was “annoying.”
  • Users reported higher averages on several outcomes than non-users, including job satisfaction (3.75 vs 3.60), flow while coding (4.20 vs 3.84), and ability to focus on satisfying work (3.77 vs 3.22); the paper reports that only the “focus on satisfying work” difference was significant, and that “over 70%” of users reported this advantage compared to “only half” of non-users.
  • Users also reported lower dependence on colleagues: the paper says nearly half of users disagreed that they rely on colleagues for help and guidance, compared with just a fifth of non-users; elsewhere it states 45% of users explicitly said they do not depend on colleagues.
  • The strongest qualitative themes were that GenAI is most attractive for repetitive/mundane tasks and when it fits existing workflows, while concerns about code quality, learning, ethics, and teamwork reduce adoption; the authors also state that adoption and dissemination are most successful when user patterns within teams are collective and similar.

## Methods

  • The study uses a mixed-methods case study in “GovTech,” a large Norwegian public-sector organization responsible for social welfare and labor services, focusing on GitHub Copilot and ChatGPT.
  • Data came from 13 semi-structured interviews conducted from November 7 to December 18, 2023, plus a survey with 114 respondents, of whom 81% were inhouse employees and 19% external consultants; 15% were women, 79% men, 1% “other,” and 5% declined to disclose gender.
  • Interview data were coded with the Constant Comparison Method and a thematic analysis approach; the survey had 57 questions, used mostly 5-point Likert scales, was adapted from GitHub Research’s Copilot survey, and was analyzed predominantly in Python. The analysis was organized using Russo’s Human-AI Collaboration and Adaptation Framework (HACAF).

## Limitations

  • The paper does not report effect sizes or p-values for the survey comparisons; it only says that most differences were not significant and that one finding was significant.
  • The evidence is from a single organization and a pilot project with volunteers, so generalizability beyond this public-sector context is limited; this is evident from the study design, even though the paper does not explicitly frame it as a limitation.
  • The survey items were adapted from a Copilot-focused instrument and generalized for users and non-users, which may limit comparability to the original survey; this is evident from the methods text, though not stated as a formal limitation.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1145/3696630.3730566

Cites

33 of 33 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Verification

Claims the paper makes

8 Valid · 4 Not valid · 12 Not checked · 75 Own results and interpretations

39 more claims

Create a free account to see all 99 claims and to verify the 4 references still open.

See all claims — free account
  • Probably not a substantial claim. 38
  • It may state two separate claims as one. 27

Create a free account to see these claims.

Citations

Cites

Research works printed but not linked: 1

  1. Robert Yin. 2017. Case Study Research and Applications: Design and Methods (6 ed.). SAGE Publications.

Cited by

See all 19

Metadata

Authors
Stray, V., Barbala, A., & Wivestad, V. T.
Affiliations
  • University of Oslo (NO)
  • SINTEF Digital (NO)
Venue
Proceedings of the 33rd ACM International Conference on the Foundations of Software Engineering
Publisher
ACM
Published
2025-06-23
Pages
1325-1332
OpenAlex
W4412703929
Cited by
19
References
13
Citations by year
2026: 19
FWCI
10.42
Type
conference paper
Language
en
Open access
Licence
cc-by
Locations
ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) (publishedVersion, cc-by)
Funders
The Research Council of Norway (321477)
Topics
  • Physical Sciences › Computer Science › Information Systems › Software Engineering Research
  • Physical Sciences › Computer Science › Information Systems › Software Engineering Techniques and Practices
  • Physical Sciences › Computer Science › Computer Science Applications › Open Source Software Innovations
Keywords
human-AI collaboration, generative AI in software development, GitHub Copilot, ChatGPT, AI coding assistants, technology adoption, organizational support, public sector software development, large language models

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.