Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 52 of the 52 references it cites were found in a registry or an index; 11 of the 32 sources its claims cite were checked for support.

The Evolution of Information Seeking in Software Development: Understanding the Role and Impact of AI Assistants

Open access
Al Haque, Ebtesam, Brown, Chris, LaToza, Thomas D., Johnson, Brittany · 2025 · Proceedings of the 33rd ACM International Conference on the Foundations of Software Engineering

Abstract

About 32% of a software practitioners' day involves seeking and using information to support task completion. Although the information needs of software practitioners have been studied extensively, the impact of AI-assisted tools on their needs and information-seeking behaviors remains largely unexplored. To addresses this gap, we conducted a mixed-method study to understand AI-assisted information seeking behavior of practitioners and its impact on their perceived productivity and skill development. We found that developers are increasingly using AI tools to support their information seeking, citing increased efficiency as a key benefit. Our findings also amplify caveats that come with effectively using AI tools for information seeking, especially for learning and skill development, such as the importance of foundational developer knowledge that can guide and inform the information provided by AI tools. Our efforts have implications for the effective integration of AI tools into developer workflows as information retrieval systems and learning aids.

Overview

CoChat Summary

TL;DR

This mixed-method study (survey n=128; 17 interviews) examines how developers use AI assistants for information seeking and how this affects perceived productivity and skill development. It finds efficiency gains and widespread AI use, but also validation burdens, overreliance risks, and trade-offs between short-term productivity and long-term learning.

Key findings

  • AI-assisted information seeking is common: 83.6% of survey respondents actively use AI tools, and the majority report using AI at least half the time in their work. Tools used include ChatGPT (118), GitHub Copilot (62), Gemini (51), Microsoft Copilot (38), Tabnine (8), Claude (4), and Other (8). Developers use AI to understand best practices, discover libraries or solutions, explore trade-offs, identify keywords, recall or explain code, and synthesize documentation.
  • For productivity, 109 respondents said their workflow was about the same when using AI tools, while 78—mostly those with fewer years of development experience—said AI improved their flow. Time savings was the most common productivity metric, followed by output quantity and solution quality; one participant reported getting a ticket scoped for several days or a week done in half an hour.
  • For skill development, 119 respondents reported using AI for learning new technologies; 114 used AI for learning about and 113 for integrating new technologies. Concerns included knowledge gaps (88), overreliance (94), and poor problem-solving skills (87). Some participants reported decreased confidence, reduced independent coding, and diminished creative problem-solving, while others reported improved creative and critical thinking.
  • AI-generated information lacks community vetting, creating validation burdens: participants described incomplete or fragmented answers, cross-referencing multiple AI systems, non-prescriptive language, inconsistent adaptive responses, and inappropriate information density. Traditional sources remain important, especially Stack Overflow, official documentation for GUI tasks, Reddit, and colleagues or mentors for niche or context-specific problems.

Methods

  • Survey: a 20-minute Qualtrics survey with four sections, piloted with 3 respondents whose data were excluded. Recruitment used LinkedIn, X, internal mailing lists, Washington D.C. developer communities, and snowball sampling. From 310 responses and 173 interview-interest responses, filtering under-three-minute or incomplete responses left 168; excluding irrelevant open-ended answers left 128 valid responses. Respondents’ median age was 25–34; experience was 0–2 years (36), 2–5 years (53), 5–7 years (21), 7–10 years (10), and more than 10 years (8).
  • Interviews: 17 participants interviewed over Zoom until theoretical saturation, with a protocol piloted with one author and two participants. Analysis used qualitative coding and thematic analysis: an initial codebook, independent labeling of one transcript by all four authors, an external auditor review of a subset of coded segments, and application of the final codebook to remaining transcripts.
  • Analysis: survey questions were mapped to research questions; experience levels were grouped into five categories. Closed-ended questions used descriptive statistics and Fisher’s exact test with Bonferroni correction; open-ended responses used thematic summaries; only significant findings were reported.

Limitations

  • Stated internal threat: self-reported survey and interview data may be subject to memory bias; the authors asked participants to ground responses in past experiences to mitigate this.
  • Stated external threats: recruitment through LinkedIn, X, and internal mailing lists may not represent the broader developer community; the authors also engaged local developer communities and used snowball sampling. They note generalizability concerns due to sample size but state the goal is transferability rather than generalizability, supported by the mixed-method design and external auditor.
  • Stated construct threat: participants may misinterpret questions; the authors piloted survey and interview protocols and provided definitions for key terms. Evident from the text: the sample skews early-career (median age 25–34; most survey respondents had 0–5 years of experience; interview participants mostly had 0–6 years except P17 with 18 years), and self-selection may limit perspectives.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1145/3696630.3731665

Cites

52 of 52 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Figures

Figure 1 Frequency of AI Use for Information Seeking
Figure 1 Frequency of AI Use for Information SeekingCC-BY · From the PDF
Figure 2 Traditional vs. AI preference for tasks
Figure 2 Traditional vs. AI preference for tasksCC-BY · From the PDF

Verification

Claims the paper makes

7 Valid · 3 Not valid · 9 Not checked · 86 Own results and interpretations

37 more claims

Create a free account to see all 105 claims and to verify the 3 references still open.

See all claims — free account
  • It may state two separate claims as one. 36
  • Probably not a substantial claim. 34
  • Borderline: one of the tests for a substantial claim is unclear. 1
  • The quote only partly states this claim. 1

Create a free account to see these claims.

Citations

Cites

Research works printed but not linked: 2

  1. Saikrishna Chinthapatla. 2024. Unleashing the Future: A Deep Dive into AI-Enhanced Productivity for Developers. Journal Homepage: http://www.ijmra.us 13, 03 (2024).
  2. Tran Minh Tung. 2024. Opening Up The Workplace: The Way Ai Tools Are Changing Productivity. Educational Administration: Theory and Practice 30, 3 (2024), 480–491.

Cited by

See all 10

Metadata

Authors
Al Haque, E., Brown, C., LaToza, T. D., & Johnson, B.
Affiliations
  • George Mason University, Fairfax, Virginia, USA
  • Virginia Tech (US)
  • George Mason University (US)
Venue
Proceedings of the 33rd ACM International Conference on the Foundations of Software Engineering
Publisher
ACM
Published
2025-06-23
Pages
1494-1502
OpenAlex
W4413268010
Cited by
9
References
26
Citations by year
2026: 4 · 2025: 5
FWCI
10.29
Type
conference paper
Language
en
Open access
Licence
cc-by
Locations
  • ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) (publishedVersion, cc-by)
  • VTechWorks (Virginia Tech) (publishedVersion, cc-by)
  • arXiv (Cornell University) (submittedVersion, cc-by)
  • arXiv (Cornell University) (cc-by)
Topics
  • Physical Sciences › Computer Science › Information Systems › Software Engineering Research
  • Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
  • Physical Sciences › Computer Science › Information Systems › Software Engineering Techniques and Practices
Keywords
information seeking, AI assistants, software developers, developer productivity, developer workflows, developer knowledge

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-09-29, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-09-29 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.