Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 63 of the 64 references it cites were found in a registry or an index.

Measuring Program Comprehension: A Large-Scale Field Study with Professionals

Xia, Xin, Bao, Lingfeng, Lo, David, Xing, Zhenchang, Hassan, Ahmed E., Li, Shanping · 2018 · IEEE Transactions on Software Engineering

Abstract

During software development and maintenance, developers spend a considerable amount of time on program comprehension activities. Previous studies show that program comprehension takes up as much as half of a developer's time. However, most of these studies are performed in a controlled setting, or with a small number of participants, and investigate the program comprehension activities only within the IDEs. However, developers' program comprehension activities go well beyond their IDE interactions. In this paper, we extend our ActivitySpace framework to collect and analyze Human-Computer Interaction (HCI) data across many applications (not just the IDEs). We follow Minelli et al.'s approach to assign developers' activities into four categories: navigation, editing, comprehension, and other. We then measure the comprehension time by calculating the time that developers spend on program comprehension, e.g., inspecting console and breakpoints in IDE, or reading and understanding tutorials in web browsers. Using this approach, we can perform a more realistic investigation of program comprehension activities, through a field study of program comprehension in practice across a total of seven real projects, on 78 professional developers, and amounting to 3,148 working hours. Our study leverages interaction data that is collected across many applications by the developers. Our study finds that on average developers spend ~58 percent of their time on program comprehension activities, and that they frequently use web browsers and document editors to perform program comprehension activities. We also investigate the impact of programming language, developers' experience, and project phase on the time that is spent on program comprehension, and we find senior developers spend significantly less percentages of time on program comprehension than junior developers. Our study also highlights the importance of several research directions needed to reduce program comprehension time, e.g., building automatic detection and improvement of low quality code and documentation, construction of software-engineering-specific search engines, designing better IDEs that help developers navigate code and browse information more efficiently, etc.

Overview

Integrity signals

Registry

Crossref · 10.1109/tse.2017.2734091

Cites

63 of 63 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Citations

Cites

Show all 64 references

Research works printed but not linked: 3

  1. whitaker, 1999, What causes IT workers to leave, Manag Rev, 88, 8
  2. spencer, 2009, Card Sorting Designing Usable Categories
  3. fjeldstad, 1983, Application program maintenance study: Report to our respondents, Proc GUIDE

Cited by

See all 326

Metadata

Authors
Xia, X., Bao, L., Lo, D., Xing, Z., Hassan, A. E., & Li, S.
Affiliations
  • Zhejiang University (CN)
  • Singapore Management University (SG)
  • Australian National University (AU)
  • Queen's University (CA)
Venue
IEEE Transactions on Software Engineering
Published
2018-10-01
Volume
44
Issue
10
Pages
951-976
ISSN
  • 0098-5589 (ISSN-L)
  • 1939-3520
  • 2326-3881
OpenAlex
W2741561716
Cited by
326
References
64
Citations by year
2026: 30 · 2025: 37 · 2024: 54 · 2023: 62 · 2022: 60 · 2021: 38 · 2020: 22 · 2019: 14 · 2018: 8
FWCI
22.89
Type
journal article
Language
en
Open access
closed
Locations
  • IEEE Transactions on Software Engineering (publishedVersion)
  • Singapore Management University Institutional Knowledge (InK) (Singapore Management University) (submittedVersion)
Funders
  • National Natural Science Foundation of China (61602403, 61572426)
  • National Key Technology R&D Program
  • Ministry of Science and Technology of China (2015BAH17F01)
Topics
  • Physical Sciences › Computer Science › Information Systems › Software Engineering Research
  • Physical Sciences › Computer Science › Information Systems › Software Engineering Techniques and Practices
  • Physical Sciences › Computer Science › Software › Advanced Software Engineering Methodologies
Keywords
program comprehension, professional developers, developer experience, project phases, software maintenance

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Measuring Program Comprehension: A Large-Scale Field Study with Professionals. https://app.cochat.ai/papers/10.1109/tse.2017.2734091
The paper
Xia, X., Bao, L., Lo, D., Xing, Z., Hassan, A. E., & Li, S. (2018). Measuring Program Comprehension: A Large-Scale Field Study with Professionals. IEEE Transactions on Software Engineering. https://doi.org/10.1109/tse.2017.2734091

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.