The DOI matches its Crossref record; 71 of the 71 references it cites were found in a registry or an index.
On the Comprehension of Program Comprehension
Abstract
Research in program comprehension has evolved considerably over the past decades. However, only little is known about how developers practice program comprehension in their daily work. This article reports on qualitative and quantitative research to comprehend the strategies, tools, and knowledge used for program comprehension. We observed 28 professional developers, focusing on their comprehension behavior, strategies followed, and tools used. In an online survey with 1,477 respondents, we analyzed the importance of certain types of knowledge for comprehension and where developers typically access and share this knowledge. We found that developers follow pragmatic comprehension strategies depending on context. They try to avoid comprehension whenever possible and often put themselves in the role of users by inspecting graphical interfaces. Participants confirmed that standards, experience, and personal communication facilitate comprehension. The team size, its distribution, and open-source experience influence their knowledge sharing and access behavior. While face-to-face communication is preferred for accessing knowledge, knowledge is frequently shared in informal comments. Our results reveal a gap between research and practice, as we did not observe any use of comprehension tools and developers seem to be unaware of them. Overall, our findings call for reconsidering the research agendas towards context-aware tool support.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Students had to comprehend and reason about the code [Maalej et al. 2014] to arrive at the solution or to formulate a relevant prompt for getting effective assistance from the bot.”
From How Do Programming Students Use Generative AI? (2025) · cites this paper as Maalej et al. 2014
Citations
Cites
Research Design: Qualitative, Quantitative, and Mixed Methods Approaches
Creswell, J. W. (1994) · Cited by 35,473
Research Design: Qualitative, Quantitative, and Mixed Methods Approaches
Sloane-Seale, A. (2009) · Canadian Journal of University Continuing Education · doi:10.21225/d54s3d · Cited by 14,328
Protocol Analysis: Verbal Reports as Data
Venkatesan, M., Ericsson, K. A., & Simon, H. A. (1986) · Journal of Marketing Research · doi:10.2307/3151491 · Cited by 4,347
Protocol Analysis: Verbal Reports as Data
Ericsson, K. A., & Simon, H. A. (1993) · doi:10.7551/mitpress/5657.001.0001 · Cited by 3,537
Cabrera, A., & Cabrera, E. F. (2002) · Organization Studies · doi:10.1177/0170840602235001 · Cited by 1,355
An empirical study of speed and communication in globally distributed software development
Herbsleb, J. D., & Mockus, A. (2003) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2003.1205177 · Cited by 997
Empirical Studies of Programming Knowledge
Soloway, E., & Ehrlich, K. (1984) · IEEE Transactions on Software Engineering · doi:10.1109/tse.1984.5010283 · Cited by 877
Towards a theory of the comprehension of computer programs
Brooks, R. (1983) · International Journal of Man-Machine Studies · doi:10.1016/s0020-7373(83)80031-5 · Cited by 756
Ko, A. J., Myers, B. A., Coblenz, M. J., & Aung, H. H. (2006) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2006.116 · Cited by 685
Maintaining mental models: a study of developer work habits
LaToza, T. D., Venolia, G., & DeLine, R. (2006) · Proceedings of the 28th international conference on Software engineering · doi:10.1145/1134285.1134355 · Cited by 628
How We Refactor, and How We Know It
Murphy-Hill, E., Parnin, C., & Black, A. P. (2012) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2011.41 · Cited by 547
Boehm (1976) · IEEE Transactions on Computers · doi:10.1109/tc.1976.1674590 · Cited by 544
Information Needs in Collocated Software Development Teams
Ko, A. J., DeLine, R., & Venolia, G. (2007) · 29th International Conference on Software Engineering (ICSE'07) · doi:10.1109/icse.2007.45 · Cited by 499
(2009) · doi:10.1145/1518701.1518944 · Cited by 447
How are Java software developers using the Eclipse IDE?
Murphy, G. C., Kersten, M., & Findlater, L. (2006) · IEEE Software · doi:10.1109/ms.2006.105 · Cited by 413
The Role of Domain Expenence in Software Design
Adelson, B., & Soloway, E. (1985) · IEEE Transactions on Software Engineering · doi:10.1109/tse.1985.231883 · Cited by 394
Expertise Browser: a quantitative approach to identifying expertise
Mockus, A., & Herbsleb, J. D. (2003) · Proceedings of the 24th International Conference on Software Engineering. ICSE 2002 · doi:10.1109/icse.2002.1007994 · Cited by 383
What Makes APIs Hard to Learn? Answers from Developers
Robillard, M. P. (2009) · IEEE Software · doi:10.1109/ms.2009.193 · Cited by 382
Designing the whyline: a debugging interface for asking questions about program behavior
Ko, A. J., & Myers, B. A. (2004) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/985692.985712 · Cited by 361
Bjørnson, F. O., & Dingsøyr, T. (2008) · Information and Software Technology · doi:10.1016/j.infsof.2008.03.006 · Cited by 360
A field study of API learning obstacles
Robillard, M. P., & DeLine, R. (2011) · Empirical Software Engineering · doi:10.1007/s10664-010-9150-8 · Cited by 352
How software engineers use documentation: the state of the practice
Lethbridge, T. C., Singer, J., & Forward, A. (2003) · IEEE Software · doi:10.1109/ms.2003.1241364 · Cited by 343
Example-centric programming: integrating web search into the development environment
Brandt, J., Dontcheva, M., Weskamp, M., & Klemmer, S. R. (2010) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1753326.1753402 · Cited by 324
Asking and Answering Questions during a Programming Change Task
Sillito, J., Murphy, G. C., & De Volder, K. (2008) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2008.26 · Cited by 319
An examination of software engineering work practices
Singer, J., Lethbridge, T., Vinson, N., & Anquetil, N. (2010) · CASCON First Decade High Impact Papers on - CASCON '10 · doi:10.1145/1925805.1925815 · Cited by 298
Debugging reinvented: asking and answering why and why not questions about program behavior
Ko, A. J., & Myers, B. A. (2008) · Proceedings of the 30th international conference on Software engineering · doi:10.1145/1368088.1368130 · Cited by 296
The relevance of software documentation, tools and technologies: a survey
Forward, A., & Lethbridge, T. C. (2002) · Proceedings of the 2002 ACM symposium on Document engineering · doi:10.1145/585058.585065 · Cited by 268
How effective developers investigate source code: an exploratory study
Robillard, M. P., Coelho, W., & Murphy, G. C. (2004) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2004.101 · Cited by 260
Delocalized Plans and Program Comprehension
Letovsky, S., & Soloway, E. (1986) · IEEE Software · doi:10.1109/ms.1986.233414 · Cited by 245
An ethnographic study of copy and paste programming practices in oopl
Kim, M., Bergman, L., Lau, T., & Notkin, D. (2004) · Cited by 240
Beginning Behavioral Research: A Conceptual Primer
Rosnow, R. L., & Rosenthal, R. (1993) · Medical Entomology and Zoology · Cited by 239
The role of concepts in program comprehension
Rajlich, V., & Wilde, N. (2003) · Proceedings 10th International Workshop on Program Comprehension · doi:10.1109/wpc.2002.1021348 · Cited by 234
Five-Point Likert Items: t test versus Mann-Whitney-Wilcoxon (Addendum added October 2012)
de Winter, J. F. C., & Dodou, D. (2010) · University of Massachusetts Amherst · doi:10.7275/bj1p-ts64 · Cited by 231
Software Design — Cognitive Aspects
Détienne, F. (2002) · Practitioner Series · doi:10.1007/978-1-4471-0111-6 · Cited by 205
Information needs in bug reports: improving cooperation between developers and users
Breu, S., Premraj, R., Sillito, J., & Zimmermann, T. (2010) · Proceedings of the 2010 ACM conference on Computer supported cooperative work · doi:10.1145/1718918.1718973 · Cited by 204
Sharing Expertise: Beyond Knowledge Management
(2004) · The Electronic Library · doi:10.1108/02640470410533461 · Cited by 200
Expertise browser: a quantitative approach to identifying expertise
Mockus, A., & Herbsleb, J. D. (2002) · Proceedings of the 24th international conference on Software engineering - ICSE '02 · doi:10.1145/581339.581401 · Cited by 199
Minimalism beyond the Nurnberg funnel
Carroll, J. M. (1998) · Cited by 184
Barriers to effective use of knowledge management systems in software engineering
Desouza, K. C. (2003) · Communications of the ACM · doi:10.1145/602421.602458 · Cited by 184
Patterns of Knowledge in API Reference Documentation
Maalej, W., & Robillard, M. P. (2013) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2013.12 · Cited by 183
Self-citationE-Research : Methods, Strategies, and Issues
Anderson, T. D., & Kanuka, H. (2003) · Medical Entomology and Zoology · Cited by 182
Mental models and software maintenance
Littman, D. C., Pinto, J., Letovsky, S., & Soloway, E. (1987) · Journal of Systems and Software · doi:10.1016/0164-1212(87)90033-1 · Cited by 181
How do professional developers comprehend software?
Roehm, T., Tiarks, R., Koschke, R., & Maalej, W. (2012) · 2012 34th International Conference on Software Engineering (ICSE) · doi:10.1109/icse.2012.6227188 · Cited by 178
Self-citationHow do professional developers comprehend software
Roehm, T., Tiarks, R., Koschke, R., & Maalej, W. (2012) · doi:10.5555/2337223.2337254 · Cited by 167
Self-citationUsing information fragments to answer the questions developers ask
Fritz, T., & Murphy, G. C. (2010) · Proceedings of the 32nd ACM/IEEE International Conference on Software Engineering - Volume 1 · doi:10.1145/1806799.1806828 · Cited by 163
Rahman, F., Bird, C., & Devanbu, P. (2012) · Empirical Software Engineering · doi:10.1007/s10664-011-9195-3 · Cited by 132
Hoffmann, R., Fogarty, J. A., & Weld, D. S. (2007) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/1294211.1294216 · Cited by 131
Identification of dynamic comprehension processes during large scale maintenance
von Mayrhauser, A., & Vans, A. M. (1996) · IEEE Transactions on Software Engineering · doi:10.1109/32.508315 · Cited by 124
Dagenais, B., & Robillard, M. P. (2010) · Proceedings of the eighteenth ACM SIGSOFT international symposium on Foundations of software engineering · doi:10.1145/1882291.1882312 · Cited by 124
Program comprehension as fact finding
LaToza, T. D., Garlan, D., Herbsleb, J. D., & Myers, B. A. (2007) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/1287624.1287675 · Cited by 105
Communication networks in geographically distributed software development
Cataldo, M., & Herbsleb, J. D. (2008) · Proceedings of the 2008 ACM conference on Computer supported cooperative work · doi:10.1145/1460563.1460654 · Cited by 98
Systematizing pragmatic software reuse
Holmes, R., & Walker, R. J. (2012) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/2377656.2377657 · Cited by 91
Towards understanding programs through wear-based filtering
DeLine, R., Khella, A., Czerwinski, M., & Robertson, G. (2005) · Proceedings of the 2005 ACM symposium on Software visualization · doi:10.1145/1056018.1056044 · Cited by 91
Can development work describe itself?
Maalej, W., & Happel, H.-J. (2010) · International Conference on Mining Software Repositories (MSR) · doi:10.1109/msr.2010.5463344 · Cited by 80
Self-citationInternet Surveys and Data Quality : A Review
Tuten, T. L., Urban, D. J., & Bošnjak, M. (2002) · Cited by 77
Peer interaction effectively, yet infrequently, enables programmers to discover new tools
Murphy-Hill, E. R., & Murphy, G. C. (2011) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/1958824.1958888 · Cited by 68
Improving software developers' fluency by recommending development environment commands
Murphy-Hill, E., Jiresal, R., & Murphy, G. C. (2012) · Proceedings of the ACM SIGSOFT 20th International Symposium on the Foundations of Software Engineering · doi:10.1145/2393596.2393645 · Cited by 67
Sjøberg, D. I. K., Anda, B., Arisholm, E., et al. (2003) · Lecture notes in computer science · doi:10.1007/978-3-540-45143-3_3 · Cited by 52
Expert Recommender: Designing for a Network Organization
Reichling, T., Veith, M., & Wulf, V. (2007) · Computer Supported Cooperative Work (CSCW) · doi:10.1007/s10606-007-9055-2 · Cited by 50
Collecting and Processing Interaction Data for Recommendation Systems
Maalej, W., Fritz, T., & Robbes, R. (2013) · Springer eBooks · doi:10.1007/978-3-642-45135-5_7 · Cited by 46
Self-citationTask-First or Context-First? Tool Integration Revisited
Maalej, W. (2009) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase.2009.36 · Cited by 41
Self-citationMinimalism Beyond the Nurnberg Funnel
Johnson, B. T. (1998) · Issues in Science and Technology Librarianship · doi:10.29173/istl1431 · Cited by 41
Managing software change tasks: an exploratory study
Sillito, J., Volder, K. D., Fisher, B., & Murphy, G. C. (2005) · International Symposium on Empirical Software Engineering · doi:10.1109/isese.2005.1541811 · Cited by 39
Proceedings. 13th International Workshop on Program Comprehension
(2005) · IEEE International Workshop on Program Comprehension (IWPC) · doi:10.1109/wpc.2005.16 · Cited by 35
Guidelines for Effective Bridging in Global Software Engineering
Milewski, A. E., Tremaine, M. M., Egan, R., Zhang, S., Köbler, F., & O’Sullivan, P. (2008) · IEEE International Conference on Global Software Engineering (ICGSE) · doi:10.1109/icgse.2008.16 · Cited by 28
Resumption strategies for interrupted programming tasks
Parnin, C., & Rugaber, S. (2009) · 2009 IEEE 17th International Conference on Program Comprehension · doi:10.1109/icpc.2009.5090030 · Cited by 24
Automatically detecting developer activities and problems in software development work
Roehm, T., & Maalej, W. (2012) · doi:10.5555/2337223.2337390 · Cited by 12
Self-citationThe Future of Social Learning in Software Engineering
Murphy-Hill, E. R. (2014) · Computer · doi:10.1109/mc.2013.406 · Cited by 9
Grubb, A. M., & Begel, A. (2012) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/2145204.2145403 · Cited by 8
Research works printed but not linked: 5
- Francoise Detienne . 2002. Software Design -- Cognitive Aspect . Springer . Francoise Detienne. 2002. Software Design -- Cognitive Aspect. Springer.
- R. K. Fjeldstad and W. T. Hamlen . 1979. Application program maintenance study: Report to our respondents . In Proceedings of GUIDE 48 . PA. R. K. Fjeldstad and W. T. Hamlen. 1979. Application program maintenance study: Report to our respondents. In Proceedings of GUIDE 48. PA.
- G. Gilder . 2013. Knowledge and Power: The Information Theory of Capitalism and How it is Revolutionizing our World . Regnery Publishing . G. Gilder. 2013. Knowledge and Power: The Information Theory of Capitalism and How it is Revolutionizing our World. Regnery Publishing.
- 10.5555/1018434.1021568 · doi:10.5555/1018434.1021568
- A. Von Mayrhauser , A. M. Vans , and S. Lang . 1998. Program comprehension and enhancement of software . In Proceedings of the IFIP World Computing Congress - Information Technology and Knowledge Engineering. A. Von Mayrhauser, A. M. Vans, and S. Lang. 1998. Program comprehension and enhancement of software. In Proceedings of the IFIP World Computing Congress - Information Technology and Knowledge Engineering.
Cited by
Using an LLM to Help With Code Understanding
Nam, D., Macvean, A., Hellendoorn, V., Vasilescu, B., & Myers, B. (2024) · Proceedings of the IEEE/ACM 46th International Conference on Software Engineering · doi:10.1145/3597503.3639187 · Cited by 487
“” This behavior might be attributed to developers’ inclination to focus on task completion rather than comprehending the software, as reported in the literature [44].”
BackgroundResultMeasuring Program Comprehension: A Large-Scale Field Study with Professionals
Xia, X., Bao, L., Lo, D., Xing, Z., Hassan, A. E., & Li, S. (2018) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2017.2734091 · Cited by 326
Application Programming Interface Documentation: What Do Software Developers Want?
Meng, M., Steinhardt, S., & Schubert, A. (2018) · Journal of Technical Writing and Communication · doi:10.1177/0047281617721853 · Cited by 78
Spreadsheet Comprehension: Guesswork, Giving Up and Going Back to the Author
Srinivasa Ragavan, S., Sarkar, A., & Gordon, A. D. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445634 · Cited by 67
Experiences with Improving the Transparency of AI Models and Services
Hind, M., Houde, S., Martino, J., Mojsilovic, A., Piorkowski, D., Richards, J., & Varshney, K. R. (2020) · Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3334480.3383051 · Cited by 44
Metadata
- Authors
- Maalej, W., Tiarks, R., Roehm, T., & Koschke, R.
- Affiliations
- Universität Hamburg (DE)
- Technical University of Munich (DE)
- University of Bremen (DE)
- Venue
- ACM Transactions on Software Engineering and Methodology
- Published
- 2014-09-05
- Volume
- 23
- Issue
- 4
- Pages
- 1-37
- ISSN
- 1049-331X (ISSN-L)
- 1557-7392
- OpenAlex
- W2144651866
- Cited by
- 180
- References
- 71
- Citations by year
- 2026: 8 · 2025: 16 · 2024: 16 · 2023: 13 · 2022: 14 · 2021: 24 · 2020: 12 · 2019: 8 · 2018: 17 · 2017: 21 · 2016: 10 · 2015: 21
- FWCI
- 47.76
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- ACM Transactions on Software Engineering and Methodology (publishedVersion)
- Funders
- Seventh Framework Programme (FP7-258109)
- Deutsche Forschungsgemeinschaft (KO 2342/3-1/BR 2906/1-1)
- European Commission
- Sixth Framework Programme (FP6-35111)
- Topics
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Physical Sciences › Computer Science › Information Systems › Software Engineering Techniques and Practices
- Physical Sciences › Computer Science › Software › Advanced Software Engineering Methodologies
- Keywords
- program comprehension, developer practices, comprehension strategies, knowledge sharing, access to knowledge, team size, face-to-face communication, research-practice gap
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.