An Exploratory Study of How Developers Seek, Relate, and Collect Relevant Information during Software Maintenance Tasks
Abstract
Much of software developers' time is spent understanding unfamiliar code. To better understand how developers gain this understanding and how software development environments might be involved, a study was performed in which developers were given an unfamiliar program and asked to work on two debugging tasks and three enhancement tasks for 70 minutes. The study found that developers interleaved three activities. They began by searching for relevant code both manually and using search tools; however, they based their searches on limited and misrepresentative cues in the code, environment, and executing program, often leading to failed searches. When developers found relevant code, they followed its incoming and outgoing dependencies, often returning to it and navigating its other dependencies; while doing so, however, Eclipse's navigational tools caused significant overhead. Developers collected code and other information that they believed would be necessary to edit, duplicate, or otherwise refer to later by encoding it in the interactive state of Eclipse's package explorer, file tabs, and scroll bars. However, developers lost track of relevant code as these interfaces were used for other tasks, and developers were forced to find it again. These issues caused developers to spend, on average, 35 percent of their time performing the mechanics of navigation within and between source files. These observations suggest a new model of program understanding grounded in theories of information foraging and suggest ideas for tools that help developers seek, relate, and collect information in a more effective and explicit manner.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Developers spend 35%-50% of their time navigating through source code during development activities.
“Empirical studies have revealed that software developers spend 35%-50% of their time navigating through source code during development activities”
From Foraging and navigations, fundamentally: developers' predictions of value and cost (2016) · cites this paper as [21]
Searches in code frequently failed to turn up desired results, and developers spent substantial time organizing and reorganizing files and bookmarks in the IDE.
“This result was consistent with an earlier study into the challenges of information foraging by developers, such as the fact that searches in code frequently failed to turn up desired results, and that developers spent substantial time organizing and reorganizing files and bookmarks in the IDE [21].”
From Foraging and navigations, fundamentally: developers' predictions of value and cost (2016) · cites this paper as [21]
Cites
Robertson, S. I. (2001) · Cited by 1,650 · see citing works
The vocabulary problem in human-system communication
Furnas, G. W., Landauer, T. K., Gomez, L. M., & Dumais, S. T. (1987) · Communications of the ACM · doi:10.1145/32206.32212 · Cited by 1,504 · see citing works
Pirolli, P. L. T., & Card, S. K. (1999) · Psychological Review · doi:10.1037/0033-295x.106.4.643 · Cited by 1,354 · see citing works
Usability Analysis of Visual Programming Environments: A ‘Cognitive Dimensions’ Framework
Green, T. R. G., & Petre, M. (1996) · Journal of Visual Languages & Computing · doi:10.1006/jvlc.1996.0009 · Cited by 1,252 · see citing works
The Time Famine: Toward a Sociology of Work Time
Perlow, L. A. (1999) · Administrative Science Quarterly · doi:10.2307/2667031 · Cited by 978 · see citing works
Programmers use slices when debugging
Weiser, M. (1982) · Communications of the ACM · doi:10.1145/358557.358577 · Cited by 763 · see citing works
"Constant, constant, multi-tasking craziness": managing multiple working spheres
González, V. M., & Mark, G. (2004) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/985692.985707 · Cited by 715 · see citing works
Does code decay? Assessing the evidence from change management data
Eick, S. G., Graves, T., Karr, A. F., Marron, J. S., & Mockus, A. (2001) · IEEE Transactions on Software Engineering · doi:10.1109/32.895984 · Cited by 637 · see citing works
Maintaining mental models: a study of developer work habits
LaToza, T. D., Venolia, G., & DeLine, R. (2006) · Proceedings of the 28th international conference on Software engineering · doi:10.1145/1134285.1134355 · Cited by 628 · see citing works
Boehm (1976) · IEEE Transactions on Computers · doi:10.1109/tc.1976.1674590 · Cited by 545 · see citing works
Stimulus structures and mental representations in expert comprehension of computer programs
Pennington, N. (1987) · Cognitive Psychology · doi:10.1016/0010-0285(87)90007-7 · Cited by 526 · see citing works
Ko, A. J., & Myers, B. A. (2004) · doi:10.1145/985692.985712 · Cited by 362 · see citing works
Self-citationHipikat: recommending pertinent software development artifacts
Čubranić, D., & Murphy, G. C. (2003) · doi:10.5555/776816.776866 · Cited by 317 · see citing works
An examination of software engineering work practices
Singer, J., Lethbridge, T. C., Vinson, N. G., & Anquetil, N. (2010) · doi:10.1145/1925805.1925815 · Cited by 298 · see citing works
Robillard, M. P., & Murphy, G. C. (2002) · Proceedings/Proceedings - International Conference on Software Engineering · doi:10.1145/581339.581390 · Cited by 288 · see citing works
How effective developers investigate source code: an exploratory study
Robillard, M. P., Coelho, W., & Murphy, G. C. (2004) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2004.101 · Cited by 263 · see citing works
Representing concerns in source code
Robillard, M. P., & Murphy, G. C. (2007) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/1189748.1189751 · Cited by 215 · see citing works
Mental models and software maintenance
Littman, D., Pinto, J., Letovsky, S. I., & Soloway, E. M. (1987) · Journal of Systems and Software · doi:10.1016/0164-1212(87)90033-1 · Cited by 209 · see citing works
A framework and methodology for studying the causes of software errors in programming systems
Ko, A. J., & Myers, B. A. (2004) · Journal of Visual Languages & Computing · doi:10.1016/j.jvlc.2004.08.003 · Cited by 195 · see citing works
Self-citationTowards a theory of the cognitive processes in computer programming
Brooks, R. (1999) · International Journal of Human-Computer Studies · doi:10.1006/ijhc.1977.0306 · Cited by 192 · see citing works
Eliciting design requirements for maintenance-oriented IDEs
Ko, A. J., Aung, H. H., & Myers, B. A. (2005) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/1062455.1062492 · Cited by 182 · see citing works
Self-citationHipikat: recommending pertinent software development artifacts
Čubranić, D., & Murphy, G. C. (2003) · Proceedings/Proceedings - International Conference on Software Engineering · doi:10.1109/icse.2003.1201219 · Cited by 174 · see citing works
Models and theories of programming strategy
Davies, S. P. (1993) · International Journal of Man-Machine Studies · doi:10.1006/imms.1993.1061 · Cited by 172 · see citing works
Program indentation and comprehensibility
Miara, R. J., Musselman, J. A., Navarro, J. A., et al. (1983) · Communications of the ACM · doi:10.1145/182.358437 · Cited by 158 · see citing works
Human factors and typography for more readable programs
(1991) · Applied Ergonomics · doi:10.1016/0003-6870(91)90038-j · Cited by 145 · see citing works
Expert problem solving strategies for program comprehension
Koenemann, J., & Robertson, S. P. (1991) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/108844.108863 · Cited by 140 · see citing works
Automatic generation of suggestions for program investigation
Robillard, M. P. (2005) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/1081706.1081711 · Cited by 129 · see citing works
Examining task engagement in sensor-based statistical models of human interruptibility
Fogarty, J., Ko, A. J., Aung, H. H., Golden, E., Tang, K. P., & Hudson, S. E. (2005) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1054972.1055018 · Cited by 111 · see citing works
Self-citationCharacteristics of the mental representations of novice and expert programmers: an empirical study
Wiedenbeck, S., Fix, V., & Scholtz, J. C. (1993) · International Journal of Man-Machine Studies · doi:10.1006/imms.1993.1084 · Cited by 100 · see citing works
Substantiating programmer variability
Curtis, B. (1981) · Proceedings of the IEEE · doi:10.1109/proc.1981.12088 · Cited by 97 · see citing works
Design requirements for more flexible structured editors from a study of programmers' text editing
Ko, A. J., Aung, H. H., & Myers, B. A. (2005) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/1056808.1056965 · Cited by 94 · see citing works
Self-citationThe Emergent Structure of Development Tasks
Murphy, G. C., Kersten, M., Robillard, M. P., & Čubranić, D. (2005) · Lecture Notes in Computer Science · doi:10.1007/11531142_2 · Cited by 69 · see citing works
Program understanding behavior during corrective maintenance of large-scale software
Vans, A. M., von Mayrhauser, A., & Somlo, G. L. (1999) · International Journal of Human-Computer Studies · doi:10.1006/ijhc.1999.0268 · Cited by 66 · see citing works
Are visual programming languages better? The role of imagery in program comprehension
Navarro-Prieto, R., & Cañas, J. J. (2001) · Int. J. Hum. Comput. Stud. · doi:10.1006/ijhc.2000.0465 · Cited by 66 · see citing works
Knowledge restructuring and the acquisition of programming expertise
Davies, S. P. (1994) · International Journal of Human-Computer Studies · doi:10.1006/ijhc.1994.1032 · Cited by 65 · see citing works
Corritore, C. L., & Wiedenbeck, S. (1999) · International Journal of Human-Computer Studies · doi:10.1006/ijhc.1998.0236 · Cited by 64 · see citing works
Corritore, C. L., & Wiedenbeck, S. (2001) · International Journal of Human-Computer Studies · doi:10.1006/ijhc.2000.0423 · Cited by 60 · see citing works
Automatically inferring concern code from program investigation activities
Robillard, M. P., & Murphy, G. C. (2004) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase.2003.1240310 · Cited by 56 · see citing works
Robillard, M. P., & Murphy, G. C. (2002) · Proceedings/Proceedings - International Conference on Software Engineering · doi:10.1145/581388.581390 · Cited by 55 · see citing works
Results from software engineering research into open source development projects using public data
Koch, S., & Schneider, G. (2000) · ePubWU Institutional Repository (Wirtschaftsuniversität Wien) · doi:10.57938/f9d9dbcd-72bc-4a8c-a864-acc0145b7466 · Cited by 53 · see citing works
The effects of naming style and expertise on program comprehension
Teasley, B. E. (1994) · International Journal of Human-Computer Studies · doi:10.1006/ijhc.1994.1036 · Cited by 44 · see citing works
Managing crosscutting concerns during software evolution tasks
Baniassad, E., Murphy, G. C., Schwanninger, C., & Kircher, M. (2002) · International Conference on Aspect-Oriented Software Development (AOSD) · doi:10.1145/508386.508401 · Cited by 44 · see citing works
Simplifying data integration: the design of the desert software development environment
Reiss, S. P. (1996) · doi:10.5555/227726.227811 · Cited by 41 · see citing works
Supporting aggregation in fine grained software configuration management
Chu-Carroll, M. C., Wright, J., & Shields, D. (2002) · doi:10.1145/587051.587067 · Cited by 41 · see citing works
A glimpse of expert programmers' mental imagery
Petre, M., & Blackwell, A. F. (1997) · doi:10.1145/266399.266409 · Cited by 36 · see citing works
The value of slicing while debugging
Francel, M. A., & Rugaber, S. (2001) · Science of Computer Programming · doi:10.1016/s0167-6423(01)00013-2 · Cited by 34 · see citing works
Near-term memory in programming: a simulation-based analysis
Altmann, E. M. (2001) · International Journal of Human-Computer Studies · doi:10.1006/ijhc.2000.0407 · Cited by 33 · see citing works
Long Term Comprehension of Software Systems: A Methodology for Study.
Douce, C. R. (2001) · Cited by 9 · see citing works
Concern Highlight: A Tool for Concern Exploration and Visualization
Nistor, E. C., & van der Hoek, A. (2006) · Cited by 3 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Large Language Models for Software Engineering: A Systematic Literature Review
Hou, X., Zhao, Y., Liu, Y., Yang, Z., Wang, K., Li, L., Luo, X., Lo, D., Grundy, J., & Wang, H. (2024) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3695988 · Cited by 1,029 · see citing works
Information Needs in Collocated Software Development Teams
Ko, A. J., DeLine, R., & Venolia, G. (2007) · 29th International Conference on Software Engineering (ICSE'07) · doi:10.1109/icse.2007.45 · Cited by 499 · see citing works
Self-citationUsing an LLM to Help With Code Understanding
Nam, D., Macvean, A., Hellendoorn, V., Vasilescu, B., & Myers, B. (2024) · Proceedings of the IEEE/ACM 46th International Conference on Software Engineering · doi:10.1145/3597503.3639187 · Cited by 345 · see citing works
“Consequently, developers spend a significant amount of time searching and foraging for the information they need and organizing and digesting the information they find [30, 31, 34, 44, 48, 57].”
BackgroundSelf-citationMeasuring Program Comprehension: A Large-Scale Field Study with Professionals
Xia, X., Bao, L., Lo, D., Xing, Z., Hassan, A. E., & Li, S. (2018) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2017.2734091 · Cited by 326 · see citing works
Debugging reinvented: asking and answering why and why not questions about program behavior
Ko, A. J., & Myers, B. A. (2008) · Proceedings of the 30th international conference on Software engineering · doi:10.1145/1368088.1368130 · Cited by 296 · see citing works
Self-citationSupporting program comprehension with source code summarization
Haiduc, S., Aponte, J., & Marcus, A. (2010) · Proceedings of the 32nd ACM/IEEE International Conference on Software Engineering - Volume 2 · doi:10.1145/1810295.1810335 · Cited by 291 · see citing works
Myers, B. A., & Stylos, J. (2016) · Communications of the ACM · doi:10.1145/2896587 · Cited by 269 · see citing works
Self-citationImproving automated source code summarization via an eye-tracking study of programmers
Rodeghero, P., McMillan, C., McBurney, P. W., Bosch, N., & D'Mello, S. (2014) · Proceedings of the 36th International Conference on Software Engineering · doi:10.1145/2568225.2568247 · Cited by 198 · see citing works
On the Comprehension of Program Comprehension
Maalej, W., Tiarks, R., Roehm, T., & Koschke, R. (2014) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/2622669 · Cited by 180 · see citing works
Code bubbles: a working set-based interface for code understanding and maintenance
Bragdon, A., Zeleznik, R., Reiss, S. P., Karumuri, S., Cheung, W., Kaplan, J., Coleman, C., Adeputra, F., & LaViola, J. J. (2010) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1753326.1753706 · Cited by 174 · see citing works
Developers ask reachability questions
LaToza, T. D., & Myers, B. A. (2010) · Proceedings of the 32nd ACM/IEEE International Conference on Software Engineering - Volume 1 · doi:10.1145/1806799.1806829 · Cited by 145 · see citing works
Self-citationProgrammers Are Users Too: Human-Centered Methods for Improving Programming Tools
Myers, B. A., Ko, A. J., LaToza, T. D., & Yoon, Y. (2016) · Computer · doi:10.1109/mc.2016.200 · Cited by 144 · see citing works
Self-citationAnalyzing and mining a code search engine usage log
Bajracharya, S. K., & Lopes, C. V. (2012) · Empirical Software Engineering · doi:10.1007/s10664-010-9144-6 · Cited by 90 · see citing works
An Information Foraging Theory Perspective on Tools for Debugging, Refactoring, and Reuse Tasks
Fleming, S. D., Scaffidi, C., Piorkowski, D., Burnett, M., Bellamy, R., Lawrance, J., & Kwan, I. (2013) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/2430545.2430551 · Cited by 79 · see citing works
Using information scent to model the dynamic foraging behavior of programmers in maintenance tasks
Lawrance, J., Bellamy, R., Burnett, M., & Rector, K. (2008) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1357054.1357261 · Cited by 76 · see citing works
The whats and hows of programmers' foraging diets
Piorkowski, D. J., Fleming, S. D., Kwan, I., Burnett, M. M., Scaffidi, C., Bellamy, R. K. E., & Jordahl, J. (2013) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/2470654.2466418 · Cited by 61 · see citing works
Reactive information foraging for evolving goals
Lawrance, J., Burnett, M., Bellamy, R., Bogart, C., & Swart, C. (2010) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1753326.1753332 · Cited by 50 · see citing works
Piorkowski, D., Fleming, S., Scaffidi, C., Bogart, C., Burnett, M., John, B., Bellamy, R., & Swart, C. (2012) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/2207676.2208608 · Cited by 50 · see citing works
Stacksplorer: call graph navigation helps increasing code maintenance efficiency
Karrer, T., Krämer, J.-P., Diehl, J., Hartmann, B., & Borchers, J. (2011) · Proceedings of the 24th annual ACM symposium on User interface software and technology · doi:10.1145/2047196.2047225 · Cited by 48 · see citing works
Developers' code context models for change tasks
Fritz, T., Shepherd, D. C., Kevic, K., Snipes, W., & Bräunlich, C. (2014) · Proceedings of the 22nd ACM SIGSOFT International Symposium on Foundations of Software Engineering · doi:10.1145/2635868.2635905 · Cited by 45 · see citing works
Henley, A. Z., & Fleming, S. D. (2014) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/2556288.2557073 · Cited by 41 · see citing works
Automated Identification of Tasks in Development Sessions
Coman, I. D., & Sillitti, A. (2008) · 2008 16th IEEE International Conference on Program Comprehension · doi:10.1109/icpc.2008.16 · Cited by 38 · see citing works
Foraging and navigations, fundamentally: developers' predictions of value and cost
Piorkowski, D., Henley, A. Z., Nabi, T., Fleming, S. D., Scaffidi, C., & Burnett, M. (2016) · Proceedings of the 2016 24th ACM SIGSOFT International Symposium on Foundations of Software Engineering · doi:10.1145/2950290.2950302 · Cited by 38 · see citing works
Focus-shifting patterns of OSS developers and their congruence with call graphs
Xuan, Q., Okano, A., Devanbu, P., & Filkov, V. (2014) · Proceedings of the 22nd ACM SIGSOFT International Symposium on Foundations of Software Engineering · doi:10.1145/2635868.2635914 · Cited by 27 · see citing works
Inference of development activities from interaction with uninstrumented applications
Bao, L., Xing, Z., Xia, X., Lo, D., & Hassan, A. E. (2018) · Empirical Software Engineering · doi:10.1007/s10664-017-9547-8 · Cited by 25 · see citing works
Linking sketches and diagrams to source code artifacts
Baltes, S., Schmitz, P., & Diehl, S. (2014) · Proceedings of the 22nd ACM SIGSOFT International Symposium on Foundations of Software Engineering · doi:10.1145/2635868.2661672 · Cited by 26 · see citing works
Students' Use of GitHub Copilot for Working with Large Code Bases
Shah, A., Chernova, A., Tomson, E., Porter, L., Griswold, W. G., & Soosai Raj, A. G. (2025) · Proceedings of the 56th ACM Technical Symposium on Computer Science Education V. 1 · doi:10.1145/3641554.3701800 · Cited by 19 · see citing works
Zou, L., & Godfrey, M. W. (2012) · 2012 28th IEEE International Conference on Software Maintenance (ICSM) · doi:10.1109/icsm.2012.6405247 · Cited by 10 · see citing works
Towards Activity-Aware Tool Support for Change Tasks
Kevic, K., & Fritz, T. (2017) · 2017 IEEE International Conference on Software Maintenance and Evolution (ICSME) · doi:10.1109/icsme.2017.48 · Cited by 5 · see citing works
- Authors
- Ko, A. J., Myers, B. A., Coblenz, M. J., & Aung, H. H.
- Affiliations
- Human-Computer Interaction Institute, School of Computer Science, Carnegie Mellon University, 5000 Forbes Ave., Pittsburgh, PA 15213
- Carnegie Mellon University (US)
- Venue
- IEEE Transactions on Software Engineering
- Published
- 2006-12
- Volume
- 32
- Issue
- 12
- Pages
- 971-987
- ISSN
- 0098-5589 (ISSN-L)
- 1939-3520
- 2326-3881
- OpenAlex
- W2151996389
- Cited by
- 685
- References
- 50
- Citations by year
- 2026: 22 · 2025: 30 · 2024: 75 · 2023: 27 · 2022: 35 · 2021: 24 · 2020: 27 · 2019: 41 · 2018: 31 · 2017: 48 · 2016: 48 · 2015: 52 · 2014: 47 · 2013: 49 · 2012: 31
- FWCI
- 52.51
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- Locations
- IEEE Transactions on Software Engineering (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Physical Sciences › Computer Science › Information Systems › Software Engineering Techniques and Practices
- Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
- Keywords
- software maintenance, program comprehension, debugging, code navigation, Eclipse IDE, information foraging, information gathering, software development environment
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review