The DOI matches its Crossref record; 80 of the 80 references it cites were found in a registry or an index.
Asking and Answering Questions during a Programming Change Task
Open accessAbstract
Little is known about the specific kinds of questions programmers ask when evolving a code base and how well existing tools support those questions. To better support the activity of programming, answers are needed to three broad research questions: 1) What does a programmer need to know about a code base when evolving a software system? 2) How does a programmer go about finding that information? 3) How well do existing tools support programmers in answering those questions? We undertook two qualitative studies of programmers performing change tasks to provide answers to these questions. In this paper, we report on an analysis of the data from these two user studies. This paper makes three key contributions. The first contribution is a catalog of 44 types of questions programmers ask during software evolution tasks. The second contribution is a description of the observed behavior around answering those questions. The third contribution is a description of how existing deployed and proposed tools do, and do not, support answering programmers' questions.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Software developers, like all humans, are cognitively diverse (e.g., they differ in attitudes toward risk, information-processing and learning preferences, motivations for using a technology), which impacts how they problem-solve, manage uncertainty in work, or even engage with genAI [7, 19, 79, 83, 101, 126].”
From What Needs Attention? Prioritizing Drivers of Developers' Trust and Adoption of Generative AI (2025) · cites this paper as [126]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The Discovery of Grounded Theory: Strategies for Qualitative Research.
Wagner, H. R., Glaser, B. G., & Strauss, A. L. (1968) · Social Forces · doi:10.2307/2575405 · Cited by 46,361
The Discovery of Grounded Theory; Strategies for Qualitative Research
Glaser, B. G., Strauss, A. L., & Strutzel, E. (1968) · Nursing Research · doi:10.1097/00006199-196807000-00014 · Cited by 39,449
(1998) · Sage eBooks · Cited by 38,631
The magical number seven, plus or minus two: Some limits on our capacity for processing information.
Miller, G. A. (1994) · Psychological Review · doi:10.1037/0033-295x.101.2.343 · Cited by 17,161
Stroustrup, B. (1985) · Cited by 4,409
The Java Language Specification
Gosling, J., Joy, B., & Steele, G. L. (1996) · Cited by 3,924
Weiser, M. (1984) · IEEE Transactions on Software Engineering · doi:10.1109/tse.1984.5010248 · Cited by 3,204
No Silver Bullet Essence and Accidents of Software Engineering
Brooks (1987) · Computer · doi:10.1109/mc.1987.1663532 · Cited by 3,186
Kernighan, B. W., & Ritchie, D. M. (1978) · Cited by 1,857
CCFinder: a multilinguistic token-based code clone detection system for large scale source code
Kamiya, T., Kusumoto, S., & Inoue, K. (2002) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2002.1019480 · Cited by 1,711
Usability Analysis of Visual Programming Environments: A ‘Cognitive Dimensions’ Framework
Green, T. R. G., & Petre, M. (1996) · Journal of Visual Languages & Computing · doi:10.1006/jvlc.1996.0009 · Cited by 1,251
Representations in Distributed Cognitive Tasks
Zhang, J., & Norman, D. A. (1994) · Cognitive Science · doi:10.1207/s15516709cog1801_3 · Cited by 1,203 · cited 1×
“Though the process still revolves around lower-level questions and no direct support is provided to identifying relevant information, the resulting visualization may help a programmer in answering higherlevel questions by externalizing more supporting information [114].”
BackgroundThe Think Aloud Method: A Practical Guide to Modelling Cognitive Processes
van Someren, M. (1994) · Cited by 973
The nature of external representations in problem solving
Zhang, J. (1997) · Cognitive Science · doi:10.1016/s0364-0213(99)80022-6 · Cited by 896
Empirical Studies of Programming Knowledge
Soloway, E., & Ehrlich, K. (1984) · IEEE Transactions on Software Engineering · doi:10.1109/tse.1984.5010283 · Cited by 877
Ritchie, D. M., Johnson, S. C., Lesk, M. E., & Kernighan, B. W. (1983) · Programming Languages · doi:10.1007/978-3-662-09507-2_22 · Cited by 860
Strengthening the case for pair programming
Williams, L., Kessler, R. R., Cunningham, W., & Jeffries, R. (2000) · IEEE Software · doi:10.1109/52.854064 · Cited by 780
Towards a theory of the comprehension of computer programs
Brooks, R. (1983) · International Journal of Man-Machine Studies · doi:10.1016/s0020-7373(83)80031-5 · Cited by 756
The think aloud method: A practical guide to modelling cognitive processes
Solomon, P. (1995) · Information Processing & Management · doi:10.1016/0306-4573(95)90031-4 · Cited by 737
SMALLTALK-80: the interactive programming environment
Goldberg, A. (1984) · HAL (Le Centre pour la Communication Scientifique Directe) · Cited by 718
Ko, A. J., Myers, B. A., Coblenz, M. J., & Aung, H. H. (2006) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2006.116 · Cited by 685
Software Psychology: Human Factors in Computer and Information Systems
Shneiderman, B. (1980) · CERN Document Server (European Organization for Nuclear Research) · Cited by 617
Program comprehension during software maintenance and evolution
Von Mayrhauser, A., & Vans, A. M. (1995) · Computer · doi:10.1109/2.402076 · Cited by 575
Locating features in source code
Eisenbarth, T., Koschke, R., & Simon, D. (2003) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2003.1183929 · Cited by 554
Stimulus structures and mental representations in expert comprehension of computer programs
Pennington, N. (1987) · Cognitive Psychology · doi:10.1016/0010-0285(87)90007-7 · Cited by 526
Information Needs in Collocated Software Development Teams
Ko, A. J., DeLine, R., & Venolia, G. (2007) · 29th International Conference on Software Engineering (ICSE'07) · doi:10.1109/icse.2007.45 · Cited by 499
Using automatic clustering to produce high-level system organizations of source code
Mancoridis, S., Mitchell, B. S., Rorres, C., Chen, Y., & Gansner, E. R. (2002) · Proceedings. 6th International Workshop on Program Comprehension. IWPC'98 (Cat. No.98TB100242) · doi:10.1109/wpc.1998.693283 · Cited by 484
Comparing the Effectiveness of Software Testing Strategies
Basili, V. R., & Selby, R. W. (1987) · IEEE Transactions on Software Engineering · doi:10.1109/tse.1987.232881 · Cited by 483
A Principled Taxonomy of Software Visualization
Price, B. A., Baecker, R. M., & Small, I. S. (1993) · Journal of Visual Languages & Computing · doi:10.1006/jvlc.1993.1015 · Cited by 476
How Many Variables Can Humans Process?
Halford, G. S., Baker, R., McCredden, J. E., & Bain, J. D. (2005) · Psychological Science · doi:10.1111/j.0956-7976.2005.00782.x · Cited by 450
Constructive Interaction and the Iterative Process of Understanding
Miyake, N. (1986) · Cognitive Science · doi:10.1207/s15516709cog1002_2 · Cited by 445
How are Java software developers using the Eclipse IDE?
Murphy, G. C., Kersten, M., & Findlater, L. (2006) · IEEE Software · doi:10.1109/ms.2006.105 · Cited by 413
Self-citationConnecting tools using message passing in the Field environment
Reiss, S. P. (1990) · IEEE Software · doi:10.1109/52.56450 · Cited by 413
Visual momentum: a concept to improve the cognitive coupling of person and computer
Woods, D. D. (1984) · International Journal of Man-Machine Studies · doi:10.1016/s0020-7373(84)80043-7 · Cited by 389
Syntactic/semantic interactions in programmer behavior: A model and experimental results
Shneiderman, B., & Mayer, R. (1979) · International Journal of Computer & Information Sciences · doi:10.1007/bf00977789 · Cited by 382
Chianti: a tool for change impact analysis of java programs
Ren, X., Shah, F., Tip, F., Ryder, B. G., & Chesley, O. (2004) · Proceedings of the 19th annual ACM SIGPLAN conference on Object-oriented programming, systems, languages, and applications · doi:10.1145/1028976.1029012 · Cited by 382
Using structural context to recommend source code examples
Holmes, R., & Murphy, G. C. (2005) · Proceedings of the 27th international conference on Software engineering - ICSE '05 · doi:10.1145/1062455.1062491 · Cited by 379
Self-citationDesigning the whyline: a debugging interface for asking questions about program behavior
Ko, A. J., & Myers, B. A. (2004) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/985692.985712 · Cited by 361
The Interlisp Programming Environment
Teitelman, W., & Masinter, L. (1981) · Computer · doi:10.1109/c-m.1981.220410 · Cited by 355
A reverse‐engineering approach to subsystem structure identification
Müller, H. A., Orgun, M. A., Tilley, S. R., & Uhl, J. S. (1993) · Journal of Software Maintenance: Research and Practice · doi:10.1002/smr.4360050402 · Cited by 353
Hypermedia and cognition: designing for comprehension
Thüring, M., Hannemann, J., & Haake, J. M. (1995) · Communications of the ACM · doi:10.1145/208344.208348 · Cited by 351
Program understanding and the concept assignment problem
Biggerstaff, T. J., Mitbander, B. G., & Webster, D. E. (1994) · Communications of the ACM · doi:10.1145/175290.175300 · Cited by 346
Cognitive processes in program comprehension
Letovsky, S. (1987) · Journal of Systems and Software · doi:10.1016/0164-1212(87)90032-x · Cited by 327
The concept assignment problem in program understanding
Biggerstaff, T. J., Mitbander, B. G., & Webster, D. E. (1993) · doi:10.5555/257572.257679 · Cited by 325
Brown, M. H. (1988) · Cited by 306
An examination of software engineering work practices
Singer, J., Lethbridge, T., Vinson, N., & Anquetil, N. (2010) · CASCON First Decade High Impact Papers on - CASCON '10 · doi:10.1145/1925805.1925815 · Cited by 298
Harold, E. R., Means, W. S., & Petrycki, L. (2000) · Cited by 295
Questions programmers ask during software evolution tasks
Sillito, J., Murphy, G. C., & De Volder, K. (2006) · Proceedings of the 14th ACM SIGSOFT international symposium on Foundations of software engineering · doi:10.1145/1181775.1181779 · Cited by 270
Self-citationHow effective developers investigate source code: an exploratory study
Robillard, M. P., Coelho, W., & Murphy, G. C. (2004) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2004.101 · Cited by 260
Self-citationCase study of feature location using dependence graph
Chen, K., & Rajlich, V. (2002) · Proceedings IWPC 2000. 8th International Workshop on Program Comprehension · doi:10.1109/wpc.2000.852498 · Cited by 247
Cognitive design elements to support the construction of a mental model during software exploration
Storey, M.-A. D., Fracchia, F. D., & Müller, H. A. (1999) · Journal of Systems and Software · doi:10.1016/s0164-1212(98)10055-9 · Cited by 237
Navigating and querying code without getting lost
Janzen, D., & De Volder, K. (2003) · Proceedings of the 2nd international conference on Aspect-oriented software development · doi:10.1145/643603.643622 · Cited by 233
Self-citationTowards a theory of the cognitive processes in computer programming
BROOKS, R. (1999) · International Journal of Human-Computer Studies · doi:10.1006/ijhc.1977.0306 · Cited by 192
Designing documentation to compensate for delocalized plans
Soloway, E. M., Lampert, R., Letovsky, S., Littman, D., & Pinto, J. (1988) · Communications of the ACM · doi:10.1145/50087.50088 · Cited by 192
Mental models and software maintenance
Littman, D. C., Pinto, J., Letovsky, S., & Soloway, E. (1987) · Journal of Systems and Software · doi:10.1016/0164-1212(87)90033-1 · Cited by 181
Ko, A. J., Aung, H., & Myers, B. A. (2005) · Proceedings of the 27th international conference on Software engineering - ICSE '05 · doi:10.1145/1062455.1062492 · Cited by 180
Eclipse Platform Technical Overview
Als-Nscort (2003) · Purdue e-Pubs (Purdue University System) · Cited by 176
How do program understanding tools affect how programmers understand programs?
Storey, M.-A., Wong, K. K., & Müller, H. A. (2000) · Science of Computer Programming · doi:10.1016/s0167-6423(99)00036-2 · Cited by 156
Unifying artifacts and activities in a visual tool for distributed software development teams
Froehlich, J. E., & Dourish, P. (2004) · doi:10.5555/998675.999443 · Cited by 154
Expert problem solving strategies for program comprehension
Koenemann, J., & Robertson, S. P. (1991) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/108844.108863 · Cited by 140
Recovering high-level views of object-oriented applications from static and dynamic information
Richner, T., & Ducasse, S. (1999) · doi:10.1109/icsm.1999.792487 · Cited by 131
Storey, M.-A., Fracchia, F. D., & Müller, H. A. (2002) · IEEE International Conference on Program Comprehension (ICPC) · doi:10.1109/wpc.1997.601257 · Cited by 101
Towards understanding programs through wear-based filtering
DeLine, R., Khella, A., Czerwinski, M., & Robertson, G. (2005) · Proceedings of the 2005 ACM symposium on Software visualization · doi:10.1145/1056018.1056044 · Cited by 91
Using structural context to recommend source code examples
Holmes, R., & Murphy, G. C. (2005) · Proceedings/Proceedings - International Conference on Software Engineering · doi:10.1109/icse.2005.1553554 · Cited by 77
Self-citationFrom code understanding needs to reverse engineering tool capabilities
von Mayrhauser, A., & Vans, A. M. (2002) · International Workshop on Computer-Aided Software Engineering (CASE) · doi:10.1109/case.1993.634824 · Cited by 76
COGNITIVE SUPPORT IN SOFTWARE ENGINEERING TOOLS: A DISTRIBUTED COGNITION FRAMEWORK
Walenstein, A. (2002) · Cited by 65
Feat: A tool for locating, describing, and analyzing concerns in source code
Robillard, M. P., & Murphy, G. C. (2003) · Cited by 65
Self-citationPreserving knowledge in design projects: what designers need to know
Herbsleb, J. D., & Kuwana, E. (1993) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/169059.169061 · Cited by 57
Code extraction algorithms which unify slicing and concept assignment
Harman, M., Gold, N., Hierons, R. M., & Binkley, D. (2003) · Working Conference on Reverse Engineering (WCRE) · doi:10.1109/wcre.2002.1173060 · Cited by 55
The Trellis programming environment
O’Brien, P., Halbert, D. C., & Kilian, M. F. (1987) · Conference on Object-Oriented Programming Systems, Languages, and Applications · doi:10.1145/38765.38815 · Cited by 51
Early field experience with the Software Reconnaissance technique for program comprehension
Wilde, N., & Casey, C. J. (2002) · doi:10.1109/wcre.1996.558934 · Cited by 49
Partial comprehension of complex programs (enough to perform maintenance)
Erdos, K., & Sneed, H. M. (2002) · doi:10.1109/wpc.1998.693322 · Cited by 46
Shaykhian, G. A. (2007) · NASA STI Repository (National Aeronautics and Space Administration) · Cited by 44
Managing software change tasks: an exploratory study
Sillito, J., Volder, K. D., Fisher, B., & Murphy, G. C. (2005) · International Symposium on Empirical Software Engineering · doi:10.1109/isese.2005.1541811 · Cited by 39
Self-citationUsing Visual Momentum to Explain Disorientation in the Eclipse IDE
de Alwis, B., & Murphy, G. C. (2006) · IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) · doi:10.1109/vlhcc.2006.49 · Cited by 36
Self-citationInteractive Explanation of Software Systems
Johnson, W. L., & Erdem, A. (1997) · Automated Software Engineering · doi:10.1023/a:1008655629091 · Cited by 33
Relo: Helping Users Manage Context during Interactive Exploratory Visualization of Large Codebases
Sinha, V., Karger, D. R., & Miller, R. C. (2006) · IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) · doi:10.1109/vlhcc.2006.40 · Cited by 33
Task oriented software understanding
Erdem, A., Johnson, W. L., & Marsella, S. (2002) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase.1998.732658 · Cited by 30
Theory-based analysis of cognitive support in software comprehension tools
Walenstein, A. (2003) · IEEE International Conference on Program Comprehension (ICPC) · doi:10.1109/wpc.2002.1021322 · Cited by 28
Kuwana, E., & Herbsleb, J. D. (2002) · IEEE International Symposium on Requirements Engineering (RE) · doi:10.1109/isre.1993.324846 · Cited by 14
Cited by
Using an LLM to Help With Code Understanding
Nam, D., Macvean, A., Hellendoorn, V., Vasilescu, B., & Myers, B. (2024) · Proceedings of the IEEE/ACM 46th International Conference on Software Engineering · doi:10.1145/3597503.3639187 · Cited by 487
“Therefore, researchers have studied to understand how developers learn and comprehend unfamiliar code and domains [30, 38, 51, 62], especially how they search for and acquire information, as developers need a variety of kinds of knowledge [43, 66, 68, 77].”
BackgroundA field study of API learning obstacles
Robillard, M. P., & DeLine, R. (2011) · Empirical Software Engineering · doi:10.1007/s10664-010-9150-8 · Cited by 352
Portfolio: finding relevant functions and their usage
McMillan, C., Grechanik, M., Poshyvanyk, D., Xie, Q., & Fu, C. (2011) · Proceedings of the 33rd International Conference on Software Engineering · doi:10.1145/1985793.1985809 · Cited by 313
Abbes, M., Khomh, F., Gueheneuc, Y.-G., & Antoniol, G. (2011) · 2011 15th European Conference on Software Maintenance and Reengineering · doi:10.1109/csmr.2011.24 · Cited by 309
Improving automated source code summarization via an eye-tracking study of programmers
Rodeghero, P., McMillan, C., McBurney, P. W., Bosch, N., & D'Mello, S. (2014) · Proceedings of the 36th International Conference on Software Engineering · doi:10.1145/2568225.2568247 · Cited by 198
Metadata
- Authors
- Sillito, J., Murphy, G. C., & De Volder, K.
- Venue
- IEEE Transactions on Software Engineering
- Published
- 2008-07
- Volume
- 34
- Issue
- 4
- Pages
- 434-451
- ISSN
- 0098-5589 (ISSN-L)
- OpenAlex
- W2097750323
- Cited by
- 319
- References
- 80
- FWCI
- 40.41
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.