Measuring Program Comprehension: A Large-Scale Field Study with Professionals
The DOI matches its Crossref record; 63 of the 64 references it cites were found in a registry or an index.
Abstract
During software development and maintenance, developers spend a considerable amount of time on program comprehension activities. Previous studies show that program comprehension takes up as much as half of a developer's time. However, most of these studies are performed in a controlled setting, or with a small number of participants, and investigate the program comprehension activities only within the IDEs. However, developers' program comprehension activities go well beyond their IDE interactions. In this paper, we extend our ActivitySpace framework to collect and analyze Human-Computer Interaction (HCI) data across many applications (not just the IDEs). We follow Minelli et al.'s approach to assign developers' activities into four categories: navigation, editing, comprehension, and other. We then measure the comprehension time by calculating the time that developers spend on program comprehension, e.g., inspecting console and breakpoints in IDE, or reading and understanding tutorials in web browsers. Using this approach, we can perform a more realistic investigation of program comprehension activities, through a field study of program comprehension in practice across a total of seven real projects, on 78 professional developers, and amounting to 3,148 working hours. Our study leverages interaction data that is collected across many applications by the developers. Our study finds that on average developers spend ~58 percent of their time on program comprehension activities, and that they frequently use web browsers and document editors to perform program comprehension activities. We also investigate the impact of programming language, developers' experience, and project phase on the time that is spent on program comprehension, and we find senior developers spend significantly less percentages of time on program comprehension than junior developers. Our study also highlights the importance of several research directions needed to reduce program comprehension time, e.g., building automatic detection and improvement of low quality code and documentation, construction of software-engineering-specific search engines, designing better IDEs that help developers navigate code and browse information more efficiently, etc.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Statistical Power Analysis for the Behavioral Sciences
Muller, K. (1989) · Technometrics · doi:10.1080/00401706.1989.10488618 · Cited by 99,167
Tabachnick, B. G., & Fidell, L. S. (1983) · Cited by 80,042
An analysis of variance test for normality (complete samples)
SHAPIRO, S. S., & WILK, M. B. (1965) · Biometrika · doi:10.1093/biomet/52.3-4.591 · Cited by 23,480
Measuring nominal scale agreement among many raters.
Fleiss, J. L. (1971) · Psychological Bulletin · doi:10.1037/h0031619 · Cited by 8,733
Software engineering: A practitioner's approach
(1983) · Advances in Engineering Software (1978) · doi:10.1016/0141-1195(83)90118-3 · Cited by 8,234
Refactoring: Improving the Design of Existing Code
Fowler, M. (2002) · Lecture Notes in Computer Science · doi:10.1007/3-540-45672-4_31 · Cited by 5,591
Manifesto for agile software development
Beck, K., Beedle, M., van Bennekum, A., Cockburn, A., Cunningham, W., Fowler, M., & Thomas, D. A. (2001) · Cited by 4,284
Dual-task interference in simple tasks: data and theory
Pashler, H. (1994) · Psychological Bulletin · doi:10.1037/0033-2909.116.2.220 · Cited by 3,112
Refactoring improving the design of existing code
Saca, M. A. (2017) · 2017 IEEE 37th Central America and Panama Convention (CONCAPAN XXXVII) · doi:10.1109/concapan.2017.8278488 · Cited by 2,054
The psychology of computer programming
(1999) · Choice Reviews Online · doi:10.5860/choice.36-4535 · Cited by 962
Research works printed but not linked: 3
- whitaker, 1999, What causes IT workers to leave, Manag Rev, 88, 8
- spencer, 2009, Card Sorting Designing Usable Categories
- fjeldstad, 1983, Application program maintenance study: Report to our respondents, Proc GUIDE
Cited by
Items 101–200 of 325
Unveiling process insights from refactoring practices
Caldeira, J., Abreu, F. B. E., Cardoso, J., & dos Reis, J. P. (2021) · Computer Standards & Interfaces · doi:10.1016/j.csi.2021.103587 · Cited by 12
Dazed and Confused: Studying the Prevalence of Atoms of Confusion in Long-Lived Java Libraries
Mendes, W., Pinheiro, O., Santos, E., Rocha, L. S., & Viana, W. (2022) · IEEE International Conference on Software Maintenance (ICSM) · doi:10.1109/icsme55016.2022.00018 · Cited by 9
Towards the Analysis and Completion of Syntactic Structure Ellipsis for Inline Comments
Zhang, X., Zou, W., Chen, L., Li, Y., & Zhou, Y. (2022) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2022.3216279 · Cited by 8
TreeCaps: Tree-Structured Capsule Networks for Program Source Code ProcessingPreprint
Jayasundara, V., Bui, N. D. Q., Jiang, L., & Lo, D. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.12306 · Cited by 9
Self-citationAn eye tracking study assessing source code readability rules for program comprehension
Park, K.-I., Johnson, J., Peterson, C. S., Yedla, N., Baysinger, I., Aponte, J., & Sharif, B. (2024) · Empirical Software Engineering · doi:10.1007/s10664-024-10532-x · Cited by 9
Code Comment Inconsistency Detection and Rectification Using a Large Language Model
Rong, G., Yu, Y., Liu, S., Tan, X., Zhang, T., Shen, H., & Hu, J. (2025) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse55347.2025.00035 · Cited by 34
Bash comment generation via data augmentation and semantic-aware CodeBERT
Shen, Y., Ju, X., Chen, X., & Yang, G. (2024) · Automated Software Engineering · doi:10.1007/s10515-024-00431-2 · Cited by 13
Source Code Summarization & Comment Generation with NLP : A New Index Proposal
Kilic, M. A. E., & Adak, M. F. (2024) · International Congress on Human-Computer Interaction, Optimization and Robotic Applications (ICHORA) · doi:10.1109/hora61326.2024.10550711 · Cited by 9
Towards Summarizing Code Snippets Using Pre-Trained Transformers
Mastropaolo, A., Ciniselli, M., Pascarella, L., Tufano, R., Aghajani, E., & Bavota, G. (2024) · IEEE International Conference on Program Comprehension (ICPC) · doi:10.1145/3643916.3644400 · Cited by 23
Comprehension effort and programming activities: related? or not related?
Rahman, A. (2018) · International Conference on Mining Software Repositories (MSR) · doi:10.1145/3196398.3196470 · Cited by 9
Code Comprehension Confounders: A Study of Intelligence and Personality
Wagner, S., & Wyrich, M. (2021) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2021.3127131 · Cited by 8
Scalable Collaborative Software Visualization as a Service: Short Industry and Experience Paper
Krause-Glau, A., & Hasselbring, W. (2022) · IEEE International Conference on Cloud Engineering · doi:10.1109/ic2e55432.2022.00026 · Cited by 14
How do Students Experience and Judge Software Comprehension Techniques?
Hebig, R., Ho-Quang, T., Jolak, R., Schröder, J., Linero, H., Ågren, M., & Maro, S. (2020) · International Conference on Program Comprehension · doi:10.1145/3387904.3389283 · Cited by 8
Semantic Code Graph—An Information Model to Facilitate Software Comprehension
Borowski, K., Baliś, B., & Orzechowski, T. M. (2024) · IEEE Access · doi:10.1109/access.2024.3351845 · Cited by 17
Zhang, Z., Xing, Z., Zhao, D., Lu, Q., Xu, X., & Zhu, L. (2024) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3597503.3639101 · Cited by 7
Self-citationSegalotto, M., Bolzan, W., & Farias, K. (2023) · Brazilian Symposium on Software Engineering · doi:10.1145/3613372.3613387 · Cited by 12
TreeCaps: Tree-Based Capsule Networks for Source Code Processing
Bui, N. D. Q., Yu, Y., & Jiang, L. (2020) · Open Research Online (The Open University) · doi:10.48550/arxiv.2009.09777 · Cited by 7
Detecting Inconsistencies in Software Architecture Documentation Using Traceability Link Recovery
Keim, J., Corallo, S., Fuchß, D., & Koziolek, A. (2023) · IEEE International Conference on Software Architecture (ICSA) · doi:10.1109/icsa56044.2023.00021 · Cited by 25
EnCoSum: enhanced semantic features for multi-scale multi-modal source code summarization
Gao, Y., Zhang, H., & Lyu, C. (2023) · Empirical Software Engineering · doi:10.1007/s10664-023-10384-x · Cited by 9
Decoding Confusing Code: Social Representations among Developers
de Mello, R., da Costa, J. A. S., de Oliveira, B., Ribeiro, M., Fonseca, B., Gheyi, R., Garcia, A., & Tiengo, W. C. (2021) · International Workshop on Cooperative and Human Aspects of Software Engineering (CHASE) · doi:10.1109/chase52884.2021.00010 · Cited by 8
Enhanced Prompting Framework for Code Summarization with Large Language Models
Fang, M., Yuan, X., Li, Y., Li, H., Fang, C., & Du, J. (2025) · Proceedings of the ACM on software engineering. · doi:10.1145/3728949 · Cited by 15
Multi-Intent Inline Code Comment Generation via Large Language Model
Zhang, X., Chen, Z., Cao, Y., Chen, L., & Zhou, Y. (2024) · International Journal of Software Engineering and Knowledge Engineering · doi:10.1142/s0218194024500050 · Cited by 7
Modelling of Cyber-Physical Systems through Domain-Specific Languages: Decision, Analysis, Design
Gerhold, M., Kouzel, A., Mangal, H., Mehmed, S., & Zaytsev, V. (2024) · ACM/IEEE International Conference on Model Driven Engineering Languages and Systems (MODELS) · doi:10.1145/3652620.3688348 · Cited by 6
Live Visualization of Dynamic Software Cities with Heat Map Overlays
Krause, A., Hansen, M., & Hasselbring, W. (2021) · IEEE Working Conference on Software Visualization (VISSOFT) · doi:10.1109/vissoft52517.2021.00024 · Cited by 6
GA-SCS: Graph-Augmented Source Code Summarization
Zhang, M., Zhou, G., Yu, W., Huang, N., & Liu, W. (2023) · ACM Transactions on Asian and Low-Resource Language Information Processing · doi:10.1145/3554820 · Cited by 8
Feature transfer learning by reinforcement learning for detecting software defect
Guo, S., Wang, J., Xu, Z., Huang, L., Li, H., & Chen, R. (2022) · Software Practice and Experience · doi:10.1002/spe.3152 · Cited by 6
Identifying Lines and Interpreting Vertical Jumps in Eye Tracking Studies of Reading Text and Code
Shamy, M., & Feitelson, D. G. (2023) · ACM Transactions on Applied Perception · doi:10.1145/3579357 · Cited by 6
Learning a holistic and comprehensive code representation for code summarization
Yang, K., Wang, J., & Song, Z. (2023) · Journal of Systems and Software · doi:10.1016/j.jss.2023.111746 · Cited by 7
Summarizing source code with Heterogeneous Syntax Graph and dual position
Guo, J., Liu, J., Liu, X., Wan, Y., & Li, L. (2023) · Information Processing & Management · doi:10.1016/j.ipm.2023.103415 · Cited by 8
Ribeiro, T. V., dos Santos, P. S. M., & Travassos, G. H. (2023) · Empirical Software Engineering · doi:10.1007/s10664-023-10360-5 · Cited by 8
How They Relate and Leave: Understanding Atoms of Confusion in Open-Source Java Projects
Pinheiro, O., Rocha, L. S., & Viana, W. (2023) · IEEE International Working Conference on Source Code Analysis and Manipulation (SCAM) · doi:10.1109/scam59687.2023.00022 · Cited by 10
On the comprehensibility of functional decomposition: An empirical study
Tempero, E. D., Denny, P. C., Finnie-Ansley, J., Luxton-Reilly, A., Kirk, D., Leinonen, J., Shakil, A., Sheehan, R., Tizard, J., Tu, Y., et al. (2024) · IEEE International Conference on Program Comprehension (ICPC) · doi:10.1145/3643916.3644432 · Cited by 7
Zhang, J., Liu, S., Luo, J., Liang, J., & Huang, Z. (2020) · IEEE Access · doi:10.1109/access.2020.3013694 · Cited by 7
Retrieve and Refine: Exemplar-based Neural Comment GenerationPreprint
Wei, B. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.10419 · Cited by 6
Task-oriented Analysis on Debugging Process Based on Eye Movements and IDE Interactions
Li, X., Liu, W., Liu, H., Xu, J., & Cheng, W. (2021) · International Conference on Computer Science & Education (ICCSE) · doi:10.1109/iccse51940.2021.9569438 · Cited by 5
APIContext2Com: Code Comment Generation by Incorporating Pre-Defined API Documentation
Shahbazi, R., & Fard, F. H. (2023) · IEEE International Conference on Program Comprehension (ICPC) · doi:10.1109/icpc58990.2023.00012 · Cited by 10
Evaluating a Language Workbench: from Working Memory Capacity to Comprehension to Acceptance
Broccia, G., Ferrari, A., ter Beek, M. H., Cazzola, W., Favalli, L., & Bertolotti, F. (2023) · IEEE International Conference on Program Comprehension (ICPC) · doi:10.1109/icpc58990.2023.00017 · Cited by 6
Liang, H.-M., & Huang, C. (2024) · IEEE Access · doi:10.1109/access.2024.3354390 · Cited by 6
Improve Code Summarization via Prompt-Tuning CodeT5
Huanzhen, L. I. (2023) · Wuhan University Journal of Natural Sciences · doi:10.1051/wujns/2023286474 · Cited by 6
Effect of Identifier Tokenization on Automatic Source Code Documentation
Rai, S., Belwal, R. C., & Gupta, A. K. (2021) · Arabian Journal for Science and Engineering · doi:10.1007/s13369-021-06149-7 · Cited by 5
Students’ Assessments in Learning Programming based on Bloom’s Taxonomy
Kadar, R., Yusoff, S. A. M., Warris, S. N., & Bakar, M. S. A. (2021) · Journal of Computing Research and Innovation · doi:10.24191/jcrinn.v6i3.223 · Cited by 9
Stereotypes as Design Patterns for Serious Games to Enhance Software Comprehension
Gökmen, R., Heidrich, D., Schreiber, A., & Bichlmeier, C. (2021) · IEEE Conference on Games (CoG) · doi:10.1109/cog52621.2021.9619060 · Cited by 5
The effect of information content and length on name recollection
Etgar, A., Friedman, R., Haiman, S., Perez, D., & Feitelson, D. G. (2022) · IEEE International Conference on Program Comprehension (ICPC) · doi:10.1145/3524610.3529159 · Cited by 7
Completing Function Documentation Comments Using Structural Information
Ciurumelea, A., Alexandru, C. V., Gall, H. C., & Proksch, S. (2023) · Empirical Software Engineering · doi:10.1007/s10664-022-10284-6 · Cited by 8
Towards Generation of Visual Attention Map for Source Code
Itoh, T., Kubo, T., Ikeda, K., Maruno, Y., Ikutani, Y., Hata, H., Matsumoto, K., & Ikeda, K. (2019) · Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC) · doi:10.1109/apsipaasc47483.2019.9023036 · Cited by 5
Predicting the understandability of computational notebooks through code metrics analysis
Ghahfarokhi, M. M., Asadi, A., Asgari, A., Mohammadi, B., & Heydarnoori, A. (2025) · Empirical Software Engineering · doi:10.1007/s10664-025-10651-z · Cited by 8
Zhao, J., Song, Y., & Cohen, E. (2025) · Journal of Systems and Software · doi:10.1016/j.jss.2025.112493 · Cited by 6
BOHR - Uma Ferramenta para a Identificação de Átomos de Confusão em Códigos Java
Mendes, W., Viana, W., & Rocha, L. S. (2021) · doi:10.5753/vem.2021.17216 · Cited by 4
Gorchakov, A. V., Demidova, L. A., & Sovietov, P. N. (2024) · Computers · doi:10.3390/computers13030075 · Cited by 8
Exploring the Effectiveness of Abstract Syntax Tree Patterns for Algorithm Recognition
Neumüller, D., Sihler, F., Straub, R., & Tichy, M. (2024) · International Conference on Code Quality (ICCQ) · doi:10.1109/iccq60895.2024.10576984 · Cited by 4
Chen, A., Wong, C., Sharif, B., & Peruma, A. (2025) · IEEE International Conference on Program Comprehension (ICPC) · doi:10.1109/icpc66645.2025.00043 · Cited by 7
Feature location enhancement based on source code augmentation with synonyms of terms
Saifan, A. A., & Obeidat, L. M. (2020) · Software Practice and Experience · doi:10.1002/spe.2900 · Cited by 4
An Enhanced Data Augmentation Approach to Support Multi-Class Code Readability Classification
Mi, Q. S., Hao, Y., Wu, M., & Ou, L. (2022) · Proceedings/Proceedings of the ... International Conference on Software Engineering and Knowledge Engineering · doi:10.18293/seke2022-130 · Cited by 4
ARSeek: identifying API resource using code and discussion on stack overflow
Luong, K., Hadi, M., Thung, F., Fard, F. H., & Lo, D. (2022) · IEEE International Conference on Program Comprehension (ICPC) · doi:10.1145/3524610.3527918 · Cited by 6
Self-citationPassSum: Leveraging paths of abstract syntax trees and self‐supervision for code summarization
Niu, C., Li, C., Ng, V., Ge, J., Huang, L., & Luo, B. (2023) · Journal of Software Evolution and Process · doi:10.1002/smr.2620 · Cited by 5
Multi-granular software annotation using file-level weak labelling
Sas, C., & Capiluppi, A. (2023) · Empirical Software Engineering · doi:10.1007/s10664-023-10423-7 · Cited by 8
Summarizing Source Code from Structure and Context
Hou, S., Chen, L., & Ye, Y. (2022) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn55064.2022.9892013 · Cited by 5
Profiling Software Developers with Process Mining and N-Gram Language ModelsPreprint
Caldeira, J., Abreu, F. B. E., Cardoso, J., Ribeiro, R., & Werner, C. M. L. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2101.06733 · Cited by 4
Needles in a Haystack: Student Struggles with Working on Large Code Bases
Shah, A., Rexin, T., Chernova, A., Allen-Perez, G., Griswold, W. G., & Raj, A. G. S. (2025) · ACM Conference on International Computing Education Research (ICER) · doi:10.1145/3702652.3744218 · Cited by 7
Towards Generating Labeled Property Graphs for Comprehending C#-based Software Projects
Heidrich, D., Schreiber, A., & Oberdörfer, S. (2022) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/3551349.3560513 · Cited by 3
ICG: A Machine Learning Benchmark Dataset and Baselines for Inline Code Comments Generation Task
Zhang, X., Chen, L., Zou, W., Cao, Y., Ren, H., Wang, Z., Li, Y., & Zhou, Y. (2023) · International Journal of Software Engineering and Knowledge Engineering · doi:10.1142/s0218194023500547 · Cited by 3
Beyond code: Is there a difference between comments in visual and textual languages?
Boll, A., Rani, P., Schultheiß, A., & Kehrer, T. (2024) · Journal of Systems and Software · doi:10.1016/j.jss.2024.112087 · Cited by 5
Nierstrasz, O., & Gîrba, T. (2024) · European Conference on Pattern Languages of Programs · doi:10.1145/3698322.3698327 · Cited by 3
Test Script Intention Generation for Mobile Application via GUI Image and Code Understanding
Yu, S., Fang, C., Liu, J., & Chen, Z. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3722105 · Cited by 10
Exploring User Experience of Automatic Documentation Tools
Wu, S., Zhou, Y., & Wang, X. (2021) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3411763.3451606 · Cited by 5
Report on a Field Experiment of the Comprehension Strategies of Computer Science MSc Students
Fekete, A., & Porkoláb, Z. (2022) · IEEE International Scientific Conference on Informatics (Informatics) · doi:10.1109/informatics57926.2022.10083413 · Cited by 3
Custom static analysis to enhance insight into the usage of in-house libraries
van de Laar, P., Corvino, R., Mooij, A. J., van Wezep, H., & Rosmalen, R. (2024) · Journal of Systems and Software · doi:10.1016/j.jss.2024.112028 · Cited by 5
Charting a Path to Efficient Onboarding: The Role of Software Visualization
Padoan, F., Santos, R. D. S., & Medeiros, R. P. (2024) · International Workshop on Cooperative and Human Aspects of Software Engineering (CHASE) · doi:10.1145/3641822.3641873 · Cited by 10
Pan, X., Liu, C., Zou, Y., Xie, T., & Xie, B. (2024) · IEEE International Conference on Program Comprehension (ICPC) · doi:10.1145/3643916.3644401 · Cited by 6
Allen, J., & Kelleher, C. L. (2024) · IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) · doi:10.1109/vl/hcc60511.2024.00025 · Cited by 7
Krause-Glau, A., Damerau, L., Hansen, M., & Hasselbring, W. (2024) · IEEE Working Conference on Software Visualization (VISSOFT) · doi:10.1109/vissoft64034.2024.00028 · Cited by 9
Early Career Developers’ Perceptions of Code Understandability: A Study of Complexity Metrics
Esposito, M., Janes, A., Kilamo, T., & Lenarduzzi, V. (2025) · IEEE Access · doi:10.1109/access.2025.3585777 · Cited by 5
A COMPARISON STUDY : THE EFFECT OF NOUNS AND VERBS IN FINDING FEATURE LOCATION
Arwan, A., Rochimah, S., & Fatichah, C. (2022) · International Conference on Sustainable Information Engineering and Technology (SIET) · doi:10.1145/3568231.3568282 · Cited by 3
The Code Generation Method Based on Gated Attention and InterAction-LSTM
Wang, Y., & Wu, J. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-87571-8_47 · Cited by 3
A Decade of Code Comment Quality Assessment: A Systematic Literature Review
Rani, P., Blasi, A., Stulova, N., Panichella, S., Gorla, A., & Nierstrasz, O. (2022) · Zürcher Hochschule für Angewandte Wissenschaften digital collection (Zurich University of Applied Sciences) · doi:10.48550/arxiv.2209.08165 · Cited by 3
A Systematic Literature Review on the Impact of Formatting Elements on Program UnderstandabilityPreprint
Oliveira, D., Bruno, R., Madeiral, F., Masuhara, H., & Castor, F. (2022) · SSRN Electronic Journal · doi:10.2139/ssrn.4182156 · Cited by 3
Label Smoothing Improves Neural Source Code Summarization
Haque, S., Bansal, A., & McMillan, C. (2023) · IEEE International Conference on Program Comprehension (ICPC) · doi:10.1109/icpc58990.2023.00025 · Cited by 7
On the Relationship between Code Verifiability and Understandability
Feldman, K., Kellogg, M., & Chaparro, O. (2023) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3611643.3616242 · Cited by 6
Learning Heterogeneous Abstract Code Graph Representations for Program Comprehension
Song, S., Zhang, M., Li, S., & Liu, H. (2024) · Asia-Pacific Software Engineering Conference (APSEC) · doi:10.1109/apsec65559.2024.00035 · Cited by 2
Self-citationAutomated Code Summarization by Training Large Language Models with Crowdsourced Knowledge
Xia, M., Maharjan, S., & Song, M. (2025) · IEEE/ACIS International Conference on Software Engineering Research, Management and Applications (SERA) · doi:10.1109/sera65747.2025.11154515 · Cited by 4
Heltweg, P., Schwarz, G., & Riehle, D. (2025) · Empirical Software Engineering · doi:10.1007/s10664-025-10746-7 · Cited by 3
CodeMap: A Graphical Note-Taking Tool Cooperating with an Integrated Development Environment
Taniguchi, R., & Masuhara, H. (2022) · International Conference on Art, Science, and Engineering of Programming (<Programming>) · doi:10.1145/3532512.3535225 · Cited by 2
RetCom: Information Retrieval-Enhanced Automatic Source-Code Summarization
Zhang, Y., Liu, Y., Fan, X., & Lu, Y. (2022) · IEEE International Conference on Software Quality, Reliability and Security (QRS) · doi:10.1109/qrs57517.2022.00099 · Cited by 3
Parameter Description Generation with the Code Parameter Flow
Chen, Q., Yang, Z., Liu, Z., Li, S., & Gao, C. (2022) · IEEE International Conference on Software Quality, Reliability and Security (QRS) · doi:10.1109/qrs57517.2022.00093 · Cited by 4
Self-citationToward Gaze-assisted Developer Tools
Kuang, P., Söderberg, E., Niehorster, D. C., & Höst, M. (2023) · IEEE/ACM International Conference on Software Engineering: New Ideas and Emerging Results (ICSE-NIER) · doi:10.1109/icse-nier58687.2023.00015 · Cited by 4
Slicito: Using Computational Notebooks for Program Comprehension
Husák, R., Kofroň, J., & Zavoral, F. (2023) · IEEE International Conference on Program Comprehension (ICPC) · doi:10.1109/icpc58990.2023.00019 · Cited by 2
Feijó, D. N., de Almeida, C. D. A., & Rocha, L. S. (2023) · doi:10.5753/vem.2023.235685 · Cited by 2
Do developer perceptions have borders? Comparing C code responses across continents
Zhuang, Y., Yu, Y., DeLong, L. A., & Yeh, M. K. ‐. (2023) · Software Quality Journal · doi:10.1007/s11219-023-09654-0 · Cited by 5
Collaborative, Code-Proximal Dynamic Software Visualization within Code Editors
Krause-Glau, A., & Hasselbring, W. (2023) · IEEE Working Conference on Software Visualization (VISSOFT) · doi:10.1109/vissoft60811.2023.00016 · Cited by 7
Statement-Grained Hierarchy Enhanced Code Summarization
Zhang, Q., Jin, D., Wang, Y., & Gong, Y. (2024) · Electronics · doi:10.3390/electronics13040765 · Cited by 1
MMCS: A Code Summarization Approach Based on Multi-Modal Feature Enhancement
Zhou, Z., & Liu, L. (2024) · International Conference on Consumer Electronics and Computer Engineering (ICCECE) · doi:10.1109/iccece61317.2024.10504240 · Cited by 4
Improving Retrieval-Augmented Code Comment Generation by Retrieving for Generation
Lu, H., & Liu, Z. (2024) · IEEE International Conference on Software Maintenance (ICSM) · doi:10.1109/icsme58944.2024.00040 · Cited by 14
Coding™: Development Task Visualization for SW Code Comprehension
Kim, T.-Y., Kim, S., & Ryu, D. (2021) · IEEE Working Conference on Software Visualization (VISSOFT) · doi:10.1109/vissoft52517.2021.00012 · Cited by 2
Query Expansion Based On User Requirements Clustering for Finding Feature Location
Arwan, A., Rochimah, S., & Fatichah, C. (2022) · International Conference on Information Technology, Information Systems and Electrical Engineering (ICITISEE) · doi:10.1109/icitisee57756.2022.10057893 · Cited by 2
Weak Labelling for File-level Source Code Classification
Sas, C., & Capiluppi, A. (2023) · IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) · doi:10.1109/saner56733.2023.00074 · Cited by 6
Q&A MAESTRO: Q&A Post Recommendation for Fixing Java Runtime Exceptions
Kimura, Y., Akazaki, T., Kikuchi, S., Mahajan, S., & Prasad, M. R. (2021) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase51524.2021.9678893 · Cited by 3
An Empirical Study on the Usage and Evolution of Identifier Styles in Practice
Zhang, J., Zou, W., & Huang, Z. (2021) · Asia-Pacific Software Engineering Conference (APSEC) · doi:10.1109/apsec53868.2021.00025 · Cited by 3
Multi-Modal Code Summarization with Retrieved Summary
Lin, L., Huang, Z., Yu, Y., & Liu, Y. (2022) · IEEE International Working Conference on Source Code Analysis and Manipulation (SCAM) · doi:10.1109/scam55253.2022.00020 · Cited by 2
Metadata
- Authors
- Xia, X., Bao, L., Lo, D., Xing, Z., Hassan, A. E., & Li, S.
- Affiliations
- Zhejiang University (CN)
- Singapore Management University (SG)
- Australian National University (AU)
- Queen's University (CA)
- Venue
- IEEE Transactions on Software Engineering
- Published
- 2018-10-01
- Volume
- 44
- Issue
- 10
- Pages
- 951-976
- ISSN
- 0098-5589 (ISSN-L)
- 1939-3520
- 2326-3881
- OpenAlex
- W2741561716
- Cited by
- 326
- References
- 64
- Citations by year
- 2026: 30 · 2025: 37 · 2024: 54 · 2023: 62 · 2022: 60 · 2021: 38 · 2020: 22 · 2019: 14 · 2018: 8
- FWCI
- 22.89
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- IEEE Transactions on Software Engineering (publishedVersion)
- Singapore Management University Institutional Knowledge (InK) (Singapore Management University) (submittedVersion)
- Funders
- National Natural Science Foundation of China (61602403, 61572426)
- National Key Technology R&D Program
- Ministry of Science and Technology of China (2015BAH17F01)
- Topics
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Physical Sciences › Computer Science › Information Systems › Software Engineering Techniques and Practices
- Physical Sciences › Computer Science › Software › Advanced Software Engineering Methodologies
- Keywords
- program comprehension, professional developers, developer experience, project phases, software maintenance
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.