Measuring the Cognitive Load of Software Developers: A Systematic Mapping Study
The DOI matches its Crossref record; 8 of the 8 references it cites were found in a registry or an index.
Abstract
Context: In recent years, several studies explored different facets of the developers' cognitive load while executing tasks related to software engineering. Researchers have proposed and assessed different ways to measure developers' cognitive load at work and some studies have evaluated the interplay between developers' cognitive load and other attributes such as productivity and software quality. Problem: However, the body of knowledge about developers' cognitive load measurement is still dispersed. That hinders the effective use of developers' cognitive load measurements by industry practitioners and makes it difficult for researchers to build new scientific knowledge upon existing results. Objective: This work aims to pinpoint gaps providing a classification and a thematic analysis of studies on the measurement of cognitive load in the context of software engineering. Method: We carried out a Systematic Mapping Study (SMS) based on well-established guidelines to investigate nine research questions. In total, 33 articles (out of 2,612) were selected from 11 search engines after a careful filtering process. Results: The main findings are that (1) 55% of the studies adopted electroencephalogram (EEG) technology for monitoring the cognitive load; (2) 51% of the studies applied machine-learning classification algorithms for predicting cognitive load; and (3) 48% of the studies measured cognitive load in the context of programming tasks. Moreover, a taxonomy was derived from the answers of research questions. Conclusion: This SMS highlighted that the precision of machine learning techniques is low for realistic scenarios, despite the combination of a set of features related to developers' cognitive load used on these techniques. Thus, this gap makes the effective integration of the measure of developers' cognitive load in industry still a relevant challenge.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Experimentation in Software Engineering
Wohlin, C., Runeson, P., Höst, M., Ohlsson, M. C., Regnell, B., & Wesslén, A. (2012) · doi:10.1007/978-3-642-29044-2 · Cited by 4,460
Requirements engineering paper classification and evaluation criteria: a proposal and a discussion
Wieringa, R., Maiden, N., Mead, N., & Rolland, C. (2006) · Requirements Engineering · doi:10.1007/s00766-005-0021-6 · Cited by 940
Using psycho-physiological measures to assess task difficulty in software development
Fritz, T., Begel, A., Müller, S. C., Yigit-Elliott, S., & Züger, M. (2014) · Proceedings of the 36th International Conference on Software Engineering · doi:10.1145/2568225.2568266 · Cited by 262 · cited 4×
“Furthermore, the primary studies chiefly used source code written in Java [2], [4], [23], [28], [30]–[33], with one study using code written in C# [5].”
BackgroundMethodInfluential(2017) · doi:10.1145/3106237.3106268 · Cited by 117
(2015) · doi:10.5555/2818754.2818838 · Cited by 111
GSR and Blink Features for Cognitive Load Classification
Nourbakhsh, N., Wang, Y., & Chen, F. (2013) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-40483-2_11 · Cited by 98
Interruptibility of Software Developers and its Prediction Using Psycho-Physiological Sensors
Züger, M., & Fritz, T. (2015) · Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems · doi:10.1145/2702123.2702593 · Cited by 96 · cited 1×
“Only one study used both C# and Java languages [34], but they where evaluated sepparetly, i.”
MethodZüger, M., Müller, S. C., Meyer, A. N., & Fritz, T. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3174165 · Cited by 66 · cited 1×
“We found out that the combinations of the data from these multiple sensors were used to improve the accuracy of the results from applying machine learning techniques [22]–[24].”
Method
Research works printed but not linked: 69
- 10.1109/SANER.2016.107 · DOI
- 10.1108/LHT-07-2016-0085 · DOI
- fowler, 1999, Refactoring Improving the Design of Existing Code
- 10.1145/3216723.3216725 · DOI
- peitek, 2018, Toward conjoint analysis of simultaneous eye-tracking and fMRI data for program-comprehension studies, Proceedings of the Workshop on Eye Movements in Programming EMIP '18, 1:1
- 10.1145/2786805.2786864 · DOI · Paper page
- 10.1038/s41598-018-21518-3 · DOI · Paper page
- 10.1007/s11063-018-9829-1 · DOI
- 10.1145/2786805.2786864 · DOI · Paper page
- solís-marcos, 2018, Event-related potentials as indices of mental workload while using an in-vehicle information system, Cognition Technology & Work, 1
- 10.1109/ICSE.2015.334 · DOI · Paper page
- 10.1038/s41598-018-21518-3 · DOI · Paper page
- lange, 2007, Assessing and improving the quality of modeling, Technische Universiteit Eindhoven
- 2011, Unified modeling language: infrastructure, version 2.4.1, document formal/2011-08-06 Technical report OMG Tech Rep
- 10.1007/s13218-011-0165-0 · doi:10.1007/s13218-011-0165-0
- 10.1145/2702123.2702593 · DOI · Paper page
- fritz, 0, Using psychophysiological measures to assess task difficulty in software development, Int Conf on Software Engineering (ICSE), 402
- 10.1016/j.ijhcs.2018.03.002 · DOI · Paper page
- 10.1371/journal.pone.0146194 · DOI
- 10.1371/journal.pone.0191673 · DOI
- 10.1145/3106237.3106268 · DOI
- 10.1109/ACCESS.2017.2787673 · DOI
- 10.3233/JCM-2010-0263 · DOI
- 10.1145/2829945 · DOI
- 10.1109/EMBC.2016.7591752 · DOI
- 10.1109/CW.2010.37 · DOI · Paper page
- 10.1109/SMC.2013.88 · DOI
- 10.1145/3196321.3196347 · DOI
- lee, 2017, Mining biometric data to predict programmer expertise and task difficulty, Cluster Computing, 1
- solís-marcos, 2018, Event-related potentials as indices of mental workload while using an in-vehicle information system, Cognition Technology & Work
- 10.1145/2884781.2884803 · DOI
- 10.1109/ICSE.2015.284 · DOI
- 10.1111/j.1467-8659.2011.01928.x · DOI · Paper page
- 10.1145/2884781.2884803 · DOI
- 10.1109/ICSE.2015.284 · DOI
- i?can, 2018, Steady state visual evoked potential (SSVEP) based brain-computer interface (BCI) performance under different perturbations, PLoS ONE, 13, 191673e
- galan, 0, EEG estimates of engagement and cognitive workload predict math problem solving outcomes, Proc Int Conf User Modeling Adaptation and Personalization, 51
- 10.1109/SNPD.2016.7515886 · DOI
- 10.1109/SPMB.2014.7002950 · DOI
- 10.1145/2567574.2567624 · DOI
- 10.1145/2968215 · DOI
- 10.1016/j.procs.2011.01.016 · DOI
- 10.1109/ACCESS.2017.2787673 · DOI
- fakhoury, 0, The effect of poor source code lexicon and readability on developers' cognitive load, Proc Int Conf on Program Compr (ICPC)
- lee, 2017, Mining biometric data to predict programmer expertise and task difficulty, Cluster Computing
- 10.1016/j.infsof.2010.12.011 · DOI · Paper page
- 10.1016/j.infsof.2015.03.007 · DOI · Paper page
- kitchenham, 2007, Guidelines for Performing Systematic Literature Reviews in Software Engineering
- 10.1109/TSE.2007.1001 · DOI · Paper page
- 10.1145/2567574.2567624 · DOI
- li, 2018, EEG emotion recognition based on graph regularized sparse linear regression, Neural Processing Letters, 1, 10.1007/s11063-017-9609-3 · DOI
- 10.1109/JCSSE.2013.6567313 · DOI
- 10.1007/978-3-319-58625-0_4 · DOI
- 10.1109/SMC.2013.88 · DOI
- 10.1109/SANER.2016.107 · DOI
- 10.1109/ACCESS.2017.2723622 · DOI · Paper page
- 10.1155/2017/5496196 · DOI
- 10.3233/JCM-2010-0263 · DOI
- 10.1016/j.ijhcs.2018.03.002 · DOI · Paper page
- 10.1049/iet-bmt.2016.0169 · DOI
- 10.1109/JCSSE.2013.6567313 · DOI
- 10.1109/CW.2010.37 · DOI · Paper page
- 10.1109/SNPD.2016.7515886 · DOI
- 10.1007/978-3-319-58625-0_4 · DOI
- 10.1109/SAMI.2014.6822382 · DOI
- 10.1109/EMBC.2016.7591752 · DOI
- 0, “Testbench™ specifications emotiv 2014 ”
- 10.1109/ICSE.2015.334 · DOI · Paper page
- 10.1145/2829945 · DOI
Cited by
Investigating and Designing for Trust in AI-powered Code Generation Tools
Wang, R., Cheng, R., Ford, D., & Zimmermann, T. (2024) · The 2024 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3630106.3658984 · Cited by 87
“Software engineering is a type of complex and high-demanding information work that often involves high cognitive load and stress [24, 29, 32, 53], and therefore demands high-quality support.”
BackgroundCheng, R., Wang, R., Zimmermann, T., & Ford, D. (2024) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3651990 · Cited by 47
“Software developers has been long considered as an important and high stake type of knowledge worker [2, 33] whose job requires years of training, are considerably stressful, and demands high mental load [26, 29].”
BackgroundMethodRazzaq, A., Buckley, J., Lai, Q., Yu, T., & Botterweck, G. (2025) · ACM Computing Surveys · doi:10.1145/3687299 · Cited by 36
Towards Decoding Developer Cognition in the Age of AI Assistants
Haque, E. A., Brown, C., LaToza, T. D., & Johnson, B. (2025) · arXiv · doi:10.48550/arxiv.2501.02684 · Cited by 1
“A systematic mapping study by Gonc¸ales et al. [12] identified 33 studies investigating cognitive load measurement in software engineering, finding that 55% of studies utilized electroencephalography (EEG), while 36% combined multiple sensors like eye tracking and electrodermal activity for improved…”
Metadata
- Authors
- Gonçales, L., Farias, K., da Silva, B., & Fessler, J.
- Venue
- 2019 IEEE/ACM 27th International Conference on Program Comprehension (ICPC)
- Publisher
- IEEE
- Published
- 2019-05
- Pages
- 42-52
- OpenAlex
- W2955991816
- Cited by
- 55
- References
- 8
- FWCI
- 4.97
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Measuring the Cognitive Load of Software Developers: A Systematic Mapping Study. https://app.cochat.ai/papers/10.1109/icpc.2019.00018
- The paper
- Gonçales, L., Farias, K., da Silva, B., & Fessler, J. (2019). Measuring the Cognitive Load of Software Developers: A Systematic Mapping Study. 2019 IEEE/ACM 27th International Conference on Program Comprehension (ICPC). https://doi.org/10.1109/icpc.2019.00018
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.