CoChat

Measuring the Cognitive Load of Software Developers: A Systematic Mapping Study

Gonçales, Lucian, Farias, Kleinner, da Silva, Bruno, Fessler, Jonathan · 2019 · 2019 IEEE/ACM 27th International Conference on Program Comprehension (ICPC)
Cited by 55

The DOI matches its Crossref record; 9 of the 9 references it cites were found in a registry or an index.

Abstract

Context: In recent years, several studies explored different facets of the developers' cognitive load while executing tasks related to software engineering. Researchers have proposed and assessed different ways to measure developers' cognitive load at work and some studies have evaluated the interplay between developers' cognitive load and other attributes such as productivity and software quality. Problem: However, the body of knowledge about developers' cognitive load measurement is still dispersed. That hinders the effective use of developers' cognitive load measurements by industry practitioners and makes it difficult for researchers to build new scientific knowledge upon existing results. Objective: This work aims to pinpoint gaps providing a classification and a thematic analysis of studies on the measurement of cognitive load in the context of software engineering. Method: We carried out a Systematic Mapping Study (SMS) based on well-established guidelines to investigate nine research questions. In total, 33 articles (out of 2,612) were selected from 11 search engines after a careful filtering process. Results: The main findings are that (1) 55% of the studies adopted electroencephalogram (EEG) technology for monitoring the cognitive load; (2) 51% of the studies applied machine-learning classification algorithms for predicting cognitive load; and (3) 48% of the studies measured cognitive load in the context of programming tasks. Moreover, a taxonomy was derived from the answers of research questions. Conclusion: This SMS highlighted that the precision of machine learning techniques is low for realistic scenarios, despite the combination of a set of features related to developers' cognitive load used on these techniques. Thus, this gap makes the effective integration of the measure of developers' cognitive load in industry still a relevant challenge.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1109/icpc.2019.00018

Cites

9 of 9 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Research works printed but not linked: 68

  1. 10.1109/SANER.2016.107 · DOI
  2. 10.1108/LHT-07-2016-0085 · DOI
  3. fowler, 1999, Refactoring Improving the Design of Existing Code
  4. 10.1145/3216723.3216725 · DOI
  5. peitek, 2018, Toward conjoint analysis of simultaneous eye-tracking and fMRI data for program-comprehension studies, Proceedings of the Workshop on Eye Movements in Programming EMIP '18, 1:1
  6. 10.1145/2786805.2786864 · DOI · Paper page
  7. 10.1038/s41598-018-21518-3 · DOI · Paper page
  8. 10.1007/s11063-018-9829-1 · DOI
  9. 10.1145/2786805.2786864 · DOI · Paper page
  10. solís-marcos, 2018, Event-related potentials as indices of mental workload while using an in-vehicle information system, Cognition Technology & Work, 1
  11. 10.1109/ICSE.2015.334 · DOI · Paper page
  12. 10.1038/s41598-018-21518-3 · DOI · Paper page
  13. lange, 2007, Assessing and improving the quality of modeling, Technische Universiteit Eindhoven
  14. 2011, Unified modeling language: infrastructure, version 2.4.1, document formal/2011-08-06 Technical report OMG Tech Rep
  15. 10.1007/s13218-011-0165-0 · doi:10.1007/s13218-011-0165-0
  16. 10.1145/2702123.2702593 · DOI · Paper page
  17. fritz, 0, Using psychophysiological measures to assess task difficulty in software development, Int Conf on Software Engineering (ICSE), 402
  18. 10.1016/j.ijhcs.2018.03.002 · DOI · Paper page
  19. 10.1371/journal.pone.0146194 · DOI
  20. 10.1371/journal.pone.0191673 · DOI
  21. 10.1145/3106237.3106268 · DOI
  22. 10.1109/ACCESS.2017.2787673 · DOI
  23. 10.3233/JCM-2010-0263 · DOI
  24. 10.1145/2829945 · DOI
  25. 10.1109/EMBC.2016.7591752 · DOI
  26. 10.1109/CW.2010.37 · DOI · Paper page
  27. 10.1109/SMC.2013.88 · DOI
  28. 10.1145/3196321.3196347 · DOI
  29. lee, 2017, Mining biometric data to predict programmer expertise and task difficulty, Cluster Computing, 1
  30. solís-marcos, 2018, Event-related potentials as indices of mental workload while using an in-vehicle information system, Cognition Technology & Work
  31. 10.1145/2884781.2884803 · DOI
  32. 10.1109/ICSE.2015.284 · DOI
  33. 10.1111/j.1467-8659.2011.01928.x · DOI · Paper page
  34. 10.1145/2884781.2884803 · DOI
  35. 10.1109/ICSE.2015.284 · DOI
  36. i?can, 2018, Steady state visual evoked potential (SSVEP) based brain-computer interface (BCI) performance under different perturbations, PLoS ONE, 13, 191673e
  37. galan, 0, EEG estimates of engagement and cognitive workload predict math problem solving outcomes, Proc Int Conf User Modeling Adaptation and Personalization, 51
  38. 10.1109/SNPD.2016.7515886 · DOI
  39. 10.1109/SPMB.2014.7002950 · DOI
  40. 10.1145/2567574.2567624 · DOI
  41. 10.1145/2968215 · DOI
  42. 10.1016/j.procs.2011.01.016 · DOI
  43. 10.1109/ACCESS.2017.2787673 · DOI
  44. fakhoury, 0, The effect of poor source code lexicon and readability on developers' cognitive load, Proc Int Conf on Program Compr (ICPC)
  45. lee, 2017, Mining biometric data to predict programmer expertise and task difficulty, Cluster Computing
  46. 10.1016/j.infsof.2010.12.011 · DOI · Paper page
  47. kitchenham, 2007, Guidelines for Performing Systematic Literature Reviews in Software Engineering
  48. 10.1109/TSE.2007.1001 · DOI · Paper page
  49. 10.1145/2567574.2567624 · DOI
  50. li, 2018, EEG emotion recognition based on graph regularized sparse linear regression, Neural Processing Letters, 1, 10.1007/s11063-017-9609-3 · DOI
  51. 10.1109/JCSSE.2013.6567313 · DOI
  52. 10.1007/978-3-319-58625-0_4 · DOI
  53. 10.1109/SMC.2013.88 · DOI
  54. 10.1109/SANER.2016.107 · DOI
  55. 10.1109/ACCESS.2017.2723622 · DOI · Paper page
  56. 10.1155/2017/5496196 · DOI
  57. 10.3233/JCM-2010-0263 · DOI
  58. 10.1016/j.ijhcs.2018.03.002 · DOI · Paper page
  59. 10.1049/iet-bmt.2016.0169 · DOI
  60. 10.1109/JCSSE.2013.6567313 · DOI
  61. 10.1109/CW.2010.37 · DOI · Paper page
  62. 10.1109/SNPD.2016.7515886 · DOI
  63. 10.1007/978-3-319-58625-0_4 · DOI
  64. 10.1109/SAMI.2014.6822382 · DOI
  65. 10.1109/EMBC.2016.7591752 · DOI
  66. 0, “Testbench™ specifications emotiv 2014 ”
  67. 10.1109/ICSE.2015.334 · DOI · Paper page
  68. 10.1145/2829945 · DOI

Cited by

See all 55

Metadata

Authors
Gonçales, L., Farias, K., da Silva, B., & Fessler, J.
Venue
2019 IEEE/ACM 27th International Conference on Program Comprehension (ICPC)
Publisher
IEEE
Published
2019-05
Pages
42-52
OpenAlex
W2955991816
Cited by
55
References
9
FWCI
4.97
Type
conference paper
Language
en
Open access
closed

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.