The DOI matches its Crossref record; 86 of the 87 references it cites were found in a registry or an index.
Understanding understanding source code with functional magnetic resonance imaging
Open accessAbstract
Program comprehension is an important cognitive process that inherently eludes direct measurement. Thus, researchers are struggling with providing suitable programming languages, tools, or coding conventions to support developers in their everyday work. In this paper, we explore whether functional magnetic resonance imaging (fMRI), which is well established in cognitive neuroscience, is feasible to soundly measure program comprehension. In a controlled experiment, we observed 17 participants inside an fMRI scanner while they were comprehending short source-code snippets, which we contrasted with locating syntax errors. We found a clear, distinct activation pattern of five brain regions, which are related to working memory, attention, and language processing---all processes that fit well to our understanding of program comprehension. Our results encourage us and, hopefully, other researchers to use fMRI in future studies to measure program comprehension and, in the long run, answer questions, such as: Can we predict whether someone will be an excellent programmer? How effective are new languages and tools for program understanding? How should we train programmers?
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“When comprehension of code was compared to detection of syntactic errors in code, activity in both fronto-parietal and putative language areas was observed (Siegmund et al., 2014, Siegmund, 2017).”
From Computer code comprehension shares neural resources with formal logical inference in the fronto-parietal network (2020) · cites this paper as Siegmund et al., 2014
Citations
Cites
Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing
Benjamini, Y., & Hochberg, Y. (1995) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1995.tb02031.x · Cited by 109,482
The assessment and analysis of handedness: The Edinburgh inventory
Oldfield, R. C. (1971) · Neuropsychologia · doi:10.1016/0028-3932(71)90067-4 · Cited by 35,900
Co-Planar Stereotaxic Atlas of the Human Brain
Talairach, J., & Tournoux, P. (1988) · Medical Entomology and Zoology · Cited by 11,798
Wing, J. M. (2006) · Communications of the ACM · doi:10.1145/1118178.1118215 · Cited by 8,102
McCabe, T. J. (1976) · IEEE Transactions on Software Engineering · doi:10.1109/tse.1976.233837 · Cited by 6,059
Kiczales, G. (1996) · ACM Computing Surveys · doi:10.1145/242224.242420 · Cited by 5,146
Binder, J. R., Desai, R. H., Graves, W. W., & Conant, L. L. (2009) · Cerebral Cortex · doi:10.1093/cercor/bhp055 · Cited by 4,060
Imaging Cognition II: An Empirical Review of 275 PET and fMRI Studies
Cabeza, R., & Nyberg, L. (2000) · Journal of Cognitive Neuroscience · doi:10.1162/08989290051137585 · Cited by 3,598
Bergmans, L., & Lopes, C. V. (1999) · Lecture notes in computer science · doi:10.1007/3-540-46589-8_17 · Cited by 2,951
Halstead, M. H. (1977) · CERN Document Server (European Organization for Nuclear Research) · Cited by 2,339
A simple objective technique for measuring flexibility in thinking
Berg, E. A. (1948) · The Journal of General Psychology · doi:10.1080/00221309.1948.9918159 · Cited by 2,255
Towards a neural basis of auditory sentence processing
Friederici, A. D. (2002) · Trends in Cognitive Sciences · doi:10.1016/s1364-6613(00)01839-8 · Cited by 2,231
Handedness and hemispheric language dominance in healthy humans
Knecht, S. (2000) · Brain · doi:10.1093/brain/123.12.2512 · Cited by 1,560
On Broca, brain, and binding: A new framework
Hagoort, P. (2005) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2005.07.004 · Cited by 1,411
Tarr, P. L., Ossher, H. L., Harrison, W., & Sutton, S. M. (1999) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/302405.302457 · Cited by 1,249
Training of Working Memory in Children With ADHD
Klingberg, T., Forssberg, H., & Westerberg, H. (2002) · Journal of Clinical and Experimental Neuropsychology · doi:10.1076/jcen.24.6.781.8395 · Cited by 1,225 · cited 1×
“Thus, a test prior to programming education [48, 41] might reveal which students might struggle with learning programming.”
Engle, R. W., Kane, M. J., & Tuholski, S. W. (1999) · Models of Working Memory · doi:10.1017/cbo9781139174909.007 · Cited by 1,195
Driven to Distraction: Dual-Task Studies of Simulated Driving and Conversing on a Cellular Telephone
Strayer, D. L., & Johnston, W. A. (2001) · Psychological Science · doi:10.1111/1467-9280.00386 · Cited by 1,134
Is working memory still working?
Baddeley, A. (2001) · American Psychologist · doi:10.1037/0003-066x.56.11.851 · Cited by 1,052
Elements of software science (Operating and programming systems series)
Halstead, M. H. (1977) · Cited by 1,037
Lesion analysis of the brain areas involved in language comprehension
Dronkers, N. F., Wilkins, D. P., Van Valin, R. D., Redfern, B. B., & Jaeger, J. J. (2004) · Cognition · doi:10.1016/j.cognition.2003.11.002 · Cited by 1,035
Turken, A. U., & Dronkers, N. F. (2011) · Frontiers in System Neuroscience · doi:10.3389/fnsys.2011.00001 · Cited by 1,012
Dissociating Verbal and Spatial Working Memory Using PET
Smith, E. E., Jonides, J., & Koeppe, R. A. (1996) · Cerebral Cortex · doi:10.1093/cercor/6.1.11 · Cited by 962
Empirical Studies of Programming Knowledge
Soloway, E. M., & Ehrlich, K. (1984) · IEEE Transactions on Software Engineering · doi:10.1109/tse.1984.5010283 · Cited by 875
Language, Thought, and Reality
(2019) · Catholic University of America Press eBooks · doi:10.2307/j.ctvt9k614.12 · Cited by 840
Activation of Extrastriate and Frontal Cortical Areas by Visual Words and Word-Like Stimuli
Petersen, S. E., Fox, P. T., Snyder, A. Z., & Raichle, M. E. (1990) · Science · doi:10.1126/science.2396097 · Cited by 821
Object-oriented metrics: measures of complexity
Henderson‐Sellers, B. (1995) · Prentice-Hall, Inc eBooks · Cited by 753
Awh, E., Jonides, J., Smith, E. E., Schumacher, E. H., Koeppe, R. A., & Katz, S. (1996) · Psychological Science · doi:10.1111/j.1467-9280.1996.tb00662.x · Cited by 705
Bottini, G., Corcoran, R., Sterzi, R., Paulesu, E., Schenone, P., Scarpa, P., Frackowiak, R. S. J., & Frith, D. (1994) · Brain · doi:10.1093/brain/117.6.1241 · Cited by 660
Stimulus structures and mental representations in expert comprehension of computer programs
Pennington, N. (1987) · Cognitive Psychology · doi:10.1016/0010-0285(87)90007-7 · Cited by 526
Working memory capacity — facets of a cognitive ability construct
Oberauer, K., Süß, H., Schulze, R., Wilhelm, O., & Wittmann, W. W. (2000) · Personality and Individual Differences · doi:10.1016/s0191-8869(99)00251-2 · Cited by 523
Prabhakaran, V., Smith, J. A., Desmond, J. E., Glover, G. H., & Gabrieli, J. D. E. (1997) · Cognitive Psychology · doi:10.1006/cogp.1997.0659 · Cited by 509
Granularity in software product lines
Kästner, C., Apel, S., & Kuhlemann, M. (2008) · Proceedings/Proceedings - International Conference on Software Engineering · doi:10.1145/1368088.1368131 · Cited by 497
Self-citationSelected Writings on Computing: A personal Perspective
Dijkstra, E. W. (1982) · Springer eBooks · doi:10.1007/978-1-4612-5695-3 · Cited by 457
Culture, language and personality
Sapir, E., & Mandelbaum, D. G. (1966) · Medical Entomology and Zoology · Cited by 450
Brain activity during reading The effects of exposure duration and task
Price, C. J., Wise, R. J. S., Watson, J. D. G., Patterson, K. E., Howard, D., & Frackowiak, R. S. J. (1994) · Brain · doi:10.1093/brain/117.6.1255 · Cited by 428
The brain basis of syntactic processes: functional imaging and lesion studies
Friederici, A. D., & Kotz, S. A. (2003) · NeuroImage · doi:10.1016/j.neuroimage.2003.09.003 · Cited by 414
Brodmann’s Localisation in the Cerebral Cortex
Brodmann, K. (2005) · Springer eBooks · doi:10.1007/b138298 · Cited by 402
Teaching objects-first in introductory computer science
Cooper, S., Dann, W. P., & Pausch, R. F. (2003) · ACM SIGCSE Technical Symposium on Computer Science Education (SIGCSE) · doi:10.1145/611892.611966 · Cited by 400
Jones, P. W. (2001) · Oxford University Press eBooks · doi:10.1093/gmo/9781561592630.article.20622 · Cited by 386
Syntactic/semantic interactions in programmer behavior: A model and experimental results
Shneiderman, B., & Mayer, R. E. (1979) · International Journal of Parallel Programming · doi:10.1007/bf00977789 · Cited by 383
Jäncke, L., Shah, N. J., & Peters, M. (2000) · Cognitive Brain Research · doi:10.1016/s0926-6410(00)00028-8 · Cited by 375
EducationPaving the way for computational thinking
Guzdial, M. (2008) · Communications of the ACM · doi:10.1145/1378704.1378713 · Cited by 350
Exploratory experimental studies comparing online and offline programming performance
Sackman, H., Erikson, W. J., & Grant, E. E. (1968) · Communications of the ACM · doi:10.1145/362851.362858 · Cited by 341
The mind of expert motor performance is cool and focused
MILTON, J. G., Solodkin, A., Hluštı́k, P., & Small, S. L. (2007) · NeuroImage · doi:10.1016/j.neuroimage.2007.01.003 · Cited by 340
Grodzinsky, Y., & Santi, A. (2008) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2008.09.001 · Cited by 334
Fiebach, C. J., Schlesewsky, M., Lohmann, G., von Cramon, D. Y., & Friederici, A. D. (2004) · Human Brain Mapping · doi:10.1002/hbm.20070 · Cited by 329
Kersten, M. A., & Murphy, G. C. (2005) · International Conference on Aspect-Oriented Software Development (AOSD) · doi:10.1145/1052898.1052912 · Cited by 299
Hierarchical artificial grammar processing engages Broca's area
Bahlmann, J., Schubotz, R. I., & Friederici, A. D. (2008) · NeuroImage · doi:10.1016/j.neuroimage.2008.04.249 · Cited by 294
Robillard, M. P., & Murphy, G. C. (2002) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/581339.581390 · Cited by 289
Task-Dependent Modulation of Regions in the Left Inferior Frontal Cortex during Semantic Processing
Roskies, A., Fiez, J., Balota, D. A., Raichle, M., & Petersen, S. (2001) · Journal of Cognitive Neuroscience · doi:10.1162/08989290152541485 · Cited by 249
Natural programming languages and environments
Myers, B. A., Pane, J. F., & Ko, A. J. (2004) · Communications of the ACM · doi:10.1145/1015864.1015888 · Cited by 247
Cerebral activation during performance of a card sorting test
Nagahama, Y., Fukuyama, H., Yamauchi, H., Matsuzaki, S., Konishi, J., Shibasaki, H., & Kimura, J. (1996) · Brain · doi:10.1093/brain/119.5.1667 · Cited by 214
University of California Press
Muto, A. (1995) · University of California Press eBooks · doi:10.1525/9780520912274 · Cited by 212
Studying programmer behavior experimentally
Brooks, R. (1980) · Communications of the ACM · doi:10.1145/358841.358847 · Cited by 208
What artificial grammar learning reveals about the neurobiology of syntax
Petersson, K. M., Folia, V., & Peter Hagoort, A. (2010) · Brain and Language · doi:10.1016/j.bandl.2010.08.003 · Cited by 203
Models and theories of programming strategy
Davies, S. P. (1993) · International Journal of Man-Machine Studies · doi:10.1006/imms.1993.1061 · Cited by 172
An Eye Tracking Study on camelCase and under_score Identifier Styles
Sharif, B., & Maletic, J. I. (2010) · IEEE International Conference on Program Comprehension (ICPC) · doi:10.1109/icpc.2010.41 · Cited by 144
Attention to One or Two Features in Left or Right Visual Field: A Positron Emission Tomography Study
Vandenberghe, R. R., Duncan, J., Dupont, P. J., Ward, R. D., Poline, J., Bormans, G. M., Michiels, J., Mortelmans, L., & Orban, G. A. (1997) · Journal of Neuroscience · doi:10.1523/jneurosci.17-10-03739.1997 · Cited by 137
Thought and Two Languages: The Impact of Bilingualism on Cognitive Development
Díaz, R. M. (1983) · Review of Research in Education · doi:10.2307/1167134 · Cited by 129
Determining language laterality by fMRI and dichotic listening
Bethmann, A., Tempelmann, C., Bleser, R. D., Scheich, H., & Brechmann, A. (2006) · Brain Research · doi:10.1016/j.brainres.2006.11.057 · Cited by 125
Self-citationMeasuring programming experience
Feigenspan, J., Kästner, C., Liebig, J., Apel, S., & Hanenberg, S. (2012) · IEEE International Conference on Program Comprehension (ICPC) · doi:10.1109/icpc.2012.6240511 · Cited by 106
Self-citationUsing a behavioral theory of program comprehension in software engineering
Brooks, R. (1978) · doi:10.5555/800099.803210 · Cited by 97
Exploiting the map metaphor in a tool for software evolution
Griswold, W. G., Yuan, J. J., & Kato, Y. (2001) · doi:10.5555/381473.381501 · Cited by 95
Neural Correlates of Artificial Grammar Learning
Skosnik, P. D., Mirza, F., Gitelman, D. R., Parrish, T. B., Mesulam, M. M., & Reber, P. J. (2002) · NeuroImage · doi:10.1006/nimg.2002.1291 · Cited by 87
How do we tell truths that might hurt?
Dijkstra, E. W. (1982) · ACM SIGPLAN Notices · doi:10.1145/947923.947924 · Cited by 82
Some basic determinants of computer programming productivity
Chrysler, E. (1978) · Communications of the ACM · doi:10.1145/359511.359523 · Cited by 80
The teaching of software engineering
Shooman, M. L. (1983) · doi:10.1145/800038.801015 · Cited by 60
The neurobiology of syntax: beyond string sets
Petersson, K. M., & Peter Hagoort, A. (2012) · Philosophical Transactions of the Royal Society B Biological Sciences · doi:10.1098/rstb.2012.0101 · Cited by 60
Chapter 2: Thought and Two Languages: The Impact of Bilingualism on Cognitive Development
Díaz, R. M. (1983) · Review of Research in Education · doi:10.3102/0091732x010001023 · Cited by 58
Using Sex Differences to Link Spatial Cognition and Program Comprehension
Fisher, M. L., Cox, A., & Zhao, L. (2006) · IEEE International Conference on Software Maintenance (ICSM) · doi:10.1109/icsm.2006.72 · Cited by 53
Creativity as a pathway to computer science
Knobelsdorf, M., & Romeike, R. (2008) · ACM Conference on Innovation and Technology in Computer Science Education (ITiCSE) · doi:10.1145/1384271.1384347 · Cited by 52
Darcy, D. P., & Ma, M. (2005) · Hawaii International Conference on System Sciences (HICSS) · doi:10.1109/hicss.2005.261 · Cited by 50
Hanenberg, S., Kleinschmager, S., & Josupeit-Walter, M. (2009) · Proceedings of the ... ACM-IEEE International Symposium on Empirical Software Engineering and Measurement/International Symposium on Empirical Software Engineering and Measurement · doi:10.1109/esem.2009.5316028 · Cited by 46
Guéhéneuc, Y. (2006) · IEEE International Conference on Collaborative Advances in Software and Computing (CASCON) · doi:10.1145/1188966.1188968 · Cited by 45
Programmer information needs after memory failure
Parnin, C., & Rugaber, S. (2012) · IEEE International Conference on Program Comprehension (ICPC) · doi:10.1109/icpc.2012.6240479 · Cited by 41
Self-citationSubvocalization - Toward Hearing the Inner Thoughts of Developers
Parnin, C. (2011) · IEEE International Conference on Program Comprehension (ICPC) · doi:10.1109/icpc.2011.49 · Cited by 39
Self-citationToward measuring program comprehension with functional magnetic resonance imaging
Siegmund, J., Brechmann, A., Apel, S., Kästner, C., Liebig, J., Leich, T., & Saake, G. (2012) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/2393596.2393624 · Cited by 28
Self-citationA Simple and Effective Measure for Complex Low-Level Dependencies
Beyer, D., & Fararooy, A. (2010) · IEEE International Conference on Program Comprehension (ICPC) · doi:10.1109/icpc.2010.49 · Cited by 26
Dyslexia and learning computer programming
Powell, N. J., Moore, D., Gray, J. M., Finlay, J., & Reaney, J. (2004) · ACM Conference on Innovation and Technology in Computer Science Education (ITiCSE) · doi:10.1145/1007996.1008072 · Cited by 20
Revisiting Information Hiding: Reflections on Classical and Nonclassical Modularity
Ostermann, K., Giarrusso, P. G., Kästner, C., & Rendel, T. (2011) · Cited by 18
Self-citationHansen, M. E., Lumsdaine, A., & Goldstone, R. L. (2012) · ACM SIGPLAN International Symposium on New Ideas, New Paradigms, and Reflections on Programming and Software (Onward!) · doi:10.1145/2384592.2384596 · Cited by 18
What Makes Code Hard to Understand?Preprint
Hansen, M., Goldstone, R. L., & Lumsdaine, A. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1304.5257 · Cited by 17
Baniassad, E., & Fleissner, S. (2006) · doi:10.1145/1176617.1176625 · Cited by 9
Creating a cognitive metric of programming task difficulty
de Alwis, B., Murphy, G. C., & Minto, S. (2008) · International Workshop on Cooperative and Human Aspects of Software Engineering (CHASE) · doi:10.1145/1370114.1370122 · Cited by 8
Wootton, S., & Horne, T. (2010) · Cited by 1
Research works printed but not linked: 1
- J. Raven . Mental Tests Used in Genetic Studies: The Performances of Related Individuals in Tests Mainly Educative and Mainly Reproductive. Master’s thesis , University of London , 1936 . J. Raven. Mental Tests Used in Genetic Studies: The Performances of Related Individuals in Tests Mainly Educative and Mainly Reproductive. Master’s thesis, University of London, 1936.
Cited by
Sorted by citations
Coding as another language: a pedagogical approach for teaching computer science in early childhood
Bers, M. U. (2019) · Journal of Computers in Education · doi:10.1007/s40692-019-00147-3 · Cited by 248
Eye Movements in Code Reading: Relaxing the Linear Order
Busjahn, T., Bednarik, R., Begel, A., Crosby, M. E., Paterson, J. H., Schulte, C., Sharif, B., & Tamm, S. (2015) · IEEE International Conference on Program Comprehension (ICPC) · doi:10.1109/icpc.2015.36 · Cited by 182
Stuck and Frustrated or in Flow and Happy: Sensing Developers' Emotions and Progress
Muller, S. C., & Fritz, T. (2015) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse.2015.334 · Cited by 150
Comprehension of Procedural Visual Business Process Models
Figl, K. (2017) · Business & Information Systems Engineering · doi:10.1007/s12599-016-0460-2 · Cited by 140
Decoding the Representation of Code in the Brain: An fMRI Study of Code Review and Expertise
Floyd, B., Santander, T., & Weimer, W. R. (2017) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse.2017.24 · Cited by 136
Metadata
- Authors
- Siegmund, J., Kästner, C., Apel, S., Parnin, C., Bethmann, A., Leich, T., Saake, G., & Brechmann, A.
- Affiliations
- University of Passau (DE)
- Carnegie Mellon University (US)
- Georgia Institute of Technology (US)
- Leibniz Institute for Neurobiology (DE)
- Metop Research Institute, Germany
- Otto-von-Guericke-Universität Magdeburg (DE)
- Venue
- Proceedings of the 36th International Conference on Software Engineering
- Publisher
- ACM
- Published
- 2014-05-31
- Pages
- 378-389
- ISSN
- 0270-5257 (ISSN-L)
- OpenAlex
- W2014232144
- Cited by
- 219
- References
- 87
- Citations by year
- 2026: 19 · 2025: 13 · 2024: 15 · 2023: 8 · 2022: 12 · 2021: 24 · 2020: 38 · 2019: 22 · 2018: 18 · 2017: 10 · 2016: 18 · 2015: 17 · 2014: 4
- FWCI
- 13.93
- Type
- conference paper
- Language
- en
- Open access
- green
- figshare.com
- Locations
- ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) (publishedVersion)
- KiltHub Repository (submittedVersion)
- Figshare (submittedVersion, cc-by)
- KiltHub Repository
- Funders
- National Science Foundation
- Deutsche Forschungsgemeinschaft
- Topics
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Physical Sciences › Computer Science › Computer Science Applications › Teaching and Learning Programming
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neuroscience, Education and Cognitive Function
- Keywords
- program comprehension, source code understanding, functional MRI, brain activation patterns, working memory, attention, language processing, programming languages, programmer training
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
