CoChat
Sign Up
Cited by 64Neurobiology of Language and BilingualismTeaching and Learning ProgrammingNeural and Behavioral Psychology Studies

Computer code comprehension shares neural resources with formal logical inference in the fronto-parietal network

Open access
Liu, Yun-Fei, Kim, Judy, Wilson, Colin, Bedny, Marina·2020·eLife
DOI 10.7554/elife.59340PMC PMC7738180

Published version of a preprint

Open this paper in CoChat — free account

Abstract

Despite the importance of programming to modern society, the cognitive and neural bases of code comprehension are largely unknown. Programming languages might ‘recycle’ neurocognitive mechanisms originally developed for natural languages. Alternatively, comprehension of code could depend on fronto-parietal networks shared with other culturally-invented symbol systems, such as formal logic and symbolic math such as algebra. Expert programmers (average 11 years of programming experience) performed code comprehension and memory control tasks while undergoing fMRI. The same participants also performed formal logic, symbolic math, executive control, and language localizer tasks. A left-lateralized fronto-parietal network was recruited for code comprehension. Patterns of activity within this network distinguish between ‘for’ loops and ‘if’ conditional code functions. In terms of the underlying neural basis, code comprehension overlapped extensively with formal logic and to a lesser degree math. Overlap with executive processes and language was low, but laterality of language and code covaried across individuals. Cultural symbol systems, including code, depend on a distinctive fronto-parietal cortical network.

AI snapshot

Written from the paper’s full text.

TL;DR

Expert programmers comprehending Python code recruited a left-lateralized fronto-parietal network that overlapped most with formal logical inference, less with math, and little with language or a simple executive-control task. Within this network, multivoxel activity patterns distinguished `for` loops from `if` conditionals.

Key findings

  • Real code > fake code activated a left-lateralized network including lateral PFC (mainly BA 44/46), IPS, pMTG/STS, and early visual cortex (p<0.01 FWER).
  • MVPA decoded `for` vs. `if` functions above chance in PFC (64.7%, p<0.001), IPS (67.4%, p<0.001), and pMTG (68.4%, p<0.001); early visual cortex was lower (55.7%, p=0.015; F(3,56)=4.78, p=0.0048).
  • Code overlap was significantly above chance with logic, math, language, and MSIT (each p<0.001 vs. chance; p<0.005 vs. code split-half), but differed across tasks (F(3,42)=5.04, p=0.0045): overlap was highest with logic, followed by math and least with MSIT and language; logic exceeded all others, while language, math, and MSIT were statistically indistinguishable.
  • Code comprehension was as left-lateralized as language (code lateralization index mean = 0.451; language = 0.393; paired t(14)=1.203, p=0.25), and individual lateralization indices for code and language were highly correlated (R²=0.658, p<0.001).

Methods

  • Fifteen expert programmers (3 women; age 20–38, mean 27.4; mean 10.7 years programming experience, 5.7 years Python) completed fMRI code comprehension (real vs. fake code; `for` vs. `if`) plus separate logic/math/language localizer and MSIT tasks.
  • fMRI analyses used GLM contrasts, functional ROIs from the real > fake code contrast, SVM-based MVPA to decode `for` vs. `if`, cosine similarity of binary activation maps to quantify overlap, and lateralization indices.
  • Behavioral pre-tests assessed Python knowledge; in-scanner accuracy was similar for real vs. fake code (92% vs. 90%, p=0.23) and for `for` vs. `if` (both 92%, p=0.77), while fake-code and `for` trials were slower.

Limitations

  • The paper states the sample was relatively small (n=15), so differences from the co-published Ivanova et al. study should be interpreted with caution and replicated.
  • The paper notes that language, math, and logic were localized partly relative to each other, so shared areas across domains and their role in code comprehension remain to be addressed; it also allows that code and logic might dissociate at a finer neural scale.
  • Evident from the text: participants were expert Python programmers, limiting generalizability to novices or other languages; the fake-code control was a memory task with different perceptual cues (yellow text) and slower responses, so it may not isolate code syntax/comprehension perfectly.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.7554/elife.59340

Cites

102 of 102 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

0 of 26 cited sources checked · 0 supported · 0 flagged

  • Findingown data

    Lateralization indices of code and language were highly correlated across individuals.

    “Overlap with executive processes and language was low, but laterality of language and code covaried across individuals.”
    • Also stated in: § fMRI results · plain
      “Moreover, lateralization indices of code and language were highly correlated across individuals (R2 = 0.658, p<0.001)”
    • Also stated in: § Code comprehension and language · plain
      “Despite lack of direct overlap, lateralization patterns of language and coding were highly correlated across individuals that is those individuals with highly left-lateralized responses to sentences also showed highly left-lateralized responses to code.”
    Quote in the paperSubstantial claim
  • Findingown data

    There was no relationship between code-math overlap and level of expertise, as measured by years of coding experience or coding assessment performance.

    “Notably there was no relationship between overlap of code and language and level of expertise, as measured either by years of experience coding (regression against code-language overlap: R2 = 0, p=0.99; regression against code-math overlap: R2 = 0.033, p=0.52) or performance on coding assessments (regression against code-language overlap: R2 = 0.033, p=0.52; regression against code-math overlap: R”
    Quote in the paperNot a substantial claim
  • Findingown data

    Patterns of activity within the code-responsive network distinguish between for loops and if conditionals.

    “Patterns of activity within this network distinguish between ‘for’ loops and ‘if’ conditional code functions.”
    • Also stated in: § fMRI results · plain
      “MVPA analysis revealed that for and if functions could be distinguished based on patterns of activity within PFC (accuracy = 64.7%, p<0.001), IPS (accuracy = 67.4%, p<0.001) and pMTG (accuracy = 68.4%, p<0.001).”
    • Also stated in: § Discussion · hedged
      “Within this code-responsive neural network, spatial patterns of activation distinguished between for vs. if code functions, suggesting that this network represents code-relevant information and is not merely activated during the coding task due to general difficulty demands.”
    Quote in the paperSubstantial claim
  • Findingown data

    Logic activated more anterior and more extensive regions in PFC than math.

    “Logic activated more anterior and more extensive regions in PFC than math.”
    Quote in the paperNot a substantial claim
  • Findingown data

    Overlap of code and MSIT was observed in IPS, precentral gyrus, and a small portion of inferior temporal sulcus.

    “Overlap of code and the MSIT (hard >easy) was also observed in the IPS, precental gyrus and a small portion of the inferior temporal sulcus.”
    Quote in the paperSubstantial claim
  • May combine two claims. 22
  • Classified as probably not a substantial claim. 39

  • May combine two claims.

    Findingown data

    The MSIT hard > easy contrast activated a fronto-parietal network including IPS, with lateral frontal activation posterior and close to the precentral gyrus.

    “The MSIT hard >easy contrast also activated a fronto-parietal network including the IPS, however, the activation in the lateral frontal cortex was posterior and close to the precentral gyrus.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Code comprehension (real > fake) overlapped significantly above chance with logic, math, language, and MSIT localizer contrasts.

    “Code comprehension (real >fake) overlapped significantly above chance with all localizer tasks: logic, math, language and MSIT (each task compared to chance p’s < 0.001, compared to code split-half overlap p’s < 0.005)”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    For vs if decoding was also above chance in early visual cortex, but decoding accuracy there was lower than in PFC, IPS, and pMTG.

    “for and if functions could also be distinguished within the early visual cortex (accuracy = 55.7%, p=0.015), however, decoding accuracy was lower than in the other regions (F(3, 56)=4.78, p=0.0048)”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Overlap with logic was significantly higher than with all other tasks, while overlaps with language, math, and MSIT were statistically indistinguishable from each other.

    “Overlap with logic was significantly higher than with all other tasks, while the overlaps with the other three tasks (language, math, MSIT) were statistically indistinguishable from each other (post-hoc paired t-tests, FDR-corrected p’s < 0.05)”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Code comprehension overlapped most with logic, followed by math, and least with MSIT and language.

    “In terms of the underlying neural basis, code comprehension overlapped extensively with formal logic and to a lesser degree math. Overlap with executive processes and language was low, but laterality of language and code covaried across individuals.”
    • Also stated in: § fMRI results · plain
      “Code comprehension overlapped most with logic (logic >language), followed by math and least with MSIT and language”
    • Also stated in: § Discussion · plain
      “In overlap analyses, the code comprehension network was most similar to the fronto-parietal system involved in formal logical reasoning and to a lesser degree math. By contrast overlap with the perisylvian fronto-temporal language network is low.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingreasoning

    The current results are consistent with the co-published study by Ivanova and colleagues.

    “The current results are consistent with the co-published study by Ivanova and colleagues.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    On average, participants answered 64.6% (SD = 16.6%) of the difficult out-of-scanner exercise questions correctly, with a range of 37.5-93.75%.

    “On average, the participants got 64.6% (SD = 16.6%) of the questions correct (range: 37.5–93.75%).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductown data

    All participants had at least 5 years of programming experience and at least 3 years of Python experience.

    “All participants had at least 5 years of programming experience (range: 5–22, mean = 10.7, SD = 5.2), and at least 3 years of experience with Python (range: 3–9, mean = 5.7, SD = 1.8).”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductreasoning

    Fake code stimuli preserved the perceptual and lexical elements of real code but were devoid of syntactic structure.

    “Each fake function preserved the perceptual and lexical elements of a real function, but was devoid of syntactic structure.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Only a handful of previous studies have looked at the neural basis of code processing.

    “Only a handful of previous studies have looked at the neural basis of code processing (Duraes et al., 2016; Floyd et al., 2017; Ikutani and Uwano, 2014; Peitek et al., 2018; Siegmund et al., 2014; Huang et al., 2019).”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites Duraes et al., 2016 source

      Not checked in the free run.

    • cites Floyd et al., 2017

      Not checked in the free run.

    • cites Ikutani and Uwano, 2014 source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S4 (based).

    Study design and conductnone given

    Overlap analyses control for the overall number of activated vertices in each contrast.

    “Note that overlap analyses control for the overall number of activated vertices in each contrast.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationown data

    The code-responsive network represents code-relevant information and is not merely activated due to general difficulty demands.

    “suggesting that this network represents code-relevant information and is not merely activated during the coding task due to general difficulty demands.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductown data

    Seventeen individuals participated; one did not complete due to claustrophobia, another was excluded for excessive movement, and data from the remaining fifteen were reported.

    “Seventeen individuals participated in the study; one did not complete the tasks due to claustrophobia, and another was excluded from analyses due to excessive movement (>2 mm). We report data from the remaining fifteen individuals (three women, age range 20–38, mean age = 27.4, SD = 5.0).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductown data

    The current sample is relatively small (n = 15), so differences between the current study and the Ivanova findings should be interpreted with caution and replicated in future work.

    “Notably, since the current sample is relatively small (n = 15), any differences between the current study and the Ivanova findings should be interpreted with caution and replicated in future work.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Within PFC, logic recruits more anterior regions associated with advanced reasoning and symbol manipulation.

    “Within the PFC, logic in particular recruits more anterior regions associated with more advanced forms of reasoning and symbol manipulation (Coetzee and Monti, 2018; Ramnani and Owen, 2004).”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites Coetzee and Monti, 2018

      Not checked in the free run.

    • cites Ramnani and Owen, 2004 source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S4 (based).

    Premisenone given

    Other culturally-invented symbol systems, such as formal logic and mathematics, do not appear to depend on the same neural network as natural language.

    “Other culturally-invented symbol systems, such as formal logic and mathematics do not appear to depend on the same neural network as natural language.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductnone given

    The order of presentation of the functions was pseudo-randomized and balanced across participants.

    “The order of the presentation of the functions was pseudo-randomized and balanced across participants.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing). May combine two claims.

    Interpretationown data

    The relationship between code and language may be ontogenetic as well as phylogenetic.

    “This intriguing observation suggests that the relationship between code and language may be ontogenetic as well as phylogenetic.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Study design and conductown data

    A bootstrapping approach was used to test whether each observed cosine similarity value was significantly above the empirical lower bound.

    “We used a bootstrapping approach to test whether each observed cosine similarity value was significantly above the empirical lower bound.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductnone given

    Identical trials never appeared in succession, and the order of the trials was different across all 8 blocks of the same kind.

    “Identical trials never appeared in succession, and the order of the trials was different across all 8 blocks of the same kind.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Study design and conductown data

    The cosine similarities of code to each of the localizer tasks were compared using repeated-measure ANOVA and post-hoc pairwise comparisons with false discovery rate (FDR) correction.

    “The cosine similarities of code to each of the localizer tasks was then compared using repeated-measure ANOVA and post-hoc pairwise comparisons with false discovery rate (FDR) correction.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The perisylvian fronto-temporal language network shows enhanced activity for stimuli that contain meaningful lexical items and sentence-level syntax.

    “The perisylvian fronto-temporal language network shows enhanced activity for stimuli that contain meaningful lexical items and sentence-level syntax (e.g. Fedorenko et al., 2016).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Fedorenko et al., 2016 source

      Not checked in the free run.

  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductreasoning

    Fake functions were derived from real functions, preserving the words, digits and operators of the real functions, but none of the scrambled lines comprised an executable Python statement.

    “Because fake functions were derived from real functions, the words, digits and operators that existed in real functions were preserved; however, none of the scrambled lines comprised an executable Python statement.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Previous studies have shown that sentences with content words lead to larger responses in the perisylvian fronto-temporal language network than spoken equations or logical statements with variables.

    “Although all of the tasks in the language/math/logic localizer contain language stimuli, previous studies have shown that sentences with content words lead to larger responses in the perisylvian fronto-temporal language network than spoken equations or logical statements with variables (Kanjlia et al., 2016; Monti et al., 2009; Monti et al., 2012).”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites Kanjlia et al., 2016 source

      Not checked in the free run.

    • cites Monti et al., 2009 source

      Not checked in the free run.

    • cites Monti et al., 2012 source

      Not checked in the free run.

  • Classified as probably not a substantial claim. May combine two claims.

    Premisecitation

    The multi-source interference task (MSIT) engages executive control processes and localizes the multiple-demand network.

    “The multi-source interference task (MSIT) was adapted from Bush and Shin, 2006 to engage executive control processes and localize the multiple-demand network.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Bush and Shin, 2006

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    A nonparametric permutation test was further implemented to cluster-correct at p<0.01 family-wise error rate.

    “A nonparametric permutation test was further implemented to cluster-correct at p<0.01 family-wise error rate.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Study design and conductreasoning

    The cosine similarity measure controls for overall vector length (i.e. the overall number of active voxels in each contrast).

    “Note that this measure controls for overall vector length (i.e. the overall number of active voxels in each contrast).”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Premisecitation

    When comprehension of code was compared to detection of syntactic errors in code, activity in both fronto-parietal and putative language areas was observed.

    “When comprehension of code was compared to detection of syntactic errors in code, activity in both fronto-parietal and putative language areas was observed (Siegmund et al., 2014, Siegmund, 2017).”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites Siegmund et al., 2014 source

      Not checked in the free run.

    • cites Siegmund, 2017

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Previous studies have found that responses to language, logic and math when compared to each other were similar to what was observed for each domain relative to independent control conditions.

    “Furthermore, previous studies have found that responses to language, logic and math when compared to each other were similar to what was observed for each domain relative to independent control conditions (e.g. sentences relative to lists of non-words for language, and easy vs. hard logic problems; Kanjlia et al., 2016; Monti et al., 2009, Monti et al., 2012).”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites Kanjlia et al., 2016 source

      Not checked in the free run.

    • cites Monti et al., 2009 source

      Not checked in the free run.

    • cites Monti et al., 2012 source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductnone given

    Participants were divided into two groups such that the variants of the functions were counterbalanced across groups, and the same participant never saw different variants of the same function.

    “Participants were divided into two groups such that the variants of the functions were counterbalanced across groups; the same participant never saw different variants of the same function.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductnone given

    For the group-level analysis across participants, random-effects models were applied, and the models were corrected for multiple comparisons at vertex level with p<0.05 false discovery rate (FDR) across the whole brain.

    “For the group-level analysis across participants, random-effects models were applied, and the models were corrected for multiple comparisons at vertex level with p<0.05 false discovery rate (FDR) across the whole brain.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Premisecitation

    Computer coding, mathematics, and logic all involve manipulation of arbitrary variables without inherent meaning according to a set of learned rules.

    “Computer coding, mathematics and logic, all involve manipulation of arbitrary variables without inherent meaning (e.g. X, Y, input, ii) according to a set of learned rules (McCoy and Burton, 1988).”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites McCoy and Burton, 1988

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    The order of the blocks was balanced within and between participants.

    “The order of the blocks was balanced within and between participants.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Study design and conductreasoning

    The empirical lower bound was calculated separately for each localizer task to account for differences in the number of activated vertices across tasks.

    “The empirical lower bound was calculated separately for each localizer task to account for differences in the number of activated vertices across tasks.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    The order of the functions was permuted systematically such that each participant saw a unique order.

    “We permuted the order of the functions systematically such that each participant saw a unique order (see the subsection ‘algorithm for stimulus permutation’ in the appendix).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    The correct response was 'true' for half of the real-code trials and half of the fake code trials.

    “The correct response was ‘true’ for half of the real-code trials and half of the fake code trials.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductreasoning

    The normalization eliminates any difference in the baselines of the for and if conditions while preserving distinctive spatial patterns.

    “The purpose of the normalization is to eliminate any difference in the baselines of the two conditions while preserving distinctive spatial patterns.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductnone given

    All 48 statement pairs from each condition were unique and appeared once throughout the experiment.

    “All 48 statement pairs from each condition were unique and appeared once throughout the experiment.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductreasoning

    To help participants distinguish between real and fake code trials and to prevent them from erroneously attempting to parse fake code, real-code appeared in white text and fake code in yellow text.

    “To help participants distinguish between real and fake code trials and to prevent the participants from erroneously attempting to parse fake code, real-code appeared in white text and fake code in yellow text.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    In half of the trials, the correct answer was 'true'.

    “In half of the trials, the correct answer was ‘true’.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Interpretationcitation

    The fronto-parietal system might be inherently more flexible throughout the lifespan and thus more capable of taking on new cultural skills.

    “The fronto-parietal system might be inherently more flexible throughout the lifespan and thus more capable of taking on new cultural skills (Riley et al., 2018).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Riley et al., 2018 source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductcitation

    The difficult out-of-scanner exercise questions were designed to cover some of the objectives specified in the Certified Associate in Python Programming Certification exam syllabus.

    “The questions were designed by the experimenter to cover some of the objectives specified in the exam syllabus of the Certified Associate in Python Programming Certification held by the Python Institute (https://pythoninstitute.org/certification/pcap-certification-associate/pcap-exam-syllabus/).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites https://pythoninstitute.org/certification/pcap-certification-associate/pcap-exam-syllabus/

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    Covariates of no interest were included to account for confounds related to white matter, cerebral spinal fluid, and motion spikes.

    “Covariates of no interest were included to account for confounds related to white matter, cerebral spinal fluid, and motion spikes.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    Order of trials was counterbalanced across participants in two lists.

    “Order of trials was counterbalanced across participants in two lists.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductnone given

    Trial order was arranged such that all 12 interference trials appeared exactly twice in an interference block, and all three control trials appeared exactly six times in a control block.

    “Trial order was arranged such that all 12 interference trials appeared exactly twice in an interference block, and all three control trials appeared exactly six times in a control block.”
    Quote in the paperNot a substantial claim
45 of 45

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • fMRI reveals that the language network responds reliably when people listen to, read, or generate sentences, but not when they perform arithmetic tasks, engage in logical reasoning, understand computer programs, listen to music, categorize objects or events, reason about people's mental states, or process non-verbal communicative information like facial expressions or gestures.

    “it responds reliably when people listen to, read, or generate sentences, but not when they perform arithmetic tasks, engage in logical reasoning, understand computer programs, listen to music, categorize objects or events, reason about people’s mental states, or process non-verbal communicative information like facial expressions or gestures [37, 41, 42, 43, 44, 45, 46, 47, 48].”

    From Dissociating language and thought in large language models (2024) · cites this paper as [46]

Statistics recomputed

Every reported test result, recomputed from its test statistic and degrees of freedom. A result shows as not adding up only once an independent check has confirmed it.

  • Not confirmedF(3, 56) = 4.78reported p = .0048 · recomputed p = .005

    “F(3, 56)=4.78, p = .0048”

  • Consistentt(73) = 2.329reported p = .023 · recomputed p = .023

    “t(73) = 2.329, p = .023”

  • Consistentt(58) = 2.127reported p = .038 · recomputed p = .038

    “t(58) = 2.127, p = .038”

  • ConsistentF(2, 42) = 18.29reported p < .001 · recomputed p < .001

    “F(2, 42)=18.29, p < .001”

  • ConsistentF(2, 42) = 7.42reported p = .0017 · recomputed p = .002

    “F(2, 42)=7.42, p = .0017”

  • Consistentt(28) = -1.363reported p = .18 · recomputed p = .184

    “t(28) = −1.363, p = .18”

  • Consistentt(28) = 8.59reported p < .001 · recomputed p < .001

    “t(28)=8.59, p < .001”

  • ConsistentF(3, 42) = 5.04reported p = .0045 · recomputed p = .005

    “F(3,42) = 5.04, p = .0045”

  • Consistentt(14) = 5.501reported p < .001 · recomputed p < .001

    “t(14) = 5.501, p < .001”

  • Consistentt(14) = 5.523reported p < .001 · recomputed p < .001

    “t(14) = 5.523, p < .001”

  • Consistentt(14) = 1.203reported p = .25 · recomputed p = .249

    “t(14) = 1.203, p = .25”

11 of 11

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

  • Control-trial counts do not fill an MSIT block

    Minor

    The paper says each control block contains 24 trials, but its stated schedule of three control-trial types appearing six times each yields only 18 trials.

    Reported 24 · recomputed 18

1 of 1

Cites

Flagged first, then by citations

1 of the 103 references couldn’t be matched to a known paper.

Research works printed but not linked: 2

  1. Siegmund J. 2017. Measuring neural efficiency of program comprehension. Proceedings of the 2017 11th Joint Meeting on Foundations of Software Engineering.
  2. Weinberg, 1971, The Psychology of Computer Programming (Vol. 29

Cited by

Sorted by citations

Related

Authors
Liu, Y.-F., Kim, J., Wilson, C., & Bedny, M.
Affiliations
  • Johns Hopkins University, Baltimore, United States
  • Johns Hopkins University (US)
ORCID
Venue
eLife
Publisher
eLife Sciences Publications, Ltd
Published
2020-12-15
Volume
9
ISSN
2050-084X (ISSN-L)
DOI
10.7554/elife.59340
PMC
PMC7738180
OpenAlex
W3112191977
Semantic Scholar
d6d1a983418a499e39c936e7e5ec2ed489645165
Cited by
64
References
103
Citations by year
2026: 10 · 2025: 13 · 2024: 18 · 2023: 5 · 2022: 7 · 2021: 8 · 2020: 2 · 2019: 1
FWCI
2.95
Type
journal article
Language
en
Open access
Licence
Locations
  • eLife (publishedVersion, cc-by)
  • bioRxiv (Cold Spring Harbor Laboratory) (acceptedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
Johns Hopkins University
Topics
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Neurobiology of Language and Bilingualism
  • Physical Sciences › Computer Science › Computer Science Applications › Teaching and Learning Programming
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
Keywords
program comprehension, programming languages, frontoparietal network, formal logic, symbolic computation, executive function, language lateralization, expert programmers, for loops, if statements
MeSH terms
Adult, Brain Mapping, Cognition, Female, Frontal Lobe, Humans, Functional Laterality, Magnetic Resonance Imaging, Male, Neural Pathways, Parietal Lobe, Software, Comprehension, Young Adult, Executive Function

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-06.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-06.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review