The Representation of Semantic Information Across Human Cerebral Cortex During Listening Versus Reading Is Invariant to Stimulus Modality

“Resulting p-values were corrected for multiple comparisons within each participant using the false discovery rate (FDR) procedure (Benjamini and Hochberg, 1995).”

“One participant was left handed, all other participants were right handed or ambidextrous according to the Edinburgh handedness inventory (Oldfield, 1971) (laterality quotient of 100: entirely left- handed, 100: entirely right-handed).”

“Semantic model construction To account for response variance caused by the semantic content of the story stimuli a 985-parameter semantic feature space based on word co-occurrence statistics in a large corpus of text (Deerwester et al., 1990; Lund and Burgess, 1996; Mitchell et al., 2008; Huth et al., 2016) was used.”

“Each functional run was motion-corrected using the FMRIB Linear Image Registration Tool (FLIRT) from FSL 5.0 (Jenkinson and Smith, 2001; Jenkinson et al., 2002).”

“Cortical surface reconstruction and visualization Cortical surface meshes were generated from the T1-weighted anatomical scans using Freesurfer software (Dale et al., 1999).”

“Banded ridge regression is a special case of the well-established statistical approach called Tikhonov regression (Tikhonov and Arsenin, 1977).”

“The contrast between faces and objects was used to define the fusiform face area (Kanwisher et al., 1997) and occipital face area (Halgren et al.”

“Each functional run was motion-corrected using the FMRIB Linear Image Registration Tool (FLIRT) from FSL 5.0 (Jenkinson and Smith, 2001; Jenkinson et al., 2002).”

“Weight maps for the foot, hand, and mouth responses were used to define primary motor and somatosensory areas for the feet (M1F, S1F), hands (M1H, S1H), and mouth (M1M, S1M); supplementary motor areas for the feet and hands; secondary somatosensory area for the feet, and, in some participants, the hands; and, in some…”

“This suggests that the human brain represents semantic information in an amodal form that is independent of input modality (Vigneau et al., 2006; Binder et al., 2009; Price, 2010, 2012).”

“Analysis and visualizations were developed using iPython (Pérez and Granger, 2007) and the interactive programming and visualization environment jupyter notebook (Kluyver et al., 2016).”

“Motion-energy model construction A spatiotemporal Gabor pyramid was used to extract low-level visual features from the sequence of word frames used in the reading experiment (Adelson and Bergen, 1985; Watson and Ahumada, 1985).”

“Voxelwise model fitting and analysis was performed using custom software tikreg (Nunez-Elizalde et al., 2019) written in Python, making heavy use of NumPy (Oliphant, 2006) and SciPy (Oliphant, 2007).”

“Analysis and visualizations were developed using iPython (Pérez and Granger, 2007) and the interactive programming and visualization environment jupyter notebook (Kluyver et al., 2016).”

“The contrast between places and objects was used to define the parahippocampal place area (Epstein and Kanwisher, 1998), occipital place area (Nakamura et al.”

“The semantic content of the stories was estimated continuously by projecting the narrative into a word embedding space based on word co-occurrence statistics (Church and Hanks, 1990; Lund and Burgess, 1996; Mitchell et al., 2008; Turney and Pantel, 2010; Wehbe et al., 2014).”

“…by listening or reading converge in semantically selective regions (Chee et al., 1999; Carpentier et al., 2001; Booth et al., 2002; Cohen et al., 2004; Constable et al., 2004; Spitsyna et al., 2006; Jobard et al., 2007; Patterson et al., 2007; Buchweitz et al., 2009; Liuzzi et al., 2017).”

“There is evidence that several cortical regions are activated during both listening and reading (for reviews, see Price, 2010, 2012).”

“This suggests that the human brain represents semantic information in an amodal form that is independent of input modality (Vigneau et al., 2006; Binder et al., 2009; Price, 2010, 2012).”

“To account for response variance caused by the semantic content of the story stimuli a 985-parameter semantic feature space based on word co-occurrence statistics in a large corpus of text (Deerwester et al., 1990; Lund and Burgess, 1996; Mitchell et al., 2008; Huth et al., 2016) was used.”

“The weight map for saccade responses was used to define the frontal eye field (Paus, 1996), frontal operculum eye movement area (Corbetta et al., 1998), intraparietal sulcus visual areas, and, in some participants, the supplementary eye field (Grosbras et al., 1999).”

“Natural speech stimuli The speech stimuli consisted of 10- to 15 min stories taken from The Moth Radio Hour and used previously (Huth et al., 2016).”

“The contrast between continuous and temporally scrambled natural movies was used to define visual motion area MT (Tootell et al., 1995).”

“…of semantically selective regions that are independent of the presentation modality, whereas previous studies reported a few amodal semantic regions located mostly in the left hemisphere (Petersen et al., 1989; Chee et al., 1999; Jobard et al., 2007; Buchweitz et al., 2009; Liuzzi et al., 2017).”

“Most imaging studies of the semantic system have examined only one input modality, either spoken or written words (Démonet et al., 1994, 1992; Vandenberghe et al., 1996; Scott et al., 2000; Booth et al., 2002; Rissman et al., 2003; Devlin et al., 2004; Nakamura et al., 2005).”

“Most imaging studies of the semantic system have examined only one input modality, either spoken or written words (Démonet et al., 1994, 1992; Vandenberghe et al., 1996; Scott et al., 2000; Booth et al., 2002; Rissman et al., 2003; Devlin et al., 2004; Nakamura et al., 2005).”

“Most imaging studies of the semantic system have examined only one input modality, either spoken or written words (Démonet et al., 1994, 1992; Vandenberghe et al., 1996; Scott et al., 2000; Booth et al., 2002; Rissman et al., 2003; Devlin et al., 2004; Nakamura et al., 2005).”

“The semantic content of the stories was estimated continuously by projecting the narrative into a word embedding space based on word co-occurrence statistics (Church and Hanks, 1990; Lund and Burgess, 1996; Mitchell et al., 2008; Turney and Pantel, 2010; Wehbe et al., 2014).”

“…to characterize the semantic selectivity of each voxel in each presentation modality and for each individual participant (see Materials and Methods and Naselaris et al., 2011; Nishimoto et al., 2011; Huth et al., 2012, 2016; Çukur et al., 2013; Stansbury et al., 2013; Lescroart et al., 2015).”

“…to characterize the semantic selectivity of each voxel in each presentation modality and for each individual participant (see Materials and Methods and Naselaris et al., 2011; Nishimoto et al., 2011; Huth et al., 2012, 2016; Çukur et al., 2013; Stansbury et al., 2013; Lescroart et al., 2015).”

“…to characterize the semantic selectivity of each voxel in each presentation modality and for each individual participant (see Materials and Methods and Naselaris et al., 2011; Nishimoto et al., 2011; Huth et al., 2012, 2016; Çukur et al., 2013; Stansbury et al., 2013; Lescroart et al., 2015).”

“Previous studies have shown that complex linguistic stimuli such as narrative stories activate many more brain regions than single words or short sentences (Mazoyer et al., 1993; Xu et al., 2005; Jobard et al., 2007; Lerner et al., 2011).”

“The weight map for saccade responses was used to define the frontal eye field (Paus, 1996), frontal operculum eye movement area (Corbetta et al.”

“Previous studies have shown that complex linguistic stimuli such as narrative stories activate many more brain regions than single words or short sentences (Mazoyer et al., 1993; Xu et al., 2005; Jobard et al., 2007; Lerner et al., 2011).”

“The temporal prior was constructed from a set of HRF basis functions (Penny et al., 2007).”

“However, the anatomical normalization procedures used in these studies tend to smooth and mask the substantial individual variability in language processing (Caramazza, 1986; Steinmetz and Seitz, 1991; Fedorenko and Kanwisher, 2009).”

“5 kHz and the Penn Phonetics Lab Forced Aligner (P2FA; Yuan and Liberman, 2008) was used to automatically align the audio to the transcript.”

“The word frames were then converted to the CIE L*A*B* color space (McLaren, 1976) and the color information was discarded.”

“Previous studies have shown that complex linguistic stimuli such as narrative stories activate many more brain regions than single words or short sentences (Mazoyer et al., 1993; Xu et al., 2005; Jobard et al., 2007; Lerner et al., 2011).”

“A pretrained neural network was used to create a parse tree for each sentence of the stories (Andor et al., 2016).”

“The weight map for speech production responses was used to define Broca’s area (Amunts et al., 2010; Zilles and Amunts, 2018) and the superior ventral premotor speech area (sPMv).”

“…to characterize the semantic selectivity of each voxel in each presentation modality and for each individual participant (see Materials and Methods and Naselaris et al., 2011; Nishimoto et al., 2011; Huth et al., 2012, 2016; Çukur et al., 2013; Stansbury et al., 2013; Lescroart et al., 2015).”

“The words of each story were presented one-by-one at the center of the screen using a rapid serial visual presentation (RSVP) procedure (Forster, 1970; Buchweitz et al., 2009).”

“However, the semantic feature space that we used here is only one possible way of representing semantics (Mitchell et al., 2008; Huth et al., 2016; Pereira et al., 2018), and it has some limitations.”

“…by listening or reading converge in semantically selective regions (Chee et al., 1999; Carpentier et al., 2001; Booth et al., 2002; Cohen et al., 2004; Constable et al., 2004; Spitsyna et al., 2006; Jobard et al., 2007; Patterson et al., 2007; Buchweitz et al., 2009; Liuzzi et al., 2017).”

“For example, when people listen to or read a story they likely employ conceptual knowledge at long time scales beyond those using for computing semantic features (Yeshurun et al., 2017).”

“A cochleogram model that accounts for the logarithmic filtering of the mammalian coclea described in (de Heer et al., 2017) was used to create the low level auditory features (80 parameters).”

“Most imaging studies of the semantic system have examined only one input modality, either spoken or written words (Démonet et al., 1994, 1992; Vandenberghe et al., 1996; Scott et al., 2000; Booth et al., 2002; Rissman et al., 2003; Devlin et al., 2004; Nakamura et al., 2005).”

“Localizers for known ROIs Known ROIs were localized separately in each participant using standard techniques (Spiridon et al., 2006; Hansen et al., 2007).”

“…by listening or reading converge in semantically selective regions (Chee et al., 1999; Carpentier et al., 2001; Booth et al., 2002; Cohen et al., 2004; Constable et al., 2004; Spitsyna et al., 2006; Jobard et al., 2007; Patterson et al., 2007; Buchweitz et al., 2009; Liuzzi et al., 2017).”

“The semantic content of the stories was estimated continuously by projecting the narrative into a word embedding space based on word co-occurrence statistics (Church and Hanks, 1990; Lund and Burgess, 1996; Mitchell et al., 2008; Turney and Pantel, 2010; Wehbe et al., 2014).”

“…pathways for processing information by listening or reading converge in semantically selective regions (Chee et al., 1999; Carpentier et al., 2001; Booth et al., 2002; Cohen et al., 2004; Constable et al., 2004; Spitsyna et al., 2006; Jobard et al., 2007; Patterson et al., 2007; Buchweitz et al.,…”

“Although the early sensory processing pathways for listening and reading are distinct, listeners and readers appear to extract very similar information about the meaning of a narrative story (Rubin et al., 2000; Diakidoy et al., 2005).”

“Therefore, it is unclear whether the differences they identified within left DLPFC are due to differences in semantic representation or some other aspect of linguistic information (e.g., syntax).”

“However, the anatomical normalization procedures used in these studies tend to smooth and mask the substantial individual variability in language processing (Caramazza, 1986; Steinmetz and Seitz, 1991; Fedorenko and Kanwisher, 2009).”

“…sensory processing, the pathways for processing information by listening or reading converge in semantically selective regions (Chee et al., 1999; Carpentier et al., 2001; Booth et al., 2002; Cohen et al., 2004; Constable et al., 2004; Spitsyna et al., 2006; Jobard et al., 2007; Patterson et…”

“Most imaging studies of the semantic system have examined only one input modality, either spoken or written words (Démonet et al., 1994, 1992; Vandenberghe et al., 1996; Scott et al., 2000; Booth et al., 2002; Rissman et al., 2003; Devlin et al., 2004; Nakamura et al., 2005).”

“Most imaging studies of the semantic system have examined only one input modality, either spoken or written words (Démonet et al., 1994, 1992; Vandenberghe et al., 1996; Scott et al., 2000; Booth et al., 2002; Rissman et al., 2003; Devlin et al., 2004; Nakamura et al., 2005).”

“, 1998), intraparietal sulcus visual areas, and, in some participants, the supplementary eye field (Grosbras et al., 1999).”

“Before surface reconstruction, anatomical surface segmentations were carefully handchecked and corrected using Blender software and pycortex (Gao et al., 2015) (http://pycortex.org).”

“In the reading condition we used an RSVP method (Forster, 1970; Buchweitz et al., 2009) to present the stories at precisely the same rate as they occurred during listening.”

“Localizers for known ROIs Known ROIs were localized separately in each participant using standard techniques (Spiridon et al., 2006; Hansen et al., 2007).”

“Most of these studies used tightly controlled stimuli, such as a set of single isolated words, sentences or curated passages, and an explicit lexical semantic task (Chee et al., 1999; Michael et al., 2001; Buchweitz et al., 2009; Liuzzi et al., 2017).”

“…to characterize the semantic selectivity of each voxel in each presentation modality and for each individual participant (see Materials and Methods and Naselaris et al., 2011; Nishimoto et al., 2011; Huth et al., 2012, 2016; Çukur et al., 2013; Stansbury et al., 2013; Lescroart et al., 2015).”

“Most imaging studies of the semantic system have examined only one input modality, either spoken or written words (Démonet et al., 1994, 1992; Vandenberghe et al., 1996; Scott et al., 2000; Booth et al., 2002; Rissman et al., 2003; Devlin et al., 2004; Nakamura et al., 2005).”

“…by listening or reading converge in semantically selective regions (Chee et al., 1999; Carpentier et al., 2001; Booth et al., 2002; Cohen et al., 2004; Constable et al., 2004; Spitsyna et al., 2006; Jobard et al., 2007; Patterson et al., 2007; Buchweitz et al., 2009; Liuzzi et al., 2017).”

“However, the anatomical normalization procedures used in these studies tend to smooth and mask the substantial individual variability in language processing (Caramazza, 1986; Steinmetz and Seitz, 1991; Fedorenko and Kanwisher, 2009).”

“BOLD responses were modeled as a linear combination of all the feature spaces using linear regression with a non-spherical spatiotemporal multivariate normal prior on the weights (Nunez-Elizalde et al., 2019).”

“…to characterize the semantic selectivity of each voxel in each presentation modality and for each individual participant (see Materials and Methods and Naselaris et al., 2011; Nishimoto et al., 2011; Huth et al., 2012, 2016; Çukur et al., 2013; Stansbury et al., 2013; Lescroart et al., 2015).”

“Although the early sensory processing pathways for listening and reading are distinct, listeners and readers appear to extract very similar information about the meaning of a narrative story (Rubin et al., 2000; Diakidoy et al., 2005).”

“…by listening or reading converge in semantically selective regions (Chee et al., 1999; Carpentier et al., 2001; Booth et al., 2002; Cohen et al., 2004; Constable et al., 2004; Spitsyna et al., 2006; Jobard et al., 2007; Patterson et al., 2007; Buchweitz et al., 2009; Liuzzi et al., 2017).”

“The weight map for speech production responses was used to define Broca’s area (Amunts et al., 2010; Zilles and Amunts, 2018) and the superior ventral premotor speech area (sPMv).”

“Although previous studies have demonstrated that most parts of cortex are active during language processing 5,22–24 , it is unclear which regions represent language at the granularity of words and phrases 25 , which regions are consistently engaged in language processing 26 and whether different…”

“Below we briefly summarize a body of evidence from cognitive science and neuroscience that supports this dissociation.”

“Second, the language areas are modality-independent: they respond to diverse kinds of input, including spoken, written and signed language 19,64–67 (Fig.”

“…Heusser et al., 2018), comparing natural language across modalities and contexts (Stephens et al., 2010; Regev et al., 2013; Yeshurun et al., 2017; Deniz et al., 2019), and using data-driven modeling to capture the complexity of naturalistic neural responses (Haxby et al., 2011, 2020; Baldassano…”

“1) that responds robustly and selectively to linguistic input (23, 24), whether auditory or visual (25, 26).”

“a memory probe task), and modality of presentation (visual and auditory: Fedorenko et al., 2010; Braze et al., 2011; Vagharchakian et al., 2012; Deniz et al., 2019).”

“25 combinatorial processing during comprehension, regardless of the input modality (e.g., Fedorenko et al., 2010, 2016; Vagharchakian et al., 2012; Regev et al., 2013; Scott et al., 2017; Deniz et al., 2019).”

“As additional effects, we modeled other variables that could modulate the BOLD response but that were not our critical variables of interest, including item (sentence), run order within a session (1-10), and sentence order within a run (1-50):”

“These findings are consistent with previous studies that mapped how different concepts in language are represented across cortex [30, 43].”

“These regions respond robustly to linguistic input, both visual and auditory (Deniz et al., 2019; Fedorenko et al., 2010; Nakai et al., 2020; Regev et al., 2013; Scott et al., 2017).”

“For example, the comprehension of narratives activates large areas of the cerebral cortex, and not simply those canonically viewed as ‘language’ areas (Deniz et al., 2019; Huth et al., 2016).”

“For example, according to the hub-and-spoke theory (Lambon Ralph et al., 2017; Patterson et al., 2007), the activated network includes both common areas (i.e., the ATL) and meaning-specific areas that are different for each word (see, e.g., Deniz et al., 2019; Huth et al., 2016).”

“The word-level semantic model uses a 985-dimensional conceptual feature space that is based on word co-occurrence statistics across a large corpus of written English(de Heer et al., 2017; Deniz et al., 2019; Huth et al., 2016).”

“To reap the benefits of both interpretable controlled experiments and generalizable naturalistic stimuli, some studies have deployed a hybrid experimental design (Chien & Honey, 2020; Deniz et al., 2019; Lerner et al., 2011; Overath et al., 2015; Yeshurun et al., 2017).”

“Note that this is similar to methods that directly compare the encoding model weights estimated for two voxels (Çukur et al., 2013; Huth et al., 2016; Deniz et al., 2019), but spatial generations looks not only at how much the weights are different, but also at how much this difference affects the ability to predict h…”

“These regions respond robustly to linguistic input (visual or auditory; Bemis & Pylkkänen, 2013; Deniz et al., 2019; Fedorenko et al., 2010) but show little or no response to tasks in non-linguistic domains, such as executive functions, math, logic, music, action observation, or non-linguistic…”

“Remarkable, it turned out that the representation of semantic information across human cerebral cortex during listening versus reading is invariant to stimulus modality [4].”

“Given that most large public brain recordings datasets are of speech-evoked language (LeBel et al., 2023; Nastase et al., 2021; Deniz et al., 2019; Momenian et al., 2024), having access to speech models with improved brain-relevant semantics is important and will provide better model organisms for…”

“have already been used to expand the understanding of the neural basis for language processing, particularly for natural language comprehension [139, 140].”

“We also use the articulation and power spectrum (PowSpec) feature vectors provided in Deniz et al. (2019).”

“…in vision (Naselaris et al., 2011; Huth et al., 2012; Yamins et al., 2014; Eickenberg et al., 2017; Wen et al., 2018; Kubilius et al., 2019; Conwell et al., 2023; Wang et al., 2022), language (Huth et al., 2016; Deniz et al., 2019), and at the interface of vision and language (Popham et al., 2021).”

“Following others (Huth et al., 2016; Deniz et al., 2019; Shain et al., 2020), we align the word-times features X , of length M , to the dynamics of the fMRI signals applying a finite impulse response (FIR) model g (cf.”

“At each permutation iteration, the timecourse of the held-out test dataset was permuted by blockwise shuffling (shuffling was performed in blocks of 10 TRs in order to account for autocorrelations in voxel responses (Deniz et al., 2019; Jain et al., 2020)).”

“…assumed nature of comprehension is supported by findings from brain imaging research comparing listening and reading stories indicating that although sensory processes clearly differ by modality, the representations of the meaning in the verbal information are similar (Deniz et al., 2019).”

“At the same time, the representation of the language semantics in the brain is independent of whether meaning is extracted from spoken or written words [91].”

“…resemblance has been demonstrated between two sets of semantic representations recalled by spoken and written stories during language comprehension (Deniz et al., 2019), and the left anterodorsal pars triangularis (Brodmann area 45, BA 45) is significantly correlated with cross-modal semantic…”

“We choose a common value of 10TRs [25].”

“…effect binomial (semantic versus phonemic) or multinomial (single phonemes vs diphone vs triphone) statistical models in the main text and, just as for d 0 , this calculation is repeated with the “ equal weight" per subject and shown in the supplementary tables for Supplementary Table 2 and 5.”

“The DMN is known to be involved in narrative comprehension and interpretation (Deniz et al., 2019; Nguyen et al., 2019; Simony et al., 2016; Yeshurun et al., 2021).”

“Some previous studies using narratives and fMRI reported that supramodal activation was not restricted to the left hemisphere (Jobard et al., 2007; Regev et al., 2013; Deniz et al., 2019).”

“The advantage of decoding semantic information, as opposed to sensory based information such as visual details, is that semantic representation is partially invariant across modalities 9–13 .”

“, sentences and speech [14,21,26]; however, it was not applicable when the response is evoked by nonlinguistic stimuli, e.”

“…in the human brain has also been highlighted through datadriven approaches, by assessing brain responses to naturally spoken narrative stories (Deniz et al., 2019; Huth et al., 2016), further mapping single word-related semantic selectivity on the whole brain surface (see also Pereira et al.,…”

“These findings, which have benefited from more precise functional mapping approaches, link with prior work suggesting broad semantic coding within PCC/DMN [72,73].”

“Only when the connection of various representations reached a particular strength could mutual activation and diffusion be realized, and the output of orthography could be realized efficiently (Deniz et al., 2019).”

“This ‘language network’ supports both comprehension (spoken, written, and signed; e.g., Deniz et al., 2019, Fedorenko et al., 2010, MacSweeney et al., 2002, Regev et al., 2013, Scott et al., 2017) and production Menenti et al., 2011, Hu et al., 2022b.”

“…a, *, Matteo Visconti di Oleggio Castello a,b, *, Jack L. Gallant a,b Imaging Neuroscience, Volume 3, 2025 Wehbe et al., 2014), and narrative language ( Caucheteux & King, 2022; de Heer et al., 2017; Deniz et al., 2019; Huth et al., 2016; Jain & Huth, 2018; Jain et al., 2020; Toneva & Wehbe, 2019).”

“…language areas, responds to both written and auditory linguistic inputs (Fedorenko et al., 2010; Vagharchakian et al., 2012; Regev et al., 2013; Deniz et al., 2019), its computations plausibly have to do with the content of the linguistic signals rather than their surface (visual or auditory)…”

“In each permutation, the test responses were shuffled in blocks of 10 TRs (19, 44, 71, 81–85).”

“This method is similar to a variance-partitioning analysis which is often employed in encoding models, e.g. Deniz et al. [2019].”

“On the other hand, many parts of temporal, parietal, and prefrontal cortex process both written and spoken language (e.g., Booth et al., 2002; Buchweitz et al., 2009; Liuzzi et al., 2017; Regev et al., 2013; Deniz et al., 2019; Nakai et al., 2021).”

“The choice of these specific permutation test configurations is based on established methodologies in previous research (Deniz et al., 2019; Reddy & Wehbe, 2021; Oota et al., 2024a).”

“However, it is possible to ensure that text and audio are presented for a precisely equal time period to ensure faithful comparison (Deniz et al., 2019).”

“Future studies could employ tasks, such as category verification or story listening/reading (Huth et al., 2016; Deniz et al., 2019; Popham et al., 2021) that encourage deep processing of words and their context in modality-independent rather than modality-focused details.”

“Researchers have adopted this approach to comprehensively examine semantic representations (de Heer et al. 2017; Deniz et al. 2019; Huth et al. 2016), visual object category representations (Çukur et al. 2013a; Çukur et al. 2016; Çelik et al. 2019; Huth et al. 2012), and how attention modulates…”

“Other studies have probed correspondences between language processing models and the brain through the use of encoding models [2,3,6,48,50].”

“2012) that represent distinct features of stimuli (Huth et al. 2012; Deniz et al. 2019; Morton et al. 2021).”

“[26] Fatma Deniz, Anwar O Nunez-Elizalde, Alexander G Huth, and Jack L Gallant.”

“Existing studieshaveexploredneuralactivationpatternsbetweenspeechandread-ing,emphasizingdistinctionsbetweenunimodalversusmultimodal regions,andmodality-speci fi c effects in language comprehension 61 – 65 .”

“This is underpinned for example by a semantic model forming accurate predictions irrespective of modality (Deniz et al., 2019) and an MEG study showing word-speci fi c neural activity shared between modalities and subjects (Arana et al., 2020).”

“This region underpins both the comprehension (Deniz et al., 2019; Scott et al., 2017; Regev et al., 2013; Fedorenko et al., 2010) and production (Menenti et al., 2011; Hu et al., 2021) of language across spoken, written, and signed modalities.”

“Significance was evaluated for each network in each participant by comparing the N -1 group’s network-predicted activity to a null distribution 40,42 .”

“, 2016), regions within a distributed, bilateral semantic network display functional specialization for conceptual categories presented in narratives and this semantic selectivity appears regardless of sensory input modality (Deniz et al., 2019; Zhang et al., 2020).”

“Despite its old age, the EHI is still extensively used in the field of psychology, especially cognitive neuroscience, as a valid measure of handedness (Voss et al., 2010; Erickson et al., 2011; Stoodley et al., 2012; Barch et al., 2013; Frolov et al., 2017; Deniz et al., 2019).”

“…models can predict brain responses to written and spoken stimuli with high accuracy (Wehbe et al., 2014a; Jain & Huth, 2018; Toneva & Wehbe, 2019; Deniz et al., 2019; Abdou et al., 2021; Toneva et al., 2022; Antonello et al., 2021; Oota et al., 2022; Aw & Toneva, 2023; Oota et al., 2024b;…”

“Brain regions supporting language comprehension are engaged by both listening and reading (Regev et al., 2013; Deniz et al., 2019), as well as speaking and listening (Stephens et al., 2014; Zada et al., 2024, 2025).”

“Many recent studies have applied them during passive language comprehension (Caucheteux & King, 2022; de Heer et al., 2017; Deniz et al., 2019; Goldstein et al., 2022; Heilbron et al., 2022; Huth et al., 2016; Kumar et al., 2024; Schrimpf et al., 2021).”

“Indeed, neuroimaging studies show that hearing or reading a given description evokes identical semantic representations in the human brain (Arana et al., 2020; Deniz et al., 2019).”

“RSA is powerful for integrating different level/scale/ modality (e.g., neural, behavioral, physical, theoretical) activities to identify cognitive manipulation (FischerBaum et al., 2017; Wang et al., 2018; Deniz et al., 2019).”

“…sentences, the development and validation of the voxelwise encoding model framework for language processing (Huth et al., 2016; de Heer et al., 2017; Deniz et al., 2019; Popham et al., 2021) has made it possible to rigorously test hypotheses about semantic processing with natural language stimuli.”

“LLMs provide computational platforms for examining modern linguistic phenomena such as semantic integration, long-distance dependencies, and predictive processing.”

“…semantics to sentence meaning, ultimately culminating in a coherent narrative (Leaner et al., 2011; Regev et al., 2013; de Heer et al., 2017; Deniz et al., 2019; Popham et al., 2021, Deniz et al., 2023; Caucheteux et al., 2023; Huth et al., 2016; Schrimpf et al., 2021; Goldstein et al.,…”

“Specifically, variance partitioning separates the brain response variance that can be attributed to two models based on their unique and overlapping contributions (Vaidya et al., 2022; Deniz et al., 2019).”

“Stimulus-driven decompositions for interpretability have also been explored, in domains such as speech and audio [14, 42, 57] and visual stimuli [7, 17, 28].”

“These results are consistent with prior naturalistic investigations of semantic processing (Thye et al., 2023; Wu et al., 2022; Deniz et al., 2019; Huth et al., 2012, 2016) and with the language comprehension network identified across diverse languages, especially in the left hemisphere…”

“…knowledge relies on speci fi c hubs, but explain how semantic information integrates between brain areas and progresses from a modality-based towards a modality-independent conceptual representation (Deniz et al., 2019; Handjaras et al., 2016; Mahon & Caramazza, 2011; Popham et al., 2021).”

“However, in a study that compared brain activation between audio-only and text-based narratives, researchers found that the same parts of the brain were activated with both audio-only and reading books [46].”

“…(meaningless) and information with “value” has been discussed for a long time under the name “semantic” information [(Bar-Hillel and Carnap, 1953; Deniz et al., 2019; Lundgren, 2019; Miłkowski, 2021), see also emphasis on the requirement for recursive modeling to account for information in the…”

“…(as shown in Figure 2): (1) these vectors are resampled at times corresponding to the fMRI acquisitions using a three-lobe Lanczos filter [13, 24]; (2) vectors are reduced to d = 90 dimensions using PCA for computational efficiency; (3) finally, based on the linearized finite impulse…”

“This language network supports both comprehension (spoken, written, and signed) (Deniz et al., 2019; Terri L. Scott and Fedorenko, 2017) and production (Menenti et al., 2011; Hu et al., 2022).”

“We found that the regions in the semantic network spanned most of the bilateral semantic system, consistent with prior findings that bilateral activation was observed in response to complex stimuli, such as narratives (Deniz et al., 2019).”