CoChat
Sign Up
Cited by 11Neurobiology of Language and BilingualismNeuroscience and Music PerceptionTactile and Sensory Interactions

Processing speech and thoughts during silent reading: direct reference effects for speech by fictional characters in voice-selective auditory cortex and a theory-of-mind network

Open access
Alderson-Day, Ben, Moffatt, Jamie, Bernini, Marco, Mitrenga, Kaja, Yao, Bo, Fernyhough, Charles·2020·Journal of Cognitive Neuroscience
DOI 10.1162/jocn_a_01571
Open this paper in CoChat — free accountOpen PDF

Abstract

Stories transport readers into vivid imaginative worlds, but understanding how readers create such worlds—populating them with characters, objects, and events—presents serious challenges across disciplines. Auditory imagery is thought to play a prominent role in this process, especially when representing characters' voices. Previous research has shown that direct reference to speech in stories (e.g., He said, “I'm over here”) may prompt spontaneous activation of voice-selective auditory cortex more than indirect speech [Yao, B., Belin, P., & Scheepers, C. Silent reading of direct versus indirect speech activates voice-selective areas in the auditory cortex. Journal of Cognitive Neuroscience, 23, 3146–3152, 2011]. However, it is unclear whether this effect reflects differential processing of speech or differences in linguistic content, source memory, or grammar. One way to test this is to compare direct reference effects for characters speaking and thinking in a story. Here, we present a multidisciplinary fMRI study of 21 readers' responses to characters' speech and thoughts during silent reading of short fictional stories. Activations relating to direct and indirect references were compared for both speaking and thinking. Eye-tracking and independent localizer tasks (auditory cortex and theory of mind [ToM]) established ROIs in which responses to stories could be tracked for individuals. Evidence of elevated auditory cortex responses to direct speech over indirect speech was observed, replicating previously reported effects; no reference effect was observed for thoughts. Moreover, a direct reference effect specific to speech was also evident in regions previously associated with inferring intentions from communication. Implications are discussed for the spontaneous representation of fictional characters and the potential roles of inner speech and ToM in this process.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1162/jocn_a_01571

Cites

18 of 18 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 74

  1. Alderson-Day, B., Bernini, M., Fernyhough, C., 2017. Uncharted features and dynamics of reading: Voices, characters, and crossing of experiences. Conscious. Cogn. 49, 98- 109. https://doi.org/10.1016/j.concog.2017.01.003 · DOI
  2. Alexander, J.D., Nygaard, L.C., 2008. Reading voices and hearing text: Talker-specific auditory imagery in reading. J. Exp. Psychol. Hum. Percept. Perform. 34, 446-459. https://doi.org/10.1037/0096-1523.34.2.446 · DOI · Paper page
  3. Barsalou, L.W., 2008. Grounded cognition. Annu. Rev. Psychol. 59, 617-645. https://doi.org/10.1146/annurev.psych.59.103006.093639 · DOI · Paper page
  4. Behne, T., Liszkowski, U., Carpenter, M., Tomasello, M., 2012. Twelve-month-olds' comprehension and production of pointing. Br. J. Dev. Psychol. 30, 359-375. https://doi.org/10.1111/j.2044-835X.2011.02043.x · DOI
  5. Belin, P., Zatorre, R.J., Lafaille, P., Ahad, P., Pike, B., 2000. Voice-selective areas in human auditory cortex. Nature 403, 309-312. https://doi.org/10.1038/35002078 · DOI
  6. Bernini, M., 2016. The opacity of fictional minds: Transparency, interpretive cognition and the exceptionality thesis, in: Garratt, P. (Ed.), The Cognitive Humanities: Embodied Mind in Literature and Culture. Palgrave Macmillan UK, London, pp. 35-54. https://doi.org/10.1057/978-1-137-59329-0_3 · DOI
  7. Borst, G., Niven, E., Logie, R.H., 2011. Visual mental image generation does not overlap with visual short-term memory: A dual-task interference study. Mem. Cognit. 40, 360-372. https://doi.org/10.3758/s13421-011-0151-7 · DOI
  8. Bortolussi, M., Dixon, P., 2003. Psychonarratology: Foundations for the empirical study of literary response. Cambridge University Press, Cambridge.
  9. Bravo, F., Cross, I., Hawkins, S., Gonzalez, N., Docampo, J., Bruno, C., Stamatakis, E.A., 2017. Neural mechanisms underlying valence inferences to sound: The role of the right angular gyrus. Neuropsychologia 102, 144-162. https://doi.org/10.1016/j.neuropsychologia.2017.05.029 · DOI
  10. Bray, J., 2007. The 'dual voice' of free indirect discourse: a reading experiment. Lang. Lit. 16, 37-52. https://doi.org/10.1177/0963947007072844 · DOI
  11. Brück, C., Kreifelts, B., Gößling-Arnold, C., Wertheimer, J., Wildgruber, D., 2014. 'Inner voices': the cerebral representation of emotional voice cues described in literary texts. Soc. Cogn. Affect. Neurosci. 9, 1819-1827. https://doi.org/10.1093/scan/nst180 · DOI
  12. Bzdok, D., Langner, R., Schilbach, L., Jakobs, O., Roski, C., Caspers, S., Laird, A.R., Fox, P.T., Zilles, K., Eickhoff, S.B., 2013. Characterization of the temporo-parietal junction by combining data-driven parcellation, complementary connectivity analyses, and functional decoding. NeuroImage 81, 381-392. https://doi.org/10.1016/j.neuroimage.2013.05.046 · DOI
  13. Caracciolo, M., Hurlburt, R.T., 2016. A passion for specificity: Confronting inner experience in literature and science, Forthcoming. ed. Ohio State University Press, Ohio.
  14. Ciaramidaro, A., Adenzato, M., Enrici, I., Erk, S., Pia, L., Bara, B.G., Walter, H., 2007. The intentional network: How the brain reads varieties of intentions. Neuropsychologia 45, 3105-3113. https://doi.org/10.1016/j.neuropsychologia.2007.05.011 · DOI · Paper page
  15. Clark, H.H., Gerrig, R.J., 1990. Quotations as demonstrations. Language 66, 764-805. https://doi.org/10.2307/414729 · DOI
  16. Cohen, J., 1969. Statistical power analysis for the behavioural sciences, 2nd ed. Academic Press, New York.
  17. Cohn, D., 1978. Transparent minds. Princeton University Press, Princeton, NJ.
  18. Daselaar, S.M., Porat, Y., Huijbers, W., Pennartz, C.M.A., 2010. Modality-specific and modality-independent components of the human imagery system. NeuroImage 52, 677-685. https://doi.org/10.1016/j.neuroimage.2010.04.239 · DOI
  19. Dickinson, E.R., Adelson, J.L., Owen, J., 2012. Gender balance, representativeness, and statistical power in sexuality research using undergraduate student samples. Arch. Sex. Behav. 41, 325-327. https://doi.org/10.1007/s10508-011-9887-1 · DOI
  20. Eerland, A., Engelen, J.A.A., Zwaan, R.A., 2013. The influence of direct and indirect speech on mental representations. PLOS ONE 8, e65480. https://doi.org/10.1371/journal.pone.0065480 · DOI
  21. Eerland, A., Zwaan, R.A., 2018. The influence of direct and indirect speech on source memory. Collabra Psychol. 4. https://doi.org/10.1525/collabra.123 · DOI
  22. Esteve-Gibert, N., Guellaï, B., 2018. Prosody in the auditory and visual domains: A developmental perspective. Front. Psychol. 9. https://doi.org/10.3389/fpsyg.2018.00338 · DOI
  23. Fernyhough, C., 2015. The Experimental Design Hackathon, in: Fernyhough, C., Woods, A., Patton, V. (Eds.), Working Knowledge: Transferable Methodology for Interdisciplinary Research. Hearing the Voice, Durham University, UK.
  24. Fernyhough, C., 2008. Getting Vygotskian about theory of mind: Mediation, dialogue, and the development of social understanding. Dev. Rev. 28, 225-262. https://doi.org/10.1016/j.dr.2007.03.001 · DOI
  25. Filik, R., Barber, E., 2011. Inner speech during silent reading reflects the reader's regional accent. PLoS ONE 6, e25782. https://doi.org/10.1371/journal.pone.0025782 · DOI · Paper page
  26. Fletcher, A., Monterosso, J., 2015. The Science of Free-Indirect Discourse: An Alternate Cognitive Effect. Narrative 24, 82-103. https://doi.org/10.1353/nar.2016.0004 · DOI
  27. Fletcher, P.C., Happé, F., Frith, U., Baker, S.C., Dolan, R.J., Frackowiak, R.S.J., Frith, C.D., 1995. Other minds in the brain: a functional imaging study of "theory of mind" in story comprehension. Cognition 57, 109-128. https://doi.org/10.1016/0010- 0277(95)00692-R Gerrig, R.J., 1993. Experiencing narrative worlds: On the psychological activities of reading. Yale University Press. · DOI
  28. Gerrig, R.J., Brennan, S.E., Ohaeri, J.O., 2001. What characters know: Projected knowledge and projected co-presence. J. Mem. Lang. 44, 81-95. https://doi.org/10.1006/jmla.2000.2740 · DOI
  29. Green, M.C., 2004. Transportation into narrative worlds: The role of prior knowledge and perceived realism. Discourse Process. 38, 247-266. https://doi.org/10.1207/s15326950dp3802_5 · DOI
  30. Green, M.C., Brock, T.C., Kaufman, G.F., 2004. Understanding Media Enjoyment: The Role of Transportation Into Narrative Worlds. Commun. Theory 14, 311-327. https://doi.org/10.1111/j.1468-2885.2004.tb00317.x · DOI
  31. Herman, D., 2013. Storytelling and the sciences of mind. MIT Press, Boston, MA.
  32. Herman, D., 2008. Narrative theory and the intentional stance. Partial Answ. J. Lit. Hist. Ideas 6, 233-260. https://doi.org/10.1353/pan.0.0019 · DOI
  33. Köder, F., Maier, E., Hendriks, P., 2015. Perspective shift increases processing effort of pronouns: a comparison between direct and indirect speech. Lang. Cogn. Neurosci. 30, 940-946. https://doi.org/10.1080/23273798.2015.1047460 · DOI
  34. Kühn, S., Fernyhough, C., Alderson-Day, B., Hurlburt, R.T., 2014. Inner experience in the scanner: Can high fidelity apprehensions of inner experience be integrated with fMRI? Front. Psychol. 5. https://doi.org/doi: 10.3389/fpsyg.2014.01393 · DOI
  35. Kuiken, D., Miall, D.S., Sikora, S., 2004. Forms of self-implication in literary reading. Poet. Today 25, 171-203. https://doi.org/10.1215/03335372-25-2-171 · DOI
  36. Kurby, C.A., Magliano, J.P., Rapp, D.N., 2009. Those voices in your head: activation of auditory images during reading. Cognition 112, 457-461. https://doi.org/10.1016/j.cognition.2009.05.007 · DOI
  37. Kurby, C.A., Zacks, J.M., 2013. The activation of modality-specific representations during discourse processing. Brain Lang. 126, 338-349. https://doi.org/10.1016/j.bandl.2013.07.003 · DOI
  38. Leech, G., Short, M., 2007. Style in fiction: A linguistic introduction to english fictional prose, 2 edition. ed. Routledge, New York.
  39. Magrassi, L., Aromataris, G., Cabrini, A., Annovazzi-Lodi, V., Moro, A., 2015. Sound representation in higher language areas during language generation. Proc. Natl. Acad. Sci. 112, 1868-1873. https://doi.org/10.1073/pnas.1418162112 · DOI
  40. Maslej, M.M., Oatley, K., Mar, R.A., 2017. Creating fictional characters: The role of experience, personality, and social processes. Psychol. Aesthet. Creat. Arts 11, 487- 499. https://doi.org/10.1037/aca0000094 · DOI
  41. Miall, D.S., 2011. Emotions and the structuring of narrative responses. Poet. Today 32, 323- 348. https://doi.org/10.1215/03335372-1162704 · DOI
  42. Molenberghs, P., Johnson, H., Henry, J.D., Mattingley, J.B., 2016. Understanding the minds of others: A neuroimaging meta-analysis. Neurosci. Biobehav. Rev. 65, 276-291. https://doi.org/10.1016/j.neubiorev.2016.03.020 · DOI
  43. Murphy, C., Jefferies, E., Rueschemeyer, S.-A., Sormaz, M., Wang, H., Margulies, D.S., Smallwood, J., 2018. Distant from input: Evidence of regions within the default mode network supporting perceptually-decoupled and conceptually-guided cognition. NeuroImage 171, 393-401. https://doi.org/10.1016/j.neuroimage.2018.01.017 · DOI
  44. Oatley, K., 2016. Fiction: Simulation of social worlds. Trends Cogn. Sci. 20, 618-628. https://doi.org/10.1016/j.tics.2016.06.002 · DOI · Paper page
  45. Oatley, K., 2011. Such stuff as dreams. Wiley, Chichester, West Sussex, U.K. ; Malden, MA.
  46. Okada, K., Matchin, W., Hickok, G., 2018. Neural evidence for predictive coding in auditory cortex during speech production. Psychon. Bull. Rev. 25, 423-430. https://doi.org/10.3758/s13423-017-1284-x · DOI
  47. Oppenheim, G.M., Dell, G.S., 2010. Motor movement matters: The flexible abstractness of inner speech. Mem. Cognit. 38, 1147-1160. https://doi.org/10.3758/MC.38.8.1147 · DOI · Paper page
  48. Palmer, A., 2004. Fictional minds. University of Nebraska Press, Lincoln.
  49. Perrone-Bertolotti, M., Kujala, J., Vidal, J.R., Hamame, C.M., Ossandon, T., Bertrand, O., Minotti, L., Kahane, P., Jerbi, K., Lachaux, J.-P., 2012. How silent is silent reading? Intracerebral evidence for top-down activation of temporal voice areas during reading. J. Neurosci. Off. J. Soc. Neurosci. 32, 17554-17562. https://doi.org/10.1523/JNEUROSCI.2982-12.2012 · DOI
  50. Perrone-Bertolotti, M., Rapin, L., Lachaux, J.-P., Baciu, M., Lœvenbruck, H., 2014. What is that little voice inside my head? Inner speech phenomenology, its role in cognitive performance, and its relation to self-monitoring. Behav. Brain Res. 261, 220-239. https://doi.org/10.1016/j.bbr.2013.12.034 · DOI · Paper page
  51. Petkov, C.I., Belin, P., 2013. Silent reading: Does the brain 'hear' both speech and voices? Curr. Biol. 23, R155-R156. https://doi.org/10.1016/j.cub.2013.01.002 · DOI
  52. Poldrack, R.A., 2011. Inferring mental states from neuroimaging data: From reverse inference to large-scale decoding. Neuron 72, 692-697. https://doi.org/10.1016/j.neuron.2011.11.001 · DOI · Paper page
  53. Polvinen, M., 2016. Enactive perception and fictional worlds, in: The Cognitive Humanities. Palgrave Macmillan, London, pp. 19-34.
  54. Prinz, J., 2011. The sensory basis of cognitive phenomenology, in: Bayne, T., Montague, M. (Eds.), Cognitive Phenomenology. Oxford University Press, pp. 174-196.
  55. Richardson, J.T.E., 2011. Eta squared and partial eta squared as measures of effect size in educational research. Educ. Res. Rev. 6, 135-147. https://doi.org/10.1016/j.edurev.2010.12.001 · DOI
  56. Ruby, P., Decety, J., 2001. Effect of subjective perspective taking during simulation of action: a PET investigation of agency. Nat. Neurosci. 4, 546-550. https://doi.org/10.1038/87510 · DOI · Paper page
  57. Saxe, R., Powell, L.J., 2006. It's the thought that counts: specific brain regions for one component of theory of mind. Psychol. Sci. 17, 692-699. https://doi.org/10.1111/j.1467-9280.2006.01768.x · DOI · Paper page
  58. Schurz, M., Radua, J., Aichhorn, M., Richlan, F., Perner, J., 2014. Fractionating theory of mind: A meta-analysis of functional brain imaging studies. Neurosci. Biobehav. Rev. https://doi.org/10.1016/j.neubiorev.2014.01.009 · DOI · Paper page
  59. Schurz, M., Tholen, M.G., Perner, J., Mars, R.B., Sallet, J., 2017. Specifying the brain anatomy underlying temporo-parietal junction activations for theory of mind: A review using probabilistic atlases from different imaging modalities. Hum. Brain Mapp. 38, 4788-4805. https://doi.org/10.1002/hbm.23675 · DOI
  60. Sharvit, Y., 2008. The puzzle of free indirect discourse. Linguist. Philos. 31, 353-395. https://doi.org/10.1007/s10988-008-9039-9 · DOI
  61. Simons, C.J.P., Tracy, D.K., Sanghera, K.K., O'Daly, O., Gilleen, J., Dominguez, M.-G., Krabbendam, L., Shergill, S.S., 2010. Functional magnetic resonance imaging of inner speech in schizophrenia. Biol. Psychiatry 67, 232-237. https://doi.org/10.1016/j.biopsych.2009.09.007 · DOI · Paper page
  62. Sotirova, V., 2004. Connectives in free indirect style: Continuity or shift? Lang. Lit. 13, 216- 234. https://doi.org/10.1177/0963947004044872 · DOI
  63. Spreng, R.N., Mar, R.A., Kim, A.S.N., 2009. The common neural basis of autobiographical memory, prospection, navigation, theory of mind, and the default mode: a quantitative meta-analysis. J. Cogn. Neurosci. 21, 489-510. https://doi.org/10.1162/jocn.2008.21029 · DOI · Paper page
  64. Tomasello, M., Carpenter, M., Call, J., Behne, T., Moll, H., 2005. Understanding and sharing intentions: The origins of cultural cognition. Behav. Brain Sci. 28, 675-691. https://doi.org/10.1017/S0140525X05000129 · DOI · Paper page
  65. Uddin, L.Q., 2016. Salience Network of the Human Brain. Academic Press, San Diego.
  66. van Dijk, T.A., Kintsch, W., 1983. Strategies of discourse comprehension. Academic Press.
  67. Vilhauer, R.P., 2016. Inner reading voices: An overlooked form of inner speech. Psychosis 8, 37-47. https://doi.org/10.1080/17522439.2015.1028972 · DOI
  68. Walter, H., Adenzato, M., Ciaramidaro, A., Enrici, I., Pia, L., Bara, B., 2004. Understanding intentions in social interaction: The role of the anterior paracingulate cortex. J. Cogn. Neurosci. 16, 1854-1863. https://doi.org/10.1162/0898929042947838 · DOI · Paper page
  69. James, D. (Ed.), The Legacies of Modernism: Historicising Postwar and Contemporary Fiction. Cambridge University Press, Cambridge, pp. 75-95.
  70. Wellman, H.M., Cross, D., Watson, J., 2001. Meta-analysis of Theory-of-Mind development: The truth about false belief. Child Dev. 72, 655-684. https://doi.org/10.1111/1467- 8624.00304 · DOI · Paper page
  71. Woodward, A.L., 1998. Infants selectively encode the goal object of an actor's reach. Cognition 69, 1-34. https://doi.org/10.1016/S0010-0277(98)00058-4 · DOI
  72. Zunshine, L., 2006. Why we read fiction: Theory of mind and the novel. Ohio State University Press, Columbus.
  73. Zwaan, R.A., 2008. Experiential traces and mental simulations in language comprehension, in: de Vega, M., Glenberg, A., Graesser, A. (Eds.), Symbols and Embodiment: Debates on Meaning and Cognition. OUP, Oxford, pp. 165-180.
  74. Zwaan, R.A., Madden, C.J., Yaxley, R.H., Aveyard, M.E., 2004. Moving words: dynamic representations in language comprehension. Cogn. Sci. 28, 611-619. https://doi.org/10.1207/

Cited by

Sorted by citations

Authors
Alderson-Day, B., Moffatt, J., Bernini, M., Mitrenga, K., Yao, B., & Fernyhough, C.
Affiliations
  • Durham University
  • University of Sussex (GB)
  • Durham University (GB)
  • University of Manchester (GB)
ORCID
Venue
Journal of Cognitive Neuroscience
Publisher
MIT Press
Published
2020-09-01
Volume
32
Issue
9
Pages
1637-1653
ISSN
  • 0898-929X (ISSN-L)
  • 1530-8898
DOI
10.1162/jocn_a_01571
OpenAlex
W3022462384
Cited by
11
References
18
Citations by year
2026: 3 · 2025: 2 · 2024: 2 · 2023: 2 · 2022: 1 · 2021: 1
FWCI
0.54
Type
journal article
Language
en
Open access
Licence
cc-by
Locations
  • Journal of Cognitive Neuroscience (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • Durham Research Online (Durham University) (acceptedVersion, cc-by)
  • Durham Research Online (Durham University) (submittedVersion, cc-by)
  • Lancaster EPrints (Lancaster University) (submittedVersion)
  • Research Explorer (The University of Manchester) (submittedVersion, other-oa)
Funders
Wellcome Trust (WT098455, 108720/Z/15/Z, WT 108720)
Topics
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Neurobiology of Language and Bilingualism
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Neuroscience and Music Perception
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Tactile and Sensory Interactions
Keywords
silent reading, reported speech, fictional characters, theory of mind, inner speech, auditory imagery, source memory, intention inference, narrative comprehension
MeSH terms
Auditory Cortex, Humans, Male, Reading, Speech, Speech Perception, Voice

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review