Panero et al. (2016): Failure to replicate methods caused the failure to replicate results.
The DOI matches its Crossref record; 8 of the 8 references it cites were found in a registry or an index.
Abstract
Contrary to Kidd and Castano (2013), Panero et al. (2016) fail to find that reading literary fiction improves performance on an advanced test of theory of mind (ToM), the Reading the Mind in the Eyes Test. However, this commentary shows that the findings presented in Panero et al. (2016) are not reliable due to two striking threats to the internal validity of their studies that were not clearly disclosed or discussed in the manuscript or supplementary materials. First, no effective strategy was implemented to ensure that participants read their assigned texts, and examination of the data revealed many participants whose reading times indicate that they were not exposed to the manipulation. Second, further examination shows that two of the largest studies contributing to Panero et al. (2016) are not valid experiments due to a clear failure of random assignment to conditions. These threats to experimental internal validity make the conclusions presented in Panero et al. (2016) untenable. After removing cases in which participants were not exposed to the manipulation and the data from the two studies without random assignment, an analysis reveals that reading literary fiction improves ToM compared to reading popular genre fiction. This result is consistent with prior studies and indicates that a failure to carefully replicate the methods of Kidd and Castano (2013) led to the failure to replicate Kidd and Castano's (2013) results. (PsycINFO Database Record
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Reading literary fiction improves theory of mind
Kidd, D. C., & Castano, E. (2013) · Science · doi:10.1126/science.1239918 · Cited by 1,642 · cited 23×
“…to Replicate Methods Caused the Failure to Replicate Results David Comer Kidd and Emanuele Castano The New School for Social Research Contrary to Kidd and Castano (2013), Panero et al. (2016) fail to find that reading literary fiction improves performance on an advanced test of theory of mind…”
BackgroundMethodResultInfluentialSelf-citationTo read or not to read: a meta-analysis of print exposure from infancy to early adulthood
Mol, S. E., & Bus, A. G. (2011) · Psychological Bulletin · doi:10.1037/a0021890 · Cited by 1,299 · cited 5×
“1 Panero et al. (2016) confirmed that no participants were excluded for low reading times (M. E. Panero, personal communication, October 30, 2016). 2 Upon being asked to clarify this matter, the first author of Panero et al. (2016) wrote, “It was not our intention to have unequal Ns per cell and it’s just the way it w…”
BackgroundMethodResultInfluentialMar, R. A., Oatley, K., & Peterson, J. B. (2009) · COMM · doi:10.1515/comm.2009.025 · Cited by 552
Panero, M. E., Weisberg, D. S., Black, J., Goldstein, T. R., Barnes, J. L., Brownell, H., & Winner, E. (2016) · Journal of Personality and Social Psychology · doi:10.1037/pspa0000064 · Cited by 289
Different stories: How levels of familiarity with literary and genre fiction relate to mentalizing.
Kidd, D., & Castano, E. (2017) · Psychology of Aesthetics, Creativity, and the Arts · doi:10.1037/aca0000069 · Cited by 124 · cited 5×
“…Mentalizing, Fiction, Replication Experimental (Black & Barnes, 2015; Kidd, Ongis, & Castano, 2016; Pino & Mazza, 2016) and correlational (Kidd & Castano, 2016) studies have replicated Kidd and Castano’s (2013) original finding that engaging with literary fiction but not popular fiction…”
BackgroundMethodSelf-citationThe Use of “Literary Fiction” to Promote Mentalizing Ability
Pino, M. C., & Mazza, M. (2016) · PLOS ONE · doi:10.1371/journal.pone.0160254 · Cited by 123 · cited 5×
“…that reading literary fiction leads to stronger performance on tests of theory of mind compared to reading nonfiction (e.g., Black & Barnes, 2015; Kidd & Castano, 2013, Exp. 1) and, most important, compared to reading popular fiction (Kidd & Castano, 2013; Kidd et al., 2016; Pino & Mazza, 2016).”
BackgroundResultInfluentialThe effects of reading material on social and non-social cognition
Black, J. E., & Barnes, J. L. (2015) · Poetics · doi:10.1016/j.poetic.2015.07.001 · Cited by 101
On literary fiction and its effects on theory of mind
Kidd, D., Ongis, M., & Castano, E. (2016) · Scientific Study of Literature · doi:10.1075/ssol.6.1.04kid · Cited by 72
Self-citation
Cited by
Mar, R. A. (2018) · Discourse Processes · doi:10.1080/0163853x.2018.1448209 · Cited by 108
Kidd, D., & Castano, E. (2019) · Social Psychological and Personality Science · doi:10.1177/1948550618775410 · Cited by 87
Self-citationvan Kuijk, I., Verkoeijen, P., Dijkstra, K., & Zwaan, R. A. (2018) · Collabra: Psychology · doi:10.1525/collabra.117 · Cited by 67
Imaginary Engagement, Real-World Effects: Fiction, Emotion, and Social Cognition
Barnes, J. L. (2018) · Review of General Psychology · doi:10.1037/gpr0000124 · Cited by 38
Watching More Closely: Shot Scale Affects Film Viewers’ Theory of Mind Tendency But Not Ability
Rooney, B., & Bálint, K. E. (2018) · Frontiers in Psychology · doi:10.3389/fpsyg.2017.02349 · Cited by 26
“The failure to distinguish between these aspects of ToM may explain why previous research has presented conflicting and ambiguous results (e.g., Kidd and Castano, 2013, 2016, 2017; Panero et al., 2016; Pino and Mazza, 2016).”
BackgroundResult
Metadata
- Authors
- Kidd, D. C., & Castano, E.
- Venue
- Journal of Personality and Social Psychology
- Published
- 2017-03
- Volume
- 112
- Issue
- 3
- Pages
- e1-e4
- ISSN
- 0022-3514 (ISSN-L)
- 1939-1315
- OpenAlex
- W2590439001
- Cited by
- 57
- References
- 8
- FWCI
- 21.16
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Panero et al. (2016): Failure to replicate methods caused the failure to replicate results. https://app.cochat.ai/papers/10.1037/pspa0000072
- The paper
- Kidd, D. C., & Castano, E. (2017). Panero et al. (2016): Failure to replicate methods caused the failure to replicate results. Journal of Personality and Social Psychology. https://doi.org/10.1037/pspa0000072
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.