Cited by 452Action Observation and SynchronizationSocial Robot Interaction and HRIFace Recognition and Perception

The thing that should not be: predictive coding and the uncanny valley in perceiving human and humanoid robot actions

Open access
Saygin, Ayse Pinar, Chaminade, Thierry, Ishiguro, Hiroshi, Driver, Jon, Frith, Chris·2012·Social Cognitive and Affective Neuroscience

Abstract

Using functional magnetic resonance imaging (fMRI) repetition suppression, we explored the selectivity of the human action perception system (APS), which consists of temporal, parietal and frontal areas, for the appearance and/or motion of the perceived agent. Participants watched body movements of a human (biological appearance and movement), a robot (mechanical appearance and movement) or an android (biological appearance, mechanical movement). With the exception of extrastriate body area, which showed more suppression for human like appearance, the APS was not selective for appearance or motion per se. Instead, distinctive responses were found to the mismatch between appearance and motion: whereas suppression effects for the human and robot were similar to each other, they were stronger for the android, notably in bilateral anterior intraparietal sulcus, a key node in the APS. These results could reflect increased prediction error as the brain negotiates an agent that appears human, but does not move biologically, and help explain the 'uncanny valley' phenomenon.

AI snapshot

Written from the paper’s full text.

TL;DR

Using fMRI repetition suppression, this study tested whether the human action perception system is selective for biological appearance or motion by comparing responses to a human, a humanoid android, and the same robot with mechanical appearance. The android—human-like appearance with mechanical movement—produced distinctive repetition suppression, especially in bilateral anterior intraparietal sulcus, which the authors interpret as prediction error relevant to the uncanny valley.

Key findings

  • The action perception system was largely not selective for appearance or motion per se; the main exception was left extrastriate body area (EBA), which showed more suppression for human-like appearance (Agent × Repetition, P = 0.03; Android > Robot, P = 0.02; Human > Robot, P = 0.07).
  • The Android condition produced stronger repetition suppression than both Human and Robot in bilateral anterior intraparietal sulcus (aIPS): left aIPS Agent × Repetition P = 0.002, Android > Robot P = 0.002, Android > Human P = 0.004; right aIPS Agent × Repetition P = 0.002, Android > Robot P = 0.003, Android > Human P = 0.02.
  • Left superior intraparietal sulcus (sIPS) also showed an Agent × Repetition interaction (P = 0.04), with Android > Robot (P = 0.01).
  • Behavioral accuracy on comprehension questions was 0.84 (s.d. = 0.28) and did not differ between conditions; eye movement measures also did not differ between conditions (P > 0.1 for all pairwise comparisons).

Methods

  • Twenty healthy right-handed adults aged 20–36 participated; data from one participant were excluded due to excessive head movement. Participants watched 2-s grayscale video clips of eight actions performed by Human, Android (Repliee Q2 with human-like appearance and mechanical movement), and Robot (same robot stripped of human-like surface appearance, with identical mechanical movement).
  • fMRI was performed on a 3 T Siemens Allegra scanner across 6 runs of 12 blocks; each block contained 12 videos, and each clip was equiprobably a repeat or nonrepeat of the previous clip.
  • Data were analyzed with SPM5 using whole-brain random-effects repetition suppression (Nonrepeat > Repeat, t ≥ 8.86, P < 0.05 FDR-corrected, cluster ≥ 30 voxels), followed by ROI analyses using 5-mm spheres and repeated-measures ANOVA on Agent × Repetition.

Limitations

  • The authors state that the experiment was not designed optimally to “explain” the uncanny valley and can make only a modest contribution to defining its neural basis; they also state that a conclusive or causal link between prediction error and the uncanny valley cannot be made from these data.
  • The authors note there is no agreed-upon localizer for the action perception system, and ventral premotor cortex did not show significant repetition suppression despite being a key node of the APS.
  • Evident from the text: the sample was small (20 recruited, 19 analyzed), and the Android and Robot conditions used the same robot with identical programmed kinematics, so appearance and motion were manipulated in a constrained, non-parametric way rather than independently across many agents.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1093/scan/nsr025

Cites

97 of 97 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

0 of 66 cited sources checked · 0 supported · 0 flagged

  • Findingown data

    With the exception of extrastriate body area (EBA), which showed more suppression for human like appearance, the APS was not selective for appearance or motion per se.

    “With the exception of extrastriate body area, which showed more suppression for human like appearance, the APS was not selective for appearance or motion per se.”
    Quote in the paperSubstantial claim
  • Findingown data

    Repetition suppression effects were stronger for the android than for human and robot, notably in bilateral anterior intraparietal sulcus (aIPS).

    “they were stronger for the android, notably in bilateral anterior intraparietal sulcus, a key node in the APS.”
    • Also stated in: § Experimental procedures and data analysis · plain
      “aIPS | 40 | −42 | −38 | 42 | 3.96 | 0.30 | 0.81 | 0.42 | Agent × repetition (P = 0.002)”
    Quote in the paperSubstantial claim
  • Findingown data

    Repetition suppression effects for the human and robot were similar to each other.

    “whereas suppression effects for the human and robot were similar to each other”
    • Also stated in: § FMRI data · plain
      “with the Human and Robot conditions leading to similar patterns of suppression”
    Quote in the paperSubstantial claim
  • Findingown data

    There was a significant Agent by Repetition interaction in the anterior portion of the intraparietal sulcus bilaterally, corresponding to area aIPS.

    “There was a significant Agent by Repetition interaction in the anterior portion of the intraparietal sulcus bilaterally, corresponding to area aIPS, the putative human homologue of macaque area AIP (Grefkes and Fink, 2005; Culham and Valyear, 2006; Grafton and Hamilton, 2007).”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites Grefkes and Fink, 2005

      Not checked in the free run.

    • cites Culham and Valyear, 2006

      Not checked in the free run.

    • cites Grafton and Hamilton, 2007

      Not checked in the free run.

  • Findingown data

    Accuracy did not differ between conditions.

    “We explored with a repeated measures analysis of variance (ANOVA) whether accuracy varied across conditions (it did not).”
    • Also stated in: § Behavioral and eye movement data · plain
      “Accuracy did not differ between conditions (P > 0.1 for all pair-wise comparisons).”
    Quote in the paperSubstantial claim
  • May combine two claims. 12
  • The quote only partly states this claim. 2
  • Classified as probably not a substantial claim. 22
  • May combine two claims.

    Findingown data

    The Agent by Repetition interaction reached significance in only a subset of regions, despite qualitatively more suppression for the Android condition in a widespread network.

    “Even though there was qualitatively more suppression for the Android condition compared with the others in a widespread network (Figure 2), the Agent by Repetition interaction reached significance in only a subset of these regions (Table 1).”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Findingown data

    In fusiform gyrus, repetition suppression was stronger for Android than Robot (P = 0.01) and Android than Human (P = 0.05).

    “Fusiform gyrus | | 46 | −44 | −16 | 4.02 | 0.22 | 0.87 | 0.41 | Agent × repetition (P = 0.075)”
    • Also stated in: § Experimental procedures and data analysis · plain
      “A > R (P = 0.01)”
    • Also stated in: § Experimental procedures and data analysis · plain
      “A > H (P = 0.05)”
    Quote in the paperSubstantial claim
  • The quote only partly states this claim.

    Findingown data

    In bilateral aIPS, repetition suppression was stronger for Android than Robot (P = 0.002/0.003).

    “aIPS | 40 | −42 | −38 | 42 | 3.96 | 0.30 | 0.81 | 0.42 | Agent × repetition (P = 0.002)”
    • Also stated in: § Experimental procedures and data analysis · plain
      “A > R (P = 0.002)”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingown data

    The main effect of Agent was found in visual cortex bilaterally, driven by a stronger response for the Robot condition compared with the other agents.

    “For completeness, we also report the main effect of Agent: this effect was found in visual cortex bilaterally (with peaks in MNI coordinates −30, −92, 2 and 38, −80, −16), and was driven by a stronger response for the Robot condition compared with the other agents.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    There were other repetition suppression foci in frontal cortex, including one that extended into dorsal premotor cortex.

    “There were other repetition suppression foci in frontal cortex, including one that extended into dorsal premotor cortex.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Findingown data

    The right hemisphere peak and a more dorsal subpeak of the left hemisphere cluster did not show significant differences between agents.

    “The right hemisphere peak, and a more dorsal subpeak of the left hemisphere cluster, did not show significant differences between agents.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Findingown data

    Repetition suppression was found in the parietal and temporal nodes of the APS, but ventral premotor cortex did not show significant repetition suppression.

    “Repetition suppression was found in the parietal and temporal nodes of the APS, but despite being a key node of the APS, ventral premotor cortex did not show significant repetition suppression (cf Chong et al., 2008; Lestou et al., 2008; Grossman et al., 2010).”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    In middle frontal gyrus, repetition suppression was stronger for Android than Human (P = 0.07).

    “Middle Frontal Gyrus | 46 | 50 | 48 | 10 | 3.93 | 0.58 | 0.80 | 0.22 | A > H (P = 0.07)”
    Quote in the paperNot a substantial claim
  • May combine two claims. The quote only partly states this claim.

    Findingown data

    In parahippocampal/amygdala, repetition suppression was greater for Android than Robot (P = 0.09) and Human than Robot (P = 0.08).

    “Parahippocampal/Amygdala | | 26 | −4 | −18 | 3.99 | 0.10 | 0.68 | 0.75 | A > R (P = 0.09)”
    • Also stated in: § Experimental procedures and data analysis · plain
      “H > R (P = 0.08)”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Findingown data

    In the repetition suppression network, all areas showed reduced responses to repeated stimuli except primary visual cortex, which showed repetition enhancement.

    “All showed reduced responses to the repeated stimuli, with the exception of primary visual cortex, which showed repetition enhancement.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    In left EBA, repetition suppression was greater for Human than Robot (P = 0.07).

    “H > R (P = 0.07)”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    In non-human primates, at least two of these regions contain 'mirror neurons', which fire during the execution as well as the observation of specific movements.

    “In non-human primates, at least two of these regions have been reported to contain ‘mirror neurons’, which fire during the execution as well as the observation of specific movements”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Rizzolatti and Craighero, 2004

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Perception of body movements in primates is supported by a network including lateral superior temporal, inferior parietal and inferior frontal brain areas.

    “In primates, perception of body movements is thought to be supported by a network including lateral superior temporal, inferior parietal and inferior frontal brain areas.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Blake and Shiffrar, 2007

      Not checked in the free run.

  • Classified as probably not a substantial claim. May combine two claims.

    Premisecitation

    Some studies have reported that artificial agents' actions affect observers' own motor processing, or activity within the APS, whereas others have argued that the APS either does not respond, or responds weakly if the perceived actor is not human.

    “Some studies have reported that artificial agents’ actions apparently affect the observers’ own motor processing, or activity within the APS, whereas others have argued that the APS either does not respond, or responds weakly if the perceived actor is not human”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites Kilner et al., 2003

      Not checked in the free run.

    • cites Tai et al., 2004

      Not checked in the free run.

    • cites Chaminade and Hodgins, 2006

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Neuroimaging studies have shown responses in lateral superior temporal, inferior parietal and inferior frontal brain areas during observation of actions.

    “Neuroimaging studies have shown responses in these areas during observation of actions”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites Saygin et al., 2004a

      Not checked in the free run.

    • cites Pobric and Hamilton, 2006

      Not checked in the free run.

    • cites Grafton and Hamilton, 2007

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Damage or disruption of lateral superior temporal, inferior parietal and inferior frontal brain areas can affect action processing.

    “neuropsychological patient and transcranial magnetic stimulation (TMS) studies have shown that damage or disruption of these areas can affect action processing”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites Saygin et al., 2004a

      Not checked in the free run.

    • cites Pobric and Hamilton, 2006

      Not checked in the free run.

    • cites Grafton and Hamilton, 2007

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Premisecitation

    The appearance of the observed agent may also be important.

    “The appearance of the observed agent may also be important”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites Buccino et al., 2004

      Not checked in the free run.

    • cites Chaminade et al., 2007

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    There are links between activity within the APS and whether the observer can perform the seen movement.

    “there are links between activity within the APS and whether the observer can perform the seen movement”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites Calvo-Merino et al., 2006

      Not checked in the free run.

    • cites Cross et al., 2006

      Not checked in the free run.

    • cites Candidi et al., 2008

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Most modern androids, including Repliee Q2 used here, are thought to fall into the uncanny valley.

    “Most modern androids, including Repliee Q2 used here, are also thought to fall into the uncanny valley”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Ishiguro, 2006

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Premisecitation

    Repetition suppression effects are thought to reflect stimulus processing rather than task demands.

    “Repetition suppression effects are thought to reflect stimulus processing rather than task demands”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Xu et al., 2007

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    These predictions can be used to correct for unanticipated events, and to account for sensory noise and delays.

    “These predictions can be used to correct for unanticipated events, and to account for sensory noise and delays.”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites Wolpert et al., 1995

      Not checked in the free run.

    • cites Wolpert and Miall, 1996

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Premisecitation

    Repetition is thought to lead to reduced responses only in neurons selective for the repeated properties.

    “Repetition is thought to lead to such reduced responses only in neurons selective for the repeated properties”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites Henson and Rugg, 2003

      Not checked in the free run.

    • cites Grill-Spector et al., 2006

      Not checked in the free run.

    • cites Krekelberg et al., 2006

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Premisecitation

    The models can be recruited to infer the meaning of a perceived action given the sensory information.

    “The models can be recruited to infer the meaning of a perceived action given the sensory information”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Wolpert et al., 2003

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Study design and conductreasoning

    Participants were not uncertain about the identity of the android by the time scanning took place.

    “They were not uncertain about the identity of the android by the time scanning took place.”
    Quote in the paperSubstantial claim
  • Classified as probably not a substantial claim.

    Premisecitation

    During perception, the error between the prediction coming from internal models and incoming visual information can be minimized by selecting models yielding accurate predictions, that therefore correspond to the observed action.

    “During perception, the error between the prediction coming from internal models and incoming visual information can be minimized by selecting models yielding accurate predictions, that therefore correspond to the observed action”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Kilner et al., 2007

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data

    Data from one participant could not be used due to excessive head movement.

    “Data from one participant could not be used due to excessive head movement.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S4 (based).

    Premisenone given

    BOLD differences for the experimental conditions can arise due to low-level stimulus factors such as differences in illumination, spatial frequency, color, or contrast that have little or nothing to do with action processing, as well as nonspecific attention or arousal effects.

    “BOLD differences for the experimental conditions (e.g. a main effect of Agent) can arise due to a number of low-level stimulus factors such as differences in illumination, spatial frequency, color, or contrast that have little or nothing to do with action processing, as well as nonspecific attention or arousal effects.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Premisecitation

    The android used in the study is often mistaken for a human at first sight, and longer exposure and dynamic viewing has been linked to the uncanny valley.

    “While the android used in our study is often mistaken for a human at first sight, longer exposure and dynamic viewing has been linked to the uncanny valley (Ishiguro, 2006).”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Ishiguro, 2006

      Not checked in the free run.

  • May combine two claims.

    Interpretationown data

    For the Android, there was a mismatch between the human-like appearance and the mechanical motion, leading to a larger prediction error, manifest as activity in relevant brain regions.

    “For the Android, however, there was a mismatch between the human-like appearance and the mechanical motion, leading to a larger prediction error, manifest as activity in relevant brain regions.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: We have a lifetime of experience that associates human appearance with biological motion, and machines (such as robots) with mechanical motion.
    • Rests on an assumption: For both the Human and Robot conditions, the observed motion kinematics was congruent with what would be predicted from the appearance of the agent.
  • May combine two claims.

    Study design and conductreasoning

    The experiments were not designed in an optimum fashion to explain the uncanny valley and can only make a modest contribution to defining its neural basis.

    “Our experiments and similar studies (e.g. Steckenfinger and Ghazanfar, 2009) were not designed in an optimum fashion to ‘explain’ the uncanny valley and as such can only make a modest contribution to defining its neural basis.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Steckenfinger and Ghazanfar, 2009

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Violating perceptual expectations can have striking effects, as illustrated by perceptual illusions.

    “It has also long been acknowledged that violating perceptual expectations can have striking effects, compellingly illustrated by perceptual illusions (e.g. Gregory, 1980).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Gregory, 1980

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The prediction error would be smaller when a stimulus was preceded by the same stimulus, consistent with neural models of repetition suppression.

    “The prediction error would be smaller when a stimulus was preceded by the same stimulus, consistent with neural models of repetition suppression (Desimone and Duncan, 1995; Friston, 2005; Grill-Spector et al., 2006).”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites Desimone and Duncan, 1995

      Not checked in the free run.

    • cites Friston, 2005

      Not checked in the free run.

    • cites Grill-Spector et al., 2006

      Not checked in the free run.

32 of 32
  • The paper relies on this without support.

    Robots can have appearance or movement kinematics that are not biological, but can nevertheless be perceived as carrying out recognizable actions.

    “robots can have appearance or movement kinematics that are not biological, but can nevertheless be perceived as carrying out recognizable actions.”
  • The paper relies on this without support.

    We have a lifetime of experience that associates human appearance with biological motion, and machines (such as robots) with mechanical motion.

    “We have a lifetime of experience that associates human appearance with biological motion, and machines (such as robots) with mechanical motion.”
  • The paper relies on this without support.

    For both the Human and Robot conditions, the observed motion kinematics was congruent with what would be predicted from the appearance of the agent.

    “For both our Human condition and our Robot condition, the observed motion kinematics was congruent with what would be predicted from the appearance of the agent.”
3 of 3

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Human-robot difference in EBA is called significant despite P = 0.07

    Major

    The discussion says suppression for the robot was significantly less than for both the human and android. Table 1 reports the human-versus-robot comparison as P = 0.07, above the paper's stated P < 0.05 significance threshold.

  • Visual-cortex enhancement is listed among regions selected for suppression

    Minor

    The methods say Table 1 contains regions identified by the positive Nonrepeat > Repeat suppression contrast. Its V1/V2 entries instead have negative suppression values, indicating greater responses to repeated stimuli.

2 of 2

Cites

Flagged first, then by citations

Research works printed but not linked: 2

  1. Levi, 2004, Why Tom Hanks is less than human: While sensors cannot capture how humans act, humans can give life to digital characters, Newsweek, 650, 305
  2. Mori, 1970, The uncanny valley, Energy, 7, 33

Cited by

Sorted by citations

Authors
Saygin, A. P., Chaminade, T., Ishiguro, H., Driver, J., & Frith, C.
Affiliations
Centre National de la Recherche Scientifique (FR)
Venue
Social Cognitive and Affective Neuroscience
Published
2011-04-22
Volume
7
Issue
4
Pages
413-422
ISSN
  • 1749-5016 (ISSN-L)
  • 1749-5024
OpenAlex
W2121184995
Cited by
452
References
97
Citations by year
2026: 36 · 2025: 34 · 2024: 27 · 2023: 22 · 2022: 30 · 2021: 35 · 2020: 24 · 2019: 36 · 2018: 35 · 2017: 34 · 2016: 29 · 2015: 38 · 2014: 27 · 2013: 22 · 2012: 20
FWCI
30.89
Type
journal article
Language
en
Open access
Licence
cc-by-nc
Locations
  • Social Cognitive and Affective Neuroscience (publishedVersion, cc-by-nc)
  • Europe PMC (PubMed Central) (publishedVersion, cc-by-nc)
  • PubMed Central (publishedVersion, cc-by-nc)
  • PubMed (publishedVersion)
  • HAL (submittedVersion, cc-by-nc)
  • UCL Discovery (University College London) (submittedVersion)
Funders
  • Royal Society
  • National Research Foundation
  • Danmarks Grundforskningsfond
  • University of California, San Diego
  • Japan Society for the Promotion of Science
  • Kavli Institute for Brain and Mind, University of California, San Diego
Topics
  • Social Sciences › Psychology › Social Psychology › Action Observation and Synchronization
  • Social Sciences › Psychology › Social Psychology › Social Robot Interaction and HRI
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Face Recognition and Perception
Keywords
predictive coding, uncanny valley, action perception, humanoid robots, Android, prediction error, intraparietal sulcus, extrastriate body area, human-robot interaction
MeSH terms
Adult, Biomechanical Phenomena, Brain Mapping, Cerebral Cortex, Eye Movements, Female, Gestures, Humans, Image Processing, Computer-Assisted, Magnetic Resonance Imaging, Male, Motion, Motion Perception, Oxygen, Photic Stimulation, Predictive Value of Tests, Random Allocation, Robotics, Young Adult

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-09.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-09.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review