Cited by 291Autism Spectrum Disorder ResearchGenetics and Neurodevelopmental DisordersChild Nutrition and Feeding Issues

Beyond Autism: A Baby Siblings Research Consortium Study of High-Risk Children at Three Years of Age

Open access
Messinger, Daniel, Young, Gregory S., Ozonoff, Sally, Dobkins, Karen, Carter, Alice, Zwaigenbaum, Lonnie, Landa, Rebecca J., Charman, Tony +9 more·2013·Journal of the American Academy of Child & Adolescent Psychiatry

Abstract

Objective First-degree relatives of persons with an autism spectrum disorder (ASD) are at increased risk for ASD-related characteristics. As little is known about the early expression of these characteristics, this study characterizes the non-ASD outcomes of 3-year-old high-risk (HR) siblings of children with ASD.Method Two groups of children without ASD participated: 507 HR siblings and 324 low-risk (LR) control subjects (no known relatives with ASD). Children were enrolled at a mean age of 8 months, and outcomes were assessed at 3 years. Outcome measures were Autism Diagnostic Observation Schedule (ADOS) calibrated severity scores, and Mullen Verbal and Non-Verbal Developmental Quotients (DQ).Results At 3 years, HR siblings without an ASD outcome exhibited higher mean ADOS severity scores and lower verbal and non-verbal DQs than LR controls. HR siblings were over-represented (21% HR versus 7% LR) in latent classes characterized by elevated ADOS severity and/or low to low-average DQs. The remaining HR siblings without ASD outcomes (79%) belonged to classes in which they were not differentially represented with respect to LR siblings.Conclusions Having removed a previously identified 18.7% of HR siblings with ASD outcomes from all analyses, HR siblings nevertheless exhibited higher mean levels of ASD severity and lower levels of developmental functioning than LR children. However, the latent class membership of four-fifths of the HR siblings was not significantly different from that of LR control subjects. One-fifth of HR siblings belonged to classes characterized by higher ASD severity and/or lower levels of developmental functioning. This empirically derived characterization of an early-emerging pattern of difficulties in a minority of 3-year-old HR siblings suggests the importance of developmental surveillance and early intervention for these children.

Integrity signals

Registry

Crossref · 10.1016/j.jaac.2012.12.011

Cites

48 of 48 references verified0 citation errors · 0 not checkable

Statistics

6 tests recomputed · 0 with the reported and recomputed p on opposite sides of .05

Claims the paper makes

0 of 45 cited sources checked · 0 supported · 0 flagged

  • Findingown data

    At age 3, high-risk siblings without an autism spectrum disorder outcome had lower mean verbal developmental quotients than low-risk controls, by an estimated 7.76 points (95% CI 4.96-10.56, d=0.46).

    “At 3 years, HR siblings without an ASD outcome exhibited higher mean ADOS severity scores and lower verbal and non-verbal DQs than LR controls.”
    • Also stated in: § Analyses of Means · hedged
      “There were significant risk group effects on verbal DQ (t = −5.43, p < .001), with the HR group scoring an estimated 7.76 points lower than the LR group (CI = 4.96–10.56, d = 0.46).”
    • Also stated in: § DISCUSSION · plain
      “Overall, HR siblings evidenced lower verbal and nonverbal DQs than LR children.”
    Quote in the paperSubstantial claim
  • Findingown data

    High-risk siblings without an autism spectrum disorder outcome were 3.3 times more likely than low-risk controls to belong to latent class 4, characterized by low ADOS severity and low developmental quotient.

    “HR siblings were more than 3 times more likely than LR children to occupy class 4, which was characterized by low severity scores and low levels of developmental functioning, particularly in the verbal domain.”
    Quote in the paperSubstantial claim
  • Findingown data

    The remaining 79% of high-risk siblings without an autism spectrum disorder outcome belonged to latent classes in which they were not differentially represented relative to low-risk controls.

    “The remaining HR siblings without ASD outcomes (79%) belonged to classes in which they were not differentially represented with respect to LR siblings.”
    • Also stated in: § DISCUSSION · plain
      “There were not significant risk group differences in the composition of classes 1 and 2 (65% of HR siblings and 83% of controls).”
    • Also stated in: § DISCUSSION · plain
      “Four-fifths of these HR siblings were not significantly less likely than control subjects to fall into classes characterized by a range of typical outcomes.”
    Quote in the paperSubstantial claim
  • Findingown data

    Males were 2.9 times more likely than females to be classified in latent class 5, characterized by elevated ADOS severity and low-average developmental quotient.

    “and 2.9 times more likely than females to be classified in class 5 (elevated ASD severity/low-average DQ).”
    • Also stated in: § DISCUSSION · hedged
      “Males were almost three times more likely than females to belong both to class 4 and to class 5.”
    Quote in the paperSubstantial claim
  • Findingown data

    High-risk siblings without an autism spectrum disorder outcome were 4.6 times more likely than low-risk controls to belong to latent class 5, characterized by elevated ADOS severity and low-average developmental quotient.

    “and 4.6 times more likely than LR controls to belong to class 5 (elevated ASD severity/low-average DQ).”
    • Also stated in: § DISCUSSION · plain
      “HR siblings were more than four times more likely than LR children to occupy class 5, which was characterized by both elevated ASD severity and low-average developmental functioning.”
    Quote in the paperSubstantial claim
  • The quote only partly states this claim. 1
  • May combine two claims. 11
  • Classified as probably not a substantial claim. 7
  • The quote only partly states this claim.

    Findingown data

    Males were 2.8 times more likely than females to be classified in latent class 4, characterized by low ADOS severity and low developmental quotient.

    “Males were almost three times more likely than females to belong both to class 4 and to class 5.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Findingown data

    ADOS severity was negatively correlated with verbal and nonverbal developmental quotients, while verbal and nonverbal developmental quotients were positively correlated with each other.

    “As shown in Table 2, ADOS severity was negatively correlated with verbal and nonverbal DQs (range, −0.22 to 0.08), whereas verbal and nonverbal DQs showed substantial positive correlations (0.54 to 0.67).32”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Findingown data

    Within the high-risk group, siblings from multiplex families exhibited substantially lower verbal and nonverbal functioning than siblings from simplex families, an effect that did not differ by infant sex.

    “Within the HR group, siblings from multiplex families exhibited substantially lower verbal and nonverbal functioning than siblings from simplex families, an effect that did not differ by infant sex.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Premisenone given

    Prospective studies indicate that high-risk siblings are at increased risk both for autism spectrum disorder outcomes and for subclinical manifestations of autism spectrum disorder-related behavioral characteristics.

    “Prospective studies indicate that high-risk (HR) siblings—the younger brothers and sisters of a child diagnosed with an ASD—are at increased risk both for ASD outcomes and for subclinical manifestations of ASD-related behavioral characteristics.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    A previous Baby Siblings Research Consortium report indicated that 18.7% of 3-year-old high-risk siblings will themselves develop an autism spectrum disorder.

    “A recent report4 from the Baby Siblings Research Consortium (BSRC) indicated that 18.7% of 3-year-old HR siblings will themselves develop an ASD.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [4] Ozonoff, 2011, Recurrence risk for autism spectrum disorders: a Baby Siblings Research Consortium study, Pediatrics source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Family multiplex status and male sex of the high-risk sibling are risk factors for autism spectrum disorder recurrence.

    “Family multiplex status—more than one older sibling with an ASD—and male sex of the HR sibling are risk factors for ASD recurrence.18”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [18] Ozonoff, 2010, A prospective study of the emergence of early behavioral signs of autism, J Am Acad Child Adolesc Psychiatry

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    Institutional review board approval was obtained to collect and analyze deidentified data from all sites.

    “Institutional review board approval was obtained to collect and analyze deidentified data from all sites.”
    Quote in the paperNot a substantial claim
  • Classified as probably not a substantial claim.

    Study design and conductcitation

    All sites verified proband diagnoses in the high-risk group, typically using the Autism Diagnostic Observation Schedule and/or parent interviews such as the Autism Diagnostic Interview-Revised or the Social Communication Questionnaire.

    “All sites verified proband diagnoses in the high-risk group, typically using the ADOS22 and/or parent interviews such as the Autism Diagnostic Interview–Revised23 or the Social Communication Questionnaire.24”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites [22] Lord, 2000, The Autism Diagnostic Observation Schedule–Generic: a standard measure of social and communication deficits associated with the spectrum of autism, J Autism Dev Disord source

      Not checked in the free run.

    • cites [23] Lord, 1994, Autism Diagnostic Interview–Revised: a revised version of a diagnostic interview for caregivers of individuals with possible pervasive developmental disorders, J Autism Dev Disord source

      Not checked in the free run.

    • cites [24] Rutter, 2003

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Approximately 20% of high-risk siblings at 3 years both exhibit difficulties with language development on standardized assessment and satisfy clinical criteria of social or language delay.

    “From a categorical perspective, approximately 20% of HR siblings at 3 years both exhibit difficulties with language development on standardized assessment11 and satisfy clinical criteria of social or language delay.13”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [11] Yirmiya, 2007, Cognitive and verbal abilities of 24- to 36-month-old siblings of children with autism, J Autism Dev Disord source

      Not checked in the free run.

    • cites [13] Landa, 2012, Latent class analysis of early developmental trajectory in baby siblings of children with autism, J Child Psychol Psychiatry

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    High-risk siblings exhibit lower levels of receptive language and other communication skills than comparison infants, but direct comparisons of intellectual functioning have not yielded consistent results.

    “Likewise, HR siblings exhibit lower levels of receptive language and other communication skills than comparison infants,2,8,9,11 but direct comparisons of intellectual functioning have not yielded consistent results.8,9,11,12”
    Quote in the paperSubstantial claim0 of 5 sources checked
    • cites [2] Landa, 2007, Social and communication development in toddlers with early and later diagnosis of autism spectrum disorders, Arch Gen Psychiatry source

      Not checked in the free run.

    • cites [8] Toth, 2007, Early social, imitation, play, and language abilities of young non-autistic siblings of children with autism, J Autism Dev Disord source

      Not checked in the free run.

    • cites [9] Stone, 2007, Early social-communicative and cognitive development of younger siblings of children with autism spectrum disorders, Arch Pediatr Adolesc Med source

      Not checked in the free run.

    • cites [11] Yirmiya, 2007, Cognitive and verbal abilities of 24- to 36-month-old siblings of children with autism, J Autism Dev Disord source

      Not checked in the free run.

    • cites [12] Zwaigenbaum, 2012, Sex differences in children with autism spectrum disorder identified within a high-risk infant cohort, J Autism Dev Disord

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Through 3 years of age, high-risk infants exhibit higher levels of autism spectrum disorder-related characteristics than controls on autism spectrum disorder screeners, but differences on examiner-administered assessments are less consistent.

    “Through 3 years of age, HR infants exhibit higher levels of ASD-related characteristics than controls on ASD screeners, but differences based on examiner-administered assessments designed to quantify these characteristics are less consistent.2,8–10”
    Quote in the paperSubstantial claim0 of 4 sources checked
    • cites [2] Landa, 2007, Social and communication development in toddlers with early and later diagnosis of autism spectrum disorders, Arch Gen Psychiatry source

      Not checked in the free run.

    • cites [8] Toth, 2007, Early social, imitation, play, and language abilities of young non-autistic siblings of children with autism, J Autism Dev Disord source

      Not checked in the free run.

    • cites [9] Stone, 2007, Early social-communicative and cognitive development of younger siblings of children with autism spectrum disorders, Arch Pediatr Adolesc Med source

      Not checked in the free run.

    • cites [10] Garon, 2009, Temperament and its relationship to autistic symptoms in a high-risk infant sib cohort, J Abnorm Child Psychol source

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Study design and conductcitation

    Mullen Scales of Early Learning developmental quotients were calculated by dividing the subscale age-equivalent score by chronological age and multiplying by 100 to avoid floor and ceiling effects.

    “For the present analyses, developmental quotients (DQs) were calculated by dividing the subscale age-equivalent score by the child’s chronological age and multiplying by 100. This procedure avoided possible floor and ceiling effects while providing a familiar IQ metric, as recommended in the literature.27,28”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [27] Lord, 2006, Autism from 2 to 9 years of age, Arch Gen Psychiatry source

      Not checked in the free run.

    • cites [28] Munson, 2008, Evidence for latent classes of IQ in young children with autism spectrum disorder, Am J Ment Retard source

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Study design and conductnone given

    Outcome was assessed relatively early, at 3 years of age.

    “In addition, outcome was assessed relatively early, at 3 years of age.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Interpretationcitation

    The findings extend previous reports of risk group differences, suggesting that overall differences in social, communication, and behavioral characteristics between high-risk and low-risk children are relatively subtle.

    “Findings of slightly elevated levels of ASD-related characteristics in HR siblings without ASD outcomes extends previous reports of risk group differences, suggesting that overall differences in social, communication, and behavioral characteristics between HR and LR children are relatively subtle.8,10,11,34”
    Quote in the paperSubstantial claim0 of 4 sources checked
    • cites [8] Toth, 2007, Early social, imitation, play, and language abilities of young non-autistic siblings of children with autism, J Autism Dev Disord source

      Not checked in the free run.

    • cites [10] Garon, 2009, Temperament and its relationship to autistic symptoms in a high-risk infant sib cohort, J Abnorm Child Psychol source

      Not checked in the free run.

    • cites [11] Yirmiya, 2007, Cognitive and verbal abilities of 24- to 36-month-old siblings of children with autism, J Autism Dev Disord source

      Not checked in the free run.

    • cites [34] Chapman, 2009, In bad taste: evidence for the oral origins of moral disgust, Science source

      Not checked in the free run.

  • May combine two claims.

    Study design and conductcitation

    Although the study relied on an observational measure of autism spectrum disorder severity, parent report measures of severity appear to yield congruent results.

    “More generally, although the study relied on an observational measure of ASD severity, parent report measures of severity appear to yield congruent results.12”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [12] Zwaigenbaum, 2012, Sex differences in children with autism spectrum disorder identified within a high-risk infant cohort, J Autism Dev Disord

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Study design and conductcitation

    ADOS total scores were converted to autism spectrum disorder-calibrated severity scores to statistically pool scores across modules.

    “To statistically pool ADOS scores across modules, total scores were converted to ASD-calibrated severity scores,25 which range from 1 to 10 and reflect the overall severity of ASD-related behavioral characteristics across the social interaction, communication, and repetitive behavior domains.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [25] Gotham, 2009, Standardizing ADOS scores for a measure of severity in autism spectrum disorders, J Autism Dev Disord source

      Not checked in the free run.

  • May combine two claims.

    Study design and conductcitation

    The ADOS-based severity measure was originally designed to distinguish among children with autism spectrum disorder, but has subsequently been used to distinguish among high-risk siblings without autism spectrum disorder diagnoses.

    “The study relied on an ADOS-based measure of ASD severity that was originally designed to distinguish among children with an ASD.22,25 However, the severity measure has subsequently been used to distinguish among HR siblings without ASD diagnoses.45”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [22] Lord, 2000, The Autism Diagnostic Observation Schedule–Generic: a standard measure of social and communication deficits associated with the spectrum of autism, J Autism Dev Disord source

      Not checked in the free run.

    • cites [25] Gotham, 2009, Standardizing ADOS scores for a measure of severity in autism spectrum disorders, J Autism Dev Disord source

      Not checked in the free run.

    • cites [45] Georgiades, 2012, A prospective study of autistic-like traits in unaffected siblings of probands with autism spectrum disorder, Arch Gen Psychiatry source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductown data

    Maternal education was not included in final models because it was available for only 53% of the sample, but its inclusion did not alter study findings.

    “As maternal education was available for only 53% of the sample (n = 445), it was not included in final models. Inclusion of maternal education in models did not alter study findings (see Table S1, available online).”
    Quote in the paperNot a substantial claim
18 of 18

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • The positive predictive value findings from this high risk study are probably conservative in nature due to the likelihood that the HR-ASD group is milder than those who are clinically-referred and diagnosed with ASD at 24 months of age, and that HR-Neg groups are known to be more heterogeneous with respect to later development of cognitive, behavioral, social-communication and motor deficits than

    “The positive predictive value findings from this high risk study are probably conservative in nature due to the likelihood that our HR-ASD group is milder than those who are clinically-referred and diagnosed with ASD at 24 months of age, and that HR-Neg groups are known to be more heterogeneous with respect to later development of cognitive, behavioral, social-communication and motor deficits than”

    From Early brain development in infants at high risk for autism spectrum disorder (2017) · cites this paper as [31]

    Split into 2 atoms — 0 confirmed, 1 partially supported, 0 disputed, 0 misattributed, 1 unverifiable. Expand the claim to see per-atom verdicts.

    Experimental — accuracy not yet measured
  • Numerous studies have indicated that some, although not all, children who are later diagnosed as having ASD have delays in motor development, social responsiveness, or language development in the early part of the second year of life, if not before.

    “First, numerous studies have indicated that some, although not all, children who are later diagnosed as having ASD have delays in motor development, social responsiveness, or language development in the early part of the second year of life, if not before (Landa & Garrett-Mayer 2006, Messinger et al. 2013, Mitchell et al. 2006, Zwaigenbaum et al. 2005).”

    From Recent Advances in Autism Research as Reflected in DSM-5 Criteria for Autism Spectrum Disorder (2015) · cites this paper as Messinger et al. 2013

    The Method passage says the study assessed outcomes at age 3 and excluded children with ASD from its analyses. The Conclusions discuss variation in when ASD characteristics emerge, but neither those passages nor the passages searched for early signs in children later diagnosed with ASD establish motor, social-responsiveness, or language delays in the early second year of life. (LLM, conf=0.72)

    Experimental — accuracy not yet measured
  • Infants at familial risk for autism are also at increased risk for altered developmental functioning in non-autism-specific domains.

    “and infants at familial risk for autism are also at increased risk for altered developmental functioning in non-autism-specific domains.”
  • Among high-risk siblings who do not develop ASD, many demonstrate lower levels of verbal, nonverbal, and motor functioning than their low-risk counterparts.

    “Among high-risk siblings who do not develop ASD, many demonstrate lower levels of verbal, nonverbal, and motor functioning than their low-risk counterparts [4–6].”
  • Infants carrying a familial high-risk ASD endophenotype can present with altered development in multiple domains.

    “This is consistent with studies suggesting that infants carrying a familial high-risk ASD endophenotype can present with altered development in multiple domains [4, 6].”

Statistics that don’t add up

Each reported p-value recomputed from its test statistic and degrees of freedom.

  • Inconsistent p-value: z = 2.79, p = .001

    Minor

    The paper reports p = .001 here; recomputing from the test statistic and its degrees of freedom gives p = .005 (two-tailed). The check cannot tell whether the statistic or the p-value is the misprint.

    Reported p = .001 · recomputed p = .005

1 of 1

Statistics recomputed

Every reported test result, recomputed from its test statistic and degrees of freedom. A result shows as not adding up only once an independent check has confirmed it.

  • Doesn’t add upz = 2.79reported p = .001 · recomputed p = .005

    “z = 2.79, p = .001”

  • Consistentz = 4.02reported p < .001 · recomputed p < .001

    “z = 4.02, p < .001”

  • Consistentz = 4.82reported p < .001 · recomputed p < .001

    “z = 4.82, p < .001”

  • Consistentz = 1.29reported p = .2 · recomputed p = .197

    “z = 1.29, p = .20”

  • Consistentz = -1.67reported p = .1 · recomputed p = .095

    “z = −1.67, p = .10”

  • Consistentz = 1.92reported p = .05 · recomputed p = .055

    “z = 1.92, p = .05”

6 of 6

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Six-class comparison does not meet the stated evidence cutoff

    Minor

    The paper says it used a value above 10 to distinguish class solutions, but calls 5.79 very strong evidence for the five-class solution over the six-class solution. That comparison falls below its stated cutoff.

  • Reported sex difference reverses direction

    Minor

    The results say that sex differences within the high-risk group favored males on both developmental quotients, whereas the discussion says males had lower verbal and nonverbal functioning than females.

2 of 2

Cites

Flagged first, then by citations

Cited by

Sorted by citations

Authors
Messinger, D., Young, G. S., Ozonoff, S., Dobkins, K., Carter, A., Zwaigenbaum, L., Landa, R. J., Charman, T., Stone, W. L., Constantino, J. N., Hutman, T., Carver, L. J., Bryson, S., Iverson, J. M., Strauss, M. S., Rogers, S. J., & Sigman, M.
Affiliations
  • University of Miami (US)
  • University of Massachusetts Boston (US)
  • University of Alberta (CA)
  • Kennedy Krieger Institute (US)
  • University of Washington (US)
  • Washington University in St. Louis (US)
  • University of California, Los Angeles (US)
  • Dalhousie University (CA)
  • University of Pittsburgh (US)
Venue
Journal of the American Academy of Child & Adolescent Psychiatry
Publisher
Elsevier BV
Published
2013-03
Volume
52
Issue
3
Pages
300-308.e1
ISSN
0890-8567 (ISSN-L)
OpenAlex
W2101103407
Cited by
290
References
48
Citations by year
2026: 14 · 2025: 13 · 2024: 14 · 2023: 15 · 2022: 26 · 2021: 13 · 2020: 22 · 2019: 31 · 2018: 32 · 2017: 29 · 2016: 24 · 2015: 33 · 2014: 16 · 2013: 6
FWCI
14.39
Type
journal article
Language
en
Open access
Locations
  • Journal of the American Academy of Child & Adolescent Psychiatry (publishedVersion)
  • PubMed (publishedVersion)
  • Europe PMC (PubMed Central) (acceptedVersion)
  • PubMed Central (acceptedVersion)
  • Research Portal (King's College London) (submittedVersion)
Funders
  • National Institute of Mental Health (U54MH066417, R01MH068398, MH059630, MH068398, R01MH059630, U54MH068172)
  • National Institute on Deafness and Other Communication Disorders (DC10290, R01DC010290)
  • National Institute of General Medical Sciences (R01GM105004)
  • Eunice Kennedy Shriver National Institute of Child Health and Human Development (HD047417, R01HD057284, HD 042541, R01HD047417, HD057284, R01HD042541, HD54979, R01HD043292, P50HD055784, R01HD052804, R01HD054979, HD043292, HD052804)
Topics
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Autism Spectrum Disorder Research
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Genetics › Genetics and Neurodevelopmental Disorders
  • Health Sciences › Medicine › Psychiatry and Mental health › Child Nutrition and Feeding Issues
Keywords
autism spectrum disorder, high-risk siblings, first-degree relatives, ASD severity, developmental functioning, latent classes, child development monitoring, early intervention, toddlers
MeSH terms
Child Development Disorders, Pervasive, Child, Preschool, Female, Humans, Infant, Intelligence Tests, Male, Phenotype, Psychiatric Status Rating Scales, Risk, Severity of Illness Index, Siblings

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-05.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-05.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review