Cited by 261Digital Mental Health InterventionsMental Health Research TopicsEmotion and Mood Recognition

Predicting Symptoms of Depression and Anxiety Using Smartphone and Wearable Data

Open access
Moshe, Isaac, Terhorst, Yannik, Opoku Asare, Kennedy, Sander, Lasse Bosse, Ferreira, Denzil, Baumeister, Harald, Mohr, David C., Pulkki-Råback, Laura·2021·Frontiers in Psychiatry

Abstract

Background: Depression and anxiety are leading causes of disability worldwide but often remain undetected and untreated. Smartphone and wearable devices may offer a unique source of data to detect moment by moment changes in risk factors associated with mental disorders that overcome many of the limitations of traditional screening methods. Objective: The current study aimed to explore the extent to which data from smartphone and wearable devices could predict symptoms of depression and anxiety. Methods: A total of N = 60 adults (ages 24–68) who owned an Apple iPhone and Oura Ring were recruited online over a 2-week period. At the beginning of the study, participants installed the Delphi data acquisition app on their smartphone. The app continuously monitored participants' location (using GPS) and smartphone usage behavior (total usage time and frequency of use). The Oura Ring provided measures related to activity (step count and metabolic equivalent for task), sleep (total sleep time, sleep onset latency, wake after sleep onset and time in bed) and heart rate variability (HRV). In addition, participants were prompted to report their daily mood (valence and arousal). Participants completed self-reported assessments of depression, anxiety and stress (DASS-21) at baseline, midpoint and the end of the study. Results: Multilevel models demonstrated a significant negative association between the variability of locations visited and symptoms of depression (beta = −0.21, p = 0.037) and significant positive associations between total sleep time and depression (beta = 0.24, p = 0.023), time in bed and depression (beta = 0.26, p = 0.020), wake after sleep onset and anxiety (beta = 0.23, p = 0.035) and HRV and anxiety (beta = 0.26, p = 0.035). A combined model of smartphone and wearable features and self-reported mood provided the strongest prediction of depression. Conclusion: The current findings demonstrate that wearable devices may provide valuable sources of data in predicting symptoms of depression and anxiety, most notably data related to common measures of sleep.

AI snapshot

Written from the paper’s full text.

TL;DR

In a 30-day observational study of 55 adults using an iPhone and Oura Ring during COVID-19 restrictions, smartphone GPS and wearable sleep/HRV data predicted depression and anxiety symptoms in multilevel models. A combined model of passive sensing plus daily self-reported mood best predicted depression, while EMA-only models were strongest for anxiety and stress.

Key findings

  • Location variance was negatively associated with depression (beta = −0.21, p = 0.037); no other GPS features were significantly associated with depression, anxiety, or stress.
  • Wearable sleep/HRV measures predicted symptoms: total sleep time (beta = 0.24, p = 0.023) and time in bed (beta = 0.26, p = 0.020) were positively associated with depression; wake after sleep onset (beta = 0.23, p = 0.035) and HRV (beta = 0.26, p = 0.035) were positively associated with anxiety.
  • EMA valence was related to depression (beta = −0.39, p = 0.001), anxiety (beta = −0.30, p = 0.014), and stress (beta = −0.39, p < 0.001); the combined EMA plus digital phenotyping model had the strongest prediction of depression, while EMA-only models were strongest for anxiety and stress.
  • Baseline DASS-21 scores were depression M = 3.78, SD = 3.48; anxiety M = 2.73, SD = 2.68; stress M = 6.00, SD = 3.82, with 67.3% normal depression, 70.9% normal anxiety, and 65.5% normal stress.

Methods

  • Longitudinal observational study with repeated measurements over 30 days; N = 60 recruited online (55 in final analysis), ages 24–68, mean age 42.8 (SD 11.6), required to own an iPhone and Oura Ring.
  • Data sources: Delphi smartphone app continuously monitored GPS and smartphone usage; Oura Ring measured activity (steps, MET), sleep (TST, SOL, WASO, TIB), and HRV; EMA mood was prompted 3 times daily (valence and arousal); DASS-21 was completed at baseline, day 16, and day 31.
  • Analysis: Spearman correlations with cluster-wise Bonferroni-Holm adjustment; multilevel models with random intercepts and slopes, z-standardized predictors, multiple imputation (20 datasets, 15 iterations), and likelihood ratio tests for combined models.

Limitations

  • Stated: observational design cannot establish causality or direction; pooling data prevented examining high-frequency processes and temporal precedence; small sample likely underpowered for small effects; sample was skewed toward white, employed, educated, iOS-using, non-clinical participants; no pre-pandemic movement data were available to confirm restricted movement.
  • Stated: missingness occurred in DASS-21 (10%) and sensing variables (9.1%), assumed missing at random and handled with multiple imputation.
  • Evident: the results text states no significant relationship between arousal and depression, anxiety, or stress, but tables report arousal in the combined depression model and the EMA stress model, indicating inconsistent reporting.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.3389/fpsyt.2021.625247

Cites

85 of 85 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

0 of 30 cited sources checked · 0 supported · 0 flagged

  • Findingown data

    No significant association was found between the other GPS-derived features (total distance, location entropy, normalized location entropy and time at home) and symptoms of depression, anxiety or stress.

    “No significant association between the other GPS-derived features (total distance, location entropy, normalized location entropy and time at home) and symptoms of depression, anxiety or stress were found.”
    Quote in the paperNot a substantial claim
  • Findingown data

    Depression could be predicted by EMA and smartphone and wearable data.

    “Depression could be predicted by EMA and smartphone and wearable data.”
    Quote in the paperSubstantial claim
  • Findingown data

    For anxiety and stress, EMA-only data models were the strongest predictors.

    “For anxiety and stress, EMA-only data models were the strongest predictors (see Tables 4, 5).”
    Quote in the paperSubstantial claim
  • Findingown data

    A multi-level model predicting daily distance traveled (Finland vs. other) revealed no significant difference between the average daily distance traveled between participants from Finland and those from other countries (β = −922.9 [in meters], p = 0.943).

    “A multi-level model predicting the daily distance traveled (using categorized country Finland vs. other as predictor) revealed no significant distance between the average daily distance traveled between participants from Finland and those from other countries (β = −922.9 [in meters], p = 0.943).”
    Quote in the paperNot a substantial claim
  • Findingown data

    Time in bed had a significant positive association with depression (beta = 0.26, SE = 0.11, t(59) = 2.39, p = 0.020).

    “time in bed and depression (beta = 0.26, p = 0.020)”
    • Also stated in: § Predicting Symptom Severity From Smartphone and Wearable Data · plain
      “time in bed and depression [beta = 0.26, SE = 0.11, t(59) = 2.39, p = 0.020]”
    Quote in the paperSubstantial claim
  • May combine two claims. 11
  • Classified as probably not a substantial claim. 18
  • May combine two claims.

    Findingown data

    EMA performed better than smartphone and wearable data models for depression, but the combination yielded the best fit.

    “A combined model of smartphone and wearable features and self-reported mood provided the strongest prediction of depression.”
    • Also stated in: § Combined Predictions · plain
      “EMA performed better than smartphone and wearable data models, but the combination yielded the best fit (see Table 3).”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Findingown data

    All other GPS features as well as variance and entropy for anxiety and stress were non-significant.

    “All other GPS features as well as variance and entropy for anxiety and stress were non-significant (see Table 2 for more details).”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductcitation

    The present study used methods from a previous study investigating the correlation between GPS variables and depressive symptom severity with a sample size of N = 40.

    “The present study used methods from a previous study (19) investigating the correlation between GPS variables and depressive symptom severity with a sample size of N = 40.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [19] Saeb, 2015, Mobile phone sensor correlates of depressive symptom severity in daily-life behavior: An exploratory study, J Med Internet Res. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductcitation

    The DASS-21 has demonstrated high internal consistency for the three subscales of depression, anxiety, and stress in previous administrations.

    “The DASS21 has demonstrated high internal consistency for the three subscales of depression, anxiety, and stress in previous administrations (42).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [42] Antony, 1998, Psychometric properties of the 42-item and 21-item versions of the depression anxiety stress scales in clinical groups and a community sample, Psychol Assess. source

      Not checked in the free run.

  • May combine two claims.

    Study design and conductcitation

    Multilevel models take into account that data is nested within persons and reduce the likelihood of Type I errors.

    “MLMs take into account that data is nested within persons, i.e., the observations are not independent (56) and reduce the likelihood of Type I errors (57).”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [56] Nezlek, 2001, multilevel random coefficient analyses of event- and interval-contingent data in social and personality psychology research, Personal Soc Psychol Bull. source

      Not checked in the free run.

    • cites [57] Musca, 2011, Data with hierarchical structure: impact of intraclass correlation and sample size on type-i error, Front Psychol. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Beiwinkel et al. found that total distance traveled had a significant negative relationship with clinical manic symptoms in patients diagnosed with bipolar disorder (beta = −0.37).

    “Beiwinkel et al. (20) found that the total distance traveled had a significant negative relationship with clinical manic symptoms in patients diagnosed with bipolar disorder (beta = −0.37).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [20] Beiwinkel, 2016, Using smartphones to monitor bipolar disorder symptoms: a pilot study, JMIR Ment Heal. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductown data

    Of the 60 participants at intake, 5 dropped out (1 due to privacy concerns, 2 due to burden of self-report, and 2 for unknown reasons).

    “Of the 60 participants at intake, 1 participant (1.7%) dropped out of the study due to concerns over privacy, 2 participants (3.4%) dropped out due to burden of self-report and 2 (3.4%) participants dropped out for unknown reasons.”
    Quote in the paperNot a substantial claim
  • Classified as probably not a substantial claim.

    Study design and conductreasoning

    Adjustment for multiple testing was only performed cluster-wise to avoid overcorrection leading to false rejection of findings.

    “However, p-value adjustment methods are sensitive and since the present study is of an exploratory nature, adjustment was only performed cluster-wise (e.g., separately for GPS features and wearable features) to avoid an overcorrection by p-value adjustment leading to a false-rejection of findings.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductreasoning

    Recruitment was continued for 2 weeks after reaching the required sample size to compensate for dropout.

    “However, as dropout rates in longitudinal observation studies using digital phenotyping data are typically high, recruitment was continued for 2 weeks after reaching the required sample size to compensate for dropout.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    A secure web-based dashboard was developed to monitor data collection and compliance during the study period.

    “To monitor data collection and compliance during the study period, a secure web-based dashboard was developed displaying the data gathered for all participants, updated at 15-min intervals.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductcitation

    P-values were adjusted for multiple testing based on the Bonferroni Holm method with a false discovery rate of 0.05.

    “P-values were adjusted for multiple testing based on the Bonferroni Holm method (50) with a false discovery rate of 0.05.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [50] Holm, 1979, Board of the foundation of the scandinavian journal of statistics a simple sequentially rejective multiple test procedure a simple sequentially rejective multiple test procedure, Source Scand J Stat Scand J Stat.

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Study design and conductreasoning

    Multiple imputations were used to handle missing values in the correlation and regression analysis.

    “To avoid bias introduced by missingness we used multiple imputations to handle missing values in the correlation and regression analysis.”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: Missingness in the DASS-21 assessment and sensing variables was missing at random (MAR), meaning missingness depended on observed data.
  • May combine two claims.

    Study design and conductcitation

    The imputation model took the nested structure of the data into account and followed guidelines for multilevel multiple imputations.

    “The imputation model took the nested structure of the data into account and followed guidelines for multilevel multiple imputations (58).”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [58] van Buuren, 2011, mice: Multivariate imputation by chained equations in R, J Stat Softw. source

      Not checked in the free run.

    • The authors state this as an assumption: Missingness in the DASS-21 assessment and sensing variables was missing at random (MAR), meaning missingness depended on observed data.
  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductown data

    Of the remaining 55 participants, 47 (85.5%) completed the midpoint questionnaire and 54 (98.2%) completed the endpoint questionnaire.

    “Of the remaining 55 participants, 47 (85.5%) completed the midpoint questionnaire and 54 (98.2%) completed the endpoint questionnaire.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductcitation

    Convergence was visually assessed and confirmed.

    “Convergence was visually assessed and confirmed (58).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [58] van Buuren, 2011, mice: Multivariate imputation by chained equations in R, J Stat Softw. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Saeb et al. found that regularity of participants' 24-h movement patterns (r = −0.63), the variance of locations visited (r = −0.58) and the proportion of time spent at home (r = 0.49) were related to depressive symptom severity in a non-clinical population.

    “Saeb et al. (19), e.g., found that regularity of participants' 24-h movement patterns (r = −0.63), the variance of locations visited (r = −0.58) and the proportion of time spent at home (r = 0.49) were related to depressive symptom severity in a non-clinical population (19).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [19] Saeb, 2015, Mobile phone sensor correlates of depressive symptom severity in daily-life behavior: An exploratory study, J Med Internet Res. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductcitation

    To avoid biases introduced by missing values, full information maximum likelihood was used as the estimator.

    “To avoid biases introduced by missing values we used full information maximum likelihood as the estimator (51, 52).”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [51] EndersCK Applied Missing Data Analysis2010

      Not checked in the free run.

    • cites [52] RevelleW 19012653Procedures for Psychological, Psychometric, and Personality Research2020

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Study design and conductnone given

    Ethical review and approval was not required for the study on human participants in accordance with the local legislation and institutional requirements.

    “Ethical review and approval was not required for the study on human participants in accordance with the local legislation and institutional requirements.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Study design and conductcitation

    Pooling data prevented exploration of high-frequency processes, so temporal precedence could not be established.

    “as pooling data prevented us from exploring high-frequency processes (e.g., the relationship between movement and mood), we were unable to establish temporal precedence (74).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [74] Kazdin, 2007, Mediators and mechanisms of change in psychotherapy research, Annu Rev Clin Psychol. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductreasoning

    As the study commenced after the onset of the pandemic, no data comparing participants' movement before and during the study were available to confirm that their movement was more limited during the study period.

    “As the study commenced after the onset of the pandemic, we were unable to provide any data comparing participants' movement before and during the study to confirm that their movement was indeed more limited during the study period.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    There are issues quantifying explained variance in multilevel models.

    “Given the issues quantifying explained variance in multilevel models (86, 87)”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [86] Rights, 2019, Quantifying explained variance in multilevel models: an integrative framework for defining R-squared measures, Psychol Methods. source

      Not checked in the free run.

    • cites [87] LaHuis, 2014, Explained variance measures for multilevel models, Organ Res Methods. source

      Not checked in the free run.

  • May combine two claims.

    Interpretationreasoning

    The current findings do not necessarily represent a causal relationship between the behavior measured by smartphone and wearable devices and symptoms of depression and anxiety, nor can they explain the direction between them.

    “thus the current findings do not necessarily represent a causal relationship between the behavior measured by smartphone and wearable devices and symptoms of depression and anxiety, nor can they explain the direction between them.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Interpretationcitation

    Sleep data from consumer wearables may provide valuable tools to identify early warning signs of mental disorders, thereby facilitating time-sensitive preventative measures.

    “sleep data from consumer wearables may thus provide valuable tools to identify early warning signs of mental disorders, thereby facilitating time-sensitive preventative measures (72).”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [72] Baglioni, 2011, Insomnia as a predictor of depression: A meta-analytic evaluation of longitudinal epidemiological studies, J Affect Disord. source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Sociodemographic status and smartphone usage behavior may differ between iOS and Android users.

    “Research has shown that sociodemographic status and smartphone usage behavior may differ between iOS and Android users (75, 76).”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [75] Ubhi, 2017, A comparison of the characteristics of iOS and Android users of a smoking cessation app, Transl Behav Med. source

      Not checked in the free run.

    • cites [76] Pryss, 2019, Ecological momentary assessment based differences between Android and iOS Users of the trackyourhearing mhealth crowdsensing platform source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    The study was open only to participants with an Apple (iOS) smartphone.

    “Related to this, the study was open only to participants with an Apple (iOS) smartphone.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    The participants provided their written informed consent to participate in this study.

    “The patients/participants provided their written informed consent to participate in this study.”
    Quote in the paperNot a substantial claim
26 of 26
  • The authors state this as an assumption.

    Missingness in the DASS-21 assessment and sensing variables was missing at random (MAR), meaning missingness depended on observed data.

    “Missingness only occurred in the DASS-21 assessment (10%) and the sensing variables (9.1%) and was assumed to be missing at random (MAR), meaning missingness depended on observed data (51, 52).”
  • The paper relies on this without support.

    Participants' movement was indeed more limited during the study period than before the pandemic.

    “we were unable to provide any data comparing participants' movement before and during the study to confirm that their movement was indeed more limited during the study period.”
2 of 2

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Statistics recomputed

Every reported test result, recomputed from its test statistic and degrees of freedom. A result shows as not adding up only once an independent check has confirmed it.

  • Consistentt(81) = -2.13reported p = .037 · recomputed p = .036

    “t(81) = −2.13, p = .037”

  • Consistentt(73) = 2.33reported p = .023 · recomputed p = .023

    “t(73) = 2.33, p = .023”

  • Consistentt(59) = 2.39reported p = .02 · recomputed p = .020

    “t(59) = 2.39, p = .020”

  • Consistentt(90) = 2.13reported p = .035 · recomputed p = .036

    “t(90) = 2.13, p = .035”

  • Consistentt(71) = 2.15reported p = .035 · recomputed p = .035

    “t(71) = 2.15, p = .035”

  • Consistentt(55) = -3.43reported p = .001 · recomputed p = .001

    “t(55) = −3.43, p = .001”

  • Consistentt(57) = -2.54reported p = .014 · recomputed p = .014

    “t(57) = −2.54, p = .014”

  • Consistentt(74) = -3.64reported p < .001 · recomputed p < .001

    “t(74) = −3.64, p < .001”

8 of 8

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Combined model is incorrectly claimed to be strongest for anxiety

    Major

    The discussion says adding sensing data to EMA had the highest predictive power for both depression and anxiety. The results instead say the EMA-only model was strongest for anxiety; Table 4 also reports a non-significant comparison of the combined model against the EMA model (p = 0.055).

  • Stress severe-category percentage does not match its count

    Minor

    Table 1 lists 2 participants with severe stress and 1 with extremely severe stress. Together, 3 of the 55 participants are in these categories, or about 5.5%, not the reported 4.4%.

    Reported 4.4 · recomputed 5.454545

  • Two-person dropout percentages use the wrong denominator

    Minor

    Each group of 2 dropouts represents 3.3% of the stated 60 participants at intake when rounded to one decimal place, not 3.4%.

    Reported 3.4 · recomputed 3.333333

  • Stress EMA model labels its significant predictor arousal

    Minor

    Table 5 reports a significant arousal coefficient for stress, whereas the results state that valence was significant for stress and arousal was not significant for any outcome.

  • Correlation method is named inconsistently

    Minor

    The methods say the correlations used Spearman's coefficient, but the results describe Table 2 as Pearson's correlation matrices.

5 of 5

Cites

Flagged first, then by citations

Research works printed but not linked: 2

  1. Holm, 1979, Board of the foundation of the scandinavian journal of statistics a simple sequentially rejective multiple test procedure a simple sequentially rejective multiple test procedure, Source Scand J Stat Scand J Stat., 6, 65
  2. Cummins, 2019, Mobile sensing and digital phenotyping in psychoinformatics

Cited by

Sorted by citations

Authors
Moshe, I., Terhorst, Y., Opoku Asare, K., Sander, L. B., Ferreira, D., Baumeister, H., Mohr, D. C., & Pulkki-Råback, L.
Affiliations
  • University of Helsinki (FI)
  • Universität Ulm (DE)
  • University of Freiburg (DE)
  • University of Oulu (FI)
  • Northwestern University (US)
Corresponding author
Moshe, Isaac
Venue
Frontiers in Psychiatry
Published
2021-01-28
Volume
12
Pages
625247-625247
ISSN
1664-0640 (ISSN-L)
OpenAlex
W3126710441
Cited by
261
References
83
Citations by year
2026: 45 · 2025: 50 · 2024: 50 · 2023: 50 · 2022: 47 · 2021: 19
FWCI
36.64
Type
journal article
Language
en
Open access
Locations
  • Frontiers in Psychiatry (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • Työväentutkimus Vuosikirja (submittedVersion)
  • OPen Access Repositorium der Universität Ulm (OPARU) (Ulm University) (submittedVersion, other-oa)
  • Universität Ulm, Kommunikations- und Informationszentrum (kiz) (cc-by)
Funders
Ministerium für Wissenschaft, Forschung und Kunst Baden-Württemberg
Topics
  • Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
  • Social Sciences › Psychology › Experimental and Cognitive Psychology › Mental Health Research Topics
  • Social Sciences › Psychology › Experimental and Cognitive Psychology › Emotion and Mood Recognition
Keywords
depression, anxiety, smartphone data, wearable devices, Oura Ring, heart rate variability, sleep, GPS location data, smartphone usage behavior, mood, mental health screening, stress, DASS-21

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-10.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-10.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review