Cited by 16Dysphagia Assessment and ManagementMedical Image Segmentation TechniquesVoice and Speech Disorders

Automated pharyngeal phase detection and bolus localization in videofluoroscopic swallowing study: killing two birds with one stone?

Open access
Bandini, Andrea, Smaoui, Sana, Steele, Catriona M.·2022·Computer Methods and Programs in Biomedicine

Abstract

Background and objective The videofluoroscopic swallowing study (VFSS) is a gold-standard imaging technique for assessing swallowing, but analysis and rating of VFSS recordings is time consuming and requires specialized training and expertise. Researchers have recently demonstrated that it is possible to automatically detect the pharyngeal phase of swallowing and to localize the bolus in VFSS recordings via computer vision approaches, fostering the development of novel techniques for automatic VFSS analysis. However, training of algorithms to perform these tasks requires large amounts of annotated data that are seldom available. In this paper, we demonstrate that the challenges of pharyngeal phase detection and bolus localization can be solved together using a single approach.Methods We propose a deep-learning framework that jointly tackles pharyngeal phase detection and bolus localization in a weakly-supervised manner, requiring only the initial and final frames of the pharyngeal phase as ground truth annotations for the training. Our approach stems from the observation that bolus presence in the pharynx is the most prominent visual feature upon which to infer whether individual VFSS frames belong to the pharyngeal phase. We conducted extensive experiments with multiple convolutional neural networks (CNNs) on a dataset of 1245 bolus-level clips from 59 healthy subjects.Results We demonstrated that the pharyngeal phase can be detected with an F1-score higher than 0.9. Moreover, by processing the class activation maps of the CNNs, we were able to localize the bolus with promising results, obtaining correlations with ground truth trajectories higher than 0.9, without any manual annotations of bolus location used for training purposes.Conclusions Once validated on a larger sample of participants with swallowing disorders, our framework will pave the way for the development of intelligent tools for VFSS analysis to support clinicians in swallowing assessment.

Integrity signals

Registry

Crossref · 10.1016/j.cmpb.2022.107058

Cites

50 of 50 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

0 of 19 cited sources checked · 0 supported · 0 flagged

  • Findingown data

    Pharyngeal phase detection reached an F1-score above 0.9.

    “We demonstrated that the pharyngeal phase can be detected with an F1-score higher than 0.9.”
    Quote in the paperSubstantial claim
  • Findingown data

    For the UESC frame, all models except CNN3 had P3UESC values above the inter-rater agreement.

    “In terms of comparison with the manual rating, all P3BPM values were below the inter-rater agreement, whereas for the UESC frame, all models but CNN3 produced P3UESC values higher than the inter-rater agreement.”
    Quote in the paperNot a substantial claim
  • Findingown data

    Bolus localization from class activation maps produced correlations with ground truth trajectories above 0.9.

    “Moreover, by processing the class activation maps of the CNNs, we were able to localize the bolus with promising results, obtaining correlations with ground truth trajectories higher than 0.9, without any manual annotations of bolus location used for training purposes.”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: The darkest pixels in grayscale VFSS frames belong to the bolus because of the radiocontrast agent.
  • Findingown data

    Every P3BPM value was lower than the inter-rater agreement.

    “In terms of comparison with the manual rating, all P3BPM values were below the inter-rater agreement, whereas for the UESC frame, all models but CNN3 produced P3UESC values higher than the inter-rater agreement.”
    Quote in the paperNot a substantial claim
  • Findingown data

    ResNet50 and VGG16 gave the two highest P3BPM and P3UESC values.

    “ResNet50 and VGG16 were also the models that yielded the top two P3BPM and P3UESC values.”
    Quote in the paperNot a substantial claim
  • May combine two claims. 26
  • Classified as probably not a substantial claim. 12
  • May combine two claims.

    Findingown data

    On the test set, ResNet50 gave the highest pharyngeal phase detection F1-score, 0.909, with VGG16 next at 0.906.

    “The best results in terms of F1-score were obtained with ResNet50 (F1 = 0.909) closely followed by VGG16 (F1 = 0.906).”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Findingown data

    The automated method easily exceeded trained rater inter-rater agreement when detecting UESC frames, but not for BPM frame detection.

    “Trained rater inter-rater agreement values could be easily exceeded by the automated method when detecting the UESC frames, but not for BPM frame detection.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Findingown data

    ResNet50 and VGG16 gave the best BPM frame detection results, but their P3BPM values remained below the trained rater inter-rater agreement.

    “ResNet50 and VGG16 produced the best results in terms of BPM frame detection, but with P3BPM values always below the trained rater inter-rater agreement.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Findingown data

    ResNet50 did not show the increase in ry with thickness; it performed consistently across the five consistencies, with ry values always above 0.9.

    “However, this increase is not visible in ResNet50, which produced consistent performance across the five consistencies, with ry values always higher than 0.9.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Findingown data

    For UESC frame detection, P3UESC values were always between 90% and 100%, and in most cases exceeded the inter-rater agreement.

    “For the UESC frame detection, it was always possible to obtain P3UESC values between 90 and 100%, which in most cases were higher than the inter-rater agreement.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Findingown data

    BPM detection performance improved as bolus thickness decreased, with the best results for slightly thick and thin boluses.

    “It is also possible to notice an increase in the BPM detection performance with the decrease of bolus thickness, with the best results obtained with slightly thick and thin boluses.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Findingown data

    CNN3 produced the lowest RMSE values, which were significantly lower than those of the other four models.

    “The lowest RMSE values were produced by CNN3, which were significantly lower than the other four models.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Findingown data

    Although a fair comparison was impossible because of different datasets, the authors confirmed Lee et al.'s results by showing that VGG16 recognizes the pharyngeal phase with high performance.

    “Although a fair comparison was not possible due to the different datasets, we confirmed their results by demonstrating that VGG16 is able to recognize the pharyngeal phase with high performance.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Findingown data

    The weakly-supervised bolus localization results are promising, with correlations with ground truth trajectories above 0.9.

    “The results obtained with our weakly-supervised approach for localizing the bolus are promising, with correlations with the ground truth trajectories higher than 0.9.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: Manual annotation of bolus location provides accurate ground truth for bolus localization.
  • May combine two claims.

    Findingown data

    InceptionV3 was the best model for thin boluses and ResNet50 for slightly thick boluses in UESC frame detection.

    “whereas InceptionV3 and ResNet50 were the best models in thin and slightly thick boluses, respectively.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Findingown data

    VGG16 was one of the top-two models for pharyngeal phase detection but was the worst for localizing the bolus.

    “The most striking example is VGG16, which was one of the top-two models for pharyngeal phase detection but was the worst for localizing the bolus”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Findingnone given

    UESC frames can be detected with an error below 3 frames in more than 90% of cases, without a clear trend across consistencies.

    “Conversely, UESC frames can be detected with error below 3 frames in more than 90% of the cases, without a clear trend across consistencies.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Findingown data

    Fig. 6 shows a cut-off at mildly thick boluses, after which performance increased for all models except ResNet50, which performed robustly across all consistencies.

    “From Fig. 6, we can detect a cut-off in correspondence to mildly thick boluses, after which the performance increased for all models except ResNet50, which produced robust performance across all consistencies.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Interpretationreasoning

    The authors' deep-learning framework performs pharyngeal phase detection and bolus localization together under weak supervision, using only the first and last frames of the pharyngeal phase as training annotations.

    “We propose a deep-learning framework that jointly tackles pharyngeal phase detection and bolus localization in a weakly-supervised manner, requiring only the initial and final frames of the pharyngeal phase as ground truth annotations for the training.”
    • Also stated in: § Introduction · plain
      “we propose a deep-learning framework that tackles pharyngeal phase detection and bolus localization simultaneously in a weakly-supervised manner, requiring only the initial and final frames of the pharyngeal phase as ground truth annotations for training.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: Bolus presence in the pharynx is the most prominent visual feature for deciding whether a VFSS frame is in the pharyngeal phase.
  • Classified as probably not a substantial claim.

    Premisecitation

    For dysphagia, VFSS is a current gold-standard clinical assessment.

    “The videofluoroscopic swallowing study (VFSS) is a current gold standard clinical assessment for dysphagia [6,7].”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [6] Costa, 2010, Videofluoroscopy: the gold standard exam for studying swallowing and its dysfunction

      Not checked in the free run.

    • cites [7] Sejdic, 2019, Computational deglutition: using signal- and image-processing methods to understand swallowing and associated disorders [Life Sciences]

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    The pharyngeal phase contains most clinical parameters of interest, and evaluating it thoroughly is essential for detecting penetration-aspiration.

    “In particular, the pharyngeal phase is the time window of the swallow event that contains most of the clinical parameters of interest, and a thorough evaluation of it is essential for detecting the presence of penetration-aspiration [10].”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [10] Steele, 2019, Reference values for healthy swallowing across the range from thin to extremely thick liquids, J. Speech Lang. Hear. Res. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationnone given

    To the authors' knowledge, this is the first time bolus localization in VFSS has been tackled in a weakly-supervised manner.

    “To the best of our knowledge, this is the first time that bolus localization in VFSS has been tackled in a weakly-supervised manner.”
    Quote in the paperNot a substantial claim
  • Classified as probably not a substantial claim.

    Study design and conductown data

    The bolus localization test set was restricted to the subset of bolus-level clips that had manual annotation of bolus location, comprising 613 frames from 25 video clips.

    “The test-set considered for these analyses was limited to the subset of bolus-level clips with available manual annotation of bolus location (i.e., 613 frames from 25 video clips).”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Interpretationown data

    Despite excellent detection performance with F1-scores often above 0.9, the end of the pharyngeal phase (UESC frame) was detected more accurately than its beginning (BPM frame).

    “Despite excellent detection performance, with F1-scores often higher than 0.9, we demonstrated that the end of the pharyngeal phase (i.e., UESC frame) could be detected more accurately than its beginning (i.e., BPM frame).”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Interpretationnone given

    Reliable detection of temporal events and spatial information would help clinicians by speeding VFSS rating, removing inter-subject variability, and supporting identification of aspiration-penetration events.

    “The reliable detection of temporal events (i.e., the pharyngeal phase) and spatial information (i.e., bolus localization) would constitute an important aid to clinicians during VFSS rating, as this would expedite the rating of VFSS recordings, remove inter subject variability, and - especially in case of bolus localization - support the identification of aspiration-penetration events.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning

    The temporal and spatial information extracted by the method can serve as input for further VFSS processing, such as automatically detecting other timing parameters between BPM and UESC.

    “The temporal and spatial information extracted with this method can be used as input to further VFSS processing, for example, to automatically detect other timing parameters between BPM and UESC”
    Quote in the paperNot a substantial claim
  • Classified as probably not a substantial claim.

    Premisecitation

    Grad-CAM can highlight the image regions a CNN uses to make its predictions.

    “Since its publication in 2017, Grad-CAM has had a huge impact in computer vision applications, as it can highlight the regions of the image where the CNN is looking to perform its predictions [20,40].”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [20] Selvaraju, 2017, Grad-CAM: visual explanations from deep networks via gradient-based localization

      Not checked in the free run.

    • cites [40] Selvaraju, 2020, Grad-CAM: visual explanations from deep networks via gradient-based localization, Int. J. Comput. Vis. source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    VFSS rating is lengthy and needs frame-by-frame inspection of the recorded video.

    “However, VFSS rating remains a lengthy process, as it requires frame-by-frame inspection of the recorded video [10].”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [10] Steele, 2019, Reference values for healthy swallowing across the range from thin to extremely thick liquids, J. Speech Lang. Hear. Res. source

      Not checked in the free run.

  • May combine two claims.

    Interpretationown data

    The proposed approach, specifically four out of five CNN models, is as accurate as a trained annotator in detecting the end of the pharyngeal phase, but further improvements are needed to automatically identify the beginning of this phase.

    “Thus, the proposed approach - specifically, 4 out of 5 CNN models - is as accurate as a trained annotator in detecting the end of the pharyngeal phase, but further improvements are needed to automatically identify the beginning of this phase.”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: Trained clinical raters’ annotations are the gold standard for pharyngeal phase detection.
  • May combine two claims.

    Interpretationown data

    The authors demonstrated for the first time that pharyngeal phase detection and bolus localization are interdependent tasks that can be addressed within the same framework.

    “For the first time, we have demonstrated that pharyngeal phase detection and bolus localization are two interdependent tasks that can be tackled within the same framework.”
    Quote in the paperSubstantial claim
  • Classified as probably not a substantial claim.

    Premisecitation

    Prior work by Lee et al. showed that the pharyngeal phase can be detected with accuracy above 90%.

    “Recent results obtained by Lee et al. [9,15] demonstrated that this phase can be detected with accuracy higher than 90%.”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [9] Lee, 2019, Automatic detection of the pharyngeal phase in raw videos for the videofluoroscopic swallowing study using efficient data collection and 3d convolutional networks, Sensors source

      Not checked in the free run.

    • cites [15] Lee, 2021, Automatic pharyngeal phase recognition in untrimmed videofluoroscopic swallowing study using transfer learning with deep convolutional neural networks, Diagnostics source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Lee et al. showed that 2DCNNs can match 3DCNNs in pharyngeal phase detection while reducing computational complexity.

    “More recently, Lee et al. [15] demonstrated that comparable performance in detecting the pharyngeal phase could be obtained with 2DCNNs [39], with a significant reduction of computational complexity compared to 3DCNNs.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [15] Lee, 2021, Automatic pharyngeal phase recognition in untrimmed videofluoroscopic swallowing study using transfer learning with deep convolutional neural networks, Diagnostics source

      Not checked in the free run.

  • May combine two claims.

    Interpretationreasoning

    The bias in bolus localization could be due to the bolus changing shape and becoming elongated during the pharyngeal phase, so the algorithm detected only part of the bolus, which would explain the lower F1-score at higher IoU thresholds.

    “This could be due to the changing shape of the bolus that becomes elongated during the pharyngeal phase; thus, the algorithm was able to detect only a portion of the entire bolus, and this would justify the reduction of F1-score at higher IoU thresholds”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductnone given

    The study focused only on the first swallow event of the bolus-level clips and did not consider residue or piecemeal swallows.

    “Another limitation was that we focused only on the first swallow event of the bolus-level clips, without considering the presence of residue or piecemeal swallows.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationnone given

    To the authors' knowledge, this is the first time pharyngeal phase detection and bolus localization performance have been calculated for different consistencies.

    “To the best of our knowledge, this is the first time that pharyngeal phase detection and bolus localization performance have been calculated for different consistencies.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Study design and conductnone given

    The study included only healthy participants and single swallow clips.

    “In this work we included only healthy participants and single swallow clips.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Interpretationreasoning

    This behavior can be explained by thinner boluses being faster and less defined in shape than thicker ones.

    “This behavior can be explained by the fact that thinner boluses are faster and less defined in shape than thicker ones.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Detecting residue is an important task in VFSS rating because residues in the oropharynx are associated with a higher risk of aspiration-penetration.

    “In particular, the detection of residue is an important task in VFSS rating, as residues in the oropharynx are associated with a higher risk of aspiration-penetration [48].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [48] Steele, 2020, The risk of penetration–aspiration related to residue in the pharynx, Am. J. Speech Lang. Pathol. source

      Not checked in the free run.

  • May combine two claims.

    Interpretationreasoning

    For the first time, the authors demonstrated that these two tasks can be solved jointly with the same model and without any manual annotations of bolus location.

    “For the first time, we have demonstrated that these two tasks can be solved jointly with the same model and without any manual annotations of bolus location.”
    Quote in the paperSubstantial claim
34 of 34
  • The paper relies on this without support.

    Bolus presence in the pharynx is the most prominent visual feature for deciding whether a VFSS frame is in the pharyngeal phase.

    “Our approach stems from the observation that bolus presence in the pharynx is the most prominent visual characteristic upon which to infer whether a VFSS frame belongs to the pharyngeal phase or not.”
    “as this is the most prominent visual feature for detecting this phase.”
  • The paper relies on this without support.

    Bolus localization by image segmentation needs large pixel-level training datasets that are rarely available.

    “In particular, the latter task requires a large amount of pixel-level training data, that are seldom available due to the lack of public datasets and the huge amount of time required to conduct the annotations.”
  • The authors state this as an assumption.

    The darkest pixels in grayscale VFSS frames belong to the bolus because of the radiocontrast agent.

    “(3) the binary maps were used as masks to seek the k darkest pixels within the grayscale VFSS frames (with k = 100) that, due to the presence of the radiocontrast agent, supposedly belonged to the bolus”
  • The authors state this as an assumption.

    Trained clinical raters’ annotations are the gold standard for pharyngeal phase detection.

    “performance close to the gold standard (i.e., trained clinical raters)”
  • The paper relies on this without support.

    Manual annotation of bolus location provides accurate ground truth for bolus localization.

    “The test-set considered for these analyses was limited to the subset of bolus-level clips with available manual annotation of bolus location”
5 of 5

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Support vector machines (SVM) and Mask-RCNN demonstrated high accuracy in classifying swallowing events, with sensitivity and specificity reaching over 90%.

    “Other methods, including support vector machines (SVM) and Mask-RCNN, have demonstrated high accuracy in classifying swallowing events, with metrics like sensitivity and specificity reaching over 90%.”

    From Artificial intelligence in the diagnosis and management of dysphagia: a scoping review (2025) · cites this paper as [69]

    The Results (Pharyngeal phase detection; page unavailable) report F1-scores above 0.90 for ResNet50 and VGG16, supporting high performance in detecting the pharyngeal phase. They do not report sensitivity or specificity above 90% for SVM or Mask-RCNN. The References identify Mask-RCNN in a separate study of bolus detection, not as a swallowing-event classifier evaluated here. (LLM, conf=0.82)

    Experimental — accuracy not yet measured
  • Deep learning models such as U-Net and CNNs achieved F1 scores exceeding 0.9 and accuracy rates of 97.8% in detecting swallowing events and anatomical structures.

    “deep learning models like U-Net and CNNs have achieved performance metrics such as F1 scores exceeding 0.9 and accuracy rates of 97.8%, indicating their robustness in detecting swallowing events and anatomical structures.”

    From Artificial intelligence in the diagnosis and management of dysphagia: a scoping review (2025) · cites this paper as [69]

    The Results (Pharyngeal phase detection; page unavailable) report F1 scores of 0.909 and 0.906 for CNN models detecting the pharyngeal phase. Our copy does not report 97.8% accuracy or results from a U-Net detecting anatomical structures; the Discussion (Limitations and future work; page unavailable) mentions U-Net only in connection with future bolus-segmentation comparisons. (LLM, conf=0.88)

    Experimental — accuracy not yet measured
  • Key studies demonstrate high performance of deep learning models, such as CNNs and Mask-RCNN, in detecting and segmenting bolus movements in VFSS with precision metrics exceeding 90% in certain frameworks.

    “Key studies demonstrate high performance of deep learning models, such as CNNs and Mask-RCNN, in detecting and segmenting bolus movements in VFSS with precision metrics exceeding 90% in certain frameworks (Appendix C).”

    From Artificial intelligence in the diagnosis and management of dysphagia: a scoping review (2025) · cites this paper as [69]

    The Results on pharyngeal phase detection report CNN F1-scores above 0.9 for some bolus consistencies, and the Results on weakly supervised bolus localization report a trajectory correlation of 0.934. Those figures are not precision above 90% for bolus segmentation. Our copy describes CNN-based localization but does not identify Mask R-CNN as an approach used in this study. (LLM, conf=0.88)

    Experimental — accuracy not yet measured
  • AI-driven tools have potential to improve diagnostic accuracy for dysphagia.

    “These findings emphasize the potential of AI-driven tools in improving diagnostic accuracy for dysphagia(51,68,69,75).”
  • AI has potential not just in diagnostics but also in automating labor-intensive aspects of analysis.

    “This highlights the potential of AI not just in diagnostics but also in automating labor-intensive aspects of analysis(65,69).”

Cites

Flagged first, then by citations

22 of the 72 references couldn’t be matched to a known paper.

Cited by

Sorted by citations

Authors
Bandini, A., Smaoui, S., & Steele, C. M.
Affiliations
University Health Network (CA)
Corresponding author
Steele, Catriona M.
Venue
Computer Methods and Programs in Biomedicine
Publisher
Elsevier BV
Published
2022-10
Volume
225
Pages
107058
ISSN
0169-2607 (ISSN-L)
OpenAlex
W3214671691
Cited by
16
References
72
Citations by year
2026: 1 · 2025: 8 · 2024: 5 · 2023: 2
FWCI
4.74
Type
journal article
Language
en
Open access
Locations
  • Computer Methods and Programs in Biomedicine (acceptedVersion)
  • PubMed (publishedVersion)
  • PubMed Central (acceptedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • IRIS UNIMORE (University of Modena and Reggio Emilia) (submittedVersion, other-oa)
  • CINECA IRIS Institutional Research Information System (Sant'Anna School of Advanced Studies) (submittedVersion)
  • arXiv (Cornell University)
Funders
  • National Institute on Deafness and Other Communication Disorders (DC011020)
  • Toronto Rehabilitation Institute
  • National Institutes of Health
Topics
  • Health Sciences › Health Professions › Speech and Hearing › Dysphagia Assessment and Management
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Medical Image Segmentation Techniques
  • Health Sciences › Medicine › Physiology › Voice and Speech Disorders
Keywords
videofluoroscopic swallowing study, dysphagia, weakly supervised learning, class activation maps, convolutional neural networks, computer vision
MeSH terms
Body Fluids, Deglutition, Deglutition Disorders, Fluoroscopy, Humans, Pharynx

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-04.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-04.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review