Cited by 244Substance Abuse Treatment and OutcomesHealth Policy Implementation ScienceOpioid Use Disorder Treatment

Screening, Brief Intervention and Referral to Treatment: implications of SAMHSA's SBIRT initiative for substance abuse policy and practice

Open access
Babor, Thomas F., Del Boca, Frances, Bray, Jeremy W.·2017·Addiction

Abstract

Aims: This paper describes the major findings and public health implications of a cross‐site evaluation of a national Screening, Brief Intervention and Referral to Treatment (SBIRT) demonstration program funded by the US Substance Abuse and Mental Health Services Administration (SAMHSA). Methods: Eleven multi‐site programs in two cohorts of SAMHSA grant recipients were each funded for 5 years to promote the adoption and sustained implementation of SBIRT. The SBIRT cross‐site evaluation used a multi‐method evaluation design to provide comprehensive information on the processes, outcomes and costs of SBIRT as implemented in a variety of medical and community settings. Findings: SBIRT programs in the two evaluated SAMHSA cohorts screened more than 1 million patients/clients. SBIRT implementation was facilitated by committed leadership and the use of substance use specialists, rather than medical generalists, to deliver services. Although the quasi‐experimental nature of the outcome evaluation does not permit causal inferences, pre–post differences were clinically meaningful and statistically significant for almost every measure of substance use. Greater intervention intensity was associated with larger decreases in substance use. Both brief intervention and brief treatment were associated with positive outcomes, but brief intervention was more cost‐effective for most substances. Sixty‐nine (67%) of the original performance sites adapted and redesigned SBIRT service delivery after initial grant funding ended. Four factors influenced SBIRT sustainability: presence of program champions, availability of funding, systemic change and effective management of SBIRT provider challenges. Conclusions: The US Substance Abuse and Mental Health Services Administration's Screening, Brief Intervention and Referral to Treatment (SBIRT) demonstration program was adapted successfully to the needs of early identification efforts for hazardous use of alcohol and illicit drugs. SBIRT is an innovative way to integrate the management of substance use disorders into primary care and general medicine. Screening, Brief Intervention and Referral to Treatment implementation was associated with improvements in treatment system equity, efficiency and economy.

Integrity signals

Registry

Crossref · 10.1111/add.13675

Cites

39 of 39 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

0 of 63 cited sources checked · 0 supported · 0 flagged

  • Findingcitation

    Across the two evaluated SAMHSA cohorts, SBIRT programs screened more than 1 million patients (cohort 1 = 528,036; cohort 3 = 489,396).

    “SBIRT programs in the two evaluated SAMHSA cohorts screened more than 1 million patients (cohort 1 = 528 036; cohort 3 = 489.396).”
    • Also stated in: § Abstract · plain
      “SBIRT programs in the two evaluated SAMHSA cohorts screened more than 1 million patients/clients.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [21] source

      Not checked in the free run.

  • Findingcitation

    Almost one-third of participants (30.0% in cohort 1; 32.1% in cohort 3) used both alcohol and illicit drugs.

    “almost onethird (30.0%; 32.1%) were dual substance users (alcohol and illicit drugs).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [21] source

      Not checked in the free run.

  • Findingcitation

    Acceptance of the contracted specialist approach was greater when SBIRT staff were well integrated into their host settings.

    “Acceptance of the CS approach was enhanced when SBIRT staff members were well integrated into their host settings.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [22] source

      Not checked in the free run.

  • Findingcitation

    Among those who screened positive, alcohol consumption was the most commonly reported substance (74.4% in cohort 1; 80.0% in cohort 3).

    “The most common type of substance use reported among those who screened positive was alcohol consumption (74.4%; 80.0%).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [21] source

      Not checked in the free run.

  • Findingown data

    Greater intervention intensity was associated with larger decreases in substance use.

    “Greater intervention intensity was associated with larger decreases in substance use.”
    • Also stated in: § PATIENT OUTCOMES · plain
      “Greater intervention intensity was associated with larger decreases in substance use.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [27] source

      Not checked in the free run.

  • May combine two claims. 37
  • Classified as probably not a substantial claim. 9
  • May combine two claims.

    Findingcitation

    In cohort 1, 22.4% of those screened screened positive, while in cohort 3, 11.1% screened positive.

    “More than one in five (22.4 %) screened positive in cohort 1 and slightly more than one in 10 (11.1%) did so in cohort 3.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [21] source

      Not checked in the free run.

  • May combine two claims.

    Findingcitation

    About one in four participants in both cohorts were black (26.2% and 23.3%), and more than a quarter of cohort 1 (28.2%) described themselves as Hispanic.

    “Approximately one in four of those in both cohorts were black (26.2%; 23.3%), and more than a quarter in cohort 1 (28.2%)described themselves as Hispanic.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [21] source

      Not checked in the free run.

  • May combine two claims.

    Findingnone given

    SBIRT implementation was facilitated by committed leadership and by using substance use specialists rather than medical generalists to deliver services.

    “SBIRT implementation was facilitated by committed leadership and the use of substance use specialists, rather than medical generalists, to deliver services.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Findingcitation

    SBIRT clients in the two cohorts tended to be middle-aged (mean ages 43.6 and 44.1 years) and more often female (56.2% and 55.9%) than male.

    “Those served tended to be middle-aged (mean age of 43.6 and 44.1 years, respectively), and more likely to be female (56.2%, 55.9%) than male.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [21] source

      Not checked in the free run.

  • May combine two claims.

    Findingcitation

    SBIRT participants tended to be white (47.7% in cohort 1; 65.6% in cohort 3), but disproportionate numbers of minority patients were also included.

    “SBIRT participants tended to be white (47.7%; 65.6%); however, disproportionate numbers of minority patients were also included.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [21] source

      Not checked in the free run.

  • May combine two claims.

    Findingcitation

    Illicit drug use was reported less often than alcohol but was still highly prevalent (41.8% in cohort 1; 45.8% in cohort 3).

    “Illicit drug use was reported less commonly, but was still highly prevalent (41.8%; 45.8%);”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [21] source

      Not checked in the free run.

  • May combine two claims.

    Findingcitation

    All cohort 1 programs migrated toward contracted specialist (CS) models in order to increase screening rates and reduce the burden on medical staff.

    “all cohort 1 programs migrated toward CS models to increase screening rates and reduce the burden on medical staff.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [22] source

      Not checked in the free run.

  • May combine two claims.

    Findingcitation

    Mean time to deliver a pre-screen was 1 minute 19 seconds, a full screen 4 minutes 28 seconds, and a brief intervention 6 minutes 51 seconds.

    “The mean time to deliver a pre-screen was 1:19 (minutes : seconds); a full screen, 4:28; and a BI, 6:51.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [26] source

      Not checked in the free run.

  • May combine two claims.

    Findingcitation

    Brief treatment had a greater impact than brief intervention on reducing the frequency of illicit drug use, although differences were not significant for measures of alcohol use.

    “Although the differences were not significant for measures of alcohol use, BT had a greater impact in reducing the frequency of illicit drug use.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [30] source

      Not checked in the free run.

  • May combine two claims.

    Findingown data

    All six FRAMES elements were present in 40% of observed brief interventions, and five elements in an additional 32%.

    “All six FRAMES elements were utilized in 40% of observed BIs, and an additional 32% contained five.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Findingown data

    Pre-post differences were clinically meaningful and statistically significant for almost every measure of substance use.

    “pre-post differences were clinically meaningful and statistically significant for almost every measure of substance use.”
    • Also stated in: § PATIENT OUTCOMES · plain
      “Although smaller than those reported previously, prepost differences were clinically meaningful and statistically significant for almost every measure of substance use.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [27] source

      Not checked in the free run.

  • May combine two claims.

    Findingcitation

    Providers spent 42% of their time on support activities and only 13% actually performing clinical services.

    “Overall, providers spent 42% of their time in support activities (e.g.reviewing the patient's chart, locating the patient, writing case-notes) and only 13% actually performing clinical services, a finding that has implications for the financial sustainability of SBIRT programs.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [26] source

      Not checked in the free run.

  • May combine two claims.

    Findingown data

    Model-adjusted means indicated that heavy drinking declined by 72% and illicit drug use by 80%.

    “Model-adjusted means indicated that heavy drinking declined by 72% and illicit drug use by 80%.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [27] source

      Not checked in the free run.

  • May combine two claims.

    Findingcitation

    For five other outcomes, brief treatment was less effective and more costly, and brief intervention would be a better use of resources.

    “For five other outcomes BT was less effective and more costly, and BI would be a better use of resources.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [31]

      Not checked in the free run.

  • May combine two claims.

    Findingcitation

    Both brief intervention and brief treatment were associated with beneficial outcomes.

    “Both BI and BT were associated with beneficial outcomes, but the cost of SBIRT was significantly higher for patients for whom BT was recommended.”
    • Also stated in: § Abstract · plain
      “Both brief intervention and brief treatment were associated with positive outcomes, but brief intervention was more cost-effective for most substances.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [31]

      Not checked in the free run.

  • May combine two claims.

    Findingcitation

    Programs that sustained services made adaptations to SBIRT components, modified SBIRT protocols, drew on multiple funding sources, and worked to maximize facilitators and overcome barriers in order to develop new SBIRT models that fit their needs and could be supported.

    “To sustain services, successful programs made a range of adaptations to SBIRT components, modified SBIRT protocols, accessed multiple funding sources and worked to maximize facilitators and overcome barriers to develop new SBIRT models that met their needs and could be supported.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [32] source

      Not checked in the free run.

  • May combine two claims.

    Findingcitation

    An additional 19 sites began operating after external support was no longer available, raising the total number of operational sites after funding to 88.

    “An additional 19 sites began operation after external support was no longer available, bringing the total number of operational sites post-funding to 88.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [32] source

      Not checked in the free run.

  • May combine two claims.

    Findingcitation

    Simulation modelling indicated that SBIRT could be sustained through health insurance in outpatient and emergency department/trauma centre settings under most staffing mixes, but that hospital in-patient settings may require patient flow above the national average to be sustained.

    “SBIRT could be sustained through health insurance in OP and ED/TC settings in most staffing mixes, but a patient flow larger than the national average may be needed to sustain SBIRT in hospital in-patient settings.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [23] source

      Not checked in the free run.

  • May combine two claims.

    Findingreasoning

    The SAMHSA SBIRT programs were well integrated with local specialty treatment systems and within their host medical settings, resulting in smooth transitions along the continuum of care.

    “SAMHSA SBIRT programs were well integrated with local specialty treatment systems, as well as within their host medical settings, resulting in smooth transitions along the continuum of care.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingnone given

    SAMHSA's SBIRT program is one of the largest dissemination efforts of its kind, with 29 states (four receiving two rounds of support), two tribal councils and one US territory funded to implement demonstration programs.

    “SAMHSA's SBIRT program is one of the largest dissemination efforts of its kind, with 29 states (four of which received two rounds of support), two tribal councils and one US territory now having been funded to implement demonstration programs.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Findingown data

    The cross-site evaluation results indicate that the SAMHSA SBIRT programs were evidence-based and well integrated within medical and specialty treatment systems.

    “The results of the cross-site evaluation indicate that the SAMHSA SBIRT programs were evidence-based and well integrated within medical and specialty treatment systems”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Findingown data

    The outcome findings showed significant reductions in substance use, but with several caveats.

    “The outcome findings showed significant reductions in substance use, but with several caveats that raise questions about the optimal ways to proceed with SBIRT as a public health program.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Interpretationown data

    The descriptive data from very large numbers of participants indicate that SBIRT can reach a broad range of patients and identify substance users whose consumption is likely to pose health risks and significant health-care costs if unaddressed.

    “These descriptive data, collected from very large numbers of participants, indicate that SBIRT is capable of reaching a broad range of patients, and identifying substance users whose consumption of alcohol and other drugs is likely to present health risks to the individual and significant health-care costs to the system of medical services, if substance abuse were not addressed.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Interpretationreasoning

    SBIRT providers can cope well with the fast-paced conditions of emergency departments/trauma centers and show adherence comparable to that seen in other settings.

    “These results suggest that SBIRT providers can contend well with the fast-paced conditions in ED/Ts and exhibit levels of adherence comparable to those observed in other settings.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Interpretationreasoning

    SBIRT implementation was associated with improvements in treatment system equity, efficiency and economy.

    “Screening, Brief Intervention and Referral to Treatment implementation was associated with improvements in treatment system equity, efficiency and economy.”
    • Also stated in: § CONCLUSIONS AND POLICY IMPLICATIONS · plain
      “Further, SBIRT implementation was associated with improvements in treatment system equity, efficiency and economy.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [21] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationnone given

    SBIRT is an innovative way to integrate the management of substance use disorders into primary care and general medicine.

    “SBIRT is an innovative way to integrate the management of substance use disorders into primary care and general medicine.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    A complementary concept was advanced that treatment for alcohol problems needed to be broadened to include screening for risk factors and brief interventions for hazardous and harmful drinkers.

    “a complementary, if not rival, concept was advanced, i.e. that there was a need to 'broaden the base' of treatment for alcohol problems to include screening for risk factors and brief interventions for hazardous and harmful drinkers [4].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [4]

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Brief treatment was designed to provide a more intensive level of care than brief intervention while offering a more flexible, accessible and convenient service than traditional out-patient therapy.

    “In the SBIRT service continuum, BT is a distinct option that differs from BI; it was designed to provide a more intensive level of care than BI, while offering a more flexible, accessible and convenient service than traditional out-patient therapy [20].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [20] source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Without a control group, screening and assessment alone may produce changes, and regression to the mean is also possible.

    “Without a control group, screening and assessment alone may produce changes; there is also the possibility of regression to the mean [29].”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [29] source

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Study design and conductcitation

    A propensity score framework commonly used in comparative effectiveness research was adopted to evaluate the impact of recommendations to brief treatment compared with brief intervention in cohort 1.

    “Aldridge et al. [30] adopted a propensity score framework used commonly in comparative effectiveness research to evaluate the impact of recommendations to BT compared with BI in cohort 1.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [30] source

      Not checked in the free run.

  • May combine two claims.

    Study design and conductcitation

    The SCORe, a protocol for assessing adherence to evidence-based SBIRT service delivery, has supporting feasibility and criterion validity.

    “In addition to supporting the feasibility and criterion validity of the SCORe, Vendetti et al. [24] found that SAMHSA cohort 3 providers tended to adhere to evidence-based protocols”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [24] source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    The earlier study of the cohort 1 programs was largely positive but did not use appropriate statistical methods to assess patient improvements.

    “The results of the previous study were largely positive, but it did not employ appropriate statistical methods to assess patient improvements.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [28] source

      Not checked in the free run.

  • May combine two claims.

    Interpretationcitation

    Settings with higher patient flow are more likely to be financially viable than low-flow settings, which is important information for planning future SBIRT services.

    “The fact that settings with higher patient flow are more likely than those with low flow to be financially viable is an important piece of information for the planning of future SBIRT services”
    Quote in the paperSubstantial claim
    • Rests on an assumption: High-volume venues are places where most patients are likely to receive their health care in the future.
  • May combine two claims.

    Premisecitation

    Such programs change the attitudes, knowledge and confidence of health workers and can reach large numbers of at-risk drinkers.

    “Research suggests that such programs not only change the attitudes, knowledge and confidence of health workers, but they are also capable of reaching large numbers of at-risk drinkers.”
    Quote in the paperNot a substantial claim
  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductcitation

    Multi-level modeling statistics were applied to matched patient samples to account for clustering of observations and to explore possible bias from sample selection and attrition.

    “Multi-level modeling statistics were applied by Aldridge et al. to matched patient samples to account for the clustering of observations and to explore possible bias from sample selection and attrition.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [27] source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Treatment services for substance use problems are typically fragmented, with both personal and societal costs and consequences.

    “Treatment services for substance use problems are typically fragmented, with both personal and societal costs and consequences [34].”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [34] source

      Not checked in the free run.

  • May combine two claims.

    Interpretationcitation

    SAMHSA's SBIRT program was part of a broader social movement that has matured to the point where national implementation programs are being advanced in countries such as Brazil, the United Kingdom and the United States.

    “SAMHSA's SBIRT program was part of a broader social movement that has now matured to the point where national implementation programs are being advanced in countries such as Brazil, the United Kingdom and the United States [11].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [21] source

      Not checked in the free run.

    • cites [11] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing). May combine two claims.

    Interpretationcitation

    Although the findings on Medicaid code activation apply primarily to the United States, the institutional framework is relevant to other countries, especially those with centralized health care systems.

    “Although the findings apply primarily to the United States, the institutional framework is relevant to other countries, especially those with centralized health care systems.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [33] source

      Not checked in the free run.

  • May combine two claims.

    Interpretationreasoning

    Efficiency was enhanced by situating SBIRT services in high-volume health-care settings such as emergency departments and by establishing partnerships with community agencies.

    “Efficiency was enhanced further by situating SBIRT services in high-volume health-care settings, such as emergency departments, and by establishing partnerships with community agencies.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Interpretationreasoning

    SBIRT enhances system equity by extending services to the uninsured and underinsured, to ethnic minorities and to those living in rural areas.

    “SBIRT enhances system equity by extending services to the uninsured and underinsured, to ethnic minorities and to those living in rural areas.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationown data

    The findings demonstrate the value of mixed methods that provide both qualitative and quantitative data as well as individual and system level data.

    “It also demonstrates the value of mixed methods that provide both qualitative and quantitative data, as well as individual and system level data.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationown data

    The findings show the value of studying implementation, costs and outcomes across multiple settings and implementation models.

    “Finally, it shows the value of studying implementation, costs and outcomes across multiple settings and implementation models.”
    Quote in the paperNot a substantial claim
42 of 42
  • The paper relies on this without support.

    The SAMHSA SBIRT demonstration programs and their cross-site evaluation are representative enough of SBIRT implementation generally that their findings can support conclusions about SBIRT policy and practice.

    “use the findings of a unique evaluation of the US Substance Abuse and Mental Health Services Administration (SAMHSA) SBIRT program to illustrate one of the most successful examples of translational research in the history of addiction science.”
  • The paper relies on this without support.

    There was a real need for a coordinated effort to promote the widespread adoption of SBIRT in the United States.

    “Responding to the need for a coordinated effort to promote the widespread adoption of SBIRT in the United States, SAMHSA's Center for Substance Abuse Treatment (CSAT) initiated the SBIRT program in 2003”
  • The paper relies on this without support.

    High-volume venues are places where most patients are likely to receive their health care in the future.

    “high-volume venues are not only more cost-effective, but are also places where most patients are likely to receive their health care in the future.”
3 of 3

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Cites

Flagged first, then by citations

2 of the 41 references couldn’t be matched to a known paper.

Research works printed but not linked: 1

  1. Miller W. R., 1994, Alcohol Use and Misuse by Young Adults. Notre Dame, 55

Cited by

Sorted by citations

Authors
Babor, T. F., Del Boca, F., & Bray, J. W.
Affiliations
  • University of Connecticut (US)
  • University of Connecticut School of Medicine Farmington CT USA
  • University of North Carolina at Greensboro (US)
Corresponding author
Babor, Thomas F.
Venue
Addiction
Publisher
Wiley
Published
2017-01-10
Volume
112
Issue
S2
Pages
110-117
ISSN
  • 0965-2140 (ISSN-L)
  • 1360-0443
OpenAlex
W2574257426
Cited by
244
References
41
Citations by year
2026: 17 · 2025: 20 · 2024: 29 · 2023: 30 · 2022: 29 · 2021: 27 · 2020: 34 · 2019: 31 · 2018: 12 · 2017: 14 · 2016: 1
FWCI
19.48
Type
journal article
Language
en
Open access
Locations
  • Addiction (publishedVersion)
  • PubMed (publishedVersion)
  • NC Digital Online Collection of Knowledge and Scholarship (The University of North Carolina at Greensboro) (submittedVersion)
Funders
National Institute on Alcohol Abuse and Alcoholism (P60-AA003510-39)
Topics
  • Health Sciences › Medicine › Epidemiology › Substance Abuse Treatment and Outcomes
  • Health Sciences › Health Professions › General Health Professions › Health Policy Implementation Science
  • Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Opioid Use Disorder Treatment
Keywords
SBIRT, substance use screening, brief intervention, short-term treatment, substance abuse policy, program implementation, program sustainability, cost-effectiveness
MeSH terms
Health Policy, Humans, Mass Screening, Referral and Consultation, United States, United States Substance Abuse and Mental Health Services Administration, Substance-Related Disorders, Motivational Interviewing

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-07.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-07.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review