Compliance with ecological momentary assessment protocols in substance users: a meta-analysis
Open accessAbstract
BACKGROUND AND AIMS: While there are considerable benefits to Ecological Momentary Assessment (EMA), poor compliance with assessment protocols has been identified as a limitation, particularly in substance users. Our aim was to identify the pooled compliance rate of EMA studies in substance users and examine variables that may influence compliance with EMA protocols, such as the length and frequency of assessments. DESIGN: A meta-analysis and meta-regression of all possible studies (randomized controlled trials and longitudinal) which incorporated EMA protocols, examining substance use. SETTING: Studies took place from 1998 to 2017, in numerous countries world-wide. PARTICIPANTS: One hundred and twenty-six studies were identified, contributing a total of 19 431 participants (52.32% male, mean age = 28.86). MEASUREMENTS: Compliance data, the proportion of responses to the study protocol, were extracted from each study alongside prompt frequency, total length of assessment period, substance use population and device used to administer EMA prompts. FINDINGS: The pooled compliance rate across all studies was 75.06% [95% confidence interval (CI) = 72.37%, 77.65%]. There was no evidence that compliance rates were significantly associated with prompt frequency [Q(3) = 7.35, P = 0.061], length of assessment period [Q(2) = 2.40, P = 0.301], substance type [Q(3) = 6.30, P = 0.098] or device administration [Q(4) = 4.28, P = 0.369]. However, dependent samples (69.80%) had lower compliance rates than non-dependent samples [76.02%; Q(1) = 4.13, P = 0.042]. CONCLUSIONS: The pooled compliance rate for Ecological Momentary Assessment studies in substance-using populations from 1998 to 2017 was lower than the recommended rate of 80%, and was not associated with frequency or duration of assessments.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
More frequent assessments may not reduce compliance, or may even increase compliance, as long as questionnaires are kept brief.
“It has also previously been reported that more frequent assessments may not reduce compliance (106), or may even increase compliance (107), as long as questionnaires are kept brief (108).”
From Don't Miss the Moment: A Systematic Review of Ecological Momentary Assessment in Suicide Research (2022) · cites this paper as [106]
A meta-analysis showed a pooled average adherence rate of 71% in substance abusers.
“A meta-analysis showed a pooled average adherence rate of 71% in substance abusers [30].”
From Experienced burden of and adherence to smartphone-based ecological momentary assessment in persons with affective disorders (2020) · cites this paper as [30]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Conducting Meta-Analyses in <i>R</i> with the <b>metafor</b> Package
Viechtbauer, W. (2010) · Journal of Statistical Software · doi:10.18637/jss.v036.i03 · Cited by 20,464 · see citing works
Duval, S., & Tweedie, R. (2000) · Biometrics · doi:10.1111/j.0006-341x.2000.00455.x · Cited by 14,099 · see citing works
(2000) · Cited by 9,198 · see citing works
Ecological momentary assessment
Shiffman, S., Stone, A. A., & Hufford, M. R. (2008) · Annual Review of Clinical Psychology · doi:10.1146/annurev.clinpsy.3.022806.091415 · Cited by 6,454 · see citing works
Missing data analysis: making it work in the real world
Graham, J. W. (2009) · Annual Review of Psychology · doi:10.1146/annurev.psych.58.110405.085530 · Cited by 6,163 · see citing works
Methods to estimate the between‐study variance and its uncertainty in meta‐analysis
Veroniki, A. A., Jackson, D., Viechtbauer, W., Bender, R., Bowden, J., Knapp, G., Kuss, O., Higgins, J. P., Langan, D., & Salanti, G. (2016) · Research Synthesis Methods · doi:10.1002/jrsm.1164 · Cited by 1,685 · see citing works
How Many Studies Do You Need?: A Primer on Statistical Power for Meta-Analysis
Valentine, J. C., Pigott, T. D., & Rothstein, H. R. (2010) · Journal of Educational and Behavioral Statistics · doi:10.3102/1076998609346961 · Cited by 1,294 · see citing works
Patient non-compliance with paper diaries
Stone, A. A., Shiffman, S., Schwartz, J. E., Broderick, J. E., & Hufford, M. R. (2002) · BMJ · doi:10.1136/bmj.324.7347.1193 · Cited by 810 · see citing works
Capturing momentary, self-report data: A proposal for reporting guidelines
Stone, A. A., & Shiffman, S. (2002) · Annals of Behavioral Medicine · doi:10.1207/s15324796abm2403_09 · Cited by 807 · see citing works
Ecological momentary assessment (EMA) in studies of substance use.
Shiffman, S. S. (2009) · Psychological Assessment · doi:10.1037/a0017074 · Cited by 764 · see citing works
Hamaker, E. L., & Wichers, M. (2017) · Current Directions in Psychological Science · doi:10.1177/0963721416666518 · Cited by 505 · see citing works
Wen, C. K. F., Schneider, S., Stone, A. A., & Spruijt-Metz, D. (2017) · Journal of Medical Internet Research · doi:10.2196/jmir.6641 · Cited by 350 · see citing works
Self-citationLiao, Y., Skelton, K. R., Dunton, G. F., & Bruening, M. M. (2016) · Journal of Medical Internet Research · doi:10.2196/jmir.4954 · Cited by 311 · see citing works
Burke, L. E., Shiffman, S., Music, E., Styn, M. A., Kriska, A., Smailagic, A., Siewiorek, D., Ewing, L. J., Chasens, E., French, B., et al. (2017) · Journal of Medical Internet Research · doi:10.2196/jmir.7138 · Cited by 267 · see citing works
Reactivity to ecological momentary assessment: an example using undergraduate problem drinkers.
Hufford, M., Shields, A., Shiffman, S., Paty, J., & Balabanis, M. H. (2002) · Psychology of Addictive Behaviors · doi:10.1037/0893-164x.16.3.205 · Cited by 258 · see citing works
Litt, M. D., Cooney, N. L., & Morse, P. A. (2000) · Addiction · doi:10.1046/j.1360-0443.2000.9568896.x · Cited by 216 · see citing works
Litt, M. D., Cooney, N. L., & Morse, P. (1998) · Health Psychology · doi:10.1037/0278-6133.17.1.48 · Cited by 214 · see citing works
Statistical models for meta-analysis: A brief tutorial
Kelley, G. A. (2012) · World Journal of Methodology · doi:10.5662/wjm.v2.i4.27 · Cited by 196 · see citing works
Effect of smoking cues and cigarette availability on craving and smoking behavior
Droungas, A., Ehrman, R. N., Childress, A. R., & O’Brien, C. P. (1995) · Addictive Behaviors · doi:10.1016/0306-4603(95)00029-c · Cited by 181 · see citing works
Boniface, S., Kneale, J., & Shelton, N. (2014) · BMC Public Health · doi:10.1186/1471-2458-14-1297 · Cited by 171 · see citing works
Cell phones for ecological momentary assessment with cocaine-addicted homeless patients in treatment
Freedman, M. J., Lester, K. M., McNamara, C. L., Milby, J. B., & Schumacher, J. E. (2006) · Journal of Substance Abuse Treatment · doi:10.1016/j.jsat.2005.10.005 · Cited by 168 · see citing works
Sex differences in cocaine/heroin users: Drug-use triggers and craving in daily life
Kennedy, A., Epstein, D. H., Phillips, K. A., & Preston, K. L. (2013) · Drug and Alcohol Dependence · doi:10.1016/j.drugalcdep.2012.12.025 · Cited by 160 · see citing works
Introduction to the special section: Ecological momentary assessment in health psychology.
Shiffman, S. M., & Stone, A. A. (1998) · Health Psychology · doi:10.1037/h0092706 · Cited by 157 · see citing works
Wray, T. B. (2014) · Alcohol research · doi:10.35946/arcr.v36.1.03 · Cited by 155 · see citing works
Marhe, R., Waters, A. J., van de Wetering, B. J. M., & Franken, I. H. A. (2012) · Journal of Consulting and Clinical Psychology · doi:10.1037/a0030754 · Cited by 151 · see citing works
Self-citationReactivity to laboratory stress provocation predicts relapse to cocaine
Back, S. E., Hartwell, K., DeSantis, S. M., Saladin, M., McRae-Clark, A. L., Price, K. L., Moran-Santa Maria, M. M., Baker, N. L., Spratt, E., Kreek, M. J., et al. (2010) · Drug and Alcohol Dependence · doi:10.1016/j.drugalcdep.2009.07.016 · Cited by 148 · see citing works
Stockwell, T. R., Zhao, J., & Macdonald, S. (2014) · Addiction · doi:10.1111/add.12609 · Cited by 122 · see citing works
Tobacco withdrawal components and their relations with cessation success
Piper, M. E., Schlam, T. R., Cook, J. W., Sheffer, M. A., Smith, S. S., Loh, W., Bolt, D. M., Kim, S.-Y., Kaye, J. T., Hefner, K. R., et al. (2011) · Psychopharmacology · doi:10.1007/s00213-011-2250-3 · Cited by 117 · see citing works
Hoeppner, B. B., Stout, R. L., Jackson, K. M., & Barnett, N. P. (2010) · Addictive Behaviors · doi:10.1016/j.addbeh.2010.08.013 · Cited by 116 · see citing works
Moore, T. M., Seavey, A. E., Ritter, K., McNulty, J. K., Gordon, K. C., & Stuart, G. L. (2013) · Psychology of Addictive Behaviors · doi:10.1037/a0034127 · Cited by 108 · see citing works
Handbook of behavioral medicine : methods and applications
Steptoe, A., Freedland, K. E., Jennings, J. R., Llabre, M. M., Manuck, S. B., & Susman, E. J. (2010) · Digital Access to Libraries (Université catholique de Louvain (UCL), l'Université de Namur (UNamur) and the Université Saint-Louis (USL-B)) · doi:10.1007/978-0-387-09488-5 · Cited by 105 · see citing works
Effects of marijuana use on impulsivity and hostility in daily life
Ansell, E. B., Laws, H. B., Roche, M. J., & Sinha, R. (2015) · Drug and Alcohol Dependence · doi:10.1016/j.drugalcdep.2014.12.029 · Cited by 104 · see citing works
Monk, R. L., Heim, D., Qureshi, A., & Price, A. (2015) · PLoS ONE · doi:10.1371/journal.pone.0126209 · Cited by 103 · see citing works
Simons, J. S., Wills, T. A., & Neal, D. J. (2014) · Journal of Abnormal Psychology · doi:10.1037/a0036926 · Cited by 99 · see citing works
Wall, A.-M., Hinson, R. E., McKee, S. A., & Goldstein, A. L. (2001) · Psychology of Addictive Behaviors · doi:10.1037/0893-164x.15.3.219 · Cited by 91 · see citing works
Wall, A.-M., McKee, S. A., & Hinson, R. E. (2000) · Psychology of Addictive Behaviors · doi:10.1037/0893-164x.14.4.367 · Cited by 91 · see citing works
Serre, F., Fatseas, M., Debrabant, R., Alexandre, J., Auriacombe, M., & Swendsen, J. D. (2012) · Drug and Alcohol Dependence · doi:10.1016/j.drugalcdep.2012.04.025 · Cited by 85 · see citing works
Berkman, E. T., Dickenson, J. A., Falk, E. B., & Lieberman, M. D. (2011) · Health Psychology · doi:10.1037/a0022201 · Cited by 85 · see citing works
Jones, A., Tiplady, B., Houben, K., Nederkoorn, C., & Field, M. (2018) · Psychopharmacology · doi:10.1007/s00213-018-4860-5 · Cited by 84 · see citing works
Self-citationFactors Predicting Compliance to Ecological Momentary Assessment Among Adolescent Smokers
Sokolovsky, A. W., Mermelstein, R. J., & Hedeker, D. (2013) · Nicotine & Tobacco Research · doi:10.1093/ntr/ntt154 · Cited by 82 · see citing works
Yang, C., Linas, B., Kirk, G., Bollinger, R., Chang, L., Chander, G., Siconolfi, D., Braxton, S., Rudolph, A., & Latkin, C. (2015) · JMIR mHealth and uHealth · doi:10.2196/mhealth.4344 · Cited by 73 · see citing works
Ecological momentary assessment of adolescent smoking cessation: A feasibility study
Gwaltney, C. J., Bartolomei, R. J., Colby, S. M., & Kahler, C. W. (2008) · Nicotine & Tobacco Research · doi:10.1080/14622200802163118 · Cited by 69 · see citing works
Daily life hour by hour, with and without cocaine: an ecological momentary assessment study
Epstein, D. H., & Preston, K. L. (2010) · Psychopharmacology · doi:10.1007/s00213-010-1884-x · Cited by 67 · see citing works
Evaluating reactivity to ecological momentary assessment during smoking cessation.
Rowan, P. J., Cofta-Woerpel, L. M., Mazas, C. A., Vidrine, J. I., Reitzel, L. R., Cinciripini, P. M., & Wetter, D. W. (2007) · Experimental and Clinical Psychopharmacology · doi:10.1037/1064-1297.15.4.382 · Cited by 65 · see citing works
Comparisons of alcohol consumption by timeline follow back vs.smartphone-based daily interviews
Dulin, P. L., Alvarado, C. E., Fitterling, J. M., & Gonzalez, V. M. (2016) · Addiction Research & Theory · doi:10.1080/16066359.2016.1239081 · Cited by 65 · see citing works
Schuster, R. M., Mermelstein, R. J., & Hedeker, D. (2016) · Addiction · doi:10.1111/add.13342 · Cited by 64 · see citing works
McGrath, E., Jones, A., & Field, M. (2016) · Psychopharmacology · doi:10.1007/s00213-016-4205-1 · Cited by 63 · see citing works
Self-citationZundert, R. M. P. V., Nijhof, L. M., & Engels, R. C. M. E. (2008) · Addictive Behaviors · doi:10.1016/j.addbeh.2008.11.004 · Cited by 62 · see citing works
Carpenter, M. J., Saladin, M. E., DeSantis, S. M., Gray, K. M., LaRowe, S. D., & Upadhyaya, H. P. (2009) · Addictive Behaviors · doi:10.1016/j.addbeh.2009.03.022 · Cited by 58 · see citing works
Berkman, E. T., Giuliani, N. R., & Pruitt, A. K. (2014) · Appetite · doi:10.1016/j.appet.2014.06.010 · Cited by 50 · see citing works
Wray, T. B., Kahler, C. W., & Monti, P. M. (2016) · AIDS and Behavior · doi:10.1007/s10461-015-1272-y · Cited by 50 · see citing works
A Real‐Time Examination of Context Effects on Alcohol Cognitions
Monk, R. L., & Heim, D. (2014) · Alcoholism Clinical and Experimental Research · doi:10.1111/acer.12504 · Cited by 46 · see citing works
Feasibility and validity of computerized ambulatory monitoring in drug-dependent women
Johnson, E. I., Barrault, M., Nadeau, L., & Swendsen, J. (2009) · Drug and Alcohol Dependence · doi:10.1016/j.drugalcdep.2008.06.010 · Cited by 45 · see citing works
Messiah, A., Grondin, O., & Encrenaz, G. (2010) · Drug and Alcohol Dependence · doi:10.1016/j.drugalcdep.2010.09.016 · Cited by 42 · see citing works
Gender Differences in Responses to Cues Presented in the Natural Environment of Cigarette Smokers
Wray, J. M., Gray, K. M., McClure, E. A., Carpenter, M. J., Tiffany, S. T., & Saladin, M. E. (2015) · Nicotine & Tobacco Research · doi:10.1093/ntr/ntu248 · Cited by 42 · see citing works
Relationship between momentary affect states and self-efficacy in adolescent smokers.
Hoeppner, B. B., Kahler, C. W., & Gwaltney, C. J. (2014) · Health Psychology · doi:10.1037/hea0000075 · Cited by 42 · see citing works
Helzer, J. E., Badger, G. J., Searles, J. S., Rose, G. L., & Mongeon, J. A. (2006) · Alcoholism Clinical and Experimental Research · doi:10.1111/j.1530-0277.2006.00093.x · Cited by 41 · see citing works
Jones, A., Rose, A. K., Cole, J. C., & Field, M. (2013) · Journal of Experimental Psychopathology · doi:10.5127/jep.031912 · Cited by 40 · see citing works
Self-citationDaily-level associations between PTSD and cannabis use among young sexual minority women
Dworkin, E. R., Kaysen, D. L., Bedard‐Gilligan, M. A., Rhew, I. C., & Lee, C. M. (2017) · Addictive Behaviors · doi:10.1016/j.addbeh.2017.06.007 · Cited by 36 · see citing works
Turner, C., Coffin, P. O., Santos, D., Huffaker, S., Matheson, T., Euren, J., DeMartini, A., Rowe, C., Batki, S. L., & Santos, G. (2017) · Journal of Substance Abuse Treatment · doi:10.1016/j.jsat.2017.01.007 · Cited by 26 · see citing works
The Promise of Intensive Longitudinal Data Capture for Behavioral Health Research
Ginexi, E. M., Riley, W. T., Atienza, A. A., & Mabry, P. L. (2014) · Nicotine & Tobacco Research · doi:10.1093/ntr/ntt273 · Cited by 24 · see citing works
Otsuki, M. (2009) · Nicotine & Tobacco Research · doi:10.1093/ntr/ntp028 · Cited by 22 · see citing works
Does Alcohol Reduce Social Anxiety in Daily Life? A 22-Day Experience Sampling Study
Battista, S. R., MacKinnon, S. P., Sherry, S. B., Barrett, S. P., MacNevin, P. D., & Stewart, S. H. (2015) · Journal of Social and Clinical Psychology · doi:10.1521/jscp.2015.34.6.508 · Cited by 20 · see citing works
Jones, A., & Robinson, E. (2017) · Frontiers in Psychology · doi:10.3389/fpsyg.2017.00002 · Cited by 16 · see citing works
Self-citationSocially anxious smokers experience greater negative affect and withdrawal during self-quit attempts
Buckner, J. D., Langdon, K. J., Jeffries, E. R., & Zvolensky, M. J. (2016) · Addictive Behaviors · doi:10.1016/j.addbeh.2016.01.004 · Cited by 14 · see citing works
Huh, J., Shin, H., Leventhal, A. M., Spruijt‐Metz, D., Abramova, Z. S., Cerrada, C. J., Hedeker, D., & Dunton, G. F. (2014) · Nicotine & Tobacco Research · doi:10.1093/ntr/ntu063 · Cited by 13 · see citing works
Roberts, M. E., Lü, B., Browning, C. R., et al. (2017) · Substance Use & Misuse · doi:10.1080/10826084.2017.1302958 · Cited by 13 · see citing works
Vachon, H., Rintala, A., Viechtbauer, W., & Myin‐Germeys, I. (2018) · Systematic Reviews · doi:10.1186/s13643-018-0673-1 · Cited by 9 · see citing works
Managing an Attractive Impression by Using Alcohol: Evidence From Two Daily Diary Studies
O’Grady, M. A., Harman, J. J., Gleason, M. E. J., & Wilson, K. (2012) · Basic and Applied Social Psychology · doi:10.1080/01973533.2011.637841 · Cited by 8 · see citing works
Perfect for a Gin and Tonic: How Context Drives Consumption Within a Modified Bogus Taste Test
Monk, R. L., Qureshi, A., McNeill, A. M., Erskine-Shaw, M., & Heim, D. (2017) · Alcohol and Alcoholism · doi:10.1093/alcalc/agx084 · Cited by 7 · see citing works
Flagged first, then by citations
Research works printed but not linked: 2
- [22]Robbins M. L., Kubiak T. Ecological Momentary Assessment in Behavioral Medicine In: Mostofsky DI, editor. The Handbook of Behavioral Medicine, 1st edn. Hoboken, NJ: John Wiley and Sons, Inc.; 2014, pp. 429–446.
- [54]Vietchtbauer W. Conducting meta‐analysis in R with the metafor package. J Stat Softw 2010; 36: 1– 48.
Cited by
Sorted by citations
Wrzus, C., & Neubauer, A. B. (2023) · Assessment · doi:10.1177/10731911211067538 · Cited by 500 · see citing works
Eisele, G., Vachon, H., Lafit, G., Kuppens, P., Houben, M., Myin-Germeys, I., & Viechtbauer, W. (2022) · Assessment · doi:10.1177/1073191120957102 · Cited by 459 · see citing works
Vachon, H., Viechtbauer, W., Rintala, A., & Myin-Germeys, I. (2019) · Journal of Medical Internet Research · doi:10.2196/14475 · Cited by 252 · see citing works
Don't Miss the Moment: A Systematic Review of Ecological Momentary Assessment in Suicide Research
Kivelä, L., van der Does, W. A. J., Riese, H., & Antypa, N. (2022) · Frontiers in Digital Health · doi:10.3389/fdgth.2022.876595 · Cited by 163 · see citing works
Investigating the feasibility of idiographic network models.
Mansueto, A. C., Wiers, R. W., van Weert, J. C. M., Schouten, B. C., & Epskamp, S. (2023) · Psychological Methods · doi:10.1037/met0000466 · Cited by 147 · see citing works
van Genugten, C. R., Schuurmans, J., Lamers, F., Riese, H., Penninx, B. W. J. H., Schoevers, R. A., Riper, H. M., & Smit, J. H. (2020) · Journal of Clinical Medicine · doi:10.3390/jcm9020322 · Cited by 71 · see citing works
Ottenstein, C., & Werner, L. (2022) · Assessment · doi:10.1177/10731911211032718 · Cited by 52 · see citing works
Hart, A., Reis, D., Prestele, E., & Jacobson, N. C. (2022) · Journal of Medical Internet Research · doi:10.2196/34015 · Cited by 22 · see citing works
A Pilot Open Trial of a Digital Mindfulness-Based Intervention for Anorexia Nervosa
Sala, M., Levinson, C. A., Kober, H., & Roos, C. R. (2023) · Behavior Therapy · doi:10.1016/j.beth.2023.01.003 · Cited by 19 · see citing works
Garland, E. L., & Jinpa, T. (2024) · Behaviour Research and Therapy · doi:10.1016/j.brat.2024.104494 · Cited by 6 · see citing works
Papp, L. M., Kouros, C. D., Witt, H. K., Curtin, J. J., Blumenstock, S. M., & Barringer, A. (2022) · Experimental and Clinical Psychopharmacology · doi:10.1037/pha0000500 · Cited by 5 · see citing works
- Authors
- Jones, A., Remmerswaal, D., Verveer, I., Robinson, E., Franken, I. H. A., Wen, C. K. F., & Field, M.
- Affiliations
- University of Liverpool (GB)
- Erasmus University Rotterdam (NL)
- University of Southern California (US)
- Corresponding author
- Jones, Andrew
- Venue
- Addiction
- Publisher
- Wiley
- Published
- 2018-12-21
- Volume
- 114
- Issue
- 4
- Pages
- 609-619
- ISSN
- 0965-2140 (ISSN-L)
- 1360-0443
- PMC
- PMC6492133
- OpenAlex
- W2907787119
- Cited by
- 319
- References
- 70
- Citations by year
- 2026: 56 · 2025: 47 · 2024: 55 · 2023: 51 · 2022: 32 · 2021: 39 · 2020: 27 · 2019: 12
- FWCI
- 3.11
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Addiction (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- EUR Research Repository (Erasmus University Rotterdam) (submittedVersion, other-oa)
- RePub (Erasmus University Rotterdam) (submittedVersion, other-oa)
- Funders
- Economic and Social Research Council (ES/N016211/1)
- Topics
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Mental Health Research Topics
- Health Sciences › Medicine › Epidemiology › Substance Abuse Treatment and Outcomes
- Health Sciences › Medicine › Family Practice › Medication Adherence and Compliance
- Keywords
- ecological momentary assessment, substance use
- MeSH terms
- Ecological Momentary Assessment, Smokers, Adult, Alcohol Drinking, Female, Humans, Male, Research Design, Substance-Related Disorders, Guideline Adherence, Drug Users, Mobile Applications
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
