Medication Adherence Reminder System for Virtual Home Assistants: Mixed Methods Evaluation Study
Open accessThe DOI matches its Crossref record; 34 of the 34 references it cites were found in a registry or an index; 10 of the 18 sources its claims cite were checked for support.
Abstract
Background Medication nonadherence is a global public health challenge that results in suboptimal health outcomes and increases health care costs. Forgetting to take medicines is one of the most common reasons for unintentional medication nonadherence. Research findings indicate that voice-activated virtual home assistants, such as Amazon Echo and Google Home devices, may be useful in promoting medication adherence. Objective This study aims to create a medication adherence app (skill), MedBuddy, for Amazon Echo devices and measure the use, usability, and usefulness of this medication-taking reminder skill. Methods A single-group, mixed methods, cohort feasibility study was conducted with women who took oral contraceptives (N=25). Participants were undergraduate students (age: mean 21.8 years, SD 6.2) at an urban university in the Southeast United States. Participants were given an Amazon Echo Dot with MedBuddy—a new medication reminder skill for Echo devices created by our team—attached to their study account, which they used for 60 days. Participants self-reported their baseline and poststudy medication adherence. MedBuddy use was objectively evaluated by tracking participants’ interactions with MedBuddy through Amazon Alexa. The usability and usefulness of MedBuddy were evaluated through a poststudy interview in which participants responded to both quantitative and qualitative questions. Results Participants’ interactions with MedBuddy, as tracked through Amazon Alexa, only occurred on half of the study days (mean 50.97, SD 29.5). At study end, participants reported missing their medication less in the past 1 and 6 months compared with baseline (χ21=0.9 and χ21=0.4, respectively; McNemar test: P<.001 for both). However, there was no significant difference in participants’ reported adherence to consistently taking medication within the same 2-hour time frame every day in the past 1 or 6 months at the end of the study compared with baseline (χ21=3.5 and χ21=0.4, respectively; McNemar test: P=.63 and P=.07, respectively). Overall feedback about usability was positive, and participants provided constructive feedback about the skill’s features that could be improved. Participants’ evaluation of MedBuddy’s usefulness was overwhelmingly positive—most (15/23, 65%) said that they would continue using MedBuddy as a medication reminder if provided with the opportunity and that they would recommend it to others. MedBuddy features that participants enjoyed were an external prompt separate from their phone, the ability to hear the reminder prompt from a separate room, multiple reminders, and verbal responses to prompts. Conclusions The findings of this feasibility study indicate that the MedBuddy medication reminder skill may be useful in promoting medication adherence. However, the skill could benefit from further usability enhancements.
Overview
CoChat Summary
## TL;DR
This single-group mixed-methods feasibility study evaluated MedBuddy, a medication reminder skill for Amazon Echo devices, in 25 women taking oral contraceptives over 60 days. Participants generally found it useful and would recommend it, but usability problems—especially speech recognition and logging issues—limited the experience.
## Key findings
- Participants’ interactions with MedBuddy occurred on only about half of study days: mean response was 50.97 days (SD 29.5), with response rates ranging from 6.6% to 86.7%.
- Self-reported adherence improved on some measures: participants reported missing their medication less in the past 1 and 6 months at study end than at baseline, and the paper reports McNemar test results of χ²1=0.9 and χ²1=0.4 with P<.001 for both.
- There was no significant change in taking medication within the same 2-hour time frame each day: χ²1=3.5 and χ²1=0.4, with P=.63 and P=.07, respectively.
- Usability/usefulness feedback was largely positive: 15/23 (65%) said they would continue using MedBuddy, and 21/23 (91%) said they would recommend it to someone with medication adherence difficulties.
## Methods
- Single-group mixed methods cohort feasibility study of MedBuddy, a medication reminder skill for Amazon Echo devices.
- Sample: 25 women taking oral contraceptives; all were undergraduate students at an urban university in the Southeast United States, with mean age 21.8 years (SD 6.2). Twenty-three (92%) completed the full study.
- Participants used an Echo Dot with MedBuddy for 60 days; the study used baseline and poststudy self-reported adherence measures, Alexa-tracked MedBuddy interactions, and structured poststudy telephone interviews with quantitative and qualitative questions. Analyses included descriptive statistics, a paired two-tailed t test for change in use over time, McNemar chi-square tests for adherence, and qualitative categorization of interview responses.
## Limitations
- The paper states several limitations: small convenience sample, all women, mostly young adult White non-Hispanic college students living off-campus; only 60 days of follow-up; self-reported adherence measures were author-developed and lacked established psychometric properties.
- The text also makes evident that COVID-19 disrupted use because some participants were separated from their Echo devices when the university moved to web-based classes, which may have affected usage and may not reflect normal circumstances.
- The paper notes that subjective adherence measures are known to overestimate adherence, so the reported improvement should be interpreted cautiously.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
6 Valid · 0 Not valid · 4 Not checked · 41 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Forgetting to take medicines is one of the most common reasons for unintentional medication nonadherence.”
- Also stated in: § Background · plain
“Among unintentional reasons for medication nonadherence, forgetting to take medicine is one of the most common reasons [8-10].”
§ Medication Adherence Reminder System for Virtual Home Assistants: Mixed Methods Evaluation Study > Abstract > Background
What the sources say
- Not checked — no full text of the sourcecites [8] source
- Partly supportedcites [9] sourceExperimental — accuracy not yet measured
The Abstract and Results (p. 1–2) state that in 840 Danish HIV-cohort patients on HAART the three most frequently reported reasons for missed doses were 'simply forgot', 'were away from home' and 'had a change in daily routines', and the Discussion (p. 3) confirms 'forgetting' was the most commonly reported reason for missing doses — so the 'most common reason for forgetting medication' part is directly supported. However, the source's evidence is confined to antiretroviral (HAART) therapy in HIV patients; it does not address medication nonadherence in general populations, and it never uses the intentional/unintentional distinction that the citing sentence carries ('unintentional medication nonadherence'), noting instead (Discussion, p. 2) that 'forgetting doses' may be explained by an interplay of lifestyle, lack of commitment, inconvenience and side effects. The generalization beyond HAART and the 'unintentional' framing are not attributable to this source. (LLM, conf=0.72)
- Partly supportedcites [10] sourceExperimental — accuracy not yet measured
The Findings (Emergent Themes; page not available) rank general forgetting third among barriers to medication adherence, with 18 occurrences. The Findings (Theme Development, Validation, and Description; page not available) also report that nine participants described forgetting to take their medicine. The study concerns high-blood-pressure medication in a specific group; it does not establish how common forgetting is among reasons for unintentional medication nonadherence generally. (LLM, conf=0.85)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § Background · plain
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“(2) nonadherence to oral contraceptives is common [28,29]”
§ Medication Adherence Reminder System for Virtual Home Assistants: Mixed Methods Evaluation Study > Methods > Recruitment and Enrollment Our team envisioned MedBuddy to be particularly useful for adults who spend most of their time at home. However, we conducted the initial usability and usefulness tests of MedBuddy with women who were college students and took oral contraceptives. We selected this population because (1) younger adults are likely to have more technological literacy than middle-aged or older adults and therefore may provide higher quality feedback about the skill with minimal frustration if it does not work as expected, (2) nonadherence to oral contraceptives is common [28,29], and (3) oral contraceptives are most effective when taken at approximately the same time every day [30]. Following the receipt of an exempt determination by the institutional review board, potential participants were recruited from an urban public university campus in the Southeastern United States. Recruitment flyers were distributed through electronic billboards, an electronic learning management system, and social media. In addition, an announcement about the research opportunity was presented to approximately 100 students who attended a university program orientation. Snowball sampling also occurred by enrolling referrals from current participants or our team’s social contacts. Potential participants were directed to contact the study coordinator by phone or email. When contacted, the study coordinator explained the study purpose, potential risks, and benefits and screened the participants for eligibility. Eligibility criteria included (1) currently taking an oral contraceptive but self-reporting difficulty remembering to take the medication some days or at about the same time each day, (2) living off-campus (because of internet inconsistency or internet overload in campus housing), and (3) owning a smartphone. Students meeting the eligibility requirements were invited to participate, and, for those interested, an enrollment appointment was scheduled. In-person enrollment took place between February 1, 2020, and March 6, 2020, at a research center on a university campus. Volunteer participants provided verbal informed consent and then completed a baseline demographic and medication information questionnaire and a 4-item questionnaire about their adherence to their prescribed oral contraceptive medicine (Multimedia Appendix 1). Each participant received an Amazon Echo Dot device and a study account for the MedBuddy skill. Participants were then instructed on setting up and using the Amazon Echo Dot and subsequently the MedBuddy skill. Participants were first familiarized with activating the MedBuddy skill, using phrases such as “Alexa, open MedBuddy.” Once participants were familiar with activating the skill, the phrases for informing MedBuddy the medication had been taken or forgotten were demonstrated to participants. These phrases included “Alexa, tell MedBuddy I took my medication at 2:00 PM today” and “Alexa, tell MedBuddy I forgot my medication.” Participants were then given the opportunity to interact with Alexa and MedBuddy. Participants were also given the option of connecting the Alexa app on their smartphone to the study account, which allowed them to interact with MedBuddy through either the Echo Dot or their smartphone. Participants received a US $15 honorarium at the completion of the baseline visit.
What the sources say
- Supportedcites [28] source
The Background reports that international studies found up to 47% of women did not fully adhere to the oral contraceptive pill. The Discussion also says the study's findings suggest over half of young women using the pill may not fully adhere to the regimen. (LLM, conf=0.93)
- Not checked — no full text of the sourcecites [29] source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Improved rates of medication adherence have been demonstrated while using smartphone apps [16-19].”
§ Medication Adherence Reminder System for Virtual Home Assistants: Mixed Methods Evaluation Study > Introduction > Background
What the sources say
- Partly supportedcites [16] sourceExperimental — accuracy not yet measured
The Results (Application features in the included interventions; page unavailable) say that one intervention using medication reminders and treatment logs improved medication adherence. But the Results (Characteristics of included studies; page unavailable) identify it as an electronic inhaler reminder system connected to a Web application, not a smartphone app. The review therefore supports an adherence benefit from a digital intervention, but not the claim that smartphone apps demonstrated it. (LLM, conf=0.86)
- Not checked — no full text of the sourcecites [17] Ehrler, F, 2018, AMIA Annu Symp Proc
- Supportedcites [18] source
The Background section reports that a study of an mHealth app found significantly improved medication adherence in the intervention group compared with controls (p=0.017). The paper reports this finding from another study, rather than demonstrating an adherence improvement in its own usability evaluation; it also notes that adherence was a secondary, self-reported measure. (LLM, conf=0.87)
- Supportedcites [19] source
The Results (AI Adherence Measures) report significantly higher short-term aromatase-inhibitor adherence with the web-based app plus weekly reminders than with the app alone (100% versus 72.7%). The Conclusion describes this as improved short-term adherence; the finding concerns a web-enabled app accessible on smartphones, not smartphone use specifically. (LLM, conf=0.88)
boostedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“Findings from a meta-analysis of medication adherence interventions among adults demonstrated that linking medication taking with existing daily routines and using behavioral strategies (eg, prompts to take medication) are the most effective strategies to promote adherence [11].”
§ Medication Adherence Reminder System for Virtual Home Assistants: Mixed Methods Evaluation Study > Introduction > Background
What the sources say
- Supportedcites [11] source
The Discussion (Links Between Adherence Outcomes and Intervention Characteristics) describes habit-based and behavioral-focused interventions as superior to cognitive-based interventions and recommends linking medication taking to daily routines and using prompts. The Results report better adherence outcomes for interventions linking medication taking with existing habits and for those using behavioral strategies. (LLM, conf=0.92)
boostedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“(3) oral contraceptives are most effective when taken at approximately the same time every day [30].”
§ Medication Adherence Reminder System for Virtual Home Assistants: Mixed Methods Evaluation Study > Methods > Recruitment and Enrollment Our team envisioned MedBuddy to be particularly useful for adults who spend most of their time at home. However, we conducted the initial usability and usefulness tests of MedBuddy with women who were college students and took oral contraceptives. We selected this population because (1) younger adults are likely to have more technological literacy than middle-aged or older adults and therefore may provide higher quality feedback about the skill with minimal frustration if it does not work as expected, (2) nonadherence to oral contraceptives is common [28,29], and (3) oral contraceptives are most effective when taken at approximately the same time every day [30]. Following the receipt of an exempt determination by the institutional review board, potential participants were recruited from an urban public university campus in the Southeastern United States. Recruitment flyers were distributed through electronic billboards, an electronic learning management system, and social media. In addition, an announcement about the research opportunity was presented to approximately 100 students who attended a university program orientation. Snowball sampling also occurred by enrolling referrals from current participants or our team’s social contacts. Potential participants were directed to contact the study coordinator by phone or email. When contacted, the study coordinator explained the study purpose, potential risks, and benefits and screened the participants for eligibility. Eligibility criteria included (1) currently taking an oral contraceptive but self-reporting difficulty remembering to take the medication some days or at about the same time each day, (2) living off-campus (because of internet inconsistency or internet overload in campus housing), and (3) owning a smartphone. Students meeting the eligibility requirements were invited to participate, and, for those interested, an enrollment appointment was scheduled. In-person enrollment took place between February 1, 2020, and March 6, 2020, at a research center on a university campus. Volunteer participants provided verbal informed consent and then completed a baseline demographic and medication information questionnaire and a 4-item questionnaire about their adherence to their prescribed oral contraceptive medicine (Multimedia Appendix 1). Each participant received an Amazon Echo Dot device and a study account for the MedBuddy skill. Participants were then instructed on setting up and using the Amazon Echo Dot and subsequently the MedBuddy skill. Participants were first familiarized with activating the MedBuddy skill, using phrases such as “Alexa, open MedBuddy.” Once participants were familiar with activating the skill, the phrases for informing MedBuddy the medication had been taken or forgotten were demonstrated to participants. These phrases included “Alexa, tell MedBuddy I took my medication at 2:00 PM today” and “Alexa, tell MedBuddy I forgot my medication.” Participants were then given the opportunity to interact with Alexa and MedBuddy. Participants were also given the option of connecting the Alexa app on their smartphone to the study account, which allowed them to interact with MedBuddy through either the Echo Dot or their smartphone. Participants received a US $15 honorarium at the completion of the baseline visit.
What the sources say
- Partly supportedcites [30] sourceExperimental — accuracy not yet measured
The Efficacy section (page unavailable) says oral contraceptives are more effective with perfect use than with typical use and identifies missed pills as a common risk. It does not say that taking a pill at approximately the same time each day, rather than at different times, improves effectiveness. (LLM, conf=0.84)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
24 more claims
Create a free account to see all 51 claims and to verify the 3 references still open.
- Probably not a substantial claim. 13
- It may state two separate claims as one. 15
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Rated major: it would touch a result — a value a result rests on, reported and recomputed p-values on opposite sides of the significance level, two passages that conflict with no sign which is right, or a cited work retracted before this paper was published.
In the paper
“Participants reported missing their medication less in the past 1 and 6 months at the end of the study compared with baseline (χ21=0.9 and χ21=0.4, respectively; McNemar test: P<.001 for both).”
§ Medication Adherence Reminder System for Virtual Home Assistants: Mixed Methods Evaluation Study > Results > MedBuddy Usefulness
Reported 0.001 · recomputed 0.5
Rated major: it would touch a result — a value a result rests on, reported and recomputed p-values on opposite sides of the significance level, two passages that conflict with no sign which is right, or a cited work retracted before this paper was published.
In the paper
“participants reported missing their medication less in the past 1 and 6 months compared with baseline (χ21=0.9 and χ21=0.4, respectively; McNemar test: P<.001 for both).”
§ Medication Adherence Reminder System for Virtual Home Assistants: Mixed Methods Evaluation Study > Abstract > Results
Reported 0.001 · recomputed 0.5
Citations
Cites
Kirk, J., MacDonald, A., Lavender, P., Dean, J., & Rubin, G. (2017) · Biomedicine Hub · doi:10.1159/000480347 · Cited by 3,887
Medication Adherence: WHO Cares?
Brown, M. T., & Bussell, J. K. (2011) · Mayo Clinic Proceedings · doi:10.4065/mcp.2010.0575 · Cited by 1,928 · cited 1×
“Medication adherence, defined as taking medicines according to agreed-upon decisions between prescribing health care professionals and patients [1,2], is a major public health challenge.”
BackgroundAdherence and health care costs
McGuire, M., & Iuga (2014) · Risk Management and Healthcare Policy · doi:10.2147/rmhp.s19801 · Cited by 708
Smartphone medication adherence apps: Potential benefits to patients and providers
Dayer, L., Heldenbrand, S., Anderson, P., Gubbins, P. O., & Martin, B. C. (2013) · Journal of the American Pharmacists Association · doi:10.1331/japha.2013.12202 · Cited by 473 · cited 1×
“Technological advances, such as smartphones and electronic pillboxes, have been increasingly used as medication reminders [12-15].”
Pradhan, A., Mehta, K., & Findlater, L. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3174033 · Cited by 448 · cited 2×
“Research findings indicate that voice-activated virtual home assistants (VHAs) such as Amazon Echo and Google Home devices may be useful in promoting medication adherence [22,23].”
BackgroundResultMedication adherence outcomes of 771 intervention trials: Systematic review and meta-analysis
Conn, V. S., & Ruppar, T. M. (2017) · Preventive Medicine · doi:10.1016/j.ypmed.2017.03.008 · Cited by 367 · cited 1×
“Findings from a meta-analysis of medication adherence interventions among adults demonstrated that linking medication taking with existing daily routines and using behavioral strategies (eg, prompts to take medication) are the most effective strategies to promote adherence [11].”
Background"What can i help you with?": infrequent users' experiences of intelligent personal assistants
Cowan, B. R., Pantidi, N., Coyle, D., Morrissey, K., Clarke, P., Al-Shehri, S., Earley, D., & Bandeira, N. (2017) · Proceedings of the 19th International Conference on Human-Computer Interaction with Mobile Devices and Services · doi:10.1145/3098279.3098539 · Cited by 351 · cited 1×
“As it seemed to be a problem for some participants and not problematic at all for others, we believe communication success may have varied based on voice tones and diction—an issue noted by users of other voice-activated devices [35].”
BackgroundMedication Adherence to Multidrug Regimens
Marcum, Z. A., & Gellad, W. F. (2012) · Clinics in Geriatric Medicine · doi:10.1016/j.cger.2012.01.008 · Cited by 264 · cited 1×
“Medication nonadherence contributes to suboptimal health management and increases overall health care costs among patients with chronic conditions [4,5].”
BackgroundHui, C. Y., Walton, R., McKinstry, B., Jackson, T., Parker, R., & Pinnock, H. (2017) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocw143 · Cited by 198 · cited 2×
“For example, there are several smartphone apps that target disease management, including medication adherence, for people with diabetes, HIV, cancer, and other chronic conditions [16-20].”
BackgroundForbes, C. A., Deshpande, S., Sorio-Vilela, F., Kutikova, L., Duffy, S., Gouni-Berthold, I., & Hagström, E. (2018) · Current Medical Research and Opinion · doi:10.1080/03007995.2018.1477747 · Cited by 149 · cited 1×
“Furthermore, subjective measures of medication adherence are known to overestimate adherence [34].”
Background
Research works printed but not linked: 2
- 2003, Adherence to Long-term Therapies: Evidence for Action, 1
- Ehrler, F, 2018, AMIA Annu Symp Proc, 2018, 413
Cited by
Redefining Virtual Assistants in Health Care: The Future With Large Language Models
Sezgin, E. (2024) · Journal of Medical Internet Research · doi:10.2196/53225 · Cited by 47
Sezgın, E., Oiler, B., Abbott, B., Noritz, G. H., & Huang, Y. (2022) · Frontiers in Public Health · doi:10.3389/fpubh.2022.849322 · Cited by 27
“…warning mechanisms for personalized health (19) and public health (20), contributing to personal medical notes as supporting the shared decision making (47), medication adherence and treatment (48) and improving patient-reported outcomes with integration into the electronic medical records (49).”
BackgroundFrom Command to Care: A Scoping Review on Utilization of Smart Speakers by Patients and Providers
Saripalle, R. K., & Patel, R. S. (2024) · Mayo Clinic Proceedings Digital Health · doi:10.1016/j.mcpdig.2024.03.002 · Cited by 17
Baertsch, M., Decker, S., Probst, L., Joneleit, S., Salwender, H. J., Frommann, F., & Buettner, H. (2022) · JMIR Cancer · doi:10.2196/35500 · Cited by 11
“Medication adherence reminder systems for digital home assistants are currently being evaluated by investigators [22].”
BackgroundSmart Medication Monitoring In Medical IoT
Ramani, K., Kumar, P. V., Jahnavi, N., et al. (2023) · doi:10.1109/vitecon58111.2023.10157688 · Cited by 4
Metadata
- Authors
- Corbett, C. F., Combs, E. M., Chandarana, P. S., Stringfellow, I., Worthy, K., Nguyen, T., Wright, P. J., & O'Kane, J. M.
- Affiliations
- University of South Carolina (US)
- Corresponding author
- Corbett, Cynthia F
- ORCID
- Venue
- JMIR Formative Research
- Publisher
- JMIR Publications Inc.
- Published
- 2021-07-13
- Volume
- 5
- Issue
- 7
- Pages
- e27327
- ISSN
- 2561-326X (ISSN-L)
- PMC
- PMC8317037
- OpenAlex
- W3166612255
- Cited by
- 14
- References
- 34
- Citations by year
- 2026: 3 · 2025: 2 · 2024: 5 · 2023: 1 · 2022: 3
- FWCI
- 2.46
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- JMIR Formative Research (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Scholar Commons (University of South Carolina) (submittedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- Topics
- Health Sciences › Medicine › Family Practice › Medication Adherence and Compliance
- Health Sciences › Health Professions › General Health Professions › Mobile Health and mHealth Applications
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Telemedicine and Telehealth Implementation
- Keywords
- medication adherence, medication reminders, Amazon Alexa, Amazon Echo, oral contraceptives, usability, perceived usefulness, women's health, digital health interventions, missed medication, university students, medication adherence support
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (qwen3.8-omni-flash and gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
