CoChat

Patient and Consumer Safety Risks When Using Conversational Assistants for Medical Information: An Observational Study of Siri, Alexa, and Google Assistant

Open access
Bickmore, Timothy W, Trinh, Ha, Olafsson, Stefan, O'Leary, Teresa K, Asadi, Reza, Rickles, Nathaniel M, Cruz, Ricardo · 2018 · Journal of Medical Internet Research

Published version of a preprint

The DOI matches its Crossref record; 28 of the 28 references it cites were found in a registry or an index.

Abstract

BACKGROUND: Conversational assistants, such as Siri, Alexa, and Google Assistant, are ubiquitous and are beginning to be used as portals for medical services. However, the potential safety issues of using conversational assistants for medical information by patients and consumers are not understood. OBJECTIVE: To determine the prevalence and nature of the harm that could result from patients or consumers using conversational assistants for medical information. METHODS: Participants were given medical problems to pose to Siri, Alexa, or Google Assistant, and asked to determine an action to take based on information from the system. Assignment of tasks and systems were randomized across participants, and participants queried the conversational assistants in their own words, making as many attempts as needed until they either reported an action to take or gave up. Participant-reported actions for each medical task were rated for patient harm using an Agency for Healthcare Research and Quality harm scale. RESULTS: Fifty-four subjects completed the study with a mean age of 42 years (SD 18). Twenty-nine (54%) were female, 31 (57%) Caucasian, and 26 (50%) were college educated. Only 8 (15%) reported using a conversational assistant regularly, while 22 (41%) had never used one, and 24 (44%) had tried one "a few times." Forty-four (82%) used computers regularly. Subjects were only able to complete 168 (43%) of their 394 tasks. Of these, 49 (29%) reported actions that could have resulted in some degree of patient harm, including 27 (16%) that could have resulted in death. CONCLUSIONS: Reliance on conversational assistants for actionable medical information represents a safety risk for patients and consumers. Patients should be cautioned to not use these technologies for answers to medical questions they intend to act on without further consultation from a health care provider.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.2196/11510

Cites

28 of 28 references verified0 citation errors · 0 not checkable

Statistics

Figures

Figure 1 Frequency of potentially harmful and fatal actions.
Figure 1 Frequency of potentially harmful and fatal actions.CC BY · PubMed Central
Figure 2 Differences in Task Outcomes by conversational assistant (% of all cases per conversational assistant). Google: Google Assistant.
Figure 2 Differences in Task Outcomes by conversational assistant (% of all cases per conversational assistant). Google: Google Assistant.CC BY · PubMed Central
Figure 3 Differences in Task Outcomes by CA (% of all cases per CA).
Figure 3 Differences in Task Outcomes by CA (% of all cases per CA).CC BY · PubMed Central

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

13 more claims about this paper

Create a free account to see every claim other papers make about this one.

See all claims — free account

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 28 references

Research works printed but not linked: 2

  1. 2018, US Census Bureau, 2018
  2. Davis, TC, 1993, Fam Med, 25, 391

Cited by

See all 308

Metadata

Authors
Bickmore, T. W., Trinh, H., Olafsson, S., O'Leary, T. K., Asadi, R., Rickles, N. M., & Cruz, R.
Affiliations
  • Northeastern University (US)
  • University of Connecticut (US)
  • Boston Medical Center (US)
Corresponding author
Bickmore, Timothy W
Venue
Journal of Medical Internet Research
Published
2018-09-04
Volume
20
Issue
9
Pages
e11510
ISSN
1438-8871 (ISSN-L)
OpenAlex
W2886572631
Cited by
308
References
28
Citations by year
2026: 62 · 2025: 36 · 2024: 32 · 2023: 47 · 2022: 35 · 2021: 44 · 2020: 32 · 2019: 20
FWCI
6.26
Type
journal article
Language
en
Open access
Licence
cc-by
Locations
  • Journal of Medical Internet Research (publishedVersion, cc-by)
  • Europe PMC (PubMed Central) (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
Topics
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
  • Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Patient Safety and Medication Errors
Keywords
chatbots, voice assistants, Siri, Amazon Alexa, Google Assistant, health information seeking, patient safety, consumer health information
MeSH terms
Adult, Consumer Behavior, Female, Humans, Male, Social Media, Health Information Exchange

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.