Clinical Advice by Voice Assistants on Postpartum Depression: Cross-Sectional Investigation Using Apple Siri, Amazon Alexa, Google Assistant, and Microsoft Cortana
Open accessThe DOI matches its Crossref record; 15 of the 15 references it cites were found in a registry or an index.
Abstract
Background: A voice assistant (VA) is inanimate audio-interfaced software augmented with artificial intelligence, capable of 2-way dialogue, and increasingly used to access health care advice. Postpartum depression (PPD) is a common perinatal mood disorder with an annual estimated cost of $14.2 billion. Only a small percentage of PPD patients seek care due to lack of screening and insufficient knowledge of the disease, and this is, therefore, a prime candidate for a VA-based digital health intervention. Objective: In order to understand the capability of VAs, our aim was to assess VA responses to PPD questions in terms of accuracy, verbal response, and clinically appropriate advice given. Methods: This cross-sectional study examined four VAs (Apple Siri, Amazon Alexa, Google Assistant, and Microsoft Cortana) installed on two mobile devices in early 2020. We posed 14 questions to each VA that were retrieved from the American College of Obstetricians and Gynecologists (ACOG) patient-focused Frequently Asked Questions (FAQ) on PPD. We scored the VA responses according to accuracy of speech recognition, presence of a verbal response, and clinically appropriate advice in accordance with ACOG FAQ, which were assessed by two board-certified physicians. Results: Accurate recognition of the query ranged from 79% to 100%. Verbal response ranged from 36% to 79%. If no verbal response was given, queries were treated like a web search between 33% and 89% of the time. Clinically appropriate advice given by VA ranged from 14% to 29%. We compared the category proportions using the Fisher exact test. No single VA statistically outperformed other VAs in the three performance categories. Additional observations showed that two VAs (Google Assistant and Microsoft Cortana) included advertisements in their responses. Conclusions: While the best performing VA gave clinically appropriate advice to 29% of the PPD questions, all four VAs taken together achieved 64% clinically appropriate advice. All four VAs performed well in accurately recognizing a PPD query, but no VA achieved even a 30% threshold for providing clinically appropriate PPD information. Technology companies and clinical organizations should partner to improve guidance, screen patients for mental health disorders, and educate patients on potential treatment.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Interrater reliability: The kappa statistic
McHugh, M. L. (2012) · Biochemia Medica · doi:10.11613/bm.2012.031 · Cited by 19,877
High-performance medicine: the convergence of human and artificial intelligence
Topol, E. J. (2019) · Nature Medicine · doi:10.1038/s41591-018-0300-7 · Cited by 10,450 · cited 1×
“While this may present privacy challenges, previous successful artificial intelligence implementations in medicine [31] and current VA apps compliant with personal health information protection laws (ie, Health Insurance Portability and Accountability Act) [32] prove that leveraging VA artificial intelligence shows pr…”
BackgroundCovid-19 and Health Care’s Digital Revolution
Keesara, S., Jonas, A., & Schulman, K. (2020) · New England Journal of Medicine · doi:10.1056/nejmp2005835 · Cited by 1,126 · cited 3×
“To enable successful collaboration, legal and regulatory actions should be developed for digital care delivery, which improved telemedicine during COVID-19 [35].”
BackgroundFirth, J., Torous, J., Nicholas, J., Carney, R., Pratap, A., Rosenbaum, S., & Sarris, J. (2017) · World Psychiatry · doi:10.1002/wps.20472 · Cited by 1,097 · cited 1×
“Mental health stands out as a prime candidate for VA-based digital health intervention to fulfill technology’s promise to provide adaptive and personalized care [8,9].”
BackgroundHollis, C., Falconer, C. J., Martin, J. L., Whittington, C., Stockton, S., Glazebrook, C., & Davies, E. B. (2017) · Journal of Child Psychology and Psychiatry · doi:10.1111/jcpp.12663 · Cited by 846 · cited 1×
“Mental health stands out as a prime candidate for VA-based digital health intervention to fulfill technology’s promise to provide adaptive and personalized care [8,9].”
BackgroundACOG Committee Opinion No. 757: Screening for Perinatal Depression
(2018) · Obstetrics & Gynecology · doi:10.1097/aog.0000000000002927 · Cited by 771
US Preventive Services Task Force, Curry, S. J., Krist, A. H., Owens, D. K., Barry, M. J., Caughey, A. B., Davidson, K. W., Doubeni, C. A., Epling, J. W., Grossman, D. C., et al. (2019) · JAMA · doi:10.1001/jama.2019.0007 · Cited by 494 · cited 2×
“In recognition of the severity of the need, the US Preventive Services Task Force, American College of Obstetricians and Gynecologists (ACOG), and American Academy of Pediatrics all recommend regular screening for PPD and early referral for treatment [10,11,14].”
BackgroundMiner, A. S., Milstein, A., Schueller, S., Hegde, R., Mangurian, C., & Linos, E. (2016) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2016.0400 · Cited by 368
Incorporating Recognition and Management of Perinatal Depression Into Pediatric Practice
Earls, M. F., Yogman, M. W., Mattson, G., Rafferty, J., COMMITTEE ON PSYCHOSOCIAL ASPECTS OF CHILD AND FAMILY HEALTH, Baum, R., Gambon, T., Lavin, A., & Wissow, L. (2019) · Pediatrics · doi:10.1542/peds.2018-3259 · Cited by 327
Luca, D. L., Margiotta, C., Staatz, C., Garlow, E., Christensen, A., & Zivin, K. (2020) · American Journal of Public Health · doi:10.2105/ajph.2020.305619 · Cited by 303
Cited by
Sezgin, E., Chekeni, F., Lee, J., & Keim, S. (2023) · Journal of Medical Internet Research · doi:10.2196/49240 · Cited by 65
“This study expands an earlier investigation on chatbot advice for PPD [8], showing that LLMs can provide clinically accurate responses to questions regarding PPD. ChatGPT provides higher-quality responses based on concordance with answers provided in the ACOG FAQ.”
Self-citationHealthcare Voice AI Assistants: Factors Influencing Trust and Intention to Use
Zhan, X., Abdi, N., Seymour, W., & Such, J. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3637339 · Cited by 46
Kavian, J. A., Wilkey, H. L., Patel, P. A., & Boyd, C. J. (2023) · The American Surgeon™ · doi:10.1177/00031348231175454 · Cited by 35
A Mixed-Methods Approach to Understanding User Trust after Voice Assistant Failures
Baughan, A., Wang, X., Liu, A., Mercurio, A., Chen, J., & Ma, X. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581152 · Cited by 32
“As the underlying technology for voice assistants continues to improve in accuracy and ability, and its applications become increasingly high stakes to human health and well-being [17, 41, 55, 66], we discuss our fndings with the goal of improving user trust and long-term engagement in voice assistants.”
BackgroundInfluential"This machine is for the aides": Tailoring Voice Assistant Design to Home Health Care Work
Bartle, V., Albright, L., & Dell, N. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581118 · Cited by 18
Metadata
- Authors
- Yang, S., Lee, J., Sezgin, E., Bridge, J., & Lin, S.
- Venue
- JMIR mHealth and uHealth
- Publisher
- JMIR Publications Inc.
- Published
- 2021-01-11
- Volume
- 9
- Issue
- 1
- Pages
- e24045
- ISSN
- 2291-5222 (ISSN-L)
- OpenAlex
- W3108615578
- Cited by
- 68
- References
- 15
- FWCI
- 1.97
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Clinical Advice by Voice Assistants on Postpartum Depression: Cross-Sectional Investigation Using Apple Siri, Amazon Alexa, Google Assistant, and Microsoft Cortana. https://app.cochat.ai/papers/10.2196/24045
- The paper
- Yang, S., Lee, J., Sezgin, E., Bridge, J., & Lin, S. (2021). Clinical Advice by Voice Assistants on Postpartum Depression: Cross-Sectional Investigation Using Apple Siri, Amazon Alexa, Google Assistant, and Microsoft Cortana. JMIR mHealth and uHealth. https://doi.org/10.2196/24045
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.