Clinical accuracy of large language models and Google search responses to postpartum depression questions: cross-sectional study
Open accessPublished version of a preprint
Abstract
UNSTRUCTURED —
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
On average, ChatGPT responses were rated at near ceiling for clinical accuracy, scoring 3.93 out of a possible 4.
“on average, ChatGPT responses were rated at near ceiling for clinical accuracy, scoring a 3.93 out of a possible 4”
From The Opportunities and Risks of Large Language Models in Mental Health (2024) · cites this paper as [11]
Board-certified human physicians rated ChatGPT responses as more in line with ACOG responses than Bard or Google Search responses.
“Board-certified human physicians rated ChatGPT responses as more in line with ACOG responses than Bard or Google Search responses”
From The Opportunities and Risks of Large Language Models in Mental Health (2024) · cites this paper as [11]
Bard's accuracy ratings were impacted by Bard's policy to advise consulting a health care provider when asked questions about mental health.
“Bard’s accuracy ratings were impacted by Bard’s policy to advise consulting a health care provider when asked questions about mental health.”
From The Opportunities and Risks of Large Language Models in Mental Health (2024) · cites this paper as [11]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
What is ‘quality of evidence’ and why is it important to clinicians?
Guyatt, G. H., Oxman, A. D., Kunz, R., Vist, G. E., Falck-Ytter, Y., & Schünemann, H. J. (2008) · BMJ · doi:10.1136/bmj.39490.551019.be · Cited by 3,250 · see citing works
The R Project in Statistical Computing
Ripley, B. D. (2001) · MSOR Connections · doi:10.11120/msor.2001.01010023 · Cited by 2,681 · see citing works
Sharma, A., Lin, I. W., Miner, A. S., Atkins, D. C., & Althoff, T. (2023) · Nature Machine Intelligence · doi:10.1038/s42256-022-00593-2 · Cited by 500 · see citing works
Sarraju, A., Bruemmer, D., Iterson, E. H. V., Cho, L., Rodríguez, F., & Laffin, L. J. (2023) · JAMA · doi:10.1001/jama.2023.1044 · Cited by 455 · see citing works
Yang, S., Lee, J., Sezgın, E., Bridge, J. A., & Lin, S. M. (2020) · JMIR mhealth and uhealth · doi:10.2196/24045 · Cited by 69 · see citing works
Self-citation
Flagged first, then by citations
Research works printed but not linked: 1
- [10]Aronson S Lieu TW Scirica BM Getting generative AI right NEJM Catalyst 2023 1 10.1056/CAT.23.0063 · DOI
Cited by
Sorted by citations
Current applications and challenges in large language models for patient care: a systematic review
Busch, F., Hoffmann, L., Rueger, C., van Dijk, E. H., Kader, R., Ortiz-Prado, E., Makowski, M. R., Saba, L., Hadamitzky, M., Kather, J. N., et al. (2025) · Communications Medicine · doi:10.1038/s43856-024-00717-2 · Cited by 358 · see citing works
Large Language Models for Chatbot Health Advice Studies: A Systematic Review
Huo, B., Boyle, A., Marfo, N., Tangamornsuksan, W., Steen, J. P., McKechnie, T., Lee, Y., Mayol, J., Antoniou, S. A., Thirunavukarasu, A. J., et al. (2025) · JAMA Network Open · doi:10.1001/jamanetworkopen.2024.57879 · Cited by 283 · see citing works
Large Language Models for Mental Health Applications: Systematic Review
Guo, Z., Lai, A., Thygesen, J. H., Farrington, J., Keen, T., & Li, K. (2024) · JMIR Mental Health · doi:10.2196/57400 · Cited by 269 · see citing works
The Opportunities and Risks of Large Language Models in Mental Health
Lawrence, H. R., Schneider, R. A., Rubin, S. B., Matarić, M. J., McDuff, D. J., & Jones Bell, M. (2024) · JMIR Mental Health · doi:10.2196/59479 · Cited by 235 · see citing works
Hua, Y., Siddals, S., Ma, Z., Galatzer‐Levy, I., Xia, W., Hau, C., Na, H., Flathers, M., Linardon, J., Ayubcha, C., et al. (2025) · World Psychiatry · doi:10.1002/wps.21352 · Cited by 117 · see citing works
The Applications of Large Language Models in Mental Health: Scoping Review
Jin, Y., Liu, J., Li, P., Wang, B., Yan, Y., Zhang, H., Ni, C., Wang, J., Li, Y., Bu, Y., et al. (2025) · Journal of Medical Internet Research · doi:10.2196/69284 · Cited by 94 · see citing works
Applications of large language models in psychiatry: a systematic review
Omar, M., Soffer, S., Charney, A. W., Landi, I., Nadkarni, G. N., & Klang, E. (2024) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2024.1422807 · Cited by 85 · see citing works
Suárez, A., Jiménez, J., de Pedro, M. L., Vázquez, C. A., García, V. D., Sánchez, M. G., & Freire, Y. (2023) · Computational and Structural Biotechnology Journal · doi:10.1016/j.csbj.2023.11.058 · Cited by 82 · see citing works
Kolding, S., Lundin, R. M., Hansen, L., & Østergaard, S. D. (2024) · Acta Neuropsychiatrica · doi:10.1017/neu.2024.50 · Cited by 80 · see citing works
Sallam, M., Barakat, M. M., & Sallam, M. (2024) · Interactive Journal of Medical Research · doi:10.2196/54704 · Cited by 88 · see citing works
Yuan, A., Colato, E. G., Pescosolido, B. A., Song, H., & Samtani, S. (2024) · ACM Transactions on Management Information Systems · doi:10.1145/3701041 · Cited by 74 · see citing works
Exploring the efficacy and potential of large language models for depression: A systematic review
Omar, M., & Levkovich, I. (2024) · Journal of Affective Disorders · doi:10.1016/j.jad.2024.11.052 · Cited by 51 · see citing works
Behavioral health and generative AI: a perspective on future of therapies and patient care
Sezgın, E., & McKay, I. T. (2024) · npj Mental Health Research · doi:10.1038/s44184-024-00067-w · Cited by 55 · see citing works
Redefining Virtual Assistants in Health Care: The Future With Large Language Models
Sezgın, E. (2024) · Journal of Medical Internet Research · doi:10.2196/53225 · Cited by 49 · see citing works
Esmailpour, H., Rasaie, V., Hemmati, Y. B., & Falahchai, M. (2025) · BMC Oral Health · doi:10.1186/s12903-025-05965-9 · Cited by 40 · see citing works
An evaluation framework for ambient digital scribing tools in clinical applications
Wang, H., Yang, R., Alwakeel, M., Kayastha, A., Chowdhury, A., Biro, J., Sorrentino, A. D., Handley, J. L., Hantzmon, S. V., Bessias, S., et al. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-01622-1 · Cited by 36 · see citing works
An Entity Extraction Pipeline for Medical Text Records Using Large Language Models: Analytical Study
Wang, L., Ma, Y., Bi, W., Lv, H., & Li, Y. (2024) · Journal of Medical Internet Research · doi:10.2196/54580 · Cited by 34 · see citing works
ChatGPT vs. Bard: A Comparative StudyPreprint
Ahmed, I., Roy, A., Kajol, M. A., Hasan, U., Datta, P. P., & Reza, M. R. (2023) · doi:10.22541/au.168923529.98827844/v1 · Cited by 33 · see citing works
Sezgin, E., Jackson, D. I., Kocaballi, A. B., Bibart, M., Zupanec, S., Landier, W., Audino, A., Ranalli, M., & Skeens, M. (2025) · Cancer Medicine · doi:10.1002/cam4.70554 · Cited by 30 · see citing works
ChatGPT versus Bard: A comparative study
Ahmed, I., Kajol, M. A., Hasan, U., Datta, P. P., Roy, A., & Reza, M. R. (2024) · Engineering Reports · doi:10.1002/eng2.12890 · Cited by 25 · see citing works
Understanding Attitudes and Trust of Generative AI Chatbots for Social Anxiety Support
Wang, Y., Wang, Y., Crace, K., & Zhang, Y. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3714286 · Cited by 25 · see citing works
Denecke, K., & Gabarrón, E. (2024) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2024.1462083 · Cited by 22 · see citing works
Application of Large Language Models in Stroke Rehabilitation Health Education: 2-Phase Study
Qiang, S., Liao, Y., Gu, Y., Gu, Y., Zhang, Y., Wang, Y., Xu, Z., Xu, Z., Han, N., Zhang, H., et al. (2025) · Journal of Medical Internet Research · doi:10.2196/73226 · Cited by 21 · see citing works
ChatGPT Clinical Use in Mental Health Care: Scoping Review of Empirical Evidence
Balan, R., & Gumpel, T. P. (2025) · JMIR Mental Health · doi:10.2196/81204 · Cited by 22 · see citing works
Detecting and Explaining Postpartum Depression in Real-Time with Generative Artificial Intelligence
García-Méndez, S., & de Arriba-Pérez, F. (2025) · Applied Artificial Intelligence · doi:10.1080/08839514.2025.2515063 · Cited by 14 · see citing works
Voultsiou, E., & Moussiades, L. (2026) · Electronics · doi:10.3390/electronics15030524 · Cited by 16 · see citing works
Hong, C., Chowdhury, A., Sorrentino, A. D., Wang, H., Agrawal, M., Bedoya, A., Bessias, S., Economou-Zavlanos, N., Wong, I., Pean, C. A., et al. (2025) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocaf023 · Cited by 12 · see citing works
Cunningham, K., Mărginean, V., & Hylock, R. H. (2025) · npj Health Systems · doi:10.1038/s44401-025-00030-7 · Cited by 14 · see citing works
García-Méndez, S., & de Arriba-Pérez, F. (2024) · Annals of Biomedical Engineering · doi:10.1007/s10439-024-03454-8 · Cited by 10 · see citing works
Dagli, M. M., Oettl, F. C., Gujral, J., Malhotra, K., Ghenbot, Y. G., Yoon, J. W., Ozturk, A. K., & Welch, W. C. (2024) · JMIR Formative Research · doi:10.2196/56165 · Cited by 10 · see citing works
Bernstein, E. E., Jaroszewski, A. C., Jacoby, R. J., Bailen, N. H., Ragan, J. D., Usmani, A., & Wilhelm, S. (2025) · Behavior Therapy · doi:10.1016/j.beth.2025.02.005 · Cited by 10 · see citing works
ChatGPT vs. Bard: A Comparative StudyPreprint
Ahmed, I., Kajol, M. A., Hasan, U., & Datta, P. P. (2023) · doi:10.36227/techrxiv.23536290 · Cited by 8 · see citing works
Potential of ChatGPT in youth mental health emergency triage: Comparative analysis with clinicians
Thotapalli, S., Yilanli, M., McKay, I. T., Leever, W. J., Youngstrom, E. A., Harvey, K., Lowder, K., Schweitzer, S., Sunderland, E., Jackson, D., et al. (2025) · Psychiatry and Clinical Neurosciences Reports · doi:10.1002/pcn5.70159 · Cited by 13 · see citing works
Meter, A. V., Wheaton, M. G., Cosgrove, V. E., Andreadis, K., & Robertson, R. E. (2025) · PLOS Digital Health · doi:10.1371/journal.pdig.0000711 · Cited by 7 · see citing works
Tan, J., Wang, L., Wang, G., Yang, Y. S., Jia, F., Chi, X., Xie, X., Li, T., Yang, B., Zhang, H., et al. (2026) · Digital Health · doi:10.1177/20552076261427505 · Cited by 6 · see citing works
Performance of chatbots in queries concerning fundamental concepts in photochemistry
Taniguchi, M., & Lindsey, J. S. (2024) · Photochemistry and Photobiology · doi:10.1111/php.14037 · Cited by 6 · see citing works
ChatGPT vsPreprint
Ahmed, I., Kajol, M. A., Hasan, U., & Datta, P. P. (2023) · doi:10.36227/techrxiv.23536290.v1 · Cited by 6 · see citing works
Learning with Conversational AI: ChatGPT and Bard/Gemini in Education
da Silva, G. S., & Ulbricht, V. R. (2024) · Cognition and exploratory learning in the digital age · doi:10.1007/978-3-031-66462-5_6 · Cited by 5 · see citing works
Trillo-Domínguez, M., Martín-Neira, J.-I., & Olvera‐Lobo, M. (2026) · Informatics · doi:10.3390/informatics13030041 · Cited by 4 · see citing works
Yang, H., & Lin, D.-M. (2026) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2026.1847854 · Cited by 3 · see citing works
Using AI to Train Future Clinicians in Depression Assessment: Feasibility Study
Holderried, F., Sonanini, A., Philipps, A., Stegemann–Philipps, C., Herschbach, L., Wietek, T. F., Zipfel, S., Erschens, R. S., & Herrmann‐Werner, A. (2025) · JMIR Medical Education · doi:10.2196/87102 · Cited by 4 · see citing works
ChatGPT vsPreprint
Ahmed, I., Kajol, M. A., Hasan, U., & Datta, P. P. (2023) · doi:10.36227/techrxiv.23536290.v2 · Cited by 3 · see citing works
Tang, Y., Huang, T., & Yin, X. (2024) · International Journal of Advanced Computer Science and Applications · doi:10.14569/ijacsa.2024.0150636 · Cited by 2 · see citing works
Augmentation or substitution: defining role of large language model in physical education
Yang, H., Huang, B., & Li, X. (2025) · Frontiers in Sports and Active Living · doi:10.3389/fspor.2025.1662056 · Cited by 4 · see citing works
Assessment of AI-generated responses to common pregnancy questions: A blinded expert evaluation
Atay, A. O., Atay, F., & Corlu, O. (2026) · European Journal of Obstetrics & Gynecology and Reproductive Biology · doi:10.1016/j.ejogrb.2026.114960 · Cited by 2 · see citing works
Huang, J., Yu, H., Chen, J., Wang, X., Huang, L., Wen, J., & Li, H. (2026) · Frontiers in Public Health · doi:10.3389/fpubh.2026.1760872 · Cited by 5 · see citing works
Jiao, R., Chen, M., & Zhang, J. P. (2026) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2026.1782288 · Cited by 2 · see citing works
Fairness Evaluation Within Large Language Models through the Lens of Depression
Han, Y. (2023) · International Conference on Machine Learning and Computer Application (ICMLCA) · doi:10.1145/3650215.3650236 · Cited by 1 · see citing works
Large Language Models for Depression Detection: A Review with Prospects of Incomplete Multimodality
Dai, A., Shi, W., Gu, X., & Huang, L. (2026) · Brain Sciences · doi:10.3390/brainsci16060593 · Cited by 1 · see citing works
Eray, O., & Polat, L. N. Ö. (2026) · Journal of Intelligent Systems Theory and Applications · doi:10.38016/jista.1831503 · Cited by 1 · see citing works
Redefining Virtual Assistants in Health Care: The Future With Large Language Models (Preprint)Preprint
Sezgın, E. (2023) · doi:10.2196/preprints.53225 · Cited by 1 · see citing works
Qiang, S., Liao, Y., Gu, Y., Gu, Y., Zhang, Y., Wang, Y., Xu, Z., Xu, Z., Han, N., Zhang, H., et al. (2025) · doi:10.2196/preprints.73226
Balan, R., & Gumpel, T. P. (2025) · doi:10.2196/preprints.81204
Application and Development of Large Language Models in Smart Inhalers
Guo, W., Cheng, Z., & Wang, J. (2025) · Pharmaceutical Fronts · doi:10.1055/a-2652-0081
Medical Diagnosis Coding Automation: Similarity Search vs. Generative Ai
Klotzman, V. (2024) · Clinical and Medical Engineering Live · doi:10.64030/2997-612x.02.02.01
Leveraging Large Language Models for Dataset Discovery
Chen, T., Schott, K., Mathiak, B., & Kern, D. (2025) · Proceedings of the Association for Information Science and Technology · doi:10.1002/pra2.1242
Kaumenova, T., Chakraborty, S., Fosler‐Lussier, E., Gofar, K., Metcalf, I., Perrault, A., & White, M. (2025) · medRxiv · doi:10.1101/2025.11.19.25340596
Inteligencia artificial generativa aplicada a la salud mental: un análisis bibliométrico
Cevallos, D. P. R. (2025) · Salmus · doi:10.61347/rcss.v1i2.e9
Min, J., Kuang, D., Sun, Y., Yao, M., & Chen, Q. (2026) · doi:10.2196/preprints.94372
Kaumenova, T., Chakraborty, S., Fosler‐Lussier, E., Gofar, K., Metcalf, I., Perrault, A., & White, M. (2026) · JMIR AI · doi:10.2196/88581
Nungo, M. A., Abubakar, R., & Makokha, C. J. (2026) · doi:10.1002/9781394282470.ch20
Yan, Y.-L., Li, Q.-Q., & Xu, M. (2026) · Frontiers in Public Health · doi:10.3389/fpubh.2026.1967813
Evaluating Harm Reduction Advice From Large Language Models
Choo, K., Smith, N., Ward, B., & Lundin, R. M. (2026) · Drug and Alcohol Review · doi:10.1111/dar.70272
Łapot, M., Hober, K., Krupnik, D., Skowron, W., Skowron, M., Bzdak, W., Szwemińska, P., Górak, J., & Neumann, W. (2026) · International Journal of Innovative Technologies in Social Science · doi:10.31435/ijitss.3(51).2026.6235
Zhao, X., Zhang, X., & Liu, L. (2026) · Frontiers in Public Health · doi:10.3389/fpubh.2026.1961647
- Authors
- Sezgin, E., Chekeni, F., Lee, J., & Keim, S.
- Affiliations
- Nationwide Children's Hospital (US)
- Venue
- Journal of Medical Internet Research
- Publisher
- JMIR Publications Inc.
- Published
- 2023-09-11
- Volume
- 25
- Pages
- e49240
- ISSN
- 1438-8871 (ISSN-L)
- PMC
- PMC10520763
- OpenAlex
- W4386304195
- Cited by
- 65
- References
- 5
- Citations by year
- 2026: 16 · 2025: 24 · 2024: 18 · 2023: 7
- FWCI
- 16.64
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- Journal of Medical Internet Research (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Topics
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Maternal Mental Health During Pregnancy and Postpartum
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Health Sciences › Health Professions › General Health Professions › Health Literacy and Information Accessibility
- Keywords
- postpartum depression, large language models, clinical accuracy, diagnostic accuracy, maternal mental health, online health information
- MeSH terms
- Cross-Sectional Studies, Female, Humans, Depression, Postpartum, Search Engine
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
