Utility of GPT-4 as an Informational Patient Resource in Otolaryngology
Open accessAbstract
We sought to understand the potential utility of ChatGPT as an informational resource for otolaryngology patients. We evaluated responses by GPT-4 to queries based on the American Academy of Otolaryngology’s Clinical Practice Guidelines. We found that while otolaryngology advice provided by ChatGPT is generally safe, it lacks accuracy and comprehensiveness, limiting its utility as an informational resource for patients. Lay Summary: As the popularity of ChatGPT explodes, patients may turn to it for medical advice. We found that while otolaryngology advice provided by ChatGPT is generally safe, it lacks accuracy and comprehensiveness, limiting its utility as an informational resource for patients.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Benefits, Limits, and Risks of GPT-4 as an AI Chatbot for Medicine
Lee, P., Bubeck, S., & Petro, J. (2023) · New England Journal of Medicine · doi:10.1056/nejmsr2214184 · Cited by 1,738 · see citing works
BioGPT: generative pre-trained transformer for biomedical text generation and mining
Luo, R., Sun, L., Xia, Y., Qin, T., Zhang, S., Poon, H., & Liu, T.-Y. (2022) · Briefings in Bioinformatics · doi:10.1093/bib/bbac409 · Cited by 1,150 · see citing works
Sarraju, A., Bruemmer, D., Iterson, E. H. V., Cho, L., Rodríguez, F., & Laffin, L. J. (2023) · JAMA · doi:10.1001/jama.2023.1044 · Cited by 455 · see citing works
AI-Generated Medical Advice—GPT and Beyond
Haupt, C. E., & Marks, M. (2023) · JAMA · doi:10.1001/jama.2023.5321 · Cited by 346 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Haltaufderheide, J., & Ranisch, R. (2024) · npj Digital Medicine · doi:10.1038/s41746-024-01157-x · Cited by 396 · see citing works
Current applications and challenges in large language models for patient care: a systematic review
Busch, F., Hoffmann, L., Rueger, C., van Dijk, E. H., Kader, R., Ortiz-Prado, E., Makowski, M. R., Saba, L., Hadamitzky, M., Kather, J. N., et al. (2025) · Communications Medicine · doi:10.1038/s43856-024-00717-2 · Cited by 358 · see citing works
Large Language Models for Chatbot Health Advice Studies: A Systematic Review
Huo, B., Boyle, A., Marfo, N., Tangamornsuksan, W., Steen, J. P., McKechnie, T., Lee, Y., Mayol, J., Antoniou, S. A., Thirunavukarasu, A. J., et al. (2025) · JAMA Network Open · doi:10.1001/jamanetworkopen.2024.57879 · Cited by 283 · see citing works
Long, C., Subburam, D., Lowe, K., dos Santos, A., Zhang, J. J., Hwang, S. W., Saduka, N., Horev, Y., Su, T., Côté, D. W. J., et al. (2024) · Otolaryngology · doi:10.1002/ohn.864 · Cited by 46 · see citing works
Dhar, S., Kothari, D. S., Vasquez, M., Clarke, T., Maroda, A. J., McClain, W. G., Sheyn, A. M., Tuliszewski, R. M., Tang, D. M., & Rangarajan, S. V. (2024) · International Journal of Pediatric Otorhinolaryngology · doi:10.1016/j.ijporl.2024.111901 · Cited by 22 · see citing works
Applications of Natural Language Processing in Otolaryngology: A Scoping Review
Banyi, N., Ma, B., Amanian, A., Bur, A. M., & Abdalkhani, A. (2025) · The Laryngoscope · doi:10.1002/lary.32198 · Cited by 11 · see citing works
Can GPT-4 revolutionize otolaryngology? Navigating opportunities and ethical considerations
Edalati, S., Vasan, V., Cheng, C., Patel, Z. M., Govindaraj, S., & Iloreta, A. M. (2024) · American Journal of Otolaryngology · doi:10.1016/j.amjoto.2024.104303 · Cited by 7 · see citing works
Assessing the research landscape and clinical utility of large language models: A scoping reviewPreprint
Park, Y., Pillai, A., Deng, J., Guo, E., Gupta, M., Paget, M., & Naugler, C. T. (2023) · Research Square · doi:10.21203/rs.3.rs-3472000/v1 · Cited by 8 · see citing works
Nazir, T. F., Ahmad, U., Mal, M. M., Rehman, M. U., Saeed, R., & Kalia, J. S. (2023) · medRxiv · doi:10.1101/2023.08.25.23294641 · Cited by 5 · see citing works
Patient perspective on large-language model responses to questions about Moyamoya
Ruppert-Gomez, M., Choi, J. H., Staffa, S. J., Holste, K. G., Xu, J. C., Stratton, C., Kocher, S., Smith, E. R., & See, A. P. (2026) · Acta Neurochirurgica · doi:10.1007/s00701-025-06743-w · Cited by 1 · see citing works
Generative artificial intelligence in otolaryngology
Kowalski, H. R., Davies, C., & Lee, S. C. (2026) · Otorhinolaryngology · doi:10.23736/s2724-6302.26.02601-0 · Cited by 1 · see citing works
- Authors
- Suresh, K., Rathi, V., Nwosu, O., Partain, M. P., Glicksman, J. T., Jowett, N., & Crowson, M. G.
- Affiliations
- Massachusetts Eye and Ear Infirmary (US)
- Indiana University Indianapolis (US)
- Corresponding author
- Suresh, Krish
- ORCID
- Venue
- medRxiv
- Publisher
- openRxiv
- Published
- 2023-05-16
- OpenAlex
- W4376637945
- Cited by
- 11
- References
- 4
- Citations by year
- 2026: 2 · 2025: 3 · 2024: 4 · 2023: 2
- Type
- posted content
- Language
- en
- Open access
- green
- doi.org
- Locations
- medRxiv (acceptedVersion)
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Health Sciences › Health Professions › General Health Professions › Health Literacy and Information Accessibility
- Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
- Keywords
- GPT-4, ChatGPT, otolaryngology, patient information resources, AI-generated medical advice, AI response accuracy, patient education
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
