Comparing the Efficacy of Large Language Models ChatGPT, BARD, and Bing AI in Providing Information on Rhinoplasty: An Observational Study
Open accessThe DOI matches its Crossref record; 11 of the 11 references it cites were found in a registry or an index.
Abstract
Background: Large language models (LLMs) are emerging artificial intelligence (AI) technologies refining research and healthcare. However, the impact of these models on presurgical planning and education remains under-explored. Objectives: This study aims to assess 3 prominent LLMs—Google's AI BARD (Mountain View, CA), Bing AI (Microsoft, Redmond, WA), and ChatGPT-3.5 (Open AI, San Francisco, CA) in providing safe medical information for rhinoplasty. Methods: Six questions regarding rhinoplasty were prompted to ChatGPT, BARD, and Bing AI. A Likert scale was used to evaluate these responses by a panel of Specialist Plastic and Reconstructive Surgeons with extensive experience in rhinoplasty. To measure reliability, the Flesch Reading Ease Score, the Flesch–Kincaid Grade Level, and the Coleman–Liau Index were used. The modified DISCERN score was chosen as the criterion for assessing suitability and reliability. A t test was performed to calculate the difference between the LLMs, and a double-sided P-value <.05 was considered statistically significant. Results: In terms of reliability, BARD and ChatGPT demonstrated a significantly (P < .05) greater Flesch Reading Ease Score of 47.47 (±15.32) and 37.68 (±12.96), Flesch–Kincaid Grade Level of 9.7 (±3.12) and 10.15 (±1.84), and a Coleman–Liau Index of 10.83 (±2.14) and 12.17 (±1.17) than Bing AI. In terms of suitability, BARD (46.3 ± 2.8) demonstrated a significantly greater DISCERN score than ChatGPT and Bing AI. In terms of Likert score, ChatGPT and BARD demonstrated similar scores and were greater than Bing AI. Conclusions: BARD delivered the most succinct and comprehensible information, followed by ChatGPT and Bing AI. Although these models demonstrate potential, challenges regarding their depth and specificity remain. Therefore, future research should aim to augment LLM performance through the integration of specialized databases and expert knowledge, while also refining their algorithms. Level of Evidence: 5:
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
Lund, B. D., Wang, T., Mannuru, N. R., Nie, B., Shimray, S., & Wang, Z. (2023) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.24750 · Cited by 919
ChatGPT in healthcare: A taxonomy and systematic review
Li, J., Dada, A., Puladi, B., Kleesiek, J., & Egger, J. (2024) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2024.108013 · Cited by 452
Use of Alar Batten Grafts for Correction of Nasal Valve Collapse
Toriumi, D. M., Josen, J., Weinberger, M., & Tardy, M. E. (1997) · Archives of Otolaryngology - Head and Neck Surgery · doi:10.1001/archotol.1997.01900080034002 · Cited by 329
Treatment of Inferior Turbinate Hypertrophy: A Randomized Clinical Trial
Passàli, D., Passàli, F. M., Passàli, G. C., Damiani, V., & Bellussi, L. (2003) · Annals of Otology, Rhinology & Laryngology · doi:10.1177/000348940311200806 · Cited by 219
Xie, Y., Seth, I., Hunter-Smith, D. J., Rozen, W. M., Ross, R., & Lee, M. (2023) · Aesthetic Plastic Surgery · doi:10.1007/s00266-023-03338-7 · Cited by 198 · cited 1×
“11 However, alongside the potential benefits, it is imperative to address the challenges and limitations associated with deploying AI-driven language models in a medical context, to ensure the reliability, accuracy, and ethical use of generated information.”
Self-citationAnterior Septal Reconstruction: Outcomes After a Modified Extracorporeal Septoplasty Technique
Most, S. P. (2006) · Archives of Facial Plastic Surgery · doi:10.1001/archfaci.8.3.202 · Cited by 166
Expanding Cosmetic Plastic Surgery Research With ChatGPT
Gupta, R., Park, J. B., Bisht, C., Herzog, I., Weisberger, J., Chao, J., Chaiyasate, K., & Lee, E. S. (2023) · Aesthetic Surgery Journal · doi:10.1093/asj/sjad069 · Cited by 112
The value of spreader grafts in rhinoplasty: a critical review
Teymoortash, A., Fasunla, J. A., & Sazgar, A. A. (2012) · European Archives of Oto-Rhino-Laryngology · doi:10.1007/s00405-011-1837-y · Cited by 72 · cited 1×
“…contextual overview, briefly discussing the mechanism and potential side effects, before identifying spreader, Alar batten, and Butterfly grafts, 15,16 but failed to discuss the spreader graft’s limitations in expanding a collapsed lateral nasal wall as noted by Teymoortash et al. Only 1 of its 5…”
Inferior Turbinate Hypertrophy in Rhinoplasty: Systematic Review of Surgical Techniques
Sinno, S., Mehta, K., Lee, Z.-H., Kidwai, S., Saadeh, P. B., & Lee, M. R. (2016) · Plastic & Reconstructive Surgery · doi:10.1097/prs.0000000000002433 · Cited by 60
Self-citationArtificial or Augmented Authorship? A Conversation with a Chatbot on Base of Thumb Arthritis
Seth, I., Sinkjær Kenney, P., Bulloch, G., Hunter-Smith, D. J., Bo Thomsen, J., & Rozen, W. M. (2023) · Plastic and Reconstructive Surgery - Global Open · doi:10.1097/gox.0000000000004999 · Cited by 26
Self-citation
Cited by
Tam, T. Y. C., Sivarajkumar, S., Kapoor, S., Stolyar, A. V., Polanska, K., McCarthy, K. R., Osterhoudt, H., Wu, X., Visweswaran, S., Fu, S., et al. (2024) · npj Digital Medicine · doi:10.1038/s41746-024-01258-7 · Cited by 424
Lee, D., Arnold, M., Srivastava, A., Plastow, K., Strelan, P., Ploeckl, F., Lekkas, D., & Palmer, E. (2024) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2024.100221 · Cited by 402
Current applications and challenges in large language models for patient care: a systematic review
Busch, F., Hoffmann, L., Rueger, C., van Dijk, E. H., Kader, R., Ortiz-Prado, E., Makowski, M. R., Saba, L., Hadamitzky, M., Kather, J. N., et al. (2025) · Communications Medicine · doi:10.1038/s43856-024-00717-2 · Cited by 358
Large Language Models for Chatbot Health Advice Studies: A Systematic Review
Huo, B., Boyle, A., Marfo, N., Tangamornsuksan, W., Steen, J. P., McKechnie, T., Lee, Y., Mayol, J., Antoniou, S. A., Thirunavukarasu, A. J., et al. (2025) · JAMA Network Open · doi:10.1001/jamanetworkopen.2024.57879 · Cited by 283
Moulaei, K., Yadegari, A., Baharestani, M., Farzanbakhsh, S., Sabet, B., & Reza Afrash, M. (2024) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2024.105474 · Cited by 244
Metadata
- Authors
- Seth, I., Lim, B., Xie, Y., Cevik, J., Rozen, W. M., Ross, R. J., & Lee, M.
- Venue
- Aesthetic Surgery Journal Open Forum
- Publisher
- Oxford University Press (OUP)
- Published
- 2023
- Volume
- 5
- Pages
- ojad084-ojad084
- ISSN
- 2631-4797 (ISSN-L)
- OpenAlex
- W4386730959
- Cited by
- 78
- References
- 11
- FWCI
- 2.70
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.