Use of Large Language Models (LLMs) in a Large Subspecialty Practice (Preprint)

Abstract

BACKGROUND: Large language models (LLMs) are increasingly being incorporated into clinical practice for tasks such as rapid evidence retrieval, documentation support, and clinical decision-making. However, real-world data on clinician adoption, trust, verification practices, and perceived ethical or security concerns remain limited. OBJECTIVE: To evaluate real-world use of LLMs among clinicians at a large academic medical center and assess perceptions regarding usefulness, reliability, ethical appropriateness, data security, and verification practices. METHODS: We conducted a web-based cross-sectional survey of clinicians within the Department of Medicine at Mayo Clinic (Rochester, Minnesota, USA) between December 2025 and February 2026. Eligible participants included attending physicians, nurse practitioners, physician assistants, residents, and fellows. The survey evaluated awareness and clinical use of LLMs, frequency and context of use, perceived usefulness and ease of use, trust and data security perceptions, verification practices, behavioral intention to use, and comparisons with traditional point-of-care reference tools. Descriptive statistics were used to summarize responses, and associations between years of clinical experience and LLM use were assessed using chi-square tests. RESULTS: A total of 254 clinicians completed the survey (response rate 11.6%). Awareness of LLMs was high (248/254, 97.6%), and 227/246 (92.3%) respondents aware of LLMs reported clinical use. Daily use was reported by 103/222 (46.4%) respondents, while 196/222 (88.3%) reported at least weekly use. OpenEvidence was the most commonly used clinical platform (187/227, 82.4%). LLMs were primarily used for rapid evidence retrieval (174/227, 76.7%), support in complex clinical scenarios (84/227, 37.0%), and guideline summarization (75/227, 33.0%). Additional reported uses included drafting clinical communications, summarizing patient histories, educational activities, and research-related tasks. Most respondents considered LLM use ethically appropriate (202/220, 91.8%) and regarded outputs as generally reliable, although confidence in data security was lower (115/217, 53.0%). Verification practices varied, with 120/217 (55.3%) reporting always or often verifying outputs. Many respondents rated LLMs more favorably than traditional reference tools such as UpToDate and PubMed. Reported use did not differ significantly across years of clinical experience. CONCLUSIONS: LLMs were widely used among respondents for both clinical and administrative tasks at a large academic medical center. Clinicians reported frequent use across diverse workflows, particularly for rapid information retrieval and support with complex clinical questions. Although perceptions of ethical appropriateness and usefulness were generally favorable, variability in verification practices and lower confidence in data security highlight the need for institutional guidance, governance frameworks, and education to support safe and consistent use of LLMs in clinical practice.