The DOI matches its Crossref record; 29 of the 29 references it cites were found in a registry or an index; 18 of the 38 sources its claims cite were checked for support.
Is there any room for ChatGPT AI bot in speech-language pathology?
Open accessAbstract
Purpose: This study investigates the potential of the ChatGPT-4.0 artificial intelligence bot to assist speech-language pathologists (SLPs) by assessing its accuracy, comprehensiveness, and relevance in various tasks related to speech, language, and swallowing disorders. Method: In this cross-sectional descriptive study, 15 practicing SLPs evaluated ChatGPT-4.0’s responses to task-specific queries across six core areas: report writing, assessment material generation, clinical decision support, therapy stimulus generation, therapy planning, and client/family training material generation. English prompts were created in seven areas: speech sound disorders, motor speech disorders, aphasia, stuttering, childhood language disorders, voice disorders, and swallowing disorders. These prompts were entered into ChatGPT-4.0, and its responses were evaluated. Using a three-point Likert-type scale, participants rated each response for accuracy, relevance, and comprehensiveness based on clinical expectations and their professional judgment. Results: The study revealed that ChatGPT-4.0 performed with predominantly high accuracy, comprehensiveness, and relevance in tasks related to speech and language disorders. High accuracy, comprehensiveness, and relevance levels were observed in report writing, clinical decision support, and creating education material. However, tasks such as creating therapy stimuli and therapy planning showed more variation with medium and high accuracy levels. Conclusions: ChatGPT-4.0 shows promise in assisting SLPs with various professional tasks, particularly report writing, clinical decision support, and education material creation. However, further research is needed to address its limitations in therapy stimulus generation and therapy planning to improve its usability in clinical practice. Integrating AI technologies such as ChatGPT could improve the efficiency and effectiveness of therapeutic processes in speech-language pathology.
Overview
CoChat Summary
## TL;DR
This cross-sectional descriptive study asked 15 practicing speech-language pathologists to rate ChatGPT-4.0’s responses to SLP tasks across speech, language, and swallowing disorders. Overall, ChatGPT-4.0 was rated highly for accuracy, comprehensiveness, and relevance, especially for report writing and clinical decision support, but it was weaker for therapy stimulus generation and some Turkish-specific assessment tasks.
## Key findings
- Across tasks and disorder categories, ChatGPT-4.0 showed predominantly high performance: mean accuracy scores ranged from 1.60 to 2.93 on a 1–3 scale.
- In Speech Sound Disorders, accuracy differed significantly across tasks (p < 0.001): report writing (M = 2.80) and clinical decision support (M = 2.93) were highest, while therapy stimuli was lowest (M = 1.60).
- For comprehensiveness, means ranged from 2.07 to 2.93; in Speech Sound Disorders, therapy planning was highest (M = 2.73) and assessment material creation was lowest (M = 2.07) with significant differences (p < 0.001).
- Relevance was generally high, with scores ranging from 2.40 to 2.93; the paper notes lower relevance for assessment/stimuli generation in speech sound disorders.
- Inter-rater reliability for the ratings was good: ICC = 0.755, 95% CI [0.677, 0.818].
## Methods
- Design: Cross-sectional descriptive study evaluating ChatGPT-4.0’s responses to SLP-related prompts.
- Sample/data: 15 participants (4 master’s students, 11 doctoral students), all practicing clinicians; prompts covered 6 tasks across 7 disorder areas. The prompts and hypothetical cases were created by five PhD-student SLP authors through three online consensus panels.
- Main analysis: Participants rated responses on a 3-point Likert scale for accuracy, comprehensiveness, and relevance; analyses used descriptives, Friedman tests, and Bonferroni-corrected Wilcoxon tests. Inter-rater reliability was assessed with a two-way mixed-effects ICC.
## Limitations
- The paper explicitly states the sample size was relatively small, which may limit generalizability.
- It also states the sample was predominantly academic (practicing SLPs who were also graduate students), which may not represent broader clinical practice.
- The survey took about 150 minutes, and the paper says this length was a barrier to participation.
- Evident from the text: the study used only ChatGPT-4.0, only text responses, and evaluation was based on participants’ judgments rather than a specific gold-standard reference; the paper itself notes that no comparison with other AI models was conducted and that the version used had data only up to September 2021.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
16 Valid · 5 Not valid · 20 Not checked · 94 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“The results of this study indicated that although ChatGPT significantly outperformed Llama, it may have harmful consequences, such as recommending medialization for patients with stridor and shortness of breath and it was shown in the study that making treatment decisions for complex conditions such as UVFP is beyond the capabilities of ChatGPT.”
§ Is there any room for ChatGPT AI bot in speech-language pathology? > Introduction
What the sources say
- Partly supportedcites [10] Dronkers EAC, Geneid A, al Yaghchi C, Lechien JR (2024) Evaluating the potential of AI chatbots in treatment Decision-making for acquired bilateral vocal fold paralysis in adults. J Voice. https://doi.org/10.1016/j.jvoice.2024.02.020 sourceExperimental — accuracy not yet measured
The Discussion reports harmful suggestions, including vocal fold medialization for patients with stridor and dyspnea, and the Conclusion says ChatGPT outperformed Llama but was inadequate for treatment decisions. The study concerns bilateral vocal fold paralysis (BVFP), not unilateral vocal fold paralysis (UVFP), so it does not support the claim as stated for UVFP. (LLM, conf=0.90)
hedgedstates a possible causen = 20- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“In this study, ChatGPT was found to be 91.67% accurate in detecting the target word and successful in combining politeness strategies [19].”
§ Is there any room for ChatGPT AI bot in speech-language pathology? > Introduction
What the sources say
- Supportedcites [19] Purohit AK, Upadhyaya A, Holzer A (2023) Chatgpt in healthcare: Exploring ai chatbot for spontaneous word retrieval in aphasia. pp 1–5 source
The Methodology identifies AphasiaBank speech samples from people with aphasia as the study material. The Preliminary Results report correct word retrieval in 11 of 12 instances (91.67%) and describe ChatGPT’s use of multiple politeness strategies. No page numbers are available in our copy. (LLM, conf=0.94)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“The results of this study indicated that although ChatGPT significantly outperformed Llama, it may have harmful consequences, such as recommending medialization for patients with stridor and shortness of breath and it was shown in the study that making treatment decisions for complex conditions such as UVFP is beyond the capabilities of ChatGPT.”
§ Is there any room for ChatGPT AI bot in speech-language pathology? > Introduction
What the sources say
- Partly supportedcites [10] Dronkers EAC, Geneid A, al Yaghchi C, Lechien JR (2024) Evaluating the potential of AI chatbots in treatment Decision-making for acquired bilateral vocal fold paralysis in adults. J Voice. https://doi.org/10.1016/j.jvoice.2024.02.020 sourceExperimental — accuracy not yet measured
The Results report that ChatGPT significantly outperformed Llama in identifying the expert-proposed treatment (50% versus 15%). However, the Methods and Conclusion specify that the cases involved bilateral vocal fold paralysis (BVFP), not unilateral vocal fold paralysis (UVFP); the comparative finding is not established for UVFP. (LLM, conf=0.88)
plainstates a causen = 20- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“In this study, ChatGPT was found to be 91.67% accurate in detecting the target word and successful in combining politeness strategies [19].”
§ Is there any room for ChatGPT AI bot in speech-language pathology? > Introduction
What the sources say
- Supportedcites [19] Purohit AK, Upadhyaya A, Holzer A (2023) Chatgpt in healthcare: Exploring ai chatbot for spontaneous word retrieval in aphasia. pp 1–5 source
The Methodology reports that 12 circumlocution instances from eight people with aphasia were drawn from AphasiaBank speech samples and provided to ChatGPT (GPT-3.5). The Preliminary Results report correct word retrieval in 11 of 12 instances (91.67%), and the Abstract also reports successful use of politeness strategies. (LLM, conf=0.94)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“The results of this study indicated that although ChatGPT significantly outperformed Llama, it may have harmful consequences, such as recommending medialization for patients with stridor and shortness of breath and it was shown in the study that making treatment decisions for complex conditions such as UVFP is beyond the capabilities of ChatGPT.”
§ Is there any room for ChatGPT AI bot in speech-language pathology? > Introduction
What the sources say
- Partly supportedcites [10] Dronkers EAC, Geneid A, al Yaghchi C, Lechien JR (2024) Evaluating the potential of AI chatbots in treatment Decision-making for acquired bilateral vocal fold paralysis in adults. J Voice. https://doi.org/10.1016/j.jvoice.2024.02.020 sourceExperimental — accuracy not yet measured
The Conclusion says treatment decision-making for complex BVFP is beyond the tested chatbots’ expertise. The citing sentence changes BVFP (bilateral vocal fold paralysis) to UVFP (unilateral vocal fold paralysis), a different condition that this conclusion does not cover. (LLM, conf=0.88)
plainn = 20- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
54 more claims · 1 with their cited sources checked
Create a free account to see all 135 claims and to verify the 2 references still open.
- Probably not a substantial claim. 55
- It may state two separate claims as one. 38
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“differences in language, dialect, and culture affect the generalizability of AI models in the diagnostic process.”
From The role of AI in the diagnosis of speech and language disorders: A systematic mapping study (2025) · cites this paper as [103]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“ChatGPT represents a new class of tools capable of supporting speechlanguage pathologists (SLPs) in a range of professional tasks, from generating text-based materials and visual content to retrieving clinical information and simulating dialogic interaction (Birol et al. 2025;Mytsyk et al. 2025).”
From Stench of Errors or the Shine of Potential: The Challenge of (Ir)Responsible Use of ChatGPT in Speech‐Language Pathology (2025) · cites this paper as Birol et al. 2025
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Birol et al. (2025) report that ChatGPT can complement SLP methods, particularly in areas such as documentation and therapy planning”
From Stench of Errors or the Shine of Potential: The Challenge of (Ir)Responsible Use of ChatGPT in Speech‐Language Pathology (2025) · cites this paper as Birol et al. 2025
The cited source says what the claim attributes to it.
In the citing paper
“but emphasize that SLPs require a comprehensive and systematic approach to reviewing data used in practice.”
From Stench of Errors or the Shine of Potential: The Challenge of (Ir)Responsible Use of ChatGPT in Speech‐Language Pathology (2025) · cites this paper as Birol et al. 2025
The cited source says what the claim attributes to it.
In the citing paper
“Birol et al. (2025) found that SLPs rated ChatGPT-4's output as highly accurate and relevant across several domains, including report writing, clinical decision support and educational resource development.”
From Stench of Errors or the Shine of Potential: The Challenge of (Ir)Responsible Use of ChatGPT in Speech‐Language Pathology (2025) · cites this paper as Birol et al. 2025
6 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
(2019) · Cited by 986
ChatGPT - Reshaping medical education and clinical management
Khan, R. A., Jawaid, M., Khan, A. R., & Sajjad, M. (2023) · Pakistan Journal of Medical Sciences · doi:10.12669/pjms.39.2.7653 · Cited by 628 · cited 1×
“In addition, chatbots can be used for tasks such as generating responses, answering questions and writing creative stories [2].”
Johnson, D., Goodman, R., Patrinely, J., Stone, C., Zimmerman, E., Donald, R., Chang, S., Berkowitz, S., Finn, A., Jahangir, E., et al. (2023) · doi:10.21203/rs.3.rs-2566942/v1 · Cited by 598 · cited 1×
“It is thought that comprehen-siveness may have been low because ChatGPT 4.0 could in data analysis and that ChatGPT had a margin of error [29 ].”
InfluentialCapabilities of GPT-4 on Medical Challenge Problems
Nori, H., King, N., McKinney, S. M., Carignan, D., & Horvitz, E. (2023) · arXiv · doi:10.48550/arxiv.2303.13375 · Cited by 509
Sharma, A., Lin, I. W., Miner, A. S., Atkins, D. C., & Althoff, T. (2023) · Nature Machine Intelligence · doi:10.1038/s42256-022-00593-2 · Cited by 500
ChatGPT in healthcare: A taxonomy and systematic review
Li, J., Dada, A., Puladi, B., Kleesiek, J., & Egger, J. (2024) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2024.108013 · Cited by 452
ChatGPT: Is this version good for healthcare and research?
Vaishya, R., Misra, A., & Vaish, A. (2023) · Diabetes & Metabolic Syndrome: Clinical Research & Reviews · doi:10.1016/j.dsx.2023.102744 · Cited by 293
Johnson, S. B., King, A. J., Warner, E. L., Aneja, S., Kann, B. H., & Bylund, C. L. (2023) · JNCI Cancer Spectrum · doi:10.1093/jncics/pkad015 · Cited by 289 · cited 1×
“…examining Chat-GPT responses to cancer myths, which can be considered similar to the task of creating relevant educational material in our study, it was observed that ChatGPT responses were relevant and had a high level of accuracy, similar to the responses of the National Cancer Institute [32].”
Evaluating ChatGPT as an Adjunct for Radiologic Decision-Making
Rao, A., Kim, J., Kamineni, M., Pang, M., Lie, W., & Succi, M. D. (2023) · doi:10.1101/2023.02.02.23285399 · Cited by 257
The exciting potential for ChatGPT in obstetrics and gynecology
Grünebaum, A., Chervenak, J., Pollet, S. L., Katz, A., & Chervenak, F. A. (2023) · American Journal of Obstetrics and Gynecology · doi:10.1016/j.ajog.2023.03.009 · Cited by 234
Research works printed but not linked: 1
- Weismer G, Brown DK (2021) Introduction to communication sciences and disorders: the scientific basis of clinical practice. Plural Publishing, San Diego, CA
Cited by
Hanna, M., & Yana, S. (2025) · International Journal of Language & Communication Disorders · doi:10.1111/1460-6984.70088 · Cited by 10
Evaluating the role of ChatGPT in rehabilitation medicine: a narrative review
Luo, M., Duan, Z., Gao, J., Sun, Y., Chen, L., & Feng, X. (2025) · Frontiers in Digital Health · doi:10.3389/fdgth.2025.1618510 · Cited by 5
Balo, E., Ökte, B., & Selvi Balo, S. (2025) · The European Research Journal · doi:10.18621/eurj.1677704 · Cited by 6
The role of AI in the diagnosis of speech and language disorders: A systematic mapping study
Tbaishat, D., Al-Shafei, R., & Odeh, M. (2025) · DIGITAL HEALTH · doi:10.1177/20552076251379769 · Cited by 4
“103 Thus, these results are in line with our observation, in the diagnosis of SLDs, differences in language, dialect, and culture affect the generalizability of AI models in the diagnostic process.”
YAŞA, İ. C., Dölek, M., Tekin, S. E., Kaya, A. S., Akgün, P., Yılmaz, S. D., & Tokalak, S. (2026) · INQUIRY The Journal of Health Care Organization Provision and Financing · doi:10.1177/00469580261445316 · Cited by 3
“21 In another study regarding stuttering, Saeedi and Bakhtiar 25 stated that even when AI responses are correct, they are sometimes overly complex and do not always fully align with clinical observation.”
Metadata
- Authors
- Birol, N. Y., Çiftci, H. B., Yılmaz, A., Çağlayan, A., & Alkan, F.
- Affiliations
- Tarsus University (TR)
- Istanbul Medipol University (TR)
- Istinye University (TR)
- Corresponding author
- Birol, Namık Yücel
- Venue
- European Archives of Oto-Rhino-Laryngology
- Published
- 2025-03-01
- Volume
- 282
- Issue
- 6
- Pages
- 3267-3280
- ISSN
- 0937-4477 (ISSN-L)
- 1434-4726
- PMC
- PMC12122639
- OpenAlex
- W4408077353
- Cited by
- 12
- References
- 28
- Citations by year
- 2026: 6 · 2025: 6
- FWCI
- 2.86
- Type
- journal article
- Language
- en
- Update notices
- correction (2025-05-08)
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/4.0 (tdm)
- creativecommons.org/licenses/by/4.0 (vor)
- Locations
- European Archives of Oto-Rhino-Laryngology (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Funders
- Tarsus University
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence Applications
- Health Sciences › Health Professions › Speech and Hearing › Voice and Speech Disorders
- Keywords
- ChatGPT, large language models, speech-language pathology, clinical decision support, speech sound disorders, motor speech disorders, aphasia, stuttering, language disorders in children, voice disorders, dysphagia, treatment planning, clinical report writing
- MeSH terms
- Generative Artificial Intelligence, Adult, Artificial Intelligence, Cross-Sectional Studies, Female, Humans, Male, Speech Disorders, Speech-Language Pathology
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-01, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-01 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Is there any room for ChatGPT AI bot in speech-language pathology? https://app.cochat.ai/papers/10.1007/s00405-025-09295-y
- The paper
- Birol, N. Y., Çiftci, H. B., Yılmaz, A., Çağlayan, A., & Alkan, F. (2025). Is there any room for ChatGPT AI bot in speech-language pathology? European Archives of Oto-Rhino-Laryngology. https://doi.org/10.1007/s00405-025-09295-y
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.