The DOI matches its Crossref record; 129 of the 129 references it cites were found in a registry or an index; 1 of the 91 sources its claims cite were checked for support.
Large language models in medical and healthcare fields: applications, advances, and challenges
Open accessAbstract
Large language models (LLMs) are increasingly recognized for their advanced language capabilities, offering significant assistance in diverse areas like medical communication, patient data optimization, and surgical planning. Our survey meticulously searched for papers with keywords such as “medical,” “clinical,” “healthcare,” and “LLMs” across various databases, including ACM and Google Scholar. It sought to delve into the latest trends and applications of LLMs in healthcare, analyzing 175 relevant publications to support both practitioners and researchers in the field. We have compiled 56 experimental datasets, various evaluation methods and reviewed cutting-edge LLMs across tasks. Our comprehensive analysis of LLMs in healthcare applications, including medical question-answering, dialogue summarization, electronic health record generation, scientific research, medical education, medical product safety monitoring, clinical health reasoning, and clinical decision support. Furthermore, we have identified the challenges, including data security, inaccurate information, fairness and bias, plagiarism, copyrights, and accountability, and the potential solutions, namely de-identification framework, references,counterfactually fair prompting,opening and ending control codes, and establishing normative standards,to address these open issues,respectively. The findings of this survey exert a profound impact on spurring innovation in practical applications and addressing inherent challenges within the academic and medical communities.
Overview
AI snapshot
Written from the paper’s full text.
TL;DR
This survey reviews large language models in medical and healthcare fields (LLMMs), covering applications, datasets, evaluation methods, model performance, and ethical/safety challenges. It analyzes 175 relevant publications and compiles 56 experimental datasets to guide practitioners and researchers.
Key findings
- The survey organizes LLMM applications into medical question-answering, medical dialog summarization, EHR/clinical letter/note generation, scientific research, medical education and translation, medical imaging recognition, clinical health reasoning, medical product safety monitoring and disease diagnosis, clinical decision support, and administrative tasks assistance.
- It reports that GPT-4 with retrieval-augmented generation (GPT4-RAG) achieved 91.4% accuracy in preoperative guidance, 5.1% higher than human-generated answers at 86.3%, with retrieval in about 1 s and generation in 15–20 s versus human physicians’ average 10 min.
- For clinical question-answering, ChatGPT reached 87% accuracy without options and 97% in multiple-choice settings on the NEJM “Image Challenge”; GPT-4 achieved 96% accuracy in disease classification with chain-of-thought and few-shot prompts.
- The paper identifies five challenge areas: data security and privacy preservation, incorrect or misleading information, fairness and bias, transparency/explainability/trustworthiness, and plagiarism/copyright/accountability, with proposed solutions such as de-identification frameworks, references, counterfactually fair prompting, opening and ending control codes, and normative standards.
Methods
- Survey design: searched Google Scholar and multi-source databases including ACM, Springer, Elsevier, arXiv, and medRxiv for papers from January 2022 to January 2024, focusing on models after ChatGPT’s November 2022 launch and parameters exceeding 10^9.
- Search strategy: combined application domains (“medical,” “clinical,” “healthcare”) with specific LLM names such as ChatGPT, LLaMA, and PaLM; reviewed titles, abstracts, and keywords, excluded papers without specific medical/healthcare LLMs, and included articles from review papers, arriving at 175 papers.
- Analysis: categorized applications, compiled 56 experimental datasets, summarized evaluation metrics (e.g., ROUGE, BERTScore, BLEU, accuracy, precision, recall, F1-score, MCC, AUM, AKCM, ASRE), and compared state-of-the-art LLMM performance across tasks.
Limitations
- The paper states that LLMMs face data security and privacy risks, inaccurate or fabricated information, fairness and bias, limited transparency/explainability/trustworthiness, and plagiarism/copyright/accountability concerns.
- It notes that extensive training of LLMMs is typically based on English-related datasets, causing lack of medical knowledge and poor performance on Chinese tasks such as disease diagnosis, drug recommendation, and clinical decision support.
- The text also makes evident that the survey is not a systematic review with a formal PRISMA-style protocol, and several sections appear truncated or garbled, limiting reproducibility of the search and synthesis process.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
85 Not checked yet · 12 Own interpretation · 9 Own result · 6 Own method · 6 No basis given · 1 Partly supported · accuracy not yet measured
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We have compiled 56 experimental datasets, various evaluation methods and reviewed cutting-edge LLMs across tasks.”
§ Large language models in medical and healthcare fields: applications > Abstract
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“The assessments and medical plans created by GPT-4 demonstrate a significant association between demographic attributes and patient differences in recommendations for expensive procedures (Zack et al. 2023).”
§ Large language models in medical and healthcare fields: applications > 6.3 Fairness and bias
What the sources say
- Not checked yetcites Zack et al. 2023
boostedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The results reveal that both ChatGPT and GPT-4, when equipped with an elaborate clinical guideline, consistently achieved higher F1 scores across the board, as detailed in Table 7.”
§ Large language models in medical and healthcare fields: applications > BERT
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In the Multiple-choice QA task, Med-PaLM 2 achieved the top performance on the MedQA (USMLE) and PubMedQA datasets, while GPT-4 excelled on MedMCQA, MMLU-Medical Genetics, and MMLU-College Biology.”
§ Large language models in medical and healthcare fields: applications > BERT
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“For the MediQA shared task, the BART-Large model yielded the highest BLEU score, and T5 SAMSum achieved the highest F1 Score.”
§ Large language models in medical and healthcare fields: applications > BERT
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“For the MediQA shared task, the BART-Large model yielded the highest BLEU score, and T5 SAMSum achieved the highest F1 Score.”
§ Large language models in medical and healthcare fields: applications > BERT
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“For example, the performance of ChatGPT with human respondents in answering genetic questions was not significantly different from human respondents (Duong and Solomon 2023).”
§ Large language models in medical and healthcare fields: applications > 6.5 Plagiarism, copyrights, and accountability
What the sources say
- Not checked yetcites Duong and Solomon 2023
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“The accuracy of Med-PaLM 2 on the MedQA dataset was 19% higher than that of Med-PaLM, achieving better performance compared to Med-PaLM in answering medical questions.”
§ Large language models in medical and healthcare fields: applications > 6.5 Plagiarism, copyrights, and accountability
What the sources say
- Not checked yetcites Singhal et al. 2023b
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“MedPaLM (Singhal et al. 2023a) was the first LLMM model to pass the USMLE exam, and Med-PaLM 2 (Singhal et al. 2023b) based on PaLM 2 fine-tuned with medical domain knowledge, introducing a new integration method to provide a prompt strategy.”
§ Large language models in medical and healthcare fields: applications > 6.5 Plagiarism, copyrights, and accountability
What the sources say
- Not checked yetcites Singhal et al. 2023a
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“As a result, Clinical Camel achieved a higher score than GPT-3.5 on the USMLE test.”
§ Large language models in medical and healthcare fields: applications > 6.5 Plagiarism, copyrights, and accountability
What the sources say
- Not checked yetcites Toma et al. 2023
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Probably not a substantial claim. 33
- It may state two separate claims as one. 45
- The quote only partly states this claim. 5
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“This improvement facilitates a deeper understanding of the internal structure and function of various organisms (e.g., (Tao et al. 2020)).”
§ Large language models in medical and healthcare fields: applications > 2.6 Medical imaging recognition and analysis
What the sources say
- Partly supportedcites Tao et al. 2020 sourceExperimental — accuracy not yet measured
The Discussion (page unavailable) says the improved PRID/GSLA system helps researchers interpret functional impacts and molecular mechanisms in rats. That supports a deeper understanding of function, but the paper does not establish an improvement in understanding internal structure or extend this result to various organisms. (LLM, conf=0.82)
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
- Partly supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Furthermore, we have identified the challenges, including data security, inaccurate information, fairness and bias, plagiarism, copyrights, and accountability, and the potential solutions, namely de-identification framework, references,counterfactually fair prompting,opening and ending control codes, and establishing normative standards,to address these open issues,respectively.”
§ Large language models in medical and healthcare fields: applications > Abstract
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Furthermore, we have identified the challenges, including data security, inaccurate information, fairness and bias, plagiarism, copyrights, and accountability, and the potential solutions, namely de-identification framework, references,counterfactually fair prompting,opening and ending control codes, and establishing normative standards,to address these open issues,respectively.”
§ Large language models in medical and healthcare fields: applications > Abstract
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We found that this approach yielded a significant number of survey and overview articles, with only a few papers containing relevant models.”
§ Large language models in medical and healthcare fields: applications > 1.2 Methodology for collecting relevant publications
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Although scholars and medical practitioners have increasingly expressed interest in the application of LLMMs, the practical utility of LLMMs in clinical and research settings is fraught with distinct challenges.These challenges include data security, privacy preservation, the risk of inaccurate information, fairness and bias issues, plagiarism concerns, copyright considerations, and accountability”
§ Large language models in medical and healthcare fields: applications > 1 Introduction
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“These datasets encompass a range of tasks, including medical question-answering, medical knowledge representation, clinical evidence understanding and integration, diagnosis generation and summarization, and others.”
§ Large language models in medical and healthcare fields: applications > 3 Available experimental datasets of LLMMs
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“For instance, GPT-4 was found not to simulate the demographics of medical conditions in various situations, consistently producing clinical hallucinations, including the differential diagnosis of standardized clinical samples, which are more likely to include stereotypes of specific racial, ethnic, and gender identities.”
§ Large language models in medical and healthcare fields: applications > 6.3 Fairness and bias
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds. It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In the Multiple-choice QA task, Med-PaLM 2 achieved the top performance on the MedQA (USMLE) and PubMedQA datasets, while GPT-4 excelled on MedMCQA, MMLU-Medical Genetics, and MMLU-College Biology.”
§ Large language models in medical and healthcare fields: applications > BERT
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“This paper presents 56 experimental datasets that are most widely used by researchers in the medical and healthcare domains.”
§ Large language models in medical and healthcare fields: applications > 3 Available experimental datasets of LLMMs
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
In the paper
“The results indicated that GPT-4 achieved a score exceeding the passing threshold on the USMLE by more than 20 points without any professional hints, outperforming GPT-3.5 and specialized medical knowledge models such as Med-PaLM and Flan-PaLM.”
§ Large language models in medical and healthcare fields: applications > 6.5 Plagiarism, copyrights, and accountability
What the sources say
- Not checked yetcites Nori et al. 2023
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“LLMs such as GPT-4 and Med-PaLM have demonstrated the ability to answer questions in the USMLE clinical knowledge test with an accuracy of over 80%.”
§ Large language models in medical and healthcare fields: applications > 6.5 Plagiarism, copyrights, and accountability
boosted- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
In the paper
“ChatGPT achieved an accuracy level of approximately 60% without any specialized training.”
§ Large language models in medical and healthcare fields: applications > 6.5 Plagiarism, copyrights, and accountability
What the sources say
- Not checked yetcites Kung et al. 2023 source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The quote only partly states this claim.
None of the sources cited for this claim have been checked yet.
In the paper
“Their findings suggested that scaling inference-time computing can enhance the reasoning performance of LLMs.”
§ Large language models in medical and healthcare fields: applications > 6.5 Plagiarism, copyrights, and accountability
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“All the papers retrieved from the above searches were included in the candidate corpus.”
§ Large language models in medical and healthcare fields: applications > 1.2 Methodology for collecting relevant publications
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
In the paper
“In the realm of medical education, LLMMs have exhibited remarkable performance, for instance, in assisting human learners and educators with United States Medical Licensing Examination (USMLE), Japanese Medical Licensing Examination (JMLE), and other medical licensing examinations, as well as in generating and evaluating multiple-choice tests (Sallam 2023b).”
§ Large language models in medical and healthcare fields: applications > 1 Introduction
What the sources say
- Not checked yetcites Sallam 2023b
boosteddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“The deployment of LLMs offers valuable insights for various stakeholders in healthcare domains, such as healthcare providers and patients (Jin and Dobry 2023)).”
§ Large language models in medical and healthcare fields: applications > 2 Application scenarios of state-of-the-art LLMMs
What the sources say
- Not checked yetcites Jin and Dobry 2023 source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“Within the medical and healthcare domains, GPT-4 can supply the most current literature in specific fields, draft discharge summaries for patients post-surgery, analyze medical image characteristics, and identify objects in photographs, revealing its significant potential in clinical trials (Waisberg et al. 2023).”
§ Large language models in medical and healthcare fields: applications > 2.3 Electronic health records (EHRs), clinical letters and medical note generation
What the sources say
- Not checked yetcites Waisberg et al. 2023
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“We then reviewed the titles, abstracts, and keywords of the candidate papers; those that did not include specific large models in the medical or healthcare fields were excluded.”
§ Large language models in medical and healthcare fields: applications > 1.2 Methodology for collecting relevant publications
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
In the paper
“The MDS aids clinicians in identifying potential health risks for patients and supports informed decision-making (Patel and Lam 2023).”
§ Large language models in medical and healthcare fields: applications > 2.2 Medical dialog summarization (MDS)
What the sources say
- Not checked yetcites Patel and Lam 2023 source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“Agrawal et al. (2022) utilized GPT-based models to extract critical variables from diverse clinical notes, demonstrating that GPT-3 outperforms other models in clinical natural language processing tasks.”
§ Large language models in medical and healthcare fields: applications > 2.2 Medical dialog summarization (MDS)
What the sources say
- Not checked yetcites Agrawal et al. 2022
boosted- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“The focus of this paper is on research work from January 2022 up until the submission of this paper in January 2024, with a particular emphasis on models introduced after the launch of ChatGPT in November 2022, as well as those with parameters exceeding 10^9.”
§ Large language models in medical and healthcare fields: applications > 1.2 Methodology for collecting relevant publications
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“Nonetheless, these studies often concentrate on a limited number of LLMs or fail to offer a thorough and expansive analysis of LLM applications and the associated potential issues, such as medical dialog summarization, scientific research, medical product safety monitoring, disease diagnosis, clinical decision support, administrative tasks assistance, and ethical concerns regarding data security”
§ Large language models in medical and healthcare fields: applications > 1.1 Distinctions of this survey from prior research
What the sources say
- Not checked yetcites Wang et al. 2023a
- Not checked yetcites Tian et al. 2023
- Not checked yetcites Liu et al. 2023a
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“In scientific research, LLMs can serve as powerful tools for data analysis (Tao et al. 2022), literature review, and hypothesis generation.”
§ Large language models in medical and healthcare fields: applications > 2.4 Scientific research
What the sources say
- Not checked yetcites Tao et al. 2022 source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Additionally, to ensure a more comprehensive coverage of research content, we also included articles discussing LLMs in the healthcare domain from multiple review papers in the candidate set, ultimately arriving at 175 papers closely related to our research.”
§ Large language models in medical and healthcare fields: applications > 1.2 Methodology for collecting relevant publications
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“Clinical notes summarization assists healthcare practitioners in identifying potential health risks within patients' electronic health records (Wornow et al. 2023), thereby reducing errors and facilitating informed decision-making.”
§ Large language models in medical and healthcare fields: applications > 2.2 Medical dialog summarization (MDS)
What the sources say
- Not checked yetcites Wornow et al. 2023 source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“To our understanding, our survey is the first to focus on LLMMs in relation to realworld application scenarios, available datasets, evaluation methods, and ethical and safety considerations.”
§ Large language models in medical and healthcare fields: applications > 1.2 Methodology for collecting relevant publications
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds.
In the paper
“LLMMs can also be trained to recognize and analyze medical images (e.g., X-rays, magnetic resonance imaging, and ultrasound), video, audio and remote photoplethysmograph signals to identify features and structures (Fan et al. 2024), aiding doctors in accurately and rapidly detecting anomalies and diagnosing diseases or injuries, thereby alleviating the workload of radiologists (Waisberg et al. 202”
§ Large language models in medical and healthcare fields: applications > 1 Introduction
What the sources say
- Not checked yetcites Waisberg et al. 2023
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“This study was conducted by searching for relevant papers on the renowned digital library Google Scholar, which encompasses literature from ACM, Springer, Elsevier, arXiv, medRxiv, and other multi-source databases.”
§ Large language models in medical and healthcare fields: applications > 1.2 Methodology for collecting relevant publications
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“Large language models in the medical and healthcare domain (LLMMs) have the potential to facilitate communication among healthcare professionals, patients, and their families, streamline the collection and analysis of patient health data, and assist in the development of surgical plans (Zhao et al. 2023).”
§ Large language models in medical and healthcare fields: applications > 1 Introduction
What the sources say
- Not checked yetcites Zhao et al. 2023 source
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“Without options, ChatGPT demonstrated an accuracy rate of 87%, while in multiple-choice scenarios, its accuracy reached 97%.”
§ Large language models in medical and healthcare fields: applications > 2.5 Medical education and language translation
What the sources say
- Not checked yetcites Ueda et al. 2023
boosted- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one. The quote only partly states this claim.
In the paper
“The results revealed that GPT-4 outperformed ChatGPT in general clinical questions, complex questions, and specific disease-related queries.”
§ Large language models in medical and healthcare fields: applications > 2.5 Medical education and language translation
What the sources say
- Not checked yetcites Takagi et al. 2023 source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“Furthermore, GPT-4 achieved a score that met the passing standard of the JMLE, demonstrating its robust reliability in clinical reasoning and medical knowledge within the Japanese context.”
§ Large language models in medical and healthcare fields: applications > 2.5 Medical education and language translation
What the sources say
- Not checked yetcites Takagi et al. 2023 source
boosted- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“(2) LLMs have become indispensable tools in scientific writing, draft generation, article summarization, language and grammar checks, and translation of multilingual content (Liebrenz et al. 2023), engaging in discussions as virtual collaborators, and offering new perspectives and research directions, thereby enhancing the efficiency and diversity of scientific and academic output (Fatani 2023).”
§ Large language models in medical and healthcare fields: applications > 2.4 Scientific research
What the sources say
- Not checked yetcites Liebrenz et al. 2023 source
- Not checked yetcites Fatani 2023
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“The study found that ChatGPT significantly outperformed GPT-3 and InstructGPT on medical question-answering tasks, with its answer level comparable to that of third-year medical students.”
§ Large language models in medical and healthcare fields: applications > 2.5 Medical education and language translation
What the sources say
- Not checked yetcites Gilson et al. 2023
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“The remarkable performance of ChatGPT on USMLE has been a significant milestone in medical education (Sallam 2023b).”
§ Large language models in medical and healthcare fields: applications > 2.5 Medical education and language translation
What the sources say
- Not checked yetcites Sallam 2023b
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“GPT-3 yielded superior diagnostic outcomes compared to laypersons without domain-specific expertise, although its performance fell short of that of professional physicians.”
§ Large language models in medical and healthcare fields: applications > 2.8 Medical product safety monitoring and disease diagnosis
What the sources say
- Not checked yetcites Levine et al. (2023)
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“GPT-4 exhibited higher scores in terms of accuracy and completeness.”
§ Large language models in medical and healthcare fields: applications > 2.5 Medical education and language translation
What the sources say
- Not checked yetcites Li et al. 2023b
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The quote only partly states this claim.
In the paper
“(1) LLMs present an exciting opportunity for researchers to streamline their research and craft influential articles by facilitating literature reviews (Chen and Li 2023), retrieving and discovering the latest scientific progress, automatically searching for academic papers based on the needs of a given field and retrieving key information tailored to the requirements of different journals.”
§ Large language models in medical and healthcare fields: applications > 2.4 Scientific research
What the sources say
- Not checked yetcites Chen and Li 2023
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“LLMs have shown remarkable proficiency in tasks involving clinical health reasoning, realworld medical question-answering, and diagnostic reasoning.”
§ Large language models in medical and healthcare fields: applications > 2.7 Clinical health reasoning and diagnostic reasoning
What the sources say
- Not checked yetcites Feng et al. (2022)
- Not checked yetcites Liévin et al. (2022
boosted- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“(4) Researchers are able to gain valuable feedback and suggestions for improvement by submitting drafts or manuscripts, a practice that is especially advantageous for academic researchers who operate independently and do not have regular access to the peer review process.”
§ Large language models in medical and healthcare fields: applications > 2.4 Scientific research
What the sources say
- Not checked yetcites Castellanos-Gomez 2023 source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“The advanced capabilities of GPT-4 present a transformative opportunity to enhance doctor-patient communication, fostering a better understanding of patients' needs, anxieties, and expectations, thereby improving the overall medical experience (Nashwan et al. 2023).”
§ Large language models in medical and healthcare fields: applications > 2.9 Clinical decision support and administrative tasks assistance
What the sources say
- Not checked yetcites Nashwan et al. 2023 source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
In the paper
“GPT-4 exhibited sufficient performance to pass the JNNE, surpassing ChatGPT, which suggests that GPT-4 is suitable for specialized medical training in the Japanese clinical setting.”
§ Large language models in medical and healthcare fields: applications > 2.5 Medical education and language translation
What the sources say
- Not checked yetcites Kaneda et al. 2023
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“The LLMs have been utilized in diverse contexts within medical education and language translation.These include applications in licensing examinations such as the USMLE and the JMLE, the generation of multiple-choice questions, the evaluation of medical tests, educational initiatives in rehabilitation, and pharmacogenomics, as well as the translation of complex medical imaging reports into layman”
§ Large language models in medical and healthcare fields: applications > 2.5 Medical education and language translation
What the sources say
- Not checked yetcites Omran et al. 2023 source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“These models can assist experimental design by providing valuable predictions, suggestions, summaries and interpretations of experimental results, thus enhancing the efficiency of the research process (Huang and Tan 2023).”
§ Large language models in medical and healthcare fields: applications > 2.4 Scientific research
What the sources say
- Not checked yetcites Huang and Tan 2023
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“This interdisciplinary approach has the potential to revolutionize various fields, including media analysis, marketing, and human-computer interaction (Zhang et al. 2023c).”
§ Large language models in medical and healthcare fields: applications > 2.4 Scientific research
What the sources say
- Not checked yetcites Zhang et al. 2023c
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“Rehabilitation education plays a vital role in the field of Physical Medicine and Rehabilitation (Peng et al. 2023).”
§ Large language models in medical and healthcare fields: applications > 2.5 Medical education and language translation
What the sources say
- Not checked yetcites Peng et al. 2023
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“The synergy between LLMs and these recognition technologies empowers systems to understand and process multiple forms of data, allowing the development of sophisticated multimodal sentiment analysis techniques (Zhang et al. 2023b).”
§ Large language models in medical and healthcare fields: applications > 2.4 Scientific research
What the sources say
- Not checked yetcites Zhang et al. 2023b
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“Some researchers also measure model performance using Medical Concept Coverage (Chintagunta et al. 2021) to test the importance and negations.”
§ Large language models in medical and healthcare fields: applications > 4 Evaluation metrics
What the sources say
- Not checked yetcites Chintagunta et al. 2021
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Commonly used metrics include ROUGE, BERTScore, BLEU scores, accuracy, precision, recall, and F1-score for precision evaluation tasks.”
§ Large language models in medical and healthcare fields: applications > 4 Evaluation metrics
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“However, the generation and training of high-quality LLMMs pose significant challenges, necessitating substantial hardware support due to the massive resource consumption and prolonged training times.”
§ Large language models in medical and healthcare fields: applications > 2.11 Summarization of state-of-the-art LLMMs application scenarios
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
In the paper
“However, this training may inadvertently integrate misinformation and biased content, potentially leading to significant drawbacks such as the generation of incorrect or fabricated information (Reddy 2023).”
§ Large language models in medical and healthcare fields: applications > 6.2 Incorrectness and risk of inaccurate information
What the sources say
- Not checked yetcites Reddy 2023 source
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“The application of LLMs is currently grappling with a crisis of trustworthiness (Liu et al. 2023d).”
§ Large language models in medical and healthcare fields: applications > 6.4 Transparency, explainability, and trustworthiness
What the sources say
- Not checked yetcites Liu et al. 2023d
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“He et al. (He et al. 2023) conducted a systematic investigation into the use of GPT-4 in lumbar disc herniation surgery and found that GPT-4 can significantly support spinal surgeons in diagnosing conditions, managing the perioperative period, conducting scientific research, and enhancing communication with patients, as well as in planning and executing surgical procedures.”
§ Large language models in medical and healthcare fields: applications > 2.9 Clinical decision support and administrative tasks assistance
What the sources say
- Not checked yetcites He et al. 2023
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“By simply concatenating the CFP with the original input prompt, sensitive information in user token embeddings can be eliminated, achieving fairness across a set of sensitive attributes without the need to retrain the entire base model.”
§ Large language models in medical and healthcare fields: applications > 6.3 Fairness and bias
What the sources say
- Not checked yetcites Lin et al. 2023
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“LLMMs have achieved significant advancements in various application scenarios, the implementation of LLMMs provides critical insights for various parties within healthcare providers and patients.”
§ Large language models in medical and healthcare fields: applications > 2.11 Summarization of state-of-the-art LLMMs application scenarios
What the sources say
- Not checked yetcites Lee et al. 2023 source
boosteddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“The GPT4-RAG model retrieved information in an average of just 1 s and generated results in an average of 15-20 s, while human physicians took an average of 10 min to produce preoperative instructions.”
§ Large language models in medical and healthcare fields: applications > 2.10 Case studies of LLMMs
What the sources say
- Not checked yetcites Ke et al. 2024
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This can lead to biases in the mechanistic models of health and disease towards understanding this process in high-income countries.”
§ Large language models in medical and healthcare fields: applications > 6.3 Fairness and bias
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The quote only partly states this claim.
- Rests on an assumption: The training data of LLMs typically comes from institutions in high-income, English-speaking countries, which may severely limit the representativeness of viewpoints from other regions of the world.
In the paper
“The average number of unsafe matches (Roy et al. 2023) offers a way to measure the effectiveness of the harm or severe consequences of the generated questions.”
§ Large language models in medical and healthcare fields: applications > 4 Evaluation metrics
What the sources say
- Not checked yetcites Roy et al. 2023 source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“Additionally, Li et al. (Li et al. 2023a) released the largest Chinese medical QA dataset, Huatuo-26 M, and Yang et al. (Yang et al. 2023a) pre-trained on this dataset.”
§ Large language models in medical and healthcare fields: applications > BERT
What the sources say
- Not checked yetcites Li et al. 2023a
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Moreover, the complexity of large-scale model architectures has heightened the difficulty in understanding and interpreting these models, particularly within the medical domain where incorrect predictions or biased recommendations could result in substantial harm to patients.”
§ Large language models in medical and healthcare fields: applications > 2.11 Summarization of state-of-the-art LLMMs application scenarios
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds. It may state two separate claims as one.
In the paper
“These strategies enhance the transparency and interpretability of both the LLM's training process and the generated text, thereby reducing errors in medical practitioners and bolstering the credibility of the strategy (Reddy 2023).”
§ Large language models in medical and healthcare fields: applications > 6.4 Transparency, explainability, and trustworthiness
What the sources say
- Not checked yetcites Reddy 2023 source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“The research found that the model augmented by GPT-4 with RAG technology was the most accurate, with the GPT4-RAG model achieving a performance of 91.4%, which is 5.1% higher than the human-generated answers at 86.3%.”
§ Large language models in medical and healthcare fields: applications > 2.10 Case studies of LLMMs
What the sources say
- Not checked yetcites Ke et al. 2024
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“LLMs exhibit considerable potential in executing a diverse range of tasks that typically require human capabilities, having been trained on extensive internet data (Harrer 2023).”
§ Large language models in medical and healthcare fields: applications > 6.2 Incorrectness and risk of inaccurate information
What the sources say
- Not checked yetcites Harrer 2023 source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
In the paper
“The findings revealed that GPT-4 achieved an accuracy rate of 96% in disease classification tasks with a thinking chain and few-shot prompts, and it could be corrected three times for four diagnostic tests.”
§ Large language models in medical and healthcare fields: applications > 2.9 Clinical decision support and administrative tasks assistance
What the sources say
- Not checked yetcites Zhang et al. 2023d
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This demonstrates the feasibility of the GPT4-RAG model in the specialized field of healthcare.”
§ Large language models in medical and healthcare fields: applications > 2.10 Case studies of LLMMs
boosteddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one. The quote only partly states this claim.
None of the sources cited for this claim have been checked yet.
In the paper
“Due to their training on a vast array of internet content, LLMs may inadvertently incorporate biases (Arora and Arora 2023) (e.g., gender bias, racial bias, geopolitical biases, religious bias, nationality bias, sexual orientation bias, and age bias, etc.) across the web, posing severe threats in sensitive fields (Korngiebel and Mooney 2021).”
§ Large language models in medical and healthcare fields: applications > 6.3 Fairness and bias
What the sources say
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The findings indicate that LLMs can furnish safe and dependable responses but may not fully address the specific information requirements of a given query.”
§ Large language models in medical and healthcare fields: applications > 2.9 Clinical decision support and administrative tasks assistance
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“The digitization of healthcare facilitates the sharing and repurposing of medical data, yet it also increases the risk of critical patient information being compromised (Liu et al. 2023c).”
§ Large language models in medical and healthcare fields: applications > 6.1 Data security and privacy-preserving
What the sources say
- Not checked yetcites Liu et al. 2023c
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“The latest achievements have witnessed a technological leap in Chinese question-answering systems, such as Doctor-GLM (Xiong et al. 2023), Zhongjing (Yang et al. 2023a), and Huatuo (Li et al. 2023a)”
§ Large language models in medical and healthcare fields: applications > 2.11 Summarization of state-of-the-art LLMMs application scenarios
What the sources say
- Not checked yetcites Xiong et al. 2023
- Not checked yetcites Yang et al. 2023a
- Not checked yetcites Li et al. 2023a
boosted- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“Given that LLMs retain and train on the information provided, the generated text introduces the potential for plagiarism, which can be illegal and threaten the integrity and copyright of the publication (Nashwan et al. 2023).”
§ Large language models in medical and healthcare fields: applications > 6.5 Plagiarism, copyrights, and accountability
What the sources say
- Not checked yetcites Nashwan et al. 2023 source
plainconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“Following adjustments to the output, ChatGPT generated persuasive but unattributed content, complicating the task of identifying plagiarism or original creation.”
§ Large language models in medical and healthcare fields: applications > 6.5 Plagiarism, copyrights, and accountability
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
In the paper
“A New York Times report (Zhang et al. 2023f) indicates that ChatGPT provided conspiracy theories and misleading responses based on researchers' queries.”
§ Large language models in medical and healthcare fields: applications > 6.5 Plagiarism, copyrights, and accountability
What the sources say
- Not checked yetcites Zhang et al. 2023f
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper relies on this without support.
Advanced LLMs do not perform well in Chinese medical question-answering systems because most LLMs are trained on English corpora.
“However, since most LLMs are trained and learned from English corpora, advanced LLMs do not perform well in Chinese medical question-answering systems.”
The paper relies on this without support.
GPT-4 exhibits superior problem-solving capabilities and an expansive knowledge base compared to ChatGPT.
“In comparison to ChatGPT, GPT-4 exhibits superior problem-solving capabilities and an expansive knowledge base.”
The paper relies on this without support.
Models such as ChatGPT and GPT-4 can serve as virtual educational companions in rehabilitation.
“Models such as ChatGPT and GPT-4 can serve as virtual educational companions in rehabilitation.”
The paper relies on this without support.
The LLMs have been trained to recognize and analyze medical images, including x-rays, MRI, and ultrasound.
“The LLMs have been trained to recognize and analyze medical images, including x-rays, magnetic resonance imaging (MRI), and ultrasound.”
The paper relies on this without support.
These models can interpret features and structures within images, assisting physicians in accurately and rapidly identifying abnormalities, diagnosing diseases, and injuries.
“These models can interpret features and structures within images, assisting physicians in accurately and rapidly identifying abnormalities, diagnosing diseases, and injuries.”
The paper relies on this without support.
The integration of LLMs can enhance radiologists' interpretive skills, facilitate communication between physicians and patients, and streamline workflow in clinical settings, particularly in hospitals.
“The integration of LLMs can enhance radiologists' interpretive skills, facilitate communication between physicians and patients, and streamline workflow in clinical settings, particularly in hospitals.”
The paper relies on this without support.
For medical device manufacturers, LLMs can serve as a valuable tool for analyzing user feedback and technical documents, providing insights that inform device development.
“For medical device manufacturers, LLMs can serve as a valuable tool for analyzing user feedback and technical documents, providing insights that inform device development.”
The paper relies on this without support.
For health insurance companies and providers, LLMs can process and analyze large datasets to identify potential fraud patterns and anomalies, offering tools for insurers to prevent fraud.
“For health insurance companies and providers, LLMs can process and analyze large datasets to identify potential fraud patterns and anomalies, offering tools for insurers to prevent fraud.”
The paper relies on this without support.
LLMs can address policyholders' queries, provide personalized recommendations to enhance customer experience, and ensure delivery of accurate and valuable information.
“Additionally, LLMs can address policyholders' queries, provide personalized recommendations to enhance customer experience, and ensure the delivery of accurate and valuable information.”
The paper relies on this without support.
For regulatory bodies, LLMs can strengthen the regulatory review process and assist in detailed scrutiny of medical product submissions.
“For regulatory bodies, LLMs can strengthen the regulatory review process and assist in the detailed scrutiny of medical product submissions.”
The paper relies on this without support.
Public health authorities can utilize the analytical capabilities of these models to analyze health data, identify disease trends and patterns, and significantly enhance disease surveillance, informing disease control and prevention strategies.
“Public health authorities can utilize the analytical capabilities of these models to analyze health data, identify disease trends and patterns, and significantly enhance disease surveillance, informing disease control and prevention strategies.”
The paper relies on this without support.
Due to the constrained scope and diversity of clinical trials for novel pharmaceuticals, comprehensive pre-market safety and efficacy assessments are often unattainable.
“Due to the constrained scope and diversity of clinical trials for novel pharmaceuticals, comprehensive pre-market safety and efficacy assessments are often unattainable.”
The paper relies on this without support.
Virtual mental health assistants are not suitable for use as diagnostic tools because they lack the ability to adhere to essential safety constraints and the professional clinical process knowledge required for accurate diagnosis.
“However, these assistants are not suitable for use as diagnostic tools because they lack the ability to adhere to essential safety constraints and the professional clinical process knowledge required for accurate diagnosis.”
The paper relies on this without support.
Despite growing interest among scholars and medical professionals in leveraging LLMs in healthcare, the examination and appraisal of their practical application and safety in clinical contexts remain limited.
“Despite growing interest among scholars and medical professionals in leveraging LLMs in healthcare, the examination and appraisal of their practical application and safety in clinical contexts remain limited.”
The paper relies on this without support.
There is currently a scarcity of data employing large-scale electronic health records (EHR) to assess the performance of LLMs and their utility in providing clinical diagnostic assistance to patients.
“Despite the exemplary performance of ChatGPT and GPT-4 models in various medical tasks, there is currently a scarcity of data employing large-scale electronic health records (EHR) to assess the performance of LLMs and their utility in providing clinical diagnostic assistance to patients.”
The paper relies on this without support.
The extensive training of LLMMs is typically based on English-related datasets, resulting in a lack of medical knowledge, which can lead to poor performance in tasks such as disease diagnosis, drug recommendation, and clinical decision support.
“However, the extensive training of LLMMs is typically based on English-related datasets, resulting in a lack of medical knowledge, which can lead to poor performance in tasks such as disease diagnosis, drug recommendation, and clinical decision support.”
The paper relies on this without support.
Existing medical datasets based on English corpora present challenges for conducting accurate experimental analyses of LLMMs on Chinese tasks.
“Existing medical datasets based on English corpora present challenges for conducting accurate experimental analyses of LLMMs on Chinese tasks.”
The paper relies on this without support.
Medical reports may inadvertently reveal private and demographic details of patient records.
“Medical reports may inadvertently reveal private and demographic details of patient records.”
The paper relies on this without support.
The training data of LLMs typically comes from institutions in high-income, English-speaking countries, which may severely limit the representativeness of viewpoints from other regions of the world.
“The training data of LLMs typically comes from institutions in high-income, English-speaking countries, which may severely limit the representativeness of viewpoints from other regions of the world.”
The paper relies on this without support.
The discrepancy between the model's output and the diagnoses of seasoned medical professionals can engender structural bias and inequitable treatment.
“The discrepancy between the model's output and the diagnoses of seasoned medical professionals can engender structural bias and inequitable treatment.”
The paper relies on this without support.
LLMs suffer from a lack of transparency, hindering their efficiency in assisting humans with complex tasks.
“Despite their impressive potential in performing various simple tasks, LLMs suffer from a lack of transparency, hindering their efficiency in assisting humans with complex tasks.”
The paper relies on this without support.
A significant challenge for healthcare practitioners is the absence of guidelines for assessing whether LLM outputs align with social norms and regulations.
“A significant challenge for healthcare practitioners is the absence of guidelines for assessing whether LLM outputs align with social norms and regulations.”
The paper relies on this without support.
As LLMs become more advanced, they are increasingly susceptible to generating harmful or inappropriate content, such as hallucinations, spam, sexist, and racist hate speech.
“As LLMs become more advanced, they are increasingly susceptible to generating harmful or inappropriate content, such as hallucinations, spam, sexist, and racist hate speech.”
The paper relies on this without support.
These models may also produce responses that sound plausible yet are incorrect or absurd.
“These models may also produce responses that sound plausible yet are incorrect or absurd.”
The paper relies on this without support.
The Health Insurance Portability and Accountability Act (HIPAA) mandates patient confidentiality and privacy, stipulating that medical records must be sanitized of sensitive information before dissemination.
“The Health Insurance Portability and Accountability Act (HIPAA) mandates patient confidentiality and privacy, stipulating that medical records must be sanitized of sensitive information before dissemination.”
The paper relies on this without support.
While rule-based and machine learning-based deidentification methods have been extensively implemented in practice, they remain limited in their versatility and effectiveness across diverse scenarios.
“While rule-based and machine learning-based deidentification methods have been extensively implemented in practice, they remain limited in their versatility and effectiveness across diverse scenarios.”
The paper relies on this without support.
Recent research has revealed a strong correlation between job opportunities and male job seekers, a correlation between negative emotions and the black race, and a correlation between positive emotions and the Asian race.
“Recent research has revealed a strong correlation between job opportunities and male job seekers, a correlation between negative emotions and the black race, and a correlation between positive emotions and the Asian race.”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Notwithstanding existing reliability and inaccuracy concerns, it has been argued that LLMs' ability to write in engaging and accessible ways will make dense medical language or medical services more accessible to the general public (78–80).”
From The benefits and dangers of anthropomorphic conversational agents (2025) · cites this paper as [78]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Large language models in medical and healthcare fields: applications > 6 Challenges and future directions
“Fine-tuned T5 1.3 AKCM QG-LSTM 1.12 QG-transformer 1.27”
§ Large language models in medical and healthcare fields: applications > BERT
“Average number of knowledge context matches (AKCM) AKCM(x, K) = |K∩t(x)| |K∪t(x)| AKCM is used to measure whether the generated questions are interpretable.K is the knowledge base concept set mapped to x”
Citations
Cites
Lewis, M., Liu, Y., Goyal, N., Ghazvininejad, M., Mohamed, A., Levy, O., Stoyanov, V., & Zettlemoyer, L. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.703 · Cited by 8,821
BioBERT: a pre-trained biomedical language representation model for biomedical text mining
Lee, J., Yoon, W., Kim, S., Kim, D., Kim, S., So, C. H., & Kang, J. (2020) · Bioinformatics · doi:10.1093/bioinformatics/btz682 · Cited by 7,218
A Survey on Evaluation of Large Language Models
Chang, Y., Wang, X., Wang, J., Wu, Y., Yang, L., Zhu, K., Chen, H., Yi, X., Wang, C., Wang, Y., et al. (2024) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3641289 · Cited by 4,254 · cited 1×
“Recent advancements in large language models (LLMs) have garnered significant interest in both academic and industrial domains due to their impressive success in language understanding and text generation (Chang et al. 2023).”
Kung, T. H., Cheatham, M., Medenilla, A., Sillos, C., De Leon, L., Elepaño, C., Madriaga, M., Aggabao, R., Diaz-Candido, G., Maningo, J., et al. (2023) · PLOS Digital Health · doi:10.1371/journal.pdig.0000198 · Cited by 4,115
Large language models encode clinical knowledge
Singhal, K., Azizi, S., Tu, T., Mahdavi, S. S., Wei, J., Chung, H. W., Scales, N., Tanwani, A., Cole-Lewis, H., Pfohl, S., et al. (2023) · Nature · doi:10.1038/s41586-023-06291-2 · Cited by 4,083
Large Language Models Encode Clinical Knowledge
Singhal, K., Azizi, S., Tu, T., Mahdavi, S. S., Wei, J., Chung, H. W., Scales, N., Tanwani, A., Cole-Lewis, H., Pfohl, S., et al. (2022) · arXiv · doi:10.48550/arxiv.2212.13138 · Cited by 4,083
Sallam, M. (2023) · Healthcare · doi:10.3390/healthcare11060887 · Cited by 3,126 · cited 1×
“The extraordinary capabilities of contemporary LLMs hold great promise for applications in the medical and health-care domain, surpassing the performance of smaller models with limited data (Sallam 2023a).”
Sallam, M. (2023) · doi:10.1101/2023.02.19.23286155 · Cited by 3,123 · cited 5×
“For instance, (Wang et al. 2023a) examined the clinical language understanding capabilities of LLMs in healthcare, Tian et al. (Tian et al. 2023) explored the potential and challenges of ChatGPT in the biomedical and health sectors, Liu et al. (Liu et al. 2023a) provided an examination of ChatGPT and GPT-4 across dive…”
InfluentialA Survey of Large Language Models
Zhao, W. X., Zhou, K., Li, J., Tang, T., Dong, Z., Hou, Y., Zhang, B., Min, Y., Zhang, J., Liu, P., et al. (2026) · Frontiers of Computer Science · doi:10.1007/s11704-026-60308-3 · Cited by 2,093 · cited 2×
“The specific names of the large language models were referenced from (Zhao et al. 2023), (Hadi et al. 2023), and others.”
A Survey of Large Language Models
Zhao, W. X., Zhou, K., Li, J., Tang, T., Wang, X., Hou, Y., Min, Y., Zhang, B., Zhang, J., Dong, Z., et al. (2023) · arXiv · doi:10.48550/arxiv.2303.18223 · Cited by 2,090
Research works printed but not linked: 54
- Arasu N et al. (2023) The survey on GPT-3 driven NLP approach for automatic medical documentation. AIP Conf Proc 10(1063/5):0152503
- Atallah S et al. (2023) How large language models including generative pre-trained transformer (GPT) 3 and 4 will impact medicine and surgery. Tech Coloproctol 23:1–6
- Chen M, Li G (2023) ChatGPT for mechanobiology and medicine: a perspective. Mech Biol Med 1(1):100005
- Chen S et al. (2020) Meddialog: a large-scale medical dialogue dataset. Preprint at https://arxiv.org/abs/3:03329
- da Mota SLA et al. (2023) Can GPT-4 be a viable alternative for discussing complex cases in digital oral radiology? a critical analysis. Excli J 22:749–751
- Davydova V, Tutubalina E (2022) Smm4h 2022 task 2: Dataset for stance and premise detection in tweets about health mandates related to covid-19. In: Proceedings of The seventh workshop on social media mining for health applications, workshop & shared task, pp 216–220
- Fatani B (2023) ChatGPT for future medical and dental research. Cureus 15(4):1–5
- Feng SY et al. (2022) CHARD: Clinical health-aware reasoning across dimensions for text generation models. Preprint at https://arxiv.org/abs/04191
- Fleming S L et al. (2023) Assessing the Potential of USMLE-Like Exam Questions Generated by GPT-4. Preprint at https://medRxiv.org/abs/23288588
- Francis S et al. (2023) Understanding the impact of label skewness and optimization on federated learning for text classification. Companion Proc of the ACM Web Conf 2023:1161–1166
- George AS et al. (2023) AI-Driven breakthroughs in healthcare: google health’s advances and the future of medical AI. Partn Univ Int Innov J 1(3):256–267
- Gilson A et al. (2023) How does ChatGPT perform on the United States medical licensing examination? The implications of large language models for medical education and knowledge assessment. JMIR Med Educ 9(1):1–9
- Hadi MU et al. (2023) A survey on large language models: Applications, challenges, limitations, and practical usage. Authorea Prepr 2023:1–31
- Han T et al. (2023) MedAlpaca: an open-source collection of medical conversational AI models and training data. Preprint at https://arxiv.org/abs/08247
- Huang J, Tan M (2023) The role of ChatGPT in scientific communication: writing better scientific review articles. AJCR 13(4):1148
- Jing C et al. (2022) Supplementing domain knowledge to BERT with semi-structured information of documents. Expert Syst Appl 2023:121054
- Kalpakchi D, Boye J (2023) SweCTRL-Mini: a data-transparent Transformer-based large language model for controllable text generation in Swedish. Preprint at https://arxiv.org/abs/13994
- Kaneda Y et al. (2023) Assessing the performance of GPT-3.5 and GPT-4 on the Japanese nursing examination. Cureus 15(8):1–7
- Klang E et al. (2023) Utilizing artificial intelligence for crafting medical examinations: a medical education study with GPT-4. Researchsquare 613:423
- Krishna K et al. (2020) Generating SOAP notes from doctor-patient conversations using modular summarization techniques. Preprint at https://arxiv.org/abs/01795
- Li X et al. (2022a) Is gpt-3 a psychopath? evaluating large language models from a psychological perspective. Preprint at https://arxiv.org/abs/10529
- Li J et al. (2023a) Huatuo-26M, a large-scale Chinese medical QA dataset. Preprint at https://arxiv.org/abs/01526
- Liévin V et al. (2022) Can large language models reason about medical questions?. Preprint at https://arxiv.org/abs/08143
- Liu Y et al. (2023a) Summary of chatgpt/gpt-4 research and perspective towards the future of large language models. Preprint at https://arxiv.org/abs/01852
- Liu H et al. (2023b) Evaluating the logical reasoning ability of chatgpt and gpt-4. Preprint at https://arxiv.org/abs/03439
- Liu Z et al. (2023c) Deid-gpt: Zero-shot medical text de-identification by gpt-4. Preprint at https://arxiv.org/abs/11032
- Liu Y et al. (2023d) Trustworthy LLMs: a survey and guideline for evaluating large language models' alignment. Preprint at https://arxiv.org/abs/05374
- Lyu Q et al. (2023) Translating radiology reports into plain language using chatgpt and gpt-4 with prompt learning: promising results, limitations, and potential. Vis Comput Ind Biomed 6(2023):1–10
- Nori H et al. (2023) Capabilities of gpt-4 on medical challenge problems. Preprint at https://arxiv.org/abs/13375
- Peng S et al. (2023) AI-ChatGPT/GPT-4: an booster for the development of physical medicine and rehabilitation in the new era! Ann Biomed Eng 62:1–5
- Pradeep R et al. (2020) Scientific claim verification with VerT5erini. Preprint at https://arxiv.org/abs/11930
- Roy K, Rawte V (2022) TDLR: top semantic-down syntactic language representation. In: NeurIPS'22 workshop on all things attention: bridging different perspectives on attention
- Sallam M (2023b) The utility of ChatGPT as an example of large language models in healthcare education, research and practice: systematic review on the future perspectives and potential limitations. Healthcare 11:887 · DOI · Paper page
- Schloss B, Konam S (2020) Towards an automated SOAP note: classifying utterances from medical conversations. In: machine learning for healthcare conference, pp 610–631
- Singhal K et al. (2023b) Towards expert-level medical question answering with large language models. Preprint at https://arxiv.org/abs/09617
- Thawkar O et al. (2023) Xraygpt: Chest radiographs summarization using medical vision-language models. Preprint at https://arxiv.org/abs/07971
- Thirunavukarasu AJ et al. (2023) Large language models in medicine. Nat Med 8:1–11
- Toma A et al. (2023) Clinical Camel: An open-source expert-level medical language model with dialogue-based knowledge encoding. Preprint at https://arxiv.org/abs/12031
- Waisberg E et al. (2023) GPT-4: a new era of artificial intelligence in medicine. Ir J Med Sci 51(2023):1645–1653
- Wang Y et al. (2023a) Are large language models ready for healthcare? a comparative study on clinical language understanding. Proc Mach Learn Res 219:1–24
- Wang Z et al. (2023b) Can LLMs like GPT-4 outperform traditional AI tools in dementia diagnosis? Maybe, but not today. Preprint at https://arxiv.org/abs/01499
- Wang H et al. (2023c) Huatuo: Tuning llama model with chinese medical knowledge. Preprint at https://arxiv.org/abs/06975
- Wang Y et al. (2023d) Are large language models ready for healthcare? A comparative study on clinical language understanding. Preprint at https://arxiv.org/abs/05368
- Wei C et al. (2023) An overview on language models: recent developments and outlook. Preprint at https://arxiv.org/abs/05759
- Wu C et al. (2023) Pmc-llama: further finetuning llama on medical papers. Preprint at https://arxiv.org/abs/14454
- Xiong H et al. (2023) Doctorglm: fine-tuning your chinese doctor is not a herculean task. Preprint at https://arxiv.org/abs/01097
- Yadav S et al. (2023) Towards understanding consumer healthcare questions on the web with semantically enhanced contrastive learning. Proc of the ACM Web Conf 2023:1773–1783
- Yadav S et al. (2022a) Chq-summ: a dataset for consumer healthcare question summarization. Preprint at https://arxiv.org/abs/06581
- Yang J et al. (2023) The Impact of ChatGPT and LLMs on medical imaging stakeholders: perspectives and use cases. MetaRadiology 1:100007
- Yang X et al. (2020) COVID-CT-dataset: a CT scan dataset about COVID-19. Preprint at https://arxiv.org/abs/13865
- Yang S et al. (2023a) Zhongjing: enhancing the Chinese medical capabilities of large language model through expert feedback and real-world multi-turn dialogue. Preprint at https://arxiv.org/abs/03549
- Zhang J et al. (2022) Fengshenbang 1.0: being the foundation of chinese cognitive intelligence. Preprint at https://arxiv.org/abs/02970
- Zhang J et al. (2023d) The potential and pitfalls of using a large language model such as ChatGPT or GPT-4 as a clinical assistant. Preprint at https://arxiv.org/abs/08152
- Zhu W et al. (2019) Panlp at mediqa 2019: Pre-trained language models, transfer learning and knowlsedge distillation. ACL Anthology. https://doi.org/10.18653/v1/W19-5039 · DOI
Cited by
Sorted by citations
Items 201–246 of 246
AI-Driven Dental Procedure Coding: A Multi-Model Framework for CDT Extraction from Clinical Text
Annareddy, P., Noori, A., Kollipara, D., & Manda, P. (2026) · Dentistry Journal · doi:10.3390/dj14060339
Kuzu, T. E., Esen, B. N., & Kurtaran, Z. (2026) · BMC Medical Education · doi:10.1186/s12909-026-09616-7
Raj, D. T., Dhanush, N., N., C. S., Poornima, P., & Raju, S. P. (2026) · AIP conference proceedings · doi:10.1063/5.0327228
Küçük, E. E., & Küçük, D. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-2600-2173-6.ch007
Principles and Applications of LLMs in Healthcare
Jaiswal, U. K., Bhatnagar, A., Jain, J., Saini, H., Vikas, Sahu, S., & Mishra, K. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-2600-2173-6.ch003
Fourati, L. C. (2026) · Journal of Systems Architecture · doi:10.1016/j.sysarc.2026.103890 · Cited by 1
AI-CURA, an automated LLM workflow for high-accuracy genetic variant classification
Ma, W., Fong, G., Lai, J., Wu, H., Hue, S. P. Y., Ying, D., Chen, L., Tang, W., Preusch, C. B., Chu, A. T. W., et al. (2026) · Science Translational Medicine · doi:10.1126/scitranslmed.adz4172 · Cited by 1
Revolutionizing healthcare systems through large language models
Divakaran, P., Saiyad, F. B. J., Chhetri, Y., Upreti, S., Sharma, V., & Goyal, K. (2026) · Elsevier eBooks · doi:10.1016/b978-0-443-44344-2.00021-2
Transforming healthcare with large language models: Innovation, integration, and impact
Joshi, G., Ghai, B., Goutam, N., & Joshi, N. (2026) · Elsevier eBooks · doi:10.1016/b978-0-443-44344-2.00010-8
Mani, N., Attaranasl, S., Sistany, B., & Cerny, T. (2026) · Smart Health · doi:10.1016/j.smhl.2026.100687
Per-FedLoRA: Personalized Federated LoRA Fine-Tuning for Multi-Task Large Language Models
Chen, Z. L., Ji, Y., Peng, H. H., Wang, B., Shi, L., Zeng, Z., & Zhang, X. (2026) · International Wireless Communications and Mobile Computing Conference (IWCMC) · doi:10.1109/iwcmc69287.2026.11580063
Hu, Y., Wang, H., Zhao, X., Zhang, Z., Wu, X., Tang, H., Zheng, F., & Fan, C. (2026) · Measurement · doi:10.1016/j.measurement.2026.122444
Wei, Y.-C., Wu, Y.-R., Chen, Y.-Y., & Tsai, C.-T. (2026) · IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW) · doi:10.1109/icstw72326.2026.00052
Dogru, D., Özdemir, M. A., & Güren, O. (2026) · International Conference on Machine Learning and Intelligent Systems Engineering (MLISE) · doi:10.1109/mlise70044.2026.11607644
EDBERT: Predicting Emergency Department Disposition Using a BERT-Based Architecture
Hossain, M. A., Chavinda, K., Woodbright, M. D., Senadheera, I., Srikandabala, K., Freeman, S., Putland, M., Akhlaghi, H., Alahakoon, D., & Rahman, M. A. (2026) · Algorithms · doi:10.3390/a19090729
Profit-aware deployment of large language model-enabled inference chains in data centers
Zhang, Y., Lv, H., Zheng, D., Yang, C., Xing, H., Cao, X., Xu, J., & Teng, F. (2026) · Computer Networks · doi:10.1016/j.comnet.2026.112609
Tan, M., Tang, S., Kan, S., Wu, B., Ni, Z., Zhang, H., & Ding, J. (2026) · Journal of Medical Internet Research · doi:10.2196/87288 · Cited by 1
Sablone, S., Costantino, A., Laurenzano, F., Ferretti, G., Croce, E. B., Vaiano, F., & Grassi, S. (2026) · Sci · doi:10.3390/sci8080182
Trust-driven consensus reaching in human-AI hybrid large-scale group decision-making
Wang, X., Xu, X., & Zhang, W. (2026) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2026.1886098
Exploring Bias in Medical Applications of Large Language Models: Protocol for a Systematic Review
Rao, S. G., Boutrous, C., Kormilitzin, A., Tai, X. Y., Hotchkiss, L., Cong, C., Meinert, E., Liang, H., Dong, H., & Harrison, J. R. (2026) · JMIR Research Protocols · doi:10.2196/91348
Artificial intelligence analytics for diagnosis, prognosis, and outcomes prediction
Jalali, S., You, Q., Xu, V., Chen, L., Zini, J., Zamorano, T., Luo, Y., Friesen, T., Fontana, G. J., & Benrimoh, D. A. (2026) · Elsevier eBooks · doi:10.1016/b978-0-443-33151-0.00009-5
Qi, Z., Saito, H., Koseki, R., Suzuki, M., Kubota, S., Naito, T., Inaba, M., Kameue, C., Matsukura, H., & Sakamoto, M. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-35387-0_33
AI-Assisted Development of Personalized Phytopharmacological Therapies
Ammara, U., Agustina, R., & Sajid, Q. U. A. (2026) · Springer eBooks · doi:10.1007/978-981-92-0875-3_9
Dai, D., Pakianathan, P. V. S., Treff, G., Sareban, M., Niebauer, J., Smeddinck, J. D., & Kuoppamäki, S. (2026) · Mensch und Computer (MuC) · doi:10.1145/3820253.3820281
Albeshri, S. (2026) · Research Square · doi:10.21203/rs.3.rs-10664547/v1
TGSFL: Trustworthy group-signature-based federated learning for LLMs in healthcare
Liu, X., Jin, C., Wu, Y., Wu, Y., Zhai, Z., Chen, G., Zhang, H., & Jian, W. (2026) · Journal of King Saud University - Computer and Information Sciences · doi:10.1007/s44443-026-01237-w
Yichen, Y., Dianqing, L., & Meng, C. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.7422696
Knowledge-guided multi-agent framework for intelligent acupuncture diagnosis and reasoning
Zhang, W., Cui, J., Zhao, M., Zhang, M., & Li, P. (2026) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2026.116139
Neves, E. B., & Pochmann, P. G. C. (2026) · InterAção · doi:10.5902/2357797597015
Dobrowolski, H. (2026) · Frontiers in Public Health · doi:10.3389/fpubh.2026.1949700
Fu, X., Chen, Z., & Lu, W. (2026) · PLoS ONE · doi:10.1371/journal.pone.0343858
Wang, D., Wang, Z., Zhao, X., Chen, Q., Chen, Q., Zhang, S., & Lu, H. (2026) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2026.111495
Self-citationKhoury, Z. H., Rokhshad, R., Sultan, M. S., Price, J. B., Tavares, T., Kazemi, K. S., Najafimakhsoos, N., & Sultan, A. S. (2026) · Cancers · doi:10.3390/cancers18183047
Nobel, M. N. A., Matar, M. G. S., Hemal, M. A. K. P., Contreras, K. A. Z., Mamun, A. A., & Uddin, R. (2026) · Technology and Innovation Conference (TIC) · doi:10.1109/tic68483.2026.11702858
CatKV: Accelerating Position-Independent Caching with Context-Adaptive KV Cache Compression
Xu, J., Liu, R., Li, Y., & Tang, B. (2026) · Proceedings of the ACM on Management of Data · doi:10.1145/3837104
Evolutionary Large Language Models Driven Cancer Cell Data Analysis
Lv, J., Ge, D., Jiang, H., Li, D., Hu, S., Yang, J., Li, P., Zang, L., & Wang, Y. (2026) · Journal of Multimedia Information System · doi:10.33851/jmis.2026.13.3.107
Leonforte, F., Cavallaro, A. S., Nicosia, V., Scapellato, A., Gambera, A., Genovese, C., Mistretta, A., Zanghi, A., Basile, G., Veroux, M., et al. (2026) · AI · doi:10.3390/ai7100396
Macharia, P. M., Hartley, A., Ding, J., Kan, S., Currie, S., Wamalwa, P., Ngugi, L., Nyongesa, T., Mukhwana, R., Mbwayo, A., et al. (2026) · Open Research Africa · doi:10.12688/openresafrica.16217.2
Application of large language models in early screening of Alzheimer’s disease: a scoping review
Tian, J., Huang, W., Wen, Y., & Liang, Y. (2026) · Frontiers in Aging Neuroscience · doi:10.3389/fnagi.2026.1906641
IPF-RAG: temporal-aware retrieval-augmented generation for idiopathic pulmonary fibrosis care
Zheng, X., Rui, X., Long, T., Wang, L., Lv, Z., Wang, J., Jin, Z., Huang, W., Dong, X., Wang, Q., et al. (2026) · Frontiers in Digital Health · doi:10.3389/fdgth.2026.1883739
Albeshri, S. (2026) · The Saudi Dental Journal · doi:10.1007/s44445-026-00247-8
Status, challenges, opportunities of digital assistants in health
Lavdaria, N. (2026) · Pan-Hellenic Conference on Progress in Computing and Informatics (PCI) · doi:10.1145/3829216.3829257
Paired response final divergence signatures of jailbreak vulnerability in large language models
Ismail, H. (2026) · Discover Artificial Intelligence · doi:10.1007/s44163-026-02593-2
Lee, J., Shin, J., & Cho, B. (2026) · Journal of Korean Medical Science · doi:10.3346/jkms.2026.41.e339
Drira, F., Tarkhani, O., & Walha, R. (2026) · Network Modeling Analysis in Health Informatics and Bioinformatics · doi:10.1007/s13721-026-00906-0
Labuschagne, N., Mahomed‐Asmail, F., Manchaiah, V. K. C., Wasmann, J.-W. A., & Swanepoel, D. W. (2026) · American Journal of Audiology · doi:10.1044/2026_aja-26-00142
Metadata
- Authors
- Wang, D., & Zhang, S.
- Affiliations
- Taizhou University (CN)
- Corresponding author
- Zhang, Shiqing
- Venue
- Artificial Intelligence Review
- Published
- 2024-09-20
- Volume
- 57
- Issue
- 11
- ISSN
- 0269-2821 (ISSN-L)
- 1573-7462
- OpenAlex
- W4402692877
- Semantic Scholar
- 70b83014b3267ddcb04d65af0f7975920d46a87a
- Cited by
- 246
- References
- 122
- Citations by year
- 2026: 124 · 2025: 119 · 2024: 3
- FWCI
- 23.05
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Artificial Intelligence Review (publishedVersion, cc-by-nc-nd)
- Funders
- Natural Science Foundation of Zhejiang Province (LQ24F020014)
- The Project of Zhejiang Province's Educational Science Planning (2024SCG176)
- National Natural Science Foundation of China (62276180)
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Keywords
- large language models, medical applications, medical question answering, medical education, clinical decision support, clinical reasoning, data security, algorithmic bias, de-identification, accountability
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
