Probabilistic medical predictions of large language models
Open accessAbstract
Large Language Models (LLMs) have shown promise in clinical applications through prompt engineering, allowing flexible clinical predictions. However, they struggle to produce reliable prediction probabilities, which are crucial for transparency and decision-making. While explicit prompts can lead LLMs to generate probability estimates, their numerical reasoning limitations raise concerns about reliability. We compared explicit probabilities from text generation to implicit probabilities derived from the likelihood of predicting the correct label token. Across six advanced open-source LLMs and five medical datasets, explicit probabilities consistently underperformed implicit probabilities in discrimination, precision, and recall. This discrepancy is more pronounced with smaller LLMs and imbalanced datasets, highlighting the need for cautious interpretation, improved probability estimation methods, and further research for clinical use of LLMs.
AI snapshot
Written from the paper’s full text.
TL;DR
Across six open-source LLMs and five medical datasets, explicit probabilities generated in text were consistently less reliable than implicit probabilities derived from token likelihood for discrimination, precision, and recall. The gap was larger for smaller LLMs and imbalanced datasets.
Key findings
- Implicit probabilities had higher AUROC and AUPRC than explicit probabilities across all LLMs and datasets, with most differences statistically significant at p < 0.05. For example, gemma-2-27b-it on USMLE had explicit AUROC 85.4% vs implicit 90.2% (difference 4.9%), and Phi-3-medium on MMLU-CM had explicit AUPRC 82.5% vs implicit 91.3% (difference 8.8%).
- All LLMs achieved at least 0.8 average accuracy on all datasets; Meta-Llama-3.1-70B-Instruct had the highest average accuracy at 91.7%, followed by Qwen2-72B-Instruct at 91.3% and Mistral-Large-Instruct-2407 at 90.0%. Qwen2-72B-Instruct performed best on MCMLE at 96.7%, while Phi-3-medium-128k-instruct performed worst on MCMLE at 69.8%.
- Small LLMs generally showed greater explicit-vs-implicit AUROC differences than their larger counterparts, while implicit probability differences for small LLMs were much smaller.
- On imbalanced USMLE reconstructions, AUPRC differences increased as datasets became more imbalanced for most LLMs; explicit probabilities were sometimes below 50% even when the model gave an answer, whereas implicit probability always selected the most probable token.
Methods
- Evaluated six large open-source LLMs (Qwen2-72B-Instruct, Meta-Llama-3.1-70B-Instruct, gemma-2-27b-it, Mistral-Large-Instruct-2407, Yi-1.5-34B-Chat, Phi-3-medium-128k-instruct) and smaller counterparts on five datasets: MMLU-CK (299 questions), MMLU-CM (200 questions), USMLE (1,000-question subset), MCMLE (1,000-question subset), and MGB-SDoH (200 patients, 9 SDoH aspects).
- Converted multiple-choice questions to binary A/B tasks, randomly assigning 50% of correct options as Option A and 50% as Option B; extracted explicit probability from generated text and implicit probability from the token probability of the predicted label token (“A” or “B”).
- Used accuracy, AUROC, and AUPRC as metrics; ran experiments three times at temperatures 0, 0.3, and 0.7; used one-sided paired t-tests; reconstructed USMLE with option-A-correct ratios of 5%, 10%, 30%, 50%, 70%, and 90%; and conducted sensitivity analysis with two additional prompts.
Limitations
- The paper states: experiments were simplified to binary classification, so conclusions do not carry over to multiple-choice question answering; Chain-of-Thought prompting was not examined because implicit probability extraction is challenging there; only open-source LLMs were studied because proprietary LLMs do not provide token probabilities; and results are primarily based on medical datasets, so generalizability to other domains needs further validation.
- The text also makes evident that implicit probability extraction is only feasible in limited settings, such as option-selection tasks or tasks without a chain of thought, and is mostly available in open-source LLMs.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
0 of 18 cited sources checked · 0 supported · 0 flagged
- Findingown data
In some cases, while the LLM provides an answer and an explicit probability as instructed, this probability is less than 50%.
“We find that in some cases, while the LLM provides an answer and an explicit probability as instructed, this probability is less than 50% (Supplementary Table 3).”
Quote in the paperSubstantial claim - Findingown data
The ROC and PRC curves for the explicit probability is a coarse step function because the LLMs usually give their probabilities in their nearest tenth.
“We observe that the ROC and PRC curves for the explicit probability is a coarse step function because the LLMs usually give their probabilities in their nearest tenth (e.g., 90%, 80%, etc.), which makes the explicit probability coarse-grained.”
Quote in the paperSubstantial claim - Findingown data
The Qwen2-72B model shows a clear advantage over other models on the MCMLE dataset.
“The Qwen2-72B model, which is trained using a rich Chinese corpus, shows a clear advantage over other models on the MCMLE dataset, which consists of Chinese medical examination questions.”
Quote in the paperNot a substantial claim - Findingown data
The Mistral-Large model's performance is much worse than the Llama-3.1-70B model on the non-English MCMLE dataset (85.2% vs 93.0%).
“The Mistral-Large model, which has comparable capability to the Llama-3.1-70B model, achieve the highest accuracy for the MMLU-CM and the MGB-SDoH dataset but its performance is much worse than the Llama-3.1-70B model on the non-English MCMLE dataset (85.2% VS 93.0%).”
Quote in the paperNot a substantial claim - Findingown data
For most LLMs, both their explicit and implicit probability distributions are very polarized, even if their predictions are incorrect.
“For most LLMs, both their explicit and implicit probability distributions are very polarized, even if their predictions are incorrect.”
Quote in the paperSubstantial claim
- Findingreasoning
Small LLMs generally exhibit greater differences between explicit and implicit probability AUROC.
“Furthermore, small LLMs generally exhibit greater differences between explicit and implicit probability AUROC, as their limited parameters make them less sensitive to explicit probabilities, reducing reliability.”
Quote in the paperNot a substantial claim - Findingown data
All LLMs achieve at least a 0.8 average accuracy on all datasets.
“According to Table 1, all LLMs achieve at least a 0.8 average accuracy on all datasets.”
Quote in the paperNot a substantial claim - Findingreasoning
For most LLMs the AUPRC difference increases as the dataset got more imbalanced.
“Based on Fig. 2, for most LLMs the AUPRC difference increases as the dataset got more imbalanced.”
Quote in the paperSubstantial claim - Findingown data
The Llama-3.1-70B model has the highest average accuracy among all the LLMs tested.
“Specifically, the Llama-3.1-70B model has the highest average accuracy among all the LLMs tested.”
Quote in the paperNot a substantial claim - Findingown data
Except for the Phi-3-medium model on the MCMLE dataset, all other LLMs have a relatively great performance on each dataset by having low probability when the true label is 0 and high probability when the true label is 1.
“According to Fig. 2b, except for the Phi-3-medium model on the MCMLE dataset, all other LLMs have a relatively great performance on each dataset by having low probability when the true label is 0 and high probability when the true label is 1, which is consistent with the consistent performance we find (Table 1).”
Quote in the paperNot a substantial claim - Findingnone given
Some exceptions to the above observation are the Mistral-Large and the Phi-3-medium model on the MCMLE and the MGB-SDoH dataset, where models show a broad spectrum of implicit probability distribution, but their explicit probability distributions are still highly polarized.
“Some exceptions to the above observation are the Mistral-Large and the Phi-3-medium model on the MCMLE and the MGB-SDoH dataset, where models show a broad spectrum of implicit probability distribution, but their explicit probability distributions are still highly polarized.”
Quote in the paperNot a substantial claim - Findingown data
The implicit probability has a higher AUROC than that of the explicit probability under all three prompts.
“It indicates that the implicit probability has a higher AUROC than that of the explicit probability under all three prompts.”
Quote in the paperNot a substantial claim - Findingown data
Across six advanced open-source LLMs and five medical datasets, explicit probabilities from text generation consistently underperformed implicit probabilities in discrimination, precision, and recall.
“Across six advanced open-source LLMs and five medical datasets, explicit probabilities consistently underperformed implicit probabilities in discrimination, precision, and recall.”
- Also stated in: § LLM Performance · plain
“Although the AUROC and AUPRC for both the explicit and implicit probabilities are high, the metrics for implicit probabilities are consistently higher than those for explicit probabilities across all LLMs, and the majority of the results show that such advantage is statistically significant.”
- Also stated in: § Discussion · boosted
“This study investigates the probabilistic prediction of LLMs in healthcare and demonstrate that the simple and flexible explicit probability (directly extracted from the generated text following the prompt) provide relatively high AUROC and AUPRC, but is consistently less reliable than the implicit probability (derived from the transition score of the predicted label token) across different langu”
Quote in the paperSubstantial claim - Also stated in: § LLM Performance · plain
- Findingown data
Compared to the explicit probability, the implicit probability has a more dispersed distribution.
“Compared to the explicit probability, the implicit probability has a more dispersed distribution, in accord with our finding on the AUROC and AUPRC curves.”
Quote in the paperNot a substantial claim - Findingreasoning
The AUROC for both probabilities largely decrease when using the second auxiliary prompt.
“The AUROC for both probabilities largely decrease when using the second auxiliary prompt.”
Quote in the paperNot a substantial claim - Findingreasoning
Except for the Yi-1.5 model on the MGB-SDoH dataset, all large LLMs outperform their smaller versions on both explicit and implicit probability AUROC.
“Except for the Yi-1.5 model on the MGB-SDoH dataset, all large LLMs outperform their smaller versions on both explicit and implicit probability AUROC, due to their richer knowledge and stronger reasoning abilities.”
Quote in the paperSubstantial claim - Findingown data
As LLM performance declines, the reliability of explicit probabilities worsens compared to implicit ones.
“We show that as LLM performance declines, the reliability of explicit probabilities worsens compared to implicit ones (Fig. 2, Supplementary Table 5).”
Quote in the paperSubstantial claim - Findingreasoning
The main prompt and the first auxiliary prompt have similar performance in terms of AUROC for both probabilities.
“We also learn that the main prompt and the first auxiliary prompt have similar performance in terms of AUROC for both probabilities, which is reasonable since both prompts use a similar multiple-choice format and contain similar information.”
Quote in the paperNot a substantial claim - Findingnone given
The Phi-3-medium model has a poor performance on the MCMLE dataset.
“On the other hand, the Phi-3-medium model has a poor performance on the MCMLE dataset.”
Quote in the paperNot a substantial claim - Findingreasoning
The implicit probabilities difference for small LLMs is much smaller than for the explicit probabilities.
“In contrast, the implicit probabilities difference for small LLMs is much smaller than for the explicit probabilities, indicating their better credible to represent the model’s actual probability to its prediction.”
Quote in the paperSubstantial claim - Findingown data
By comparing explicit and implicit probabilities across multiple advanced LLMs and medical datasets, the study uncovered consistent discrepancies in reliability, particularly under conditions of small model size and dataset imbalance.
“By comparing explicit and implicit probabilities across multiple advanced LLMs and medical datasets, we have uncovered consistent discrepancies in reliability, particularly under conditions of small model size and dataset imbalance.”
Quote in the paperSubstantial claim - Interpretationown data
The implicit probability is a better indicator of the model answer's confidence than the explicit probability.
“Overall, this result indicates that for LLMs, the implicit probability is a better indicator of the model answer’s confidence (Tables 2 and 3).”
- Also stated in: § LLM Performance · boosted
“This further proves that the implicit probability is a better indicator than the explicit probability.”
Quote in the paperSubstantial claim - Also stated in: § LLM Performance · boosted
- Interpretationown data
Using explicit probabilities can amplify biases inherent in LLMs, especially when their performance is poor.
“This suggests that using explicit probabilities can amplify biases inherent in LLMs, especially when their performance is poor.”
Quote in the paperSubstantial claim - Interpretationreasoning
Such confidence does not originate from the data leakage, but is an intrinsic property of the LLMs.
“Since such polarization still persists on the MGB-SDoH dataset, a private dataset that can’t be touched during LLM training, we argue that such confidence does not originate from the data leakage31, but is an intrinsic property of the LLMs.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [31] Xu, R., Wang, Z., Fan, R.-Z. & Liu, P. Benchmarking benchmark leakage in large language models. arXiv preprint arXiv:2404.18824 (2024).
Not checked in the free run.
- Rests on an assumption: The MGB-SDoH dataset was not used in LLM training.
- Interpretationreasoning
The LLMs are overly confident about most of their predictions, regardless of their actual correctness.
“This indicates that the LLMs are overly confident about most of their predictions, regardless of their actual correctness.”
Quote in the paperSubstantial claim - Interpretationreasoning
In clinical scenarios with poor LLM performance and increased biases, accurate interpretation of explicit probabilities is hindered.
“In such clinical scenarios, the combination of LLMs’ poor performance and increased biases hinders accurate interpretation of explicit probabilities.”
Quote in the paperSubstantial claim- Rests on an assumption: Many countries’ medical systems use minority languages that most LLMs are not familiar with.
- Premisecitation
LLMs struggle to produce reliable prediction probabilities.
“However, they struggle to produce reliable prediction probabilities, which are crucial for transparency and decision-making.”
- Also stated in: § Introduction · plain
“However, this generative framework does not naturally output probabilities of the predictions, as it converts all the tasks as a text generation process, rather than assigning probabilities to fixed candidate labels.”
- Also stated in: § Introduction · hedged
“However, given the challenges LLMs face with numerical reasoning28,29, the reliability of text-generated explicit probability values may be questionable and has not yet been thoroughly examined.”
Quote in the paperSubstantial claim0 of 2 sources checked- cites [28] Shen, R. et al. Positional description matters for transformers arithmetic. arXiv preprint arXiv:2311.14737 (2023).
Not checked in the free run.
- cites [29] Schwartz, E. et al. NumeroLogic: Number Encoding for Enhanced LLMs’ Numerical Reasoning. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, 206–212, Miami, Florida, USA (Association for Computational Linguistics 2024). source
Not checked in the free run.
- Also stated in: § Introduction · plain
- Interpretationreasoning
The study's conclusions about binary options do not carry over to other tasks such as multiple-choice question answering.
“As a result, our conclusions about binary options do not carry over to other tasks such as multiple-choice questions answering.”
Quote in the paperSubstantial claim - Interpretationreasoning
Low explicit probability may signal a higher risk of hallucination.
“Thus, low explicit probability may signal a higher risk of hallucination.”
Quote in the paperSubstantial claim
- May combine two claims. 18
- Classified as probably not a substantial claim. 13
May combine two claims.
Findingown dataThe Mistral-Large model achieves the highest accuracy for the MMLU-CM and the MGB-SDoH dataset.
“The Mistral-Large model, which has comparable capability to the Llama-3.1-70B model, achieve the highest accuracy for the MMLU-CM and the MGB-SDoH dataset but its performance is much worse than the Llama-3.1-70B model on the non-English MCMLE dataset (85.2% VS 93.0%).”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataThe only exception is the Llama-3.1-70B model on the USMLE dataset, which is mainly due to the high standard deviation of the implicit probability.
“The only exception is the Llama-3.1-70B model on the USMLE dataset, which is mainly due to the high standard deviation of the implicit probability.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataThe discrepancy between explicit and implicit probabilities is more pronounced with smaller LLMs and imbalanced datasets.
“This discrepancy is more pronounced with smaller LLMs and imbalanced datasets, highlighting the need for cautious interpretation, improved probability estimation methods, and further research for clinical use of LLMs.”
- Also stated in: § Large LLM vs. small LLM · plain
“Furthermore, small LLMs generally exhibit greater differences between explicit and implicit probability AUROC, as their limited parameters make them less sensitive to explicit probabilities, reducing reliability.”
- Also stated in: § Imbalanced datasets analysis · hedged
“Based on Fig. 2, for most LLMs the AUPRC difference increases as the dataset got more imbalanced.”
Quote in the paperSubstantial claim- Also stated in: § Large LLM vs. small LLM · plain
May combine two claims.
Findingown dataThe explicit probability, even with its narrow distribution, may give values that are close to 50%, while the more widely distributed implicit probability values rarely fall within this area.
“However, we notice that the explicit probability, even with its narrow distribution, may give values that are close to 50%, while the more widely distributed implicit probability values rarely fall within this area.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim.
FindingreasoningThe steps of the implicit probability are barely discernible in the plot.
“In contrast, the steps of the implicit probability are barely discernible in the plot.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataThis finding is valid on both a non-English dataset (MCMLE) and a dataset from private electronic health records (MGB-SDoH).
“This finding is valid on both a non-English dataset (MCMLE) and a dataset from private electronic health records (MGB-SDoH).”
Quote in the paperSubstantial claimClassified as probably not a substantial claim.
PremisecitationDuring LLM inference, there is a correlation between predictive uncertainty and hallucination.
“Recent work has shown that during LLM inference, there is a correlation between predictive uncertainty and hallucination — a behavior where LLMs generate false facts or knowledge not supported by input39.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [39] Xiao, Y. & Wang, W. Y. Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume.
Not checked in the free run.
May combine two claims.
PremisecitationIn the clinical domain, LLMs often underperform due to limited training data for minority languages, lack of domain knowledge, and dataset imbalances.
“In the clinical domain, due to limited training data for minority languages, lack of domain knowledge, and dataset imbalances, LLMs often underperforms34.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [34] Wu, J. et al. Clinical text datasets for medical artificial intelligence and large language models—a systematic review. NEJM AI 1, AIra2400012 (2024). source
Not checked in the free run.
- Rests on an assumption: Many countries’ medical systems use minority languages that most LLMs are not familiar with.
May combine two claims.
InterpretationreasoningThe implicit probability always selects the most probable token, avoiding such contradictions and supporting its reliability over the explicit probability.
“In contrast, the implicit probability always selects the most probable token, avoiding such contradictions and supporting its reliability over the explicit probability.”
Quote in the paperSubstantial claimMay combine two claims.
Study design and conductown dataAll other models have a less than 10% fail rate on the validation process, with 78.6% of the model inference on one dataset have a less than 1% fail rate on the validation process.
“All other models have a less than 10% fail rate on the validation process, with 78.6% of the model inference on one dataset have a less than 1% fail rate on the validation process.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationMost LLMs are trained mainly on English corpora.
“Most LLMs are trained mainly on English corpora35, while many countries’ medical systems use minority languages that most LLMs are not familiar with.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [35] Blevins, T. & Zettlemoyer, L. Language contamination explains the cross-lingual capabilities of English pretrained models. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, 3563–3574, Abu Dhabi, United Arab Emirates (Association for Computational Linguistics 2022). source
Not checked in the free run.
May combine two claims.
PremisecitationAdvanced LLMs often rely on general domain datasets and lack exposure to real-world EHR notes, which limits their effectiveness in clinical settings.
“Furthermore, due to data access and privacy concerns34, advanced LLMs often rely on general domain datasets and lack exposure to real-world EHR notes, which limits their effectiveness in clinical settings24,36.”
Quote in the paperNot a substantial claim0 of 3 sources checked- cites [34] Wu, J. et al. Clinical text datasets for medical artificial intelligence and large language models—a systematic review. NEJM AI 1, AIra2400012 (2024). source
Not checked in the free run.
- cites [24] Wu, J. et al. Large language models leverage external knowledge to extend clinical insight beyond language boundaries. J. American Medical Informatics Association, ocae079 (2024). source
Not checked in the free run.
- cites [36] Wu, J., Wu, X. & Yang, J. in Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence, {IJCAI-24} (ed Kate L.) 7491–7499 (International Joint Conferences on Artificial Intelligence Organization, 2024). source
Not checked in the free run.
Classified as probably not a substantial claim. May combine two claims.
PremisecitationIncorrect estimation can lead to misinformed decisions, compromising patient safety and treatment efficacy.
“Incorrect estimation can lead to misinformed decisions, compromising patient safety and treatment efficacy32,33.”
Quote in the paperSubstantial claim0 of 2 sources checked- cites [32] Koçak, B. et al. Bias in artificial intelligence for medical imaging: fundamentals, detection, avoidance, mitigation, challenges, ethics, and prospects. Diagn Interv Radiol (2024). source
Not checked in the free run.
- cites [33] Albahri, A. S. et al. A systematic review of trustworthy and explainable artificial intelligence in healthcare: Assessment of quality, bias risk, and data fusion. Inf. Fusion 96, 156–191 (2023). source
Not checked in the free run.
May combine two claims.
PremisecitationMedical datasets frequently suffer from imbalances in phenotypes, like rare diseases or serious clinical outcomes, and cohort identification.
“Medical datasets also frequently suffer from imbalances in phenotypes, like rare diseases or serious clinical outcomes37, and cohort identification7.”
Quote in the paperNot a substantial claim0 of 2 sources checked- cites [37] Schubach, M., Re, M., Robinson, P. N. & Valentini, G. Imbalance-aware machine learning for predicting rare and common disease-associated non-coding variants. Sci. Rep. 7, 2959 (2017). source
Not checked in the free run.
- cites [7] Yang, J. et al. Development and validation of a deep learning model for detection of allergic reactions using safety event reports across hospitals. JAMA Netw. Open 3, e2022836–e2022836 (2020). source
Not checked in the free run.
May combine two claims.
Study design and conductown dataAll LLMs follow instructions well, with outputs strictly adhering to the format in the prompt, except for the Qwen2-7B model on the USMLE dataset.
“All LLMs follow instructions well, with outputs strictly adhering to the format in the prompt, except for the Qwen2-7B model on the USMLE dataset.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
InterpretationreasoningThe decrease in AUROC when using the second auxiliary prompt is due to the introduction of the setting of a student proposing answers to the LLM, which introduces additional complexity and lowered the model's performance.
“We propose that this is due to the introduction of the setting of a student proposing answers to the LLM, which introduces additional complexity and lowered the model’s performance.”
Quote in the paperNot a substantial claimMay combine two claims.
PremisecitationA primary cause of low explicit probability is insufficient knowledge, which can drive LLMs to produce hallucinated information for predictions.
“A primary cause of low explicit probability is insufficient knowledge, which can drive LLMs to produce hallucinated information for predictions40,41.”
Quote in the paperSubstantial claim0 of 2 sources checked- cites [40] Martino, A., Iannelli, M. & Truong, C. in European Semantic Web Conference. 182–185 (Springer).
Not checked in the free run.
- cites [41] Liu, X. et al. Uncovering language disparity of ChatGPT on retinal vascular disease classification: cross-sectional study. J. Med. Internet Res. 26, e51926 (2024). source
Not checked in the free run.
May combine two claims.
Interpretationown dataThis implies that although the model proposes an answer, it assigns a higher probability to the alternative, undermining its reliability.
“This implies that although the model proposes an answer, it assigns a higher probability to the alternative, undermining its reliability.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductnone givenMass General Brigham Institutional Review Board approved the study protocol (2020P001486).
“Mass General Brigham (MGB) Institutional Review Board approved the study protocol (2020P001486).”
Quote in the paperNot a substantial claimMay combine two claims.
Study design and conductreasoningThe study only applies to open-sourced LLMs since it is difficult to obtain the implicit probability of tokens using proprietary LLMs.
“Additionally, our study only applies to open-sourced LLMs since it is difficult to obtain the implicit probability of the tokens using proprietary LLMs.”
Quote in the paperSubstantial claimClassified as probably not a substantial claim.
Study design and conductcitationTo eliminate the impact of option order on LLM performance, 50% of correct options were randomly assigned as Option A and the remaining 50% as Option B.
“To eliminate the impact of the option order to LLMs performance55, we randomly assign 50% of the correct options as Option A and the remaining 50% of the correct options as Option B.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [55] Pezeshkpour, P. & Hruschka, E. in Findings of the Association for Computational Linguistics: NAACL 2024. 2006–2017.
Not checked in the free run.
Classified as probably not a substantial claim.
Study design and conductown dataTo ensure robustness, a sensitivity analysis was conducted using two additional prompts on the USMLE dataset to analyze if the relative difference between explicit and implicit probability persists under different prompt settings.
“To ensure the robustness of our results, we conduct a sensitivity analysis by using two additional prompts (Supplementary Table 1) on the USMLE dataset and analyze if the relative difference between the explicit and the implicit probability persists under different prompt settings.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Study design and conductreasoningThe study does not examine probability performance using Chain of Thought (CoT) prompting, as it is challenging to extract implicit probabilities in these cases.
“Besides, we do not examine the probability performance using Chain of Thought (CoT) prompting, as it is challenging to extract implicit probabilities in these cases.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim. May combine two claims.
Study design and conductnone givenValidation scripts were implemented to ensure that extracted content aligns with model predictions and falls within the two candidate options.
“Validation scripts are implemented to ensure that the extracted content aligns with the model predictions and falls within the two candidate options.”
Quote in the paperSubstantial claimClassified as probably not a substantial claim.
PremisecitationAUPRC is a commonly used metric to evaluate model performance on datasets with imbalanced labels.
“Since the metric of AUPRC is a commonly used metric to evaluate model performance on datasets with imbalanced labels56,57, we calculate the AUPRC of the two models and compare their differences.”
Quote in the paperSubstantial claim0 of 2 sources checked- cites [56] Wu, X., Huang, F. & Huang, H. in 2022 IEEE International Conference on Data Mining (ICDM). 578–587 (IEEE). source
Not checked in the free run.
- cites [57] Qi, Q., Luo, Y., Xu, Z., Ji, S. & Yang, T. Stochastic optimization of areas under precision-recall curves with provable convergence. Adv. neural Inf. Process. Syst. 34, 1752–1765 (2021).
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Interpretationown dataThe results are primarily based on medical datasets, and generalizability to different domains needs confirmation by further validation studies.
“Finally, our results are primarily based on medical datasets, the generalizability to different domains needs to be confirmed by further validation studies.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductnone givenConsent to participate was not deemed required for this observational investigation.
“Consent to participate was not deemed required for this observational investigation.”
Quote in the paperNot a substantial claim
The paper relies on this without support.
The MGB-SDoH dataset was not used in LLM training.
“Since such polarization still persists on the MGB-SDoH dataset, a private dataset that can’t be touched during LLM training, we argue that such confidence does not originate from the data leakage31, but is an intrinsic property of the LLMs.”
The paper relies on this without support.
Labels in clinical practice often have imbalanced distributions, and the performance of AI models is affected by such imbalance.
“Labels in clinical practice often have imbalanced distributions, and the performance of AI models are affected by such imbalance.”
The paper relies on this without support.
Many countries’ medical systems use minority languages that most LLMs are not familiar with.
“Most LLMs are trained mainly on English corpora35, while many countries’ medical systems use minority languages that most LLMs are not familiar with.”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
LLMs can serve as classifiers in clinical prediction tasks and have achieved outstanding performance.
“When provided with clinical text inputs, LLMs can also serve as classifiers in clinical prediction tasks and have achieved outstanding performance31–33.”
From MoMA: a mixture-of-multimodal-agents architecture for enhancing clinical prediction modelling (2025) · cites this paper as [33]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Qwen2-72B is an unacknowledged exception to higher implicit AUROC
MinorThe text names Llama-3.1-70B on USMLE as the only exception, but Table 2 shows another: on USMLE, Qwen2-72B has a lower implicit AUROC (89.5%) than explicit AUROC (90.5%).
“The only exception is the Llama-3.1-70B model on the USMLE dataset, which is mainly due to the high standard deviation of the implicit probability.”
“| Qwen2-72B-Instruct | Explicit | 92.8 ± 0.8 | 88.9 ± 0.8 | 90.5 ± 0.2 | 97.1 ± 0.3 | 95.9 ± 0.2 | | Qwen2-72B-Instruct | Implicit | 93.1 ± 3.5 | 90.9 ± 3.3 | 89.5 ± 3.2 | 97.7 ± 1.1 | 97.0 ± 1.2 |”
Cites
Cortes, C., & Vapnik, V. (1995) · Machine Learning · doi:10.1007/bf00994018 · Cited by 59,421 · see citing works
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E., Le, Q., & Zhou, D. (2022) · arXiv · doi:10.48550/arxiv.2201.11903 · Cited by 4,263 · cited 1× · see citing works
“The simplicity of explicit probability ensures that it can be applied in any advanced prompt engi-neeringtechniquessuch asChain-of-Thought(CoT) 22 ,retrievalaugmented generation (RAG) 23,24 , self consistency 25 , and LLM agents 26,27 .”
Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., Küttler, H., Lewis, M., Yih, W.-T., Rocktäschel, T., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.11401 · Cited by 3,101 · cited 1× · see citing works
“The simplicity of explicit probability ensures that it can be applied in any advanced prompt engi-neeringtechniquessuch asChain-of-Thought(CoT) 22 ,retrievalaugmented generation (RAG) 23,24 , self consistency 25 , and LLM agents 26,27 .”
A survey on large language model based autonomous agents
Wang, L., Ma, C., Feng, X., Zhang, Z., Yang, H., Zhang, J., Chen, Z., Tang, J., Chen, X., Lin, Y., et al. (2024) · Frontiers of Computer Science · doi:10.1007/s11704-024-40231-1 · Cited by 1,941 · cited 1× · see citing works
“The simplicity of explicit probability ensures that it can be applied in any advanced prompt engi-neeringtechniquessuch asChain-of-Thought(CoT) 22 ,retrievalaugmented generation (RAG) 23,24 , self consistency 25 , and LLM agents 26,27 .”
The Rise and Potential of Large Language Model Based Agents: A Survey
Xi, Z., Chen, W., Guo, X., He, W., Ding, Y., Hong, B., Zhang, M., Wang, J., Jin, S., Zhou, E., et al. (2023) · arXiv · doi:10.48550/arxiv.2309.07864 · Cited by 972 · cited 1× · see citing works
“The simplicity of explicit probability ensures that it can be applied in any advanced prompt engi-neeringtechniquessuch asChain-of-Thought(CoT) 22 ,retrievalaugmented generation (RAG) 23,24 , self consistency 25 , and LLM agents 26,27 .”
Jin, D., Pan, E., Oufattole, N., Weng, W.-H., Fang, H., & Szolovits, P. (2021) · Applied Sciences · doi:10.3390/app11146421 · Cited by 718 · see citing works
Albahri, A. S., Duhaim, A. M., Fadhel, M. A., Alnoor, A., Baqer, N. S., Alzubaidi, L., Albahri, O. S., Alamoodi, A. H., Bai, J., Salhi, A., et al. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.03.008 · Cited by 716 · see citing works
Measuring Massive Multitask Language Understanding
Hendrycks, D., Burns, C., Basart, S., Zou, A., Mazeika, M., Song, D., & Steinhardt, J. (2020) · arXiv · doi:10.48550/arxiv.2009.03300 · Cited by 291 · cited 5× · see citing works
“…(MMLU-CK) 42,43 , Measuring Massive Multitask Language Understanding College Medicine (MMLU-CM) 42,43 , United States Medical Licensing Examination (USMLE) 44 , Mainland China Medical Licensing Examination (MCMLE) 44 — and one internal EHR…”
InfluentialFoundation and large language models: fundamentals, challenges, opportunities, and social impacts
Myers, D., Mohawesh, R., Chellaboina, V. I., Sathvik, A. L., Venkatesh, P., Ho, Y.-H., Henshaw, H., Al-Hawawreh, M. S., Berdik, D., & Jararweh, Y. (2023) · Cluster Computing · doi:10.1007/s10586-023-04203-7 · Cited by 197 · see citing works
Koçak, B., Ponsiglione, A., Stanzione, A., Bluethgen, C., Santinha, J., Ugga, L., Huisman, M., Klontzas, M. E., Cannella, R., & Cuocolo, R. (2024) · Diagnostic and Interventional Radiology · doi:10.4274/dir.2024.242854 · Cited by 168 · see citing works
Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone
Abdin, M., Aneja, J., Awadalla, H., Awadallah, A., Awan, A. A., Bach, N., Bahree, A., Bakhtiari, A., Bao, J., Behl, H., et al. (2024) · arXiv · doi:10.48550/arxiv.2404.14219 · Cited by 166 · see citing works
Self-citationWysocki, O., Davies, J. K., Vigo, M., Armstrong, A. C., Landers, D., Lee, R., & Freitas, A. (2023) · Artificial Intelligence · doi:10.1016/j.artint.2022.103839 · Cited by 161 · see citing works
Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference
Chiang, W.-L., Zheng, L., Sheng, Y., Angelopoulos, A. N., Li, T., Li, D., Zhang, H., Zhu, B., Jordan, M., Gonzalez, J. E., et al. (2024) · arXiv · doi:10.48550/arxiv.2403.04132 · Cited by 118
A comprehensive evaluation of large Language models on benchmark biomedical text processing tasks
Jahan, I., Laskar, M. T. R., Peng, C., & Huang, J. X. (2024) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2024.108189 · Cited by 116 · see citing works
Schubach, M., Re, M., Robinson, P. N., & Valentini, G. (2017) · Scientific Reports · doi:10.1038/s41598-017-03011-5 · Cited by 97 · cited 1× · see citing works
“Medical datasets also frequently suffer from imbalances in phenotypes, like rare diseases or serious clinical outcomes 37 , and cohort identi fi cation 7 .”
Explainability in medicine in an era of AI-based clinical decision support systems
Pierce, R. L., Van Biesen, W., Van Cauwenberge, D., Decruyenaere, J., & Sterckx, S. (2022) · Frontiers in Genetics · doi:10.3389/fgene.2022.903600 · Cited by 95 · cited 1× · see citing works
“Check for updates Bowen Gu 1 , Rishi J. Desai 1 , Kueiyu Joshua Lin 1,5,6 & Jie Yang 1,2,3,4,5,6 Large Language Models (LLMs) have shown promise in clinical applications through prompt engineering, allowing fl exible clinical predictions.”
Machine learning for patient risk stratification for acute respiratory distress syndrome
Zeiberg, D., Prahlad, T., Nallamothu, B. K., Iwashyna, T. J., Wiens, J., & Sjoding, M. W. (2019) · PLoS ONE · doi:10.1371/journal.pone.0214465 · Cited by 89 · see citing works
Large language models for biomedicine: foundations, opportunities, challenges, and best practices
Sahoo, S. S., Plasek, J. M., Xu, H., Uzuner, Ö., Cohen, T., Yetişgen, M., Liu, H., Meystre, S. M., & Wang, Y. (2024) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocae074 · Cited by 63 · cited 1× · see citing works
“These models are particularly advantageous in clinical applications where tasks are diverse and annotated data is scarce and expensive to generate 12,13 .”
Yang, A., Yang, B., Hui, B., Zheng, B., Yu, B., Zhou, C., Li, C., Li, C., Liu, D., Huang, F., et al. (2024) · arXiv · doi:10.48550/arxiv.2407.10671 · Cited by 63 · see citing works
Self-citationScalable information extraction from free text electronic health records using large language models
Gu, B., Shao, V., Liao, Z., Carducci, V., Brufau, S. R., Yang, J., & Desai, R. J. (2025) · BMC Medical Research Methodology · doi:10.1186/s12874-025-02470-z · Cited by 57 · cited 2× · see citing works
“…Measuring Massive Multitask Language Understanding College Medicine (MMLU-CM) 42,43 , United States Medical Licensing Examination (USMLE) 44 , Mainland China Medical Licensing Examination (MCMLE) 44 — and one internal EHR dataset, Mass General Brigham - Social Determinant of Health (MGB-SDoH) 45 .”
Self-citationLiu, X., Wu, J., Shao, A., Shen, W., Ye, P., Wang, Y., Ye, J., Jin, K., & Yang, J. (2024) · Journal of Medical Internet Research · doi:10.2196/51926 · Cited by 52 · see citing works
Self-citationYi: Open Foundation Models by 01.AI
AI, 0., :, Young, A., Chen, B., Li, C., Huang, C., Zhang, G., Zhang, G., Wang, G., Li, H., et al. (2024) · arXiv · doi:10.48550/arxiv.2403.04652 · Cited by 50 · see citing works
The impact of prompt engineering in large language model performance: a psychiatric example
Grabb, D. (2023) · Journal of Medical Artificial Intelligence · doi:10.21037/jmai-23-71 · Cited by 48 · cited 1× · see citing works
“Prompt-based LLMs represent the most common usage of LLMs, as they can fl exibly instruct the models to perform different tasks by simply de fi ning prompt text without heavy model training 14 – 16 .”
Yang, J., Wang, L., Phadke, N. A., Wickner, P. G., Mancini, C. M., Blumenthal, K. G., & Zhou, L. (2020) · JAMA Network Open · doi:10.1001/jamanetworkopen.2020.22836 · Cited by 42 · see citing works
Self-citationWu, J., Wu, X., Qiu, Z., Li, M., Lin, S., Zhang, Y., Zheng, Y., Yuan, C., & Yang, J. (2024) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocae079 · Cited by 41 · cited 2× · see citing works
“Furthermore, due to data access and privacy concerns 34 , advanced LLMsoftenrelyongeneraldomaindatasetsandlackexposuretoreal-world EHRnotes,whichlimitstheireffectivenessinclinicalsettings 24,36 .”
Self-citationWu, J., Liu, X., Li, M., Li, W., Su, Z., Lin, S., Garay, L., Zhang, Z., Zhang, Y., Zeng, Q., et al. (2024) · NEJM AI · doi:10.1056/aira2400012 · Cited by 41 · cited 2× · see citing works
“Furthermore, due to data access and privacy concerns 34 , advanced LLMsoftenrelyongeneraldomaindatasetsandlackexposuretoreal-world EHRnotes,whichlimitstheireffectivenessinclinicalsettings 24,36 .”
Self-citationNiaf, É., Flamary, R., Rouvière, O., et al. (2014) · IEEE Transactions on Image Processing · doi:10.1109/tip.2013.2295759 · Cited by 38 · cited 1× · see citing works
“This paradigm naturally generates the probability of the predicted label and has been widely adopted in medical AI applications 5,7 – 9 .”
Evaluating Large Language Model (LLM) Performance on Established Breast Classification Systems
Haider, S. A., Pressman, S. M., Borna, S., Gomez-Cabello, C. A., Sehgal, A., Leibovich, B. C., & Forte, A. J. (2024) · Diagnostics · doi:10.3390/diagnostics14141491 · Cited by 29 · see citing works
Language Contamination Helps Explains the Cross-lingual Capabilities of English Pretrained Models
Blevins, T., & Zettlemoyer, L. (2022) · Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2022.emnlp-main.233 · Cited by 27 · see citing works
NumeroLogic: Number Encoding for Enhanced LLMs’ Numerical Reasoning
Schwartz, E., Choshen, L., Shtok, J., Doveh, S., Karlinsky, L., & Arbelle, A. (2024) · Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2024.emnlp-main.12 · Cited by 14 · see citing works
LCD benchmark: long clinical document benchmark on mortality prediction for language models
Yoon, W., Chen, S., Gao, Y., Zhao, Z., Dligach, D., Bitterman, D. S., Afshar, M., & Miller, T. A. (2024) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocae287 · Cited by 13 · see citing works
Proceedings of the Twenty-Second Conference on Uncertainty in Artificial Intelligence (2006)Preprint
Dechter, R., & Richardson, T. S. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1208.5160 · Cited by 12 · see citing works
Benchmarking LLMs via Uncertainty QuantificationPreprint
Ye, F., Yang, M., Pang, J., et al. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2401.12794 · Cited by 9 · see citing works
Fast Stochastic Recursive Momentum Methods for Imbalanced Data Mining
Wu, X., Huang, F., & Huang, H. (2022) · 2022 IEEE International Conference on Data Mining (ICDM) · doi:10.1109/icdm54844.2022.00068 · Cited by 7 · see citing works
Guiding Clinical Reasoning with Large Language Models via Knowledge Seeds
Wu, J., Wu, X., & Yang, J. (2024) · Proceedings of the Thirty-ThirdInternational Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2024/829 · Cited by 7 · see citing works
Self-citationBenchmarking Benchmark Leakage in Large Language Models
Xu, R., Wang, Z., Fan, R.-Z., & Liu, P. (2024) · arXiv · doi:10.48550/arxiv.2404.18824 · Cited by 6 · see citing works
Positional Description Matters for Transformers ArithmeticPreprint
Shen, R., Bubeck, S., Eldan, R., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2311.14737 · Cited by 3 · see citing works
Does Biomedical Training Lead to Better Medical Performance?Preprint
Dada, A., Bauer, M., Contreras, A. B., et al. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2404.04067 · Cited by 3 · see citing works
Combining Constraint Programming Reasoning with Large Language Model PredictionsPreprint
Régin, F., MARIA, E. D., & Bonlarron, A. (2024) · arXiv (Cornell University) · doi:10.4230/lipics.cp.2024.25 · Cited by 2 · see citing works
Implicit Chain of Thought Reasoning via Knowledge Distillation
Deng, Y., Prasad, K., Fernandez, R., Smolensky, P., Chaudhary, V., & Shieber, S. (2023) · arXiv · doi:10.48550/arxiv.2311.01460 · Cited by 1 · see citing works
Understanding Token Probability Encoding in Output EmbeddingsPreprint
Cho, H., Sakai, Y., Tanaka, K., et al. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2406.01468 · Cited by 1 · see citing works
Flagged first, then by citations
Research works printed but not linked: 11
- Devlin, J., Chang, M.-W., Lee, K. & Toutanova, K. in Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers). 4171–4186.
- Brown, T. et al. Language models are few-shot learners. Adv. neural Inf. Process. Syst. 33, 1877–1901 (2020).
- Zhou, W. & Ngo, T. H. Using Pretrained Large Language Model with Prompt Engineering to Answer Biomedical Questions. In Proceedings of the Conference and Labs of the Evaluation Forum, 3740, 253–268 (2024).
- Radford, A. et al. Language models are unsupervised multitask learners. OpenAI blog 1, 9 (2019).
- Wightman, G. P., Delucia, A. & Dredze, M. in Proceedings of the 3rd Workshop on Trustworthy Natural Language Processing (TrustNLP 2023). 326–362.
- Wang, X. et al. The Eleventh International Conference on Learning Representations.
- Xiao, Y. & Wang, W. Y. Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume.
- Martino, A., Iannelli, M. & Truong, C. in European Semantic Web Conference. 182–185 (Springer).
- Hendrycks, D. et al. Aligning AI with shared human values. In Proceedings of the International Conference on Learning Representations (2021).
- Pezeshkpour, P. & Hruschka, E. in Findings of the Association for Computational Linguistics: NAACL 2024. 2006–2017.
- Qi, Q., Luo, Y., Xu, Z., Ji, S. & Yang, T. Stochastic optimization of areas under precision-recall curves with provable convergence. Adv. neural Inf. Process. Syst. 34, 1752–1765 (2021).
Cited by
Sorted by citations
Howard, A., Reza, N., Green, P. L., Mo, Y., Duffy, E. M., Mwandumba, H. C., Gerada, A., & Hope, W. W. (2025) · The Lancet Infectious Diseases · doi:10.1016/s1473-3099(25)00313-5 · Cited by 31 · see citing works
Shyr, C., Ren, B., Hsu, C.-Y., Yan, C., Tinker, R. J., Cassini, T. A., Hamid, R., Wright, A., Bastarache, L., Peterson, J. F., et al. (2025) · medRxiv · doi:10.1101/2025.08.06.25333170 · Cited by 28 · see citing works
“This can raise concerns in diagnostic settings, as an LLM can generate divergent diagnostic suggestions for the same patient case across multiple runs, potentially undermining clinician trust and limiting the model’s utility for decision support [12, 13].”
Implications of integrating large language models into clinical decision making
Christof, M., & Armoundas, A. A. (2025) · Communications Medicine · doi:10.1038/s43856-025-01216-8 · Cited by 16 · see citing works
Kiguchi, K., Tu, Y., Ajito, K., Alnajjar, F. S. K., & Murase, K. (2025) · IEEE Access · doi:10.1109/access.2025.3582853 · Cited by 13 · see citing works
“Large language models (LLMs) have demonstrated remarkable capabilities in processing and synthesizing biomedical literature, contextualizing complex relationships, and generating testable hypotheses [19], [20].”
Sun, D., Hadjiiski, L., Bruno, G., Gormley, J., Chan, H.-P., Caoili, E. M., Cohan, R. H., Alva, A., Mihalcea, R., Zhou, C., et al. (2025) · Scientific Reports · doi:10.1038/s41598-025-27593-7 · Cited by 9 · see citing works
“Model performance variability, particularly due to the inherent probabilistic nature 8,9 of LLMs and differences in input prompts 10,11 or model updates, can potentially lead to outcome inconsistencies.”
Chang, Y., Liu, Y., Huang, S., & Hsu, W. (2025) · Psychiatry Research · doi:10.1016/j.psychres.2025.116501 · Cited by 6 · see citing works
Lee, S., Lee, H., Lee, H., et al. (2025) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocaf099 · Cited by 5 · see citing works
Implicit and explicit research quality score probabilities from ChatGPT
Thelwall, M., & Yang, Y. (2025) · Quantitative Science Studies · doi:10.1162/qss.a.393 · Cited by 5 · see citing works
Dundas, N. E., Law, T. J., Brender, T. D., Mills, H., Espejo, E., Heintz, T. A., Wallace, A. W., & Cobert, J. M. (2025) · Anesthesiology · doi:10.1097/aln.0000000000005795 · Cited by 7 · see citing works
BRIDGE: benchmarking large language models for understanding real-world clinical practice texts
Wu, J., Gu, B., Zhou, R., Xie, K., Snyder, D., Jiang, Y., Carducci, V., Wyss, R., Desai, R. J., Alsentzer, E., et al. (2026) · Nature Biomedical Engineering · doi:10.1038/s41551-026-01719-2 · Cited by 14 · see citing works
Self-citationChen, Z., Gao, Y., Gu, S., Yong, Z.-P., Lei, L., Wang, R., Lei, Q., & Zeng, S. (2026) · Anesthesiology and Perioperative Science · doi:10.1007/s44254-025-00140-3 · Cited by 4 · see citing works
“This study delves into the application of large language models (LLMs) in addressing these challenges [3], focusing on their potential to generate personalized and clinically relevant anesthesia plans [4].”
Zhang, H., Wang, A., He, C., Qiu, S., & An, Z. (2026) · Journal of the American Pharmacists Association · doi:10.1016/j.japh.2026.103077 · Cited by 3 · see citing works
Gorenshtein, A., Perek, S., Vaisbuch, Y., & Shelly, S. (2025) · Scientific Reports · doi:10.1038/s41598-025-22769-7 · Cited by 3 · see citing works
“These results suggest that LLMs exhibit difficulties in producing reliable prediction probabilities 29 .”
Sirisha, U., Kumar, C. K., Durgam, R., Eswaraiah, P., & Nagamani, G. M. (2025) · Computers, materials & continua/Computers, materials & continua (Print) · doi:10.32604/cmc.2025.063721 · Cited by 6 · see citing works
Tang, J., Abedi, A., Colella, T. J. F., & Khan, S. S. (2025) · Communications in computer and information science · doi:10.1007/978-981-95-0568-5_5 · Cited by 4 · see citing works
MoMA: a mixture-of-multimodal-agents architecture for enhancing clinical prediction modelling
Gao, J., Rahman, M., Caskey, J., Oguss, M., O’Rourke, A., Brown, R., Stey, A., Mayampurath, A., Churpek, M. M., Chen, G., et al. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-02219-4 · Cited by 2 · see citing works
“Whenprovidedwithclinicaltextinputs,LLMscanalsoserve as classi fi ers in clinical prediction tasks and have achieved outstanding performance 31 – 33 .”
Sepsis as Seen through the Eyes of AI: A Comparative evaluation of ChatGPT and Gemini
Bal, T., & Güner, R. (2025) · Infectious Diseases Now · doi:10.1016/j.idnow.2025.105228 · Cited by 2 · see citing works
Georgiou, G. P. (2025) · Journal of Autism and Developmental Disorders · doi:10.1007/s10803-025-07176-1 · Cited by 2 · see citing works
van Zanten, S., Boel, T. T., de Jong, J. S. Y., Bais, B., Fedorowski, A., Sutton, R., Selder, J. L., Giele, F., Geertsma, C., Scheffer, M. G., et al. (2025) · JACC Advances · doi:10.1016/j.jacadv.2025.102426 · Cited by 2 · see citing works
Generative Artificial Intelligence and Large Language Models in Clinical Oncology
Yu, Y., Zhao, Z., Wang, Z., Lin, R., Tan, Y., Chen, Y., Li, T., Baptista‐Hon, D. T., Zhang, X., Wu, C., et al. (2026) · MedComm · doi:10.1002/mco2.70833 · Cited by 2 · see citing works
“Moreover, current LLMs still struggle with atypical or edge cases due to the underrepresentation of such examples in training data, leading to compromised prediction reliability [ 87 ].”
Karaman, A. K., Yılmaz, B., Acar, K. H., Alizada, S., Korkmazer, B., Kartum, T. A., Samancı, C., Dağdelen, M., Çomunoğlu, N., Arslan, S. A., et al. (2026) · Journal of Imaging Informatics in Medicine · doi:10.1007/s10278-025-01793-z · Cited by 1 · see citing works
Small, W. R., Crowley, R. J., Pariente, C. S., Zhang, J., Eaton, K. P., Jiang, L. Y., Oermann, E. K., & Aphinyanaphongs, Y. (2026) · npj Health Systems · doi:10.1038/s44401-025-00059-8 · Cited by 1 · see citing works
“To compare the performance of generative and fi ne-tuned discriminative models, we used an implicit probability prediction method shown to outperform self-reported con fi dence measures 56 .”
Gupta, A. K., Adams, L. C., & Rangarajan, K. (2026) · Journal of Imaging Informatics in Medicine · doi:10.1007/s10278-026-01904-4 · Cited by 1 · see citing works
Léon, M., Feng, R., Quiroz-Flores, M., Pelletier, G. J., Bethencourt, D. M., Shibata, M., He, H., & Ruaengsri, C. (2026) · Frontiers in Digital Health · doi:10.3389/fdgth.2026.1769467 · Cited by 1 · see citing works
Large Language Models Generate Stigmatizing Language During Reasoning Over Real-World Clinical DataPreprint
Yang, Y., Gu, B., Hathaway, D. B., Wyss, R., Marengo, L., Gibbons, J. B., Lyndon, S., Wu, J., Chen, Q., Liu, N., et al. (2026) · medRxiv · doi:10.64898/2026.08.12.26360210 · Cited by 1 · see citing works
Self-citationKnowledge-Driven Hypothesis Generation for Burn Diagnosis from Ultrasound with Vision-Language Model
Rahman, M. M., Masry, M. S. E., Gordillo, G. M., & Wachs, J. P. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-98688-8_19
Mikoláš, P., Huth, F., Bröckel, K. L., Berndt, C., Martini, J., Maicher, B., Marxen, M., Verhees, F. G., Henneberg, P. M., Vogelbacher, C., et al. (2025) · doi:10.2196/preprints.84424
Mikoláš, P., Huth, F., Bröckel-Bundt, K., Berndt, C., Martini, J., Maicher, B., Marxen, M., Verhees, F. G., Henneberg, P. M., Vogelbacher, C., et al. (2025) · JMIR Mental Health · doi:10.2196/84424
“A significant disadvantage of LLMs is the lack of interpretability, bias risk due to nondisclosed training data [51] as well as of reliable probability estimates [52].”
Zhang, S., Wang, Z., & Kharrazi, H. H. K. (2025) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm66473.2025.11356803
“Although these models are not optimized for direct probabilistic prediction [15], their ability to generate reasoning-rich narratives provides a powerful source of complementary knowledge that can be leveraged during training to guide predictive models toward more clinically meaningful…”
Nakamura, F., Tsuchida, N., Matsuda, K., et al. (2026) · Health Care Management Science · doi:10.1007/s10729-026-09757-7
Large language models in radiology report quality: from linguistic adequacy to clinical validity
Büyükceran, E. U., Seyfettin, A., Babatürk, A., et al. (2026) · Clinical Radiology · doi:10.1016/j.crad.2026.107353
The vexatious black box of psychiatry may be smaller because of this issue
Grimes, T. (2026) · Current research in psychiatry · doi:10.46439/psychiatry.6.044
Huang, A., Vurgun, U., Thompson, A., Kuo, P. X., Ye, M., Batugo, A., Smith, A. M., Xia, C., Mowery, D. L., & Ky, B. (2026) · doi:10.2196/preprints.99147
Wang, Y., Zheng, J., Jiang, Y., Lee, C., & Li, Y. (2026) · Advanced Functional Materials · doi:10.1002/adfm.202526000 · Cited by 1 · see citing works
“This combination of real-time implantable sensing with large-scale predictive intelligence helps establish a closed-loop system for personalized diagnosis and adaptive treatment, paving the way for the next generation of intelligent bioelectronic medicine [358, 359].”
Haimovich, A. D., Erion, G., Nathanson, L. A., Cohen, C., Orcutt, R., Desai, S., Rubins, D. M., Hwang, U., Taylor, R. A., Shapiro, N. I., et al. (2026) · Annals of Emergency Medicine · doi:10.1016/j.annemergmed.2026.05.006
On-premises open-source large language models for privacy-preserving multimodal depression screening
Kwon, S., Kim, Y., Jhon, M., Park, J.-H., Lim, B., Jeon, E., Kim, J.-M., Kim, J.-W., & Lee, H. (2026) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2026.106577 · Cited by 1 · see citing works
Can AI assist in reducing diagnostic error? A narrative review
Scott, I. A. (2026) · Diagnosis · doi:10.1515/dx-2026-0021
“Using self-supervised learning, LLMs statistically correlate observed clinical features with potential diagnoses by large-scale pattern recognition based on vast training datasets [17].”
Shi, J., Ma, Y., Liu, Y., & Shi, B. (2026) · Frontiers in Public Health · doi:10.3389/fpubh.2026.1904062
“Simultaneously, recent studies emphasize persistent problems in calibration, hallucination, adversarial susceptibility, and insufficient expression of uncertainty (6–8, 21, 22).”
Performance and limitations of four large language models in genetic counseling for thalassemia
Zhong, W., Huang, J., Wei, M., et al. (2026) · Frontiers in Digital Health · doi:10.3389/fdgth.2026.1902480
Li, J., Zhang, J., Tao, X., Chen, Y., Xun, S., Qin, G., Diao, K., & Qin, C. (2026) · Journal of Medical Internet Research · doi:10.2196/91257
“These models perform well in text understanding and generation [11,12], and their application in clinical prediction tasks is increasing [13,14].”
Prat, Y., Mahmud, H. N., Tsur, A., et al. (2026) · World Journal of Urology · doi:10.1007/s00345-026-06761-y
Wang, B., Franciosi, A. N., O’Neill, L. M., Holden, S., McDonald, K., Gallagher, J., & Doran, P. (2026) · Journal of the American Heart Association · doi:10.1161/jaha.126.053665
Lu, J., McConeghy, K. W., Zullo, A. R., & Chen, J. (2026) · JMIR Medical Informatics · doi:10.2196/91151
Kızıltoprak, M., KARACA, M., & Uslu, M. Ö. (2026) · Journal of Clinical Medicine · doi:10.3390/jcm15197585
- Authors
- Gu, B., Desai, R. J., Lin, K. J., & Yang, J.
- Affiliations
- Brigham and Women's Hospital (US)
- Broad Institute (US)
- Corresponding author
- Yang, Jie
- Venue
- npj Digital Medicine
- Published
- 2024-12-19
- Volume
- 7
- Issue
- 1
- Pages
- 367-367
- ISSN
- 2398-6352 (ISSN-L)
- PMC
- PMC11659327
- OpenAlex
- W4405597099
- Semantic Scholar
- e85a458ee283dc1bd0bbeb44e1774e0908add8fa
- Cited by
- 50
- References
- 38
- Citations by year
- 2026: 30 · 2025: 20
- FWCI
- 4.67
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- npj Digital Medicine (publishedVersion, cc-by-nc-nd)
- PubMed Central (publishedVersion, cc-by-nc-nd)
- PubMed (publishedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by-nc-nd)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Institutes of Health (R01LM013204)
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
- Keywords
- large language models, medical prediction, predicted probabilities, clinical decision support, imbalanced medical data, model size effects
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-09.
Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-09.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
