Can Large Language Models Self-Correct in Medical Question Answering? An Exploratory Study
Open accessAbstract
Large language models (LLMs) have achieved strong performance on medical question answering (medical QA), and chain-of-thought (CoT) prompting has further improved results by eliciting explicit intermediate reasoning; meanwhile, self-reflective (self-corrective) prompting has been widely claimed to enhance model reliability by prompting LLMs to critique and revise their own reasoning, yet its effectiveness in safety-critical medical settings remains unclear. In this work, we conduct an exploratory analysis of self-reflective reasoning for medical multiple-choice question answering: using GPT-4o and GPT-4o-mini, we compare standard CoT prompting with an iterative self-reflection loop and track how predictions evolve across reflection steps on three widely used medical QA benchmarks (MedQA, HeadQA, and PubMedQA). We analyze whether self-reflection leads to error correction, error persistence, or the introduction of new errors. Our results show that self-reflective prompting does not consistently improve accuracy and its impact is highly dataset- and model-dependent: it yields modest gains on MedQA but provides limited or negative benefits on HeadQA and PubMedQA, and increasing the number of reflection steps does not guarantee better performance. These findings highlight a gap between reasoning transparency and reasoning correctness, suggesting that self-reflective reasoning is better viewed as an analytical tool for understanding model behavior rather than a standalone solution for improving medical QA reliability.
AI snapshot
Written from the paper’s full text.
TL;DR
This exploratory study compares chain-of-thought prompting with iterative self-reflection for medical multiple-choice QA using GPT-4o and GPT-4o-mini. Self-reflection does not consistently improve accuracy and is highly dataset- and model-dependent, suggesting it is better viewed as an analytical tool than a standalone reliability solution.
Key findings
- Self-reflection produced modest gains on MedQA: GPT-4o improved from 87.80% to 88.80%, and GPT-4o-mini from 74.80% to 76.00%.
- Effects were limited or negative on other benchmarks: HeadQA changed from 91.39% to 90.57% for GPT-4o and remained 79.10% for GPT-4o-mini; PubMedQA dropped from 70.80% to 69.00% for GPT-4o and from 71.60% to 69.60% for GPT-4o-mini.
- Accuracy typically stabilized after a small number of reflection steps and did not improve monotonically up to 10 iterations; most instances terminated with 0–1 reflection steps, and over 85% of GPT-4o instances on each dataset required no additional reflection beyond the initial answer.
- The paper concludes that self-reflection can correct some errors but can also preserve incorrect reasoning or introduce new errors, with larger models showing fewer reflection steps and more stable performance.
Methods
- Exploratory design comparing a standard chain-of-thought baseline against an iterative self-reflection loop that prompts the model to review and revise its own reasoning, with up to K = 10 reflection attempts unless otherwise specified.
- Evaluated GPT-4o and GPT-4o-mini via the OpenAI API in a zero-shot, black-box setting without parameter updates or fine-tuning, using official test splits of MedQA (1,273), HeadQA (244), and PubMedQA (500).
- Tracked predictions at each reflection step and computed accuracy against ground-truth answers; if the model did not report satisfaction within the maximum attempts, a majority-vote answer over generated predictions was also computed for analysis.
Limitations
- Stated: the study uses only prompt-based self-reflection without external supervision, retrieval, or verification signals, so reflection relies entirely on internal representations and may reinforce incorrect reasoning.
- Stated: reflection is evaluated primarily at the answer level, which may limit intervention in earlier reasoning steps where errors originate, and experiments cover a limited set of multiple-choice medical QA benchmarks rather than open-ended clinical reasoning or real-world decision support.
- Evident from the text: the analysis is exploratory and limited to two GPT-4o-family models accessed as black boxes, with no fine-tuning or training, so findings may not generalize to other model families or scales.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
0 of 6 cited sources checked · 0 supported · 0 flagged
- Findingown data
ChatGPT-4o consistently requires fewer reflection steps than ChatGPT-4o-mini.
“ChatGPT-4o consistently requires fewer reflection steps than ChatGPT-4o-mini”
Quote in the paperSubstantial claim - Findingown data
For ChatGPT-4o, over 85% of instances on each dataset require no additional reflection beyond the initial answer.
“For ChatGPT-4o, over 85% of instances on each dataset require no additional reflection beyond the initial answer.”
Quote in the paperSubstantial claim - Findingown data
On HeadQA, ChatGPT-4o accuracy decreased slightly after self-reflection, from 91.39% to 90.57%.
“on HeadQA, ChatGPT-4o slightly decreases after self-reflection (from 91.39% to 90.57%)”
Quote in the paperSubstantial claim - Findingown data
On PubMedQA, self-reflection decreased ChatGPT-4o accuracy from 70.80% to 69.00%.
“on PubMedQA, self-reflection slightly degrades performance for both models (ChatGPT-4o: 70.80% → 69.00%”
Quote in the paperSubstantial claim - Findingown data
On MedQA, self-reflection increased ChatGPT-4o accuracy from 87.80% to 88.80%.
“On MedQA, self-reflection yields modest gains for both models, increasing accuracy from 87.80% to 88.80% for ChatGPT-4o”
Quote in the paperSubstantial claim
- Findingown data
Across all datasets, most instances terminate with 0-1 reflection steps.
“Across all datasets, most instances terminate with 0-1 reflection steps”
Quote in the paperSubstantial claim - Findingown data
PubMedQA requires the fewest reflection steps overall.
“whereas PubMedQA requires the fewest steps overall.”
Quote in the paperNot a substantial claim - Findingown data
For HeadQA and MedQA, ChatGPT-4o accuracy remains relatively stable across reflection steps with only minor fluctuations.
“For HeadQA and MedQA, ChatGPT-4o remains relatively stable across reflection steps, with only minor fluctuations.”
Quote in the paperSubstantial claim - Findingown data
Deep reflection near the maximum of 10 steps remains uncommon in all settings.
“However, deep reflection (near the maximum of 10 steps) remains uncommon in all settings.”
Quote in the paperNot a substantial claim - Findingown data
ChatGPT-4o-mini shows larger variance across reflection steps, especially on MedQA where accuracy oscillates rather than steadily improving.
“ChatGPT-4o-mini shows larger variance, especially on MedQA, where accuracy oscillates across iterations rather than steadily improving.”
Quote in the paperSubstantial claim - Findingown data
On MedQA, self-reflection increased ChatGPT-4o-mini accuracy from 74.80% to 76.00%.
“and from 74.80% to 76.00% for ChatGPT-4o-mini”
Quote in the paperSubstantial claim - Findingown data
On PubMedQA, self-reflection decreased ChatGPT-4o-mini accuracy from 71.60% to 69.60%.
“ChatGPT-4o-mini: 71.60% → 69.60%)”
Quote in the paperSubstantial claim - Findingown data
On PubMedQA, both models produce nearly flat accuracy curves across reflection steps.
“On PubMedQA, both models produce nearly flat curves”
Quote in the paperSubstantial claim - Findingown data
On HeadQA, ChatGPT-4o-mini accuracy was unchanged by self-reflection, at 79.10% in both conditions.
“while ChatGPT-4o-mini shows no change (79.10% in both settings)”
Quote in the paperSubstantial claim - Findingown data
MedQA induces the highest average reflection depth for both models.
“MedQA induces the highest average reflection depth for both models”
Quote in the paperNot a substantial claim - Interpretationreasoning
The effectiveness of self-reflection is closely tied to model capacity and reasoning robustness.
“This highlights that the effectiveness of self-reflection is closely tied to model capacity and reasoning robustness.”
Quote in the paperSubstantial claim - Interpretationreasoning
Once a model commits to an initial interpretation, subsequent reflection steps may serve to rationalize that decision rather than critically challenge it, resembling confirmation bias.
“Once a model commits to an initial interpretation, subsequent reflection steps may serve to rationalize that decision rather than critically challenge it, resembling confirmation bias in human reasoning.”
Quote in the paperSubstantial claim - Interpretationreasoning
Self-reflective prompting may not function as a true error-correction mechanism, but instead may reinforce existing reasoning trajectories.
“self-reflective prompting may not function as a true error-correction mechanism, but instead may reinforce existing reasoning trajectories.”
Quote in the paperSubstantial claim - Interpretationown data
Self-reflective reasoning, as a purely prompt-based inference strategy, does not reliably improve medical multiple-choice question answering performance.
“self-reflective reasoning is not a reliable performance enhancement mechanism for medical multiple-choice question answering.”
- Also stated in: § IV. DISCUSSION · boosted
“Our results provide empirical evidence that self-reflective reasoning, when applied as a purely prompt-based inference strategy, does not reliably improve medical multiple-choice question answering performance.”
Quote in the paperSubstantial claim - Also stated in: § IV. DISCUSSION · boosted
- Interpretationreasoning
The impact of self-reflection is highly dependent on dataset characteristics and model scale.
“The impact of selfreflection is highly dependent on both dataset characteristics and model scale”
Quote in the paperSubstantial claim - Interpretationown data
On PubMedQA, additional reflection rarely changes the final answer.
“suggesting that additional reflection rarely changes the final answer.”
Quote in the paperSubstantial claim - Interpretationown data
Increasing the number of reflection steps does not guarantee improved accuracy.
“increasing the number of reflection steps does not guarantee improved accuracy.”
Quote in the paperSubstantial claim - Interpretationreasoning
The findings highlight a gap between reasoning transparency and reasoning correctness.
“These results highlight a critical gap between reasoning transparency and reasoning correctness”
Quote in the paperSubstantial claim - Premisecitation
HeadQA consists of medical examination questions from Spanish medical specialization exams covering diverse clinical specialties and realistic decision-making scenarios.
“HeadQA [25] consists of medical examination questions collected from Spanish medical specialization exams, covering diverse clinical specialties and realistic decision-making scenarios.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [25]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- May combine two claims. 9
- Classified as probably not a substantial claim. 9
May combine two claims.
Findingown dataChatGPT-4o-mini engages in reflection more often than ChatGPT-4o, most notably on MedQA where a substantial portion of examples require multiple reflection steps.
“ChatGPT-4o-mini engages in reflection more often, most notably on MedQA, where a substantial portion of examples require multiple reflection steps.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataAcross datasets, accuracy typically stabilizes after a small number of reflection steps and does not improve monotonically as reflection continues.
“Across datasets, accuracy typically stabilizes after a small number of steps and does not exhibit monotonic improvement as reflection continues.”
Quote in the paperSubstantial claimMay combine two claims.
Interpretationown dataReflection can occasionally correct initial errors, but it can also preserve incorrect reasoning or introduce new errors.
“While reflection can occasionally correct initial errors, it can also preserve incorrect reasoning or introduce new errors.”
Quote in the paperSubstantial claimMay combine two claims.
Interpretationown dataLarger models such as ChatGPT-4o tend to require fewer reflection steps and exhibit more stable performance, suggesting higher internal consistency in their initial chain-of-thought reasoning.
“Larger models such as ChatGPT-4o tend to require fewer reflection steps and exhibit more stable performance, suggesting higher internal consistency in their initial chain-of-thought reasoning.”
Quote in the paperSubstantial claimMay combine two claims.
Interpretationown dataSmaller models are more prone to engaging in reflection and show greater variability across reflection steps, indicating iterative self-evaluation may introduce instability rather than correction when underlying reasoning is weak.
“Smaller models, while more prone to engaging in reflection, show greater variability across reflection steps, indicating that iterative selfevaluation may introduce instability rather than correction when underlying reasoning is weak.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationChain-of-thought prompting offers a practical benefit for interpretability and transparency by revealing an inspectable reasoning trace.
“Beyond performance improvements, CoT prompting offers a practical benefit for interpretability and transparency [18]: it reveals a reasoning trace that can be inspected, compared, and analyzed”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [18]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductnone givenThe study evaluated self-reflection primarily at the answer level, which may limit its ability to intervene in earlier reasoning steps where errors originate.
“we evaluate self-reflection primarily at the answer level, which may limit its ability to intervene in earlier reasoning steps where errors originate.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim.
Study design and conductnone givenAll datasets were evaluated using their official test splits.
“All datasets are evaluated using their official test splits”
Quote in the paperSubstantial claimMay combine two claims.
Interpretationown dataSelf-reflection does not consistently improve performance; its effect depends on both the dataset and the model.
“Overall, self-reflection does not consistently improve performance; instead, its effect depends on both the dataset and the model.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationChain-of-thought prompting has been shown to improve accuracy on tasks that require step-by-step reasoning.
“By explicitly instructing LLMs to produce intermediate reasoning steps before committing to a final answer, CoT prompting has been shown to improve accuracy on tasks that require step-by-step reasoning.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [14]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Classified as probably not a substantial claim.
PremisecitationMedQA-USMLE is designed to assess multi-step clinical reasoning, differential diagnosis, and hypothesis evaluation rather than merely retrieving isolated facts.
“MedQA-USMLE is designed to assess whether models can perform multi-step clinical reasoning, differential diagnosis, and hypothesis evaluation, rather than merely retrieving isolated facts.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [13]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
May combine two claims.
InterpretationreasoningSelf-reflection encourages explicit reasoning and self-evaluation but does not ensure error correction.
“although self-reflection encourages explicit reasoning and self-evaluation, it does not ensure error correction.”
Quote in the paperSubstantial claimMay combine two claims.
InterpretationreasoningMedQA questions often involve multi-step clinical reasoning and differential diagnosis, where revisiting intermediate assumptions may occasionally help identify overlooked inconsistencies.
“MedQA questions often involve multi-step clinical reasoning and differential diagnosis, where revisiting intermediate assumptions may occasionally help identify overlooked inconsistencies.”
Quote in the paperSubstantial claimClassified as probably not a substantial claim. May combine two claims.
PremisecitationPubMedQA is a biomedical QA dataset constructed from PubMed abstracts, requiring reasoning grounded in scientific evidence rather than exam-style clinical heuristics.
“PubMedQA [26] is a biomedical question answering dataset constructed from PubMed abstracts, requiring reasoning grounded in scientific evidence rather than exam-style clinical heuristics.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [26]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim: may not pass S2 (checkable).
PremisecitationMedical QA is inherently challenging because it requires broad biomedical knowledge, domain-specific terminology understanding, multi-step reasoning, and integrating evidence under uncertainty.
“medical QA is inherently challenging because it requires a combination of broad biomedical knowledge, domain-specific terminology understanding, multi-step reasoning, and the ability to integrate evidence under uncertainty [12].”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [12]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Classified as probably not a substantial claim.
InterpretationreasoningThe dataset-dependent behavior may be explained by the nature of the reasoning required by different medical QA benchmarks.
“One possible explanation for the dataset-dependent behavior observed in our results lies in the nature of the reasoning required by different medical QA benchmarks.”
Quote in the paperSubstantial claimClassified as probably not a substantial claim.
InterpretationreasoningHeadQA, while exam-based, tends to be more structured, leaving fewer opportunities for reflection-driven revision once an initial reasoning path has been formed.
“HeadQA, while also exam-based, tends to be more structured, leaving fewer opportunities for reflection-driven revision once an initial reasoning path has been formed.”
Quote in the paperSubstantial claim
The paper relies on this without support.
Incorrect conclusions accompanied by coherent and confident explanations may increase user trust despite being wrong.
“incorrect conclusions accompanied by coherent and confident explanations may increase user trust despite being wrong.”
The paper relies on this without support.
Self-reflection operates purely at inference time without additional training data, fine-tuning, or parameter updates, making it broadly applicable to black-box models accessed via APIs.
“it operates purely at inference time, without requiring additional training data, fine-tuning, or parameter updates, making it broadly applicable to black-box models accessed via APIs.”
The paper relies on this without support.
The three medical QA benchmarks cover complementary sources of medical knowledge and reasoning demands.
“three widely used medical QA benchmarks that cover complementary sources of medical knowledge and reasoning demands.”
The paper relies on this without support.
The effectiveness and reliability of self-reflection in medical question answering remain insufficiently characterized.
“its effectiveness and reliability in medical question answering remain insufficiently characterized.”
The paper relies on this without support.
Medicine is a safety-critical domain, so evaluation of LLMs must consider reliability, robustness, and trustworthiness in addition to accuracy.
“because medicine is a safety-critical domain, the evaluation of LLMs must consider not only accuracy but also reliability, robustness, and trustworthiness”
Cites
Chain-Of-Thought Prompting Elicits Reasoning in Large Language Models
Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E., Le, Q. V., & Zhou, D. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-1800 · Cited by 4,505 · cited 2× · see citing works
“A key driver behind recent performance gains in complex reasoning tasks, including medical QA, has been the development of reasoning-oriented prompting strategies, most notably chain-of-thought (CoT) prompting [14].”
Jin, D., Pan, E., Oufattole, N., Weng, W.-H., Fang, H., & Szolovits, P. (2021) · Applied Sciences · doi:10.3390/app11146421 · Cited by 718 · see citing works
Jin, D., Pan, E., Oufattole, N., Weng, W.-H., Fang, H., & Szolovits, P. (2021) · doi:10.20944/preprints202105.0498.v1 · Cited by 718 · see citing works
PubMedQA: A Dataset for Biomedical Research Question Answering
Jin, Q., Dhingra, B., Liu, Z., Cohen, W., & Lu, X. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1259 · Cited by 685 · see citing works
Vrdoljak, J., Boban, Z., Vilović, M., Kumrić, M., & Božić, J. (2025) · Healthcare · doi:10.3390/healthcare13060603 · Cited by 179 · see citing works
Large language models for disease diagnosis: a scoping review
Zhou, S., Xu, Z., Zhang, M., Xu, C., Guo, Y., Zhan, Z., Fang, Y., Ding, S., Wang, J., Xu, K., et al. (2025) · npj Artificial Intelligence · doi:10.1038/s44387-025-00011-z · Cited by 143 · cited 1× · see citing works
“…domain, these models have demonstrated strong performance on medical question answering (medical QA) benchmarks and related clinical NLP tasks [11], prompting increasing interest in their potential use for clinical reasoning assistance, educational support, and downstream decision-making…”
Self-citationChain of Thought Utilization in Large Language Models and Application in Nephrology
Miao, J., Thongprayoon, C., Suppadungsuk, S., Krisanapan, P., Radhakrishnan, Y., & Cheungpasitporn, W. (2024) · Medicina · doi:10.3390/medicina60010148 · Cited by 108 · see citing works
Iterative Reasoning Preference Optimization
Pang, R. Y., Yuan, W., Cho, K., He, H., Sukhbaatar, S., & Weston, J. (2024) · neural information processing systems · doi:10.52202/079017-3702 · Cited by 57 · see citing works
Zhang, Z., Yao, Y., Zhang, A., Tang, X., Ma, X., He, Z., Wang, Y., Gerstein, M., Wang, R., Liu, G., et al. (2025) · ACM Computing Surveys · doi:10.1145/3719341 · Cited by 51 · see citing works
Wang, L., Hu, Y., He, J., Xu, X., Liu, N., Liu, H., & Shen, H. T. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i17.29884 · Cited by 50 · see citing works
Generative Models in Medical Visual Question Answering: A Survey
Dong, W., Shen, S., Han, Y., Tao, T., Wu, J., & Xu, H. (2025) · Applied Sciences · doi:10.3390/app15062983 · Cited by 35 · see citing works
Automating expert-level medical reasoning evaluation of large language models
Zhou, S., Xie, W., Li, J., Zhan, Z., Song, M., Yang, H., Espinoza, C., Welton, L., Mai, X., Jin, Y., et al. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-02208-7 · Cited by 30 · see citing works
Self-citationZhan, Z., Zhou, S., Li, M., & Zhang, R. (2025) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocaf002 · Cited by 27 · see citing works
Self-citationLi, M., Zhan, Z., Yang, H., Xiao, Y., Zhou, H., Huang, J., & Zhang, R. (2025) · Science Advances · doi:10.1126/sciadv.adr1443 · Cited by 21 · see citing works
Self-citationFang, J., Liu, C.-T., Kim, J., et al. (2024) · Recent Advances in Natural Language Processing · doi:10.48550/arxiv.2412.15487 · Cited by 13
Zhan, Z., Wang, J., Zhou, S., Deng, J., & Zhang, R. (2025) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocaf128 · Cited by 11 · see citing works
Self-citationBenchmarking GPT-5 for biomedical natural language processingPreprint
Hou, Y., Zhan, Z.-F., Zeng, M., Wu, Y.-F., Zhou, S., & Zhang, R. (2025) · arXiv.org · doi:10.48550/arxiv.2509.04462 · Cited by 7 · see citing works
Self-citationIs Depth All You Need? An Exploration of Iterative Reasoning in LLMsPreprint
Wu, Z.-Q., Li, T., Xu, B., Yang, J., Zhan, M., Zhu, X., & Feng, L. (2025) · arXiv.org · doi:10.48550/arxiv.2502.10858 · Cited by 6 · see citing works
Large Language Models for Disease Diagnosis: A Scoping ReviewPreprint
Zhou, S., Xu, Z., Zhang, M., et al. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2409.00097 · Cited by 6 · see citing works
An evaluation of DeepSeek Models in Biomedical Natural Language ProcessingPreprint
Zhan, Z., Zhou, S., Zhou, H., et al. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2503.00624 · Cited by 2 · see citing works
Self-citationPEER: Towards reliable and efficient inference via Patience-Based Early Exiting with Rejection
Zhan, Z., Zhou, S., & Zhang, R. (2026) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2026.104988 · Cited by 2 · see citing works
Self-citationMedCoT: Medical Chain of Thought via Hierarchical ExpertPreprint
Liu, J., Wang, Y., Du, J., et al. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2412.13736 · Cited by 1 · see citing works
Iteration of Thought: Leveraging Inner Dialogue for Autonomous Large Language Model ReasoningPreprint
Radha, S. K., Jelyani, Y. N., Ghukasyan, A., et al. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2409.12618
MedReflect: Teaching Medical LLMs to Self-Improve via Reflective CorrectionPreprint
Huang, Y., Chen, Y., Xu, D., et al. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2510.03687
Cao, S., Zhang, Z., Jiao, J., et al. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2504.17213
Lin, Z., Xu, Z., Song, X., Wan, Y., Yao, X., Lin, T.-H., Song, S., Subbaraman, P., Zhou, B., Chang, K.-W., et al. (2025) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2025.findings-acl.975
OpenAI, Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F. L., Almeida, D., Altenschmidt, J., Altman, S., et al. (2023) · arXiv · doi:10.48550/arxiv.2303.08774
Flagged first, then by citations
Research works printed but not linked: 1
- Vilares, HEAD-QA: A healthcare dataset for complex reasoning, 960
Cited by
Sorted by citations
Constructing and Evaluating Clinical Reasoning Trajectories for Medical Agent
Zhu, Y.-Q., Zhang, W.-S., & Yang, X.-B. (2026) · doi:10.48550/arxiv.2609.05090
“However, the effectiveness of self-reflective reasoning in medical settings remains dataset-and model-dependent [30], indicating a gap between reasoning transparency and reasoning correctness.”
Xu, Y., Wu, J., Zhang, B.-W., et al. (2026) · arXiv.org · doi:10.48550/arxiv.2605.20867
- Authors
- Zhan, Z., Cui, M., & Zhang, R.
- Affiliations
- University of Minnesota (US)
- University of St. Thomas
- Venue
- 2026 IEEE 14th International Conference on Healthcare Informatics (ICHI)
- Publisher
- IEEE
- Published
- 2026-06
- Pages
- 1388-1390
- OpenAlex
- W7148483564
- Semantic Scholar
- a6541e27274e536e95486c83cdc8969a17227125
- Cited by
- 0
- References
- 3
- FWCI
- 0.00
- Type
- conference paper
- Open access
- green
- arxiv.org
- Licence
- Locations
- IEEE International Conference on Healthcare Informatics (ICHI) (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Center for Complementary and Integrative Health
- National Institute on Aging
- National Cancer Institute
- National Institute of Diabetes and Digestive and Kidney Diseases
- National Institute on Minority Health and Health Disparities
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Keywords
- large language models, medical question answering, chain-of-thought prompting, multiple-choice question answering, error correction, GPT-4o, MedQA, PubMedQA
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-07.
Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-07.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
