MedARC: Adaptive multi-agent refinement and collaboration for enhanced medical reasoning in large language models
Abstract
BACKGROUND Large Language Models (LLMs) have shown remarkable potential in medical question answering (QA), yet their deployment in clinical settings remains limited by hallucinations, inconsistent reasoning, and difficulties in handling complex biomedical information. METHODS To address these challenges, we propose MedARC (Medical Agent Refinement and Collaboration), a novel multi-agent framework that enhances medical QA through structured debate among multiple LLM agents. MedARC introduces two key mechanisms: (1) structured inter-agent summarization to extract and refine key agreements and disagreements across agents, and (2) confidence-aware aggregation to synthesize final answers based on the most reliable and well-reasoned contributions. RESULTS We evaluate MedARC on three representative medical QA benchmarks, spanning yes/no, factoid, and open-ended QA tasks. Experimental results demonstrate that MedARC significantly improves performance over zero-shot and chain-of-thought (CoT) prompting baselines. For instance, in the yes/no QA task on PubMedQA, MedARC boosts accuracy from 72.9 % (zero-shot) to 77.2 % using DeepSeek-V3 as the backbone model. Human evaluation further confirms gains in factual consistency and completeness with MedARC. We conduct extensive ablation studies and sensitivity analyses, revealing that MedARC's structured summarization and aggregation modules contribute independently to performance improvements. Additionally, we explore the impact of heterogeneous agent configurations and varying numbers of debate rounds on overall system effectiveness. CONCLUSIONS Overall, MedARC provides a reliable and scalable solution for enhancing LLM-based medical QA systems. The code for this study is publicly available on GitHub (https://github.com/asdmiao/MedARC).
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 48,293 · see citing works
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372 · see citing works
Kaugeranna, R., Kaugeranna, E., & Aion, (. S. 4. 6. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.giscience.2023.43 · Cited by 13,695 · see citing works
Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks
Reimers, N., & Gurevych, I. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1410 · Cited by 12,778 · see citing works
BioBERT: a pre-trained biomedical language representation model for biomedical text mining
Lee, J., Yoon, W., Kim, S., Kim, D., Kim, S., So, C. H., & Kang, J. (2020) · Bioinformatics · doi:10.1093/bioinformatics/btz682 · Cited by 7,218 · see citing works
Large language models encode clinical knowledge
Singhal, K., Azizi, S., Tu, T., Mahdavi, S. S., Wei, J., Chung, H. W., Scales, N., Tanwani, A., Cole-Lewis, H., Pfohl, S., et al. (2023) · Nature · doi:10.1038/s41586-023-06291-2 · Cited by 4,083 · see citing works
Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., Küttler, H., Lewis, M., Yih, W.-T., Rocktäschel, T., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.11401 · Cited by 3,101 · see citing works
Domain-specific language model pretraining for biomedical natural language processing
Gu, Y., Tinn, R., Cheng, H., Lucas, M., Usuyama, N., Liu, X., Naumann, T., Gao, J., & Poon, H. (2022) · ACM Transactions on Computing for Healthcare · doi:10.1145/3458754 · Cited by 2,170 · see citing works
Huang, L., Yu, W., Ma, W., Zhong, W., Feng, Z., Wang, H., Chen, Q., Peng, W., Feng, X., Qin, B., et al. (2025) · ACM Transactions on Information Systems · doi:10.1145/3703155 · Cited by 2,142 · see citing works
Self-Consistency Improves Chain of Thought Reasoning in Language Models
Wang, X., Wei, J., Schuurmans, D., Le, Q., Chi, E., Narang, S., Chowdhery, A., & Zhou, D. (2022) · arXiv · doi:10.48550/arxiv.2203.11171 · Cited by 829 · see citing works
PubMedQA: A Dataset for Biomedical Research Question Answering
Jin, Q., Dhingra, B., Liu, Z., Cohen, W., & Lu, X. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1259 · Cited by 685 · see citing works
Analyzing Data Measured By Individual Likert-Type Items
Clason, D. L., & Dormody, T. J. (1994) · Journal of Agricultural Education · doi:10.5032/jae.1994.04031 · Cited by 642 · see citing works
Large language models in healthcare and medical domain: A review
Nazi, Z. A., & Peng, W. (2024) · Informatics · doi:10.3390/informatics11030057 · Cited by 415 · see citing works
Opportunities and Challenges for ChatGPT and Large Language Models in Biomedicine and Health
Tian, S., Jin, Q., Yeganova, L., Lai, P.-T., Zhu, Q., Chen, X., Yang, Y., Chen, Q., Kim, W., Comeau, D. C., et al. (2023) · Briefings in Bioinformatics · doi:10.1093/bib/bbad493 · Cited by 359 · see citing works
Comparing ChatGPT and GPT-4 performance in USMLE soft skill assessments
Brin, D., Sorin, V., Vaid, A., Soroush, A., Glicksberg, B. S., Charney, A. W., Nadkarni, G., & Klang, E. (2023) · Scientific Reports · doi:10.1038/s41598-023-43436-9 · Cited by 319 · see citing works
CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Society
Li, G., Hammoud, H. A. A. K., Itani, H., Khizbullin, D., & Ghanem, B. (2023) · arXiv · doi:10.48550/arxiv.2303.17760 · Cited by 315 · see citing works
Camel: Communicative agents for" mind" exploration of large language model society
Li, G., Hammoud, H., Itani, H., Khizbullin, D., & Ghanem, B. (2023) · Advances in Neural Information Processing Systems 36 · doi:10.52202/075280-2264 · Cited by 314 · see citing works
Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debate
Liang, T., He, Z., Jiao, W., Wang, X., Wang, Y., Wang, R., Yang, Y., Shi, S., & Tu, Z. (2024) · Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2024.emnlp-main.992 · Cited by 274 · see citing works
A Survey of Confidence Estimation and Calibration in Large Language Models . -
Geng, J., Cai, F., Wang, Y., Koeppl, H., Nakov, P., & Gurevych, I. (2024) · Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers) · doi:10.18653/v1/2024.naacl-long.366 · Cited by 88 · see citing works
MDAgents: an adaptive collaboration of LLMs for medical decision-making
Kim, Y., Park, C., Jeong, H., Chan, Y., Xu, X., McDuff, D., Lee, H., Ghassemi, M., Breazeal, C., & Park, H. (2024) · Advances in Neural Information Processing Systems 37 · doi:10.52202/079017-2522 · Cited by 81 · see citing works
Benchmarking Large Language Models in Evidence-Based Medicine
Li, J., Deng, Y., Sun, Q., Zhu, J., Tian, Y., Li, J., & Zhu, T. (2025) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2024.3483816 · Cited by 35 · see citing works
Self-citationConfidence Calibration and Rationalization for LLMs via Multi-Agent Deliberation
Yang, R., Rajagopal, D., Hayati, S. A., et al. (2024) · arXiv.org · doi:10.48550/arxiv.2404.09127 · Cited by 29
Nentidis, A., Katsimpras, G., Vandorou, E., Krithara, A., Miranda-Escalada, A., Gascó, L., Krallinger, M., & Παλιούρας, Γ. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-13643-6_22 · Cited by 25 · see citing works
de Oliveira, R., Garber, M., Gwinnutt, J. M., Rashidi, E., Hwang, J.-H. (., Gilmour, W., Nanavati, J., Abidine, K. Z. E., & Mack, C. D. (2025) · JAMIA Open · doi:10.1093/jamiaopen/ooaf058 · Cited by 17 · see citing works
Nentidis, A., Katsimpras, G., Krithara, A., et al. (2024) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-71908-0_1
Flagged first, then by citations
Cited by
Sorted by citations
Artificial intelligence agents in healthcare research: A scoping review
Njei, B., Al-Ajlouni, Y. A., Sidney Kanmounye, U., Boateng, S., Loic Nguefang, G., Njei, N., Hamouri, S., & Al-Ajlouni, A. F. (2026) · PLOS One · doi:10.1371/journal.pone.0342182 · Cited by 27 · see citing works
“Concerns regarding safety and trustworthiness persist, particularly the problem of LLM hallucination, which can compromise output quality despite the use of self-correction loops [20,21,30].”
InfluentialApplications of DeepSeek in Medicine: Bibliometric Analysis and Scoping Review
Zhang, H., Wang, D., Xu, Y., Han, S., & Wang, G. (2026) · Journal of Medical Internet Research · doi:10.2196/93354 · Cited by 1 · see citing works
Enhancing Medical Question Answering via a Multi-Agent Debate Framework
Lu, D., Jiang, V., Wang, S., Zhu, M., & Zhai, S. (2026) · IEEE Annual Computing and Communication Workshop and Conference (CCWC) · doi:10.1109/ccwc67433.2026.11393824
“MedARC [26] structures debate into three stages — independent reasoning, summarization of agreement/disagreement, and confidence-weighted aggregation — to improve transparency and accuracy.”
Agentic AI in Healthcare 5.0: Reference Architecture, Applications, and Challenges
Gadekallu, T. R., Huynh‐The, T., Chehri, A., Mohammed, S. Y., Saleh, H. M., Bhattacharya, P., Kodati, D., Donta, P. K., Masuduzzaman, M., Yenduri, G., et al. (2026) · IEEE Internet of Things Journal · doi:10.1109/jiot.2026.3718925
“MedARC [52] uses structured multiagent deliberation, summarization, and confidence-aware aggregation to improve medical question answering.”
Li, J., Zhang, J., Tao, X., Chen, Y., Xun, S., Qin, G., Diao, K., & Qin, C. (2026) · Journal of Medical Internet Research · doi:10.2196/91257
“The multiagent framework represents an innovative approach that significantly enhances the capabilities of LLMs to handle complex tasks [32,33] by enabling multiple agents to engage in discussions around the same problem and ultimately reach a consensus on the output.”
Self-citationMACA-PA: A Modular Agent-based Clinical Assistant for Perioperative Anesthesia
Jiang, Y., Ma, C., Xu, Z.-Z., et al. (2026) · Proceedings of the 2026 2nd International Conference on Health Informatization and Data Analysis · doi:10.1145/3819113.3819136
Zhong, R., Liu, D., Dai, Y., Li, Z., Song, Z., Li, Z., Zhang, L., Chen, S., Su, Y., Sun, Y., et al. (2026) · Chinese Medicine · doi:10.1186/s13020-026-01512-y
“Recent advances in LLMs and related techniques, including retrieval-augmented generation, knowledge graphs, and multi-agent systems, have created new opportunities for improving knowledge grounding, reasoning, and interaction in medical applications [15–18].”
- Authors
- Miao, Y., Wen, J., Luo, Y., & Li, J.
- Affiliations
- Nanjing University of Information Science and Technology (CN)
- Corresponding author
- Li, Jin
- Venue
- International Journal of Medical Informatics
- Publisher
- Elsevier BV
- Published
- 2026-02
- Volume
- 206
- Pages
- 106136
- ISSN
- 1386-5056 (ISSN-L)
- OpenAlex
- W4415226090
- Semantic Scholar
- e52f3455056b5067d568304f74b10469348a1b68
- Cited by
- 7
- References
- 15
- Citations by year
- 2026: 7
- FWCI
- 1.56
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- Locations
- International Journal of Medical Informatics (publishedVersion)
- PubMed (publishedVersion)
- Funders
- National Natural Science Foundation of China (82302352, 82441029)
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Biomedical Text Mining and Ontologies
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Keywords
- multi-agent systems, large language models, medical question answering, hallucination mitigation, LLM agents, clinical reasoning, factual consistency
- MeSH terms
- Clinical Reasoning, Large Language Models, Humans, Language, Natural Language Processing
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
