CoChat
Sign Up
PreprintCited by 6Artificial Intelligence in HealthcareArtificial Intelligence in Healthcare and EducationMachine Learning in Healthcare

Beyond Direct Diagnosis: LLM-based Multi-Specialist Agent Consultation for Automatic Diagnosis

Open access
Wang, Haochun, Zhao, Sendong, Qiang, Zewen, Xi, Nuwa, Qin, Bing, Liu, Ting·2024·arXiv
DOI 10.48550/arxiv.2401.16107arXiv 2401.16107
Open this paper in CoChat — free accountOpen PDF

Abstract

Automatic diagnosis is a significant application of AI in healthcare, where diagnoses are generated based on the symptom description of patients. Previous works have approached this task directly by modeling the relationship between the normalized symptoms and all possible diseases. However, in the clinical diagnostic process, patients are initially consulted by a general practitioner and, if necessary, referred to specialists in specific domains for a more comprehensive evaluation. The final diagnosis often emerges from a collaborative consultation among medical specialist groups. Recently, large language models have shown impressive capabilities in natural language understanding. In this study, we adopt tuning-free LLM-based agents as medical practitioners and propose the Agent-derived Multi-Specialist Consultation (AMSC) framework to model the diagnosis process in the real world by adaptively fusing probability distributions of agents over potential diseases. Experimental results demonstrate the superiority of our approach compared with baselines. Notably, our approach requires significantly less parameter updating and training time, enhancing efficiency and practical utility. Furthermore, we delve into a novel perspective on the role of implicit symptoms within the context of automatic diagnosis.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2401.16107

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

No references could be resolved yet.

Research works printed but not linked: 26

  1. Zhijie Bao, Wei Chen, Shengze Xiao, Kuang Ren, Jiaao Wu, Cheng Zhong, Jiajie Peng, Xu- anjing Huang, and Zhongyu Wei. Disc-medllm: Bridg- ing general large language models and real-world medical consultation. arXiv preprint arXiv:2308.14346, 2023. · arXiv
  2. Tom Brown, Benjamin Mann, Nick Ry- der, Melanie Subbiah, Jared D Kaplan, Prafulla Dhari- wal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing sys- tems, 33:1877-1901, 2020.
  3. Junying Chen, Dongfang Li, Qingcai Chen, Wenxiu Zhou, and Xin Liu. Diaformer: Automatic diagnosis via symptoms sequence generation. In Proceed- ings of the AAAI Conference on Artificial Intelligence, vol- ume 36, pages 4432-4440, 2022.
  4. Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et al. Palm: Scaling lan- guage modeling with pathways. Journal of Machine Learning Research, 24(240):1-113, 2023.
  5. Hou et al., 2021] Zhengxu Hou, Bang Liu, Ruihui Zhao, Zi- jing Ou, Yafei Liu, Xi Chen, and Yefeng Zheng. Imperfect also deserves reward: Multi-level and sequential reward modeling for better dialog management. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 2993-3001, 2021.
  6. Zhenyu Hou, Yukuo Cen, Ziding Liu, Dongxue Wu, Baoyan Wang, Xuanhe Li, Lei Hong, and Jie Tang. Mtdiag: An effective multi-task framework for automatic diagnosis. In Proceedings of the AAAI Confer- ence on Artificial Intelligence, volume 37, pages 14241- 14248, 2023.
  7. Qiao Jin, Yifan Yang, Qingyu Chen, and Zhiyong Lu. Genegpt: Augmenting large language mod- els with domain tools for improved access to biomedical information. ArXiv, 2023.
  8. Minki Kang, Seanie Lee, Jinheon Baek, Kenji Kawaguchi, and Sung Ju Hwang. Knowledge- augmented reasoning distillation for small language models in knowledge-intensive tasks. arXiv preprint arXiv:2305.18395, 2023. · arXiv
  9. Hao-Cheng Kao, Kai-Fu Tang, and Ed- ward Chang. Context-aware symptom checking for dis- ease diagnosis using hierarchical reinforcement learning. In Proceedings of the AAAI conference on artificial intel- ligence, volume 32, 2018.
  10. Kangenbei Liao, Qianlong Liu, Zhongyu Wei, Baolin Peng, Qin Chen, Weijian Sun, and Xuan- jing Huang. Task-oriented dialogue system for automatic disease diagnosis via hierarchical reinforcement learning. arXiv preprint arXiv:2004.14254, 2020. · arXiv
  11. Valentin Liévin, Christoffer Egeberg Hother, and Ole Winther. Can large language mod- els reason about medical questions? arXiv preprint arXiv:2207.08143, 2022. · arXiv
  12. Robinson and Wingate, 2023] Joshua Robinson and David Wingate. Leveraging large language models for multiple choice question answering. In The Eleventh International Conference on Learning Representations, 2023.
  13. John Schulman, Filip Wolski, Pra- fulla Dhariwal, Alec Radford, and Oleg Klimov. Prox- imal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017. · arXiv
  14. Karan Singhal, Shekoofeh Azizi, Tao Tu, S Sara Mahdavi, Jason Wei, Hyung Won Chung, Nathan Scales, Ajay Tanwani, Heather Cole-Lewis, Stephen Pfohl, et al. Large language models encode clini- cal knowledge. Nature, 620(7972):172-180, 2023.
  15. Kai-Fu Tang, Hao-Cheng Kao, Chun- Nan Chou, and Edward Y Chang. Inquire and diagnose: Neural symptom checking ensemble using deep reinforce- ment learning. In NIPS workshop on deep reinforcement learning, 2016.
  16. Liyan Tang, Zhaoyi Sun, Betina Idnay, Jordan G Nestor, Ali Soroush, Pierre A Elias, Ziyang Xu, Ying Ding, Greg Durrett, Justin F Rousseau, et al. Evalu- ating large language models on medical evidence summa- rization. npj Digital Medicine, 6(1):158, 2023.
  17. Milene Santos Teixeira, Vinícius Maran, and Mauro Dragoni. The interplay of a conver- sational ontology and ai planning for health dialogue management. In Proceedings of the 36th annual ACM symposium on applied computing, pages 611-619, 2021.
  18. Arun James Thirunavukarasu, Darren Shu Jeng Ting, Kabilan Elangovan, Laura Gutierrez, Ting Fang Tan, and Daniel Shu Wei Ting. Large language models in medicine. Nature medicine, 29(8):1930-1940, 2023.
  19. Touvron et al., 2023] Hugo Touvron, Thibaut Lavril, Gau- tier Izacard, Xavier Martinet, Marie-Anne Lachaux, Tim- othée Lacroix, Baptiste Rozière, Naman Goyal, Eric Ham- bro, Faisal Azhar, et al. Llama: Open and efficient founda- tion language models. arXiv preprint arXiv:2302.13971, 2023. · arXiv
  20. Wang et al., 2023b] Haochun Wang, Sendong Zhao, Zewen Qiang, Zijian Li, Nuwa Xi, Yanrui Du, MuZhen Cai, Hao- qiang Guo, Yuhan Chen, Haoming Xu, et al. Knowledge- tuning large language models with structured medical knowledge bases for reliable response generation in chi- nese. arXiv preprint arXiv:2309.04175, 2023. · arXiv
  21. Wang et al., 2023c] Huimin Wang, Wai Chung Kwan, Kam-Fai Wong, and Yefeng Zheng. Coad: Automatic di- agnosis through symptom and disease collaborative gener- ation. In Proceedings of the 61st Annual Meeting of the As- sociation for Computational Linguistics (Volume 1: Long Papers), pages 6348-6361, 2023.
  22. Zhongyu Wei, Qianlong Liu, Baolin Peng, Huaixiao Tou, Ting Chen, Xuan-Jing Huang, Kam-Fai Wong, and Xiang Dai. Task-oriented dialogue system for automatic diagnosis. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 201-207, 2018.
  23. Yuan Xia, Jingbo Zhou, Zhenhui Shi, Chao Lu, and Haifeng Huang. Generative adversarial regular- ized mutual information policy gradient framework for au- tomatic diagnosis. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pages 1062-1069, 2020.
  24. Lin Xu, Qixian Zhou, Ke Gong, Xiao- dan Liang, Jianheng Tang, and Liang Lin. End-to-end knowledge-routed relational dialogue system for auto- matic diagnosis. In Proceedings of the AAAI conference on artificial intelligence, volume 33, pages 7346-7353, 2019.
  25. Aiyuan Yang, Bin Xiao, Bingning Wang, Borong Zhang, Ce Bian, Chao Yin, Chenxu Lv, Da Pan, Dian Wang, Dong Yan, et al. Baichuan 2: Open large- scale language models. arXiv preprint arXiv:2309.10305, 2023. · arXiv
  26. Zheng et al., 2023] Chujie Zheng, Hao Zhou, Fandong Meng, Jie Zhou, and Minlie Huang. On large language models' selection bias in multi-choice questions. arXiv preprint arXiv:2309.03882, 2023. · arXiv

Cited by

Sorted by citations

Authors
Wang, H., Zhao, S., Qiang, Z., Xi, N., Qin, B., & Liu, T.
Affiliations
Harbin Institute of Technology (CN)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2024
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2401.16107
arXiv
2401.16107
OpenAlex
W4391376774
Cited by
6
References
0
Citations by year
2026: 1 · 2025: 2 · 2024: 3
Type
posted content
Language
en
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
Keywords
automated diagnosis, large language models, symptom-based diagnosis, LLM agents, general practitioner referral, parameter-efficient inference

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review