ENTAgents: AI Agents for Complex Knowledge Otolaryngology

Abstract

Various healthcare applications based on large language models (LLMs) have emerged as LLMs show improved efficiency and error reduction. Recently, retrieval augmented generation (RAG) has been adopted frequently for LLM applications to solve the problem of hallucinations. Despite the success of RAG, it has its drawbacks, including incomplete semantic meanings, and large-scale dataset requirements. AI Agents have shown great potential in medicine and healthcare applications by leveraging their rich background knowledge and reasoning capabilities. In this paper, we introduce ENTAgents, a framework that utilizes both the RAG and multi-agent systems. To achieve better decision-making and enhanced explainability, we use the verbal reinforcement learning framework, Reflexion, as a reference for the task planning of ENTAgents. With the verbal feedback received from the last agent node, the agentic system can decide the best agent to utilize to improve the response of ENTAgents. We tested ENTAgents with various types of questions, including short questions, essay questions, and multiple-choice questions. ENTAgents improve accuracy by over 11.3% in handling short questions, with 2.78 folds in the length of the text and explaining clearly. ENTAgents shows higher accuracy and is more comprehensive in its answer compared to other LLM models. ENTAgents also demonstrates its ability to refine its answer with additional information to make it more thorough and educational. It also presents the capability to change its response to the correct answer in multiple-choice questions according to its self-reflection. Overall, ENTAgents is easy to use, can self-correct, and can provide detailed information in complex knowledge to users. We look forward to integrating ENTAgents into various healthcare scenarios, such as medical education and clinical decision support, particularly otolaryngology.