MedAgents: Large Language Models as Collaborators for Zero-shot Medical Reasoning

Reliance on CoT in isolation can inadvertently result in hallucinations, especially in specialist domains such as medicine.

“We have found that reliance on CoT in isolation can inadvertently result in hallucinations -spurious outputs typically associated with the misapplication of medical terminologies.”

• Also stated in: § A Analysis on the Addition of CoT · hedged

  “As demonstrated in the example, for specialist domains that demand considerable expert knowledge such as the medical domain knowledge, employing a CoT approach might sometimes lead to hallucination (Bubeck et al., 2023;Guerreiro et al., 2023;Ji et al., 2023;Maynez et al., 2020).”

• cites Bubeck et al., 2023 sourcePartly supported

  Section 9.1 describes LLM hallucinations, and Section 10.2 warns that relying on hallucinated factual claims can be costly in healthcare. Figure 1.8 also shows a medical-note example containing claims not established by the patient's facts. The cited passages do not link those errors to reliance on CoT in isolation or characterize them as misapplied medical terminology. (LLM, conf=0.85)

  Experimental — accuracy not yet measured
• cites Guerreiro et al., 2023 sourcePartly supported

  Section 5.4 (p. 10) investigates hallucinations in medical-domain translation, supporting the general point that hallucinations can occur in that setting. It does not examine chain-of-thought (CoT) or attribute hallucinations to misapplied medical terminology. It also reports that hallucination rates were not consistently higher for medical data than for the comparison data. (LLM, conf=0.82)

  Experimental — accuracy not yet measured
• cites Ji et al., 2023Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

The demand for extensive domain knowledge and advanced reasoning skills makes eliciting medical expertise via simple prompting challenging, preventing LLMs from effectively handling medical tasks.

“(ii) The demand for extensive domain knowledge (Schmidt and Rikers, 2007) and advanced reasoning skills (Liévin et al., 2022) makes eliciting medical expertise via simple prompting challenging (Kung et al., 2023;Singhal et al., 2023a).”

• cites Schmidt and Rikers, 2007 sourceNot checked — no full text of the source
• cites Liévin et al., 2022 sourceSupported
• cites Kung et al., 2023 sourcePartly supported

  The Introduction describes USMLE questions as testing extensive medical knowledge and clinical reasoning, and the Results report that ChatGPT gave some inaccurate answers to questions posed as natural user queries. That supports the idea that eliciting reliable answers is challenging. But the Discussion (“The rising accuracy of ChatGPT”; page unavailable) says ChatGPT approached the passing range without prompt tuning or training, so the paper does not support the broader suggestion that simple prompting prevents LLMs from handling medical tasks effectively. (LLM, conf=0.78)

  Experimental — accuracy not yet measured

Failures in medical question answering originate from a lack of domain knowledge rather than from reasoning rationale.

“Such failures in medical question-answering originate from a lack of domain knowledge (Harris, 2023;Kung et al., 2023;Tian et al., 2024) instead of reasoning rationale.”

• cites Harris, 2023 sourceNot checked — no full text of the source
• cites Kung et al., 2023 sourcePartly supported

  The Discussion (‘The rising accuracy of ChatGPT’; page unavailable) says inaccurate responses were driven primarily by missing information, supporting the domain-knowledge part of the claim. It does not establish that reasoning played no role: it also associates poorer accuracy with lower explanation concordance, and the Discussion (‘The potential for AI-assisted human learning in medical education’; page unavailable) says further failure-mode analysis is needed. (LLM, conf=0.88)

  Experimental — accuracy not yet measured
• cites Tian et al., 2024Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

CoT improvements are significantly limited in medical question answering.

“For example, the results from Liévin et al. (2022) demonstrate that CoT improvements are significantly limited.”

• cites Liévin et al., 2022 sourcePartly supported

  The Discussion (p. 9) says single-sample CoT prompting was not competitive with direct prompting, supporting a limitation for that setting. But the Conclusion (p. 10) calls CoT a powerful tool that improved performance, and the Discussion (p. 10) reports gains from combining many CoTs. The unqualified claim that CoT improvements are significantly limited overstates the paper's finding. (LLM, conf=0.86)

  Experimental — accuracy not yet measured

The optimal number of agents is relatively consistent across different datasets.

“We find that the optimal number of agents is relatively consistent across different datasets, pointing to its potential applicability to other datasets beyond those we test on.”

MedAgents outperforms the zero-shot baseline methods by a large margin.

“Notably, our proposed MEDAGENTS framework outperforms the zero-shot baseline methods by a large margin, indicating the effectiveness of our MEDAGENTS framework in real-world application scenarios.”

• AssumptionRests on an assumption: LLMs contain latent medical expertise that can be elicited through role-playing and multi-agent discussion.
• AssumptionRests on an assumption: The zero-shot setting is applicable in real-world scenarios.

Large-scale experiment performance remains largely robust against the effect of domain changes.

“Results in Table 6 depict minor variance for different sizes of data with random removing, reinforcing the notion that large-scale experiment performance remains largely robust against the effect of domain changes.”

Open-source domain-adapted medical models fell short of the baseline in Table 2.

“Results in Table 4 demonstrate that the open-source methods fell short of the baseline in Table 2, which leads us to focus on the more effective methods.”

Performance improves significantly with the introduction of any number of expert agents compared to the baseline.

“Our key observation lies in that the performance improves significantly with the introduction of any number of expert agents compared to our baseline, thus verifying the consistent contribution of multiple expert agents.”

MedAgents achieves better performance under the zero-shot setting compared with the 5-shot baselines.

“Most notably, our approach achieves better performance under the zero-shot setting compared with the 5-shot baselines.”

• AssumptionRests on an assumption: The zero-shot setting is applicable in real-world scenarios.

The majority (77%) of error examples are due to confusion about domain knowledge, including lack and mis-retrieval of domain knowledge.

“As is shown in Figure 4, the majority (77%) of the error examples are due to confusion about the domain knowledge (including the lack and mis-retrieval of domain knowledge)”

MedAgents achieves comparable performance under the zero-shot setting compared with the strong baseline Few-shot CoT+SC.

“Furthermore, our approach achieves comparable performance under the zero-shot setting compared with the strong baseline Few-shot CoT+SC.”

• AssumptionRests on an assumption: LLMs contain latent medical expertise that can be elicited through role-playing and multi-agent discussion.

A minor 8% of errors are due to CoT.

“This was also observed in our experiments, with a substantial 77% of errors related to domain knowledge, compared to a minor 8% due to CoT (as shown in Figure 4).”

The introduction of CoT occasionally leads to a surprising degradation in performance.

“Interestingly, the introduction of CoT occasionally leads to a surprising degradation in performance.”

There are four common categories of errors in MedAgents: lack of domain knowledge, mis-retrieval of domain knowledge, consistency errors, and CoT errors.

“We find four common categories of errors: (i) lack of domain knowledge, (ii) mis-retrieval of domain knowledge, (iii) consistency errors, and (iv) CoT errors.”

Multiple runs on 300 samples for GPT-4 and GPT-3.5 showed small variance between runs.

“we have performed multiple runs on a set of 300 samples for GPT-4 and GPT-3.5 to account for variability. The preliminary tests involved the average score of 5 repetitions for each sample, and the results indicated a decent small variance between runs.”

MedAgents corrects previous errors in analyzed instances.

“We analyze instances where our proposed MEDAGENTS framework manages to correct previous errors.”

MedAgents' collaborative synthesis distills and integrates diverse expert opinions into a cohesive and accurate conclusion.

“These corrections showcase the MEDAGENTS framework's strength in collaborative synthesis; it distills and integrates diverse expert opinions into a cohesive and accurate conclusion.”

• AssumptionRests on an assumption: A unanimous report generated by iterative expert consultation best represents the collective expert knowledge and reasoning on the given medical problem.

Multi-agent collaboration enhances the model's reasoning capabilities throughout multiple rounds of interaction.

“This process subsequently enhances the model's reasoning capabilities throughout multiple rounds of interaction (Wang et al., 2023c;Fu et al., 2023).”

• cites Wang et al., 2023cNot matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Fu et al., 2023Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

MedAgents' multi-agent role-playing methodology effectively mitigates CoT-induced hallucinations.

“In contrast, our multi-agent roleplaying methodology effectively mitigates these issues, thus underscoring its potential as a more robust approach in medically oriented LLM applications.”

• AssumptionRests on an assumption: LLMs contain latent medical expertise that can be elicited through role-playing and multi-agent discussion.

There still exists a portion of domain knowledge that is explicitly beyond the intrinsic knowledge of LLMs, leading to a bottleneck of MedAgents.

“which illustrates that there still exists a portion of domain knowledge that is explicitly beyond the intrinsic knowledge of LLMs, leading to a bottleneck of our proposed method.”

• AssumptionRests on an assumption: LLMs contain latent medical expertise that can be elicited through role-playing and multi-agent discussion.

Limited volume and specificity of medical training data compared to vast general text data, due to cost and privacy concerns, is a major challenge preventing LLMs from effectively handling medical tasks.

“Two major challenges prevent LLMs from effectively handling tasks in the medical sphere: (i) Limited volume and specificity of medical training data compared to the vast general text data, due to cost and privacy concerns (Thirunavukarasu et al., 2023).”

• cites Thirunavukarasu et al., 2023 sourceNot checked — no full text of the source

Strategies used in the medical field have been shown to induce hallucination.

“strategies used in the medical field have been shown to induce hallucination (Umapathi et al., 2023;Harris, 2023;Ji et al., 2023)”

• cites Umapathi et al., 2023Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Harris, 2023 sourceNot checked — no full text of the source
• cites Ji et al., 2023Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Collaborative consultation actively refines and rectifies initial analyses, aligning the decision-making process closer to clinical accuracy.

“By interweaving a variety of perspectives, the collaborative consultation actively refines and rectifies initial analyses, thereby aligning the decision-making process closer to clinical accuracy.”

• AssumptionRests on an assumption: A unanimous report generated by iterative expert consultation best represents the collective expert knowledge and reasoning on the given medical problem.

In specialist domains, the use of CoT does not improve but hinders overall performance.

“Consequently, in these instances, the use of the CoT method does not improve but hindrances the overall performance.”

MedAgents eliminates the dependency on knowledge bases.

“Our approach differs from most existing methods by eliminating the dependency on knowledge bases, instead uniquely integrating medical knowledge through role-playing agents.”

• AssumptionRests on an assumption: LLMs contain latent medical expertise that can be elicited through role-playing and multi-agent discussion.

MedAgents uniquely integrates medical knowledge through role-playing agents.

“Our approach differs from most existing methods by eliminating the dependency on knowledge bases, instead uniquely integrating medical knowledge through role-playing agents.”

• AssumptionRests on an assumption: LLMs contain latent medical expertise that can be elicited through role-playing and multi-agent discussion.

The consistency observed in the multiple runs provides confidence in the stability of the reported results.

“In summary, the consistency observed in this set provides confidence in the stability of our reported results.”

Role-playing allows the LLM to explicitly reason with accurate knowledge without the need for retrieval augmented generation (RAG).

“And we demonstrate that role-playing allows LLM to explicitly reason with accurate knowledge, without the need for retrieval augmented generation (RAG).”

• Also stated in: § A Analysis on the Addition of CoT · boosted

  “Role-playing in our MEDAGENTS framework allows the model to explicitly reason with accurate knowledge, remarkably bypassing the need for RAG.”

• AssumptionRests on an assumption: LLMs contain latent medical expertise that can be elicited through role-playing and multi-agent discussion.

The collaborative consultation refines and corrects the initial interpretation, leading to the correct answer.

“Furthermore, the collaborative consultation refines and corrects the initial interpretation, leading us toward the correct answer, thus demonstrating the effectiveness of our method in correcting errors and ensuring accuracy.”

MedAgents enjoys enhanced faithfulness and interpretability by harnessing role-playing and collaborative agent discussion.

“Our proposed MEDAGENTS framework enjoys enhanced faithfulness and interpretability by harnessing role-playing and collaborative agent discussion.”

• AssumptionRests on an assumption: LLMs contain latent medical expertise that can be elicited through role-playing and multi-agent discussion.