Auditing medical multi-agent AI reveals risks of false consensus

May combine two claims.

The auditor achieved an F1-score of 0.974 and Cohen's kappa of 0.950 for role-task mismatch (F-2.1.1).

“For role-task mismatch (F-2.1.1),the auditor asks whether the assigned specialty matches the clinical question and the data type, and the auditor records an F1-score of 0.974 and a Cohen's kappa of 0.950.”

May combine two claims.

Phase 1 failures persist in subsequent analyses and can become more frequent than in the first analysis.

“When a flawed reading appears in the first analysis, phase 1 failures persist in subsequent analyses and can become more frequent than in the first analysis.”

The quote only partly states this claim.

In MedAgents, the audited-step rate of unresolved conflicts rises from 4.7% at R1-Review to 76.4% at R2-Analysis and 82.2% at R3-Analysis.

“In MedAgents, the audited-step rate rises from 4.7% at”

May combine two claims.

MedAgents and ColaCare reach 100.00% and 99.96% repetition of initial views on a per-case basis.

“MedAgents and ColaCare reach 100.00% and 99.96% on a per-case basis”

May combine two claims.

The auditor achieved F1-scores of 0.743 for factual hallucinations (F-1.1.1) and 0.750 for failure to activate specialist knowledge (F-2.1.2).

“the auditor maintains F1-scores of 0.743 and 0.750.”

May combine two claims.

Across 24 dataset-LLM combinations, multi-agent collaboration shows uneven effects on diagnostic accuracy relative to matched single-LLM baselines: some combinations improve, while many score lower when the baseline is already strong.

“Across 24 dataset-LLM combinations, the matched baseline comparison shows uneven effects of MAS collaboration: some combinations improve on the single-LLM baseline, whereas many score lower when that baseline is already strong (Supplementary Table 2).”

• Also stated in: § Abstract · plain

  “Across systems, collaboration yields uneven accuracy gains and frequent process failures.”

May combine two claims.

All three QA datasets have authority bias rates below 25%, while PathVQA, VQA-RAD, and SLAKE reach 44.60%, 47.55%, and 41.25%, respectively.

“all three QA datasets remain below 25%; PathVQA, VQA-RAD, and SLAKE reach 44.60%, 47.55%, and 41.25%, respectively”

May combine two claims.

The pattern of minority suppression appears in both QA and VQA, with MedXpertQA and VQA-RAD reaching 6.81% and 7.42% on a per-case basis.

“This pattern appears in both QA and VQA, with MedXpertQA and VQA-RAD reaching 6.81% and 7.42% on a per-case basis”

May combine two claims.

Authority bias is the most frequent phase 3 failure and is more common in VQA datasets.

“(1) Authority bias is the most frequent phase 3 failure and is more common in VQA datasets.”

May combine two claims.

VQA-RAD has a per-case self-contradiction rate of 26.92%, and ColaCare reaches 47.14%.

“VQA-RAD reaches 26.92% on a per-case basis, and ColaCare reaches 47.14%”

May combine two claims.

Self-contradiction across rounds (F-3.2.1) affects 18.53% of audited cases with cross-round synthesis or decision comparisons, making it the second most frequent phase 3 failure after authority bias.

“Self-contradiction across rounds (F-3.2.1) affects 18.53% of audited cases with cross-round synthesis or decision comparisons, making it the second most frequent phase 3 failure after authority bias”

May combine two claims.

In that case, the first-round option matched the ground truth, but the fabricated visual premise was not corrected in later discussion, and the system ended with the wrong final answer B under full inter-agent consensus.

“The first-round option matches the ground truth, yet the fabricated visual premise is not corrected in subsequent discussion, and the system ends with the wrong final answer”

• AssumptionRests on an assumption: Human expert majority vote is a valid reference standard for evaluating the automated auditor's diagnostic performance.

Probably not a substantial claim: may not pass S3 (load-bearing). The quote only partly states this claim.

For the ColaCare framework, repetition of initial views during collaborative discussion (F-2.2.1) occurred at a per-audit rate of 95.83% (7345/7665).

“95.83 (7345/7665)”

• AssumptionRests on an assumption: Familiarity with the failure mode definitions ensures valid annotations.

Probably not a substantial claim: may not pass S3 (load-bearing).

The automated auditing system provided quantitative estimates across 14,400 audited instances.

“While the automated auditing system provides quantitative estimates across 14,400 audited instances”

May combine two claims.

In the MedXpertQA case, the meta agent synthesized the specialists' views by stating that all three specialists agreed on galactosemia, that Neonatology further supported this, and that the transition to lactose-containing baby foods was the critical trigger.

“The meta agent then synthesizes their views by stating that”

May combine two claims. The quote only partly states this claim.

In the MedXpertQA case, the final MAS answer remained 'A', and the case received unanimous human failure labels.

“The final MAS answer remains”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

In the PubMedQA case, the agent used the text modality correctly, read the abstract, compared MRI with arthroscopy, and answered the stated yes-or-no question instead of drifting into unrelated background.

“The agent uses the text modality correctly.It reads the abstract, compares MRI with arthroscopy, and answers the stated yes-or-no question instead of drifting into unrelated background.”

Probably not a substantial claim: may not pass S3 (load-bearing).

In the per-audit evaluation of unresolved conflicts during collaborative discussion (F-2.2.2), the overall rate across all datasets and models was 8.39% (2755/32819).

“Grand Total 8.39 (2755/32819)”

May combine two claims. The quote only partly states this claim.

In that case, two agents answered B and one answered A, but the audit failure was already present before this disagreement because the team lacked the specialty that normally interprets histopathology slides.

“Two agents answer”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

In that case, the agent's claims were traceable to the source text, it did not add unsupported patient details or invented statistics, and its final answer A matched the ground truth.

“Each claim is traceable to the source text, and the agent does not add unsupported patient details or invented statistics.The final answer remains”

May combine two claims.

Repetition of initial views dominated phase 2; role-task mismatch plus failure to activate specialist knowledge occurred at role assignment and role execution and could leave the discussion without the needed specialty or without role-appropriate clinical reasoning, while unresolved conflicts were less frequent overall.

“Summary cards showing that repetition of initial views dominates phase 2; role-task mismatch and failure to activate specialist knowledge show two failures at role assignment and role execution that can leave the discussion without the needed specialty or without role-appropriate clinical reasoning, and unresolved conflicts are less frequent overall.”

The quote only partly states this claim.

For the MAC framework, repetition of initial views during collaborative discussion (F-2.2.1) occurred at a per-audit rate of 71.40% (734/1028).

“71.40 (734/1028)”

• AssumptionRests on an assumption: Familiarity with the failure mode definitions ensures valid annotations.

May combine two claims.

For medical QA datasets, the per-case rate of neglect of contradictions in reasoning during decision was 2.17% for GPT-5.2, 0.64% for Gemini-3-Flash, 3.13% for DeepSeek-V3.2, and 12.21% for Qwen-3.

“LLM Avg. (QA) (MAS & QA datasets aggr.) | GPT-5.2 2.17 (42/1939) | | Gemini-3-Flash 0.64 (12/1868) | DeepSeek-V3.2 3.13 (64/2042) | Qwen-3 12.21 (260/2129)”

• AssumptionRests on an assumption: Human expert majority vote is a valid reference standard for evaluating the automated auditor's diagnostic performance.

May combine two claims.

A validated auditor evaluated 14,400 multi-agent cases from 144 combinations of architectures, datasets, and underlying LLMs, judging predefined failure modes at predefined interaction steps, and achieved a macro-average F1-score of 0.845 on 400 sampled cases.

“A validated auditor runs alongside 14,400 MAS cases from 144 combinations of architectures, datasets, and underlying LLMs and judges the predefined failure modes at predefined interaction steps, reaching a macro-average F1-score of 0.845 on 400 sampled cases.”

May combine two claims.

Phase 1 failures usually entered at the initial reading step and often persisted in subsequent steps.

“Round-step trajectories across datasets, showing that phase 1 failures usually enter at the initial reading step and often persist in subsequent steps.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

In the PubMedQA EQ-5D acute coronary syndrome case, the ColaCare agent using Gemini-3-Flash in Round 1 restated the study objective, cited the reported correlations, noted statistical significance, and concluded EQ-5D is valid for ACS patients.

“The agent restates the study objective, cites the reported correlations, notes that all are statistically significant, and concludes that EQ-5D is a valid tool for patients with acute coronary syndrome.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Supplementary Figures 19 and 20 show one case identified as a failure through auditing and one case not flagged as a failure, illustrating F-1.2.1 and how modality adherence differs from it.

“Supplementary Figures 19 and 20 show one case identified as a failure through auditing and one case not flagged as a failure by the auditor, illustrating how F-1.2.1 appears in practice and how modality adherence differs from it.”

• cites Supplementary Figures 19 and 20Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

May combine two claims.

In phase 3 examples, one trajectory had the system adopt an answer from fewer agents because one role was given extra weight, and another had the system produce a single consensus answer because several agents chose the same option even though the reasoning behind that option was mutually incompatible across agents.

“In the phase 3 examples, one trajectory shows the system following an answer from fewer agents because extra weight is given to one role, and the other shows the system writing one consensus answer because several agents select the same option even when the reasoning used to support that option is mutually incompatible across agents.”

May combine two claims.

Coding 720 sampled logs from 3,600 multi-agent collaboration logs, then double-blind annotating 360 holdout logs, produced ten collaborative failure modes across three audit phases, with a macro-average Cohen's kappa of 0.76.

“Open coding of 720 sampled logs from 3,600 MAS collaboration logs, followed by double-blind annotation of 360 holdout logs, yields ten collaborative failure modes across three audit phases (macro-average Cohen's kappa = 0.76).”

May combine two claims. The quote only partly states this claim.

The final system answer was B, the ground truth was A, and the case received unanimous human failure labels.

“The final system answer is”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Supplementary Figures 21 and 22 show one case identified as a failure through auditing and one not flagged, illustrating F-2.1.1 and how appropriate role assignment differs from it.

“Supplementary Figures 21 and 22 show one case identified as a failure through auditing and one case not flagged as a failure by the auditor, illustrating how F-2.1.1 appears in practice and how appropriate role assignment differs from it.”

• cites Supplementary Figures 21 and 22Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

May combine two claims. The quote only partly states this claim.

In the PathVQA case, the final system answer remained option 'A' under unanimous inter-agent agreement, while the ground truth was option 'B'.

“The final system answer remains option”

May combine two claims. The quote only partly states this claim.

Across 400 validation instances, the automated auditor's macro-average sensitivity was 0.965, specificity 0.808, F1-score 0.845, and Cohen's kappa 0.730 against human majority vote.

“Macro-Average | 400 | 0.631 | 0.965 | 0.808 | 0.845 | 0.730 | 152 194 48 | 6”

• AssumptionRests on an assumption: Human expert majority vote is a valid reference standard for evaluating the automated auditor's diagnostic performance.

Probably not a substantial claim: may not pass S3 (load-bearing).

The audit result for that EQ-5D case was non-failure because the agent added no unsupported symptoms, numbers, or patient details.

“Audit Result: Non-Failure Explanation: No unsupported symptoms, numbers, or patient details are added.”

The quote only partly states this claim.

Role-task mismatch (F-2.1.1) occurred at an overall per-audit rate of 22.71% (652/2871).

“Grand Total 22.71 (652/2871)”

• AssumptionRests on an assumption: Familiarity with the failure mode definitions ensures valid annotations.

The quote only partly states this claim.

Failure to activate specialist knowledge during role execution (F-2.1.2) occurred at an overall per-case rate of 42.73% (5320/12451).

“Grand Total 42.73 (5320/12451)”

• AssumptionRests on an assumption: Familiarity with the failure mode definitions ensures valid annotations.

The quote only partly states this claim.

Human inter-rater reliability among three independent medical experts had a macro-average Fleiss' kappa of 0.631 across 400 validation instances.

“Macro-Average | 400 | 0.631 | 0.965 | 0.808 | 0.845 | 0.730 | 152 194 48 | 6”

• AssumptionRests on an assumption: Human expert majority vote is a valid reference standard for evaluating the automated auditor's diagnostic performance.

May combine two claims. The quote only partly states this claim.

In the PathVQA case, the MedAgents agent using GLM-4.6V answered 'A' and recited the general fact about amyloid fibrils having a cross-β pleated-sheet pattern, but did not point to the boxed element, arrows, or fibril bundles in the image.

“In Round 1, the audited agent in MedAgents with GLM-4.6Vanswers”

Classified as probably not a substantial claim.

The highlighted white rectangle in that case is a specimen label, not a surgical instrument.

“The highlighted white rectangle is a specimen label; treating it as a surgical instrument introduces a visual fact unsupported by the image.”

• AssumptionRests on an assumption: Human expert majority vote is a valid reference standard for evaluating the automated auditor's diagnostic performance.

The quote only partly states this claim.

In a PathVQA case, the audited agent in MAC with Qwen-3VL answered A in Round 1 but justified it by describing the image as a surgical exposure and the specimen label as a surgical clamp.

“In Round 1, the audited agent in MAC with Qwen-3VL answers”

• AssumptionRests on an assumption: Human expert majority vote is a valid reference standard for evaluating the automated auditor's diagnostic performance.

May combine two claims.

Unresolved conflicts were less common at the first discussion step but often persisted once they appeared.

“Round-step trajectories showing that unresolved conflicts are less common at the first discussion step but often persist once they appear.”

May combine two claims.

Factual hallucinations accounted for more phase 1 failures than modality neglect, and VQA had higher failure rates than textual QA.

“Summary cards showing that factual hallucinations account for more phase 1 failures than modality neglect and that VQA has higher failure rates than textual QA.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims. The quote only partly states this claim.

In the F-1.2.1 failure case, the agent's answer never referred to the boxed element, the arrows, or the visual separation between the green central strands and the pink bundles, and that missing visual check was the audit failure.

“The answer never refers to the boxed element, the arrows, or the visual separation between the green central strands and the pink bundles.That missing visual check is the audit failure.”

May combine two claims.

For medical VQA datasets, the per-case rate of neglect of contradictions in reasoning during decision was 3.07% for GPT-5.2, 3.32% for Gemini-3-Flash, 3.25% for GLM-4.6V, and 9.03% for Qwen-3VL.

“LLM Avg. (VQA) (MAS & VQA datasets aggr.) | GPT-5.2 3.07 (59/1922) | | Gemini-3-Flash 3.32 (63/1898) | GLM-4.6V 3.25 (61/1876) | Qwen-3VL 9.03 (174/1927)”

• AssumptionRests on an assumption: Human expert majority vote is a valid reference standard for evaluating the automated auditor's diagnostic performance.

May combine two claims. The quote only partly states this claim.

In the MedXpertQA case, the Pediatrics, Genetics, and Neonatology agents in MedAgents with Qwen-3 all answered 'A' for galactosemia and repeated the same incorrect premise that baby foods introduce lactose.

“In Round 1, the Pediatrics, Genetics, and Neonatology agents in MedAgents with Qwen-3 all answer”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The response stayed tied to the provided text and the yes-or-no question, and the final system answer remained A and matched the ground truth.

“The response stays tied to the provided text and the yes-or-no question.The final system answer remains option”

The quote only partly states this claim.

In the per-case evaluation of self-contradiction in viewpoints across rounds during decision-making (F-3.2.1), the overall rate was 18.53% (101/545).

“Grand Total 18.53 (101/545)”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims. The quote only partly states this claim.

In that case, the MedAgents agent with Qwen-3 in Round 1 followed the abstract's objective, cohort, MRI protocol, reported lesion findings, and conclusion, then answered A by stating MRI serves as a diagnostic substitute while arthroscopy still provides direct visualization and intervention.

“In Round 1, the audited agent in MedAgents with Qwen-3 follows the abstract's study objective, cohort and MRI protocol, reported lesion findings, and conclusion, then answers”

Probably not a substantial claim: may not pass S1 (asserted), S3 (load-bearing), S4 (based). May combine two claims.

The final MAS answer remained A, which matched the ground truth.

“The final MAS answer remains”

• Also stated in: § Failure Mode 2.2.2: Unresolved Conflicts During Collaborative Discussion · plain

  “The final MAS answer remains”

May combine two claims.

In the VQA-RAD case asking whether the ventricles are enlarged, the Radiology agent in MedAgents with GLM-4.6V answered A and identified ventricular enlargement with possible mass effect, whereas the Cardiology and Internal Medicine agents both answered B and described the ventricles as normal.

“In Round 1, the Radiology agent in MedAgents with GLM-4.6Vanswers”

May combine two claims. The quote only partly states this claim.

During review, the audited doctor_1 response in ColaCare with Qwen-3 stated that all analyses consistently supported option F and repeated the rightward-rotation reading without addressing doctor_2's opposing mechanistic claim.

“During review, the audited doctor_1 response in ColaCare with Qwen-3 states that all analyses consistently support option”

Classified as probably not a substantial claim. May combine two claims.

In the July effect case, the meta agent selected answer 'A' in Round 1 and answer 'C' in Round 2.

“In round 1, the agent selected 'A' by noting that the question asks for 'any' evidence and that the significant increase in hematologic complications constitutes such evidence.In round 2, the agent changes its final answer to 'C'.”

Probably not a substantial claim: may not pass S4 (based). May combine two claims.

It did not address the main limitation already visible in the initial opinions: most patients in the study showed no benefit, so the paper supports C (Maybe) rather than A (Yes) for the full study question.

“It does not address the main limitation already visible in the initial opinions: most patients in the study show no benefit, so the paper supports”

May combine two claims. The quote only partly states this claim.

In the MedXpertQA case asking which lymph node station in esophageal cancer is nonregional or distant, all three MedAgents specialists with Qwen-3 answered E, but their justifications were not mutually compatible.

“In Round 1, all three MedAgents specialists with Qwen-3 answer”

May combine two claims.

These were mutually exclusive interpretations of the same clinical sign, yet all three agents still chose option F.

“These are mutually exclusive interpretations of the same clinical sign, yet all three agents still choose option”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The meta agent preserved this shared reasoning and again answered B.

“The meta agent preserves this shared reasoning and again answers”

May combine two claims. The quote only partly states this claim.

In the VQA-RAD case asking whether air is present in the bowels on an abdominal CT image, the Internal Medicine, Surgery, and Radiology agents in ColaCare with Gemini-3-Flash all answered A and independently pointed to the same image content: low-density black intraluminal regions in the bowel loops, especially in the ascending colon.

“In Round 1, the Internal Medicine, Surgery, and Radiology agents in ColaCare with Gemini-3-Flash all answer”

May combine two claims. The quote only partly states this claim.

The final MAS answer remained unanimous at A, although the ground truth was C.

“The final MAS answer remains unanimous at”

May combine two claims.

In the PathVQA case asking whether a nodular tumor is bloody, the audited Internal Medicine agent in ColaCare with GLM-4.6V answered A and justified the answer only by saying the specimen looked reddish and moist.

“In Round 1, the audited Internal Medicine agent in ColaCare with GLM-4.6Vanswers”

Classified as probably not a substantial claim. May combine two claims.

The meta agent nonetheless stated that all three specialists agreed and reduced these incompatible staging definitions to terminology differences.

“The meta agent nonetheless states that all three specialists agree and reduces these incompatible staging definitions to terminology differences.”

Classified as probably not a substantial claim.

The audited Radiology review in MedAgents with DeepSeek-V3.2 agreed with the synthesis and repeated the same subgroup contrast, only adding generic statements about radiology best practice and guideline support.

“The audited Radiology review in MedAgents with DeepSeek-V3.2then agrees with the synthesis and repeats the same subgroup contrast, only adding generic statements about radiology best practice and guideline support.”

May combine two claims.

In the MedXpertQA case about cranial asymmetry involving the sphenoid and occiput, doctor_1 and doctor_3 stated that the greater wing appearing more lateral on the right implies rightward sphenoid rotation, whereas doctor_2 stated that the same finding implies leftward rotation.

“In Round 1, doctor_1 and doctor_3 state that the greater wing appearing more lateral on the right implies rightward sphenoid rotation, whereas doctor_2 states that the same finding implies leftward rotation.”

Classified as probably not a substantial claim. May combine two claims.

The Radiology agent argued that level 1 lies outside the primary regional drainage groups, whereas the Surgery agent further stated that celiac nodes (level 20) are also nonregional.

“The Radiology agent argues that level 1 lies outside the primary regional drainage groups, whereas the Surgery agent further states that celiac nodes (level 20) are also nonregional.”

Classified as probably not a substantial claim.

Because the history contained agreement about the relevant clinical finding rather than mutually exclusive statements, the audit did not flag unresolved conflict.

“Because the history contains agreement about the relevant clinical finding rather than mutually exclusive statements, the audit does not flag unresolved conflict.”

May combine two claims.

In the trachea case, during review, the Pulmonology agent briefly objected and argued for rightward deviation, but that contradiction did not come from the meta agent.

“During review, the Pulmonology agent briefly objects and argues for rightward deviation, but that contradiction does not come from the meta agent.”

Probably not a substantial claim: may not pass S3 (load-bearing). The quote only partly states this claim.

In the PubMedQA case about oral endotracheal intubation in a helicopter environment, the audited General Medicine agent in ReConcile with Gemini-3-Flash first answered B by focusing on the non-significant difference in successful tracheal placement.

“The audited General Medicine agent first answers B by focusing on the non-significant difference in successful tracheal placement.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims. The quote only partly states this claim.

The meta agent also answered A, but unlike the positive example, it did not rely on a role hierarchy or a more elaborate narrative; instead, it restated the CT principle that air appears black because of low radiodensity and tied the answer to the visible bowel gas in the provided image.

“The meta agent also answers”

Probably not a substantial claim: may not pass S1 (asserted), S3 (load-bearing), S4 (based). May combine two claims.

The final MAS answer remained F, although the ground truth was A.

“The final MAS answer remains”

May combine two claims. The quote only partly states this claim.

In the PathVQA case asking whether splenomegaly with cirrhosis shows hepatitis B virus or hepatocellular carcinoma, all three MedAgents specialists with GLM-4.6V answered A and justified the choice with the same cirrhosis-HBV-hepatocellular-carcinoma association, and the meta agent preserved that unanimous trajectory at synthesis.

“In Round 1, all three MedAgents specialists with GLM-4.6Vanswer”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Each agent also distinguished the visible renal cystic changes from the liver, with one agent explicitly noting that any small hepatic hypodensities were likely simple non-enhancing cysts.

“Each agent also distinguishes the visible renal cystic changes from the liver, with one agent explicitly noting that any small hepatic hypodensities are likely simple non-enhancing cysts.”

Classified as probably not a substantial claim. May combine two claims.

The Internal Medicine agent stated that stations 1 through 20 are regional yet still treated level 1 as nonregional in some frameworks.

“The Internal Medicine agent states that stations 1 through 20 are regional yet still treats level 1 as nonregional in some frameworks.”

May combine two claims.

In the MedXpertQA galactosemia case, the meta agent demonstrated authority bias by primarily grounding its conclusion in the consensus of the specialists, and its reasoning failed to independently verify the clinical facts provided in the question.

“The meta agent demonstrates authority bias by primarily grounding its conclusion in the consensus of the”

May combine two claims. The quote only partly states this claim.

The final MAS answer was A, whereas the dataset ground truth was B, but the audit did not flag the case because the interaction history contained no disagreement and therefore no correct minority view to suppress.

“The final MAS answer is”

Probably not a substantial claim: may not pass S3 (load-bearing).

In the trachea case, the meta agent stayed with answer 'A' throughout.

“Here the meta agent stays with A throughout.”

Probably not a substantial claim: may not pass S3 (load-bearing).

During review, the audited agent restated the same benign diagnosis and management plan.

“During review, the audited agent restates the same benign diagnosis and management plan.”

Probably not a substantial claim: may not pass S3 (load-bearing).

In the trachea case, the human audit label was 0/0/0, indicating non-failure.

“Human Audit Label: 0 / 0 / 0 (1 is Failure, 0 is Non-Failure)”

• AssumptionRests on an assumption: The human audit labels are a reliable ground truth for whether a case is a failure.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

It recommended ice and ibuprofen, corresponding to option E and the ground truth.

“It then recommends ice and ibuprofen, which corresponds to option”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims. The quote only partly states this claim.

During review, the same agent changed to A and added three concrete pieces of reasoning: the sample of 30 attempts may be too small to detect the clinical importance of two esophageal intubations, efficacy in emergency airway management includes procedural speed as well as placement success, and the study question concerns impairment in the in-flight environment rather than only statistical non-in

“During review, the same agent changes to”

May combine two claims.

In the MedXpertQA case about acute knee pain and effusion after intra-articular methylprednisolone injection, the audited Internal Medicine agent in MedAgents with GPT-5.2 identified a post-injection corticosteroid flare (pseudoseptic reaction), cited the six-hour onset, synovial leukocyte count of 12,000/mm3 with neutrophil predominance, negative Gram stain, and mixed birefringent crystals, and,

“In Round 1, the audited Internal Medicine agent in MedAgents with GPT-5.2 identifies a post-injection corticosteroid flare (pseudoseptic reaction), cites the six-hour onset, the synovial leukocyte count of 12,000/mm 3 with neutrophil predominance, the negative Gram stain, and the mixed birefringent crystals, and explains why these findings argue against septic arthritis, gout, and calcium pyroph”

May combine two claims.

The meta agent explicitly noted this disagreement and acknowledged that imaging expertise was relevant, but still selected B because two of the three consulted agents supported that option.

“The meta agent explicitly notes this disagreement and acknowledges that imaging expertise is relevant, but it still selects”

Probably not a substantial claim: may not pass S3 (load-bearing).

In the trachea case, the agent's conclusion remained consistent across rounds.

“The agent's conclusion has remained consistent throughout the process.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims. The quote only partly states this claim.

The final MAS answer became A, which matched the ground truth.

“The final MAS answer becomes”

May combine two claims.

In the MedQA case about an asymptomatic cervical lesion, all three ReConcile agents identified a Nabothian cyst from the small, firm, non-tender yellow mass at the exocervical os and chose option A, no further intervention.

“In Round 1, all three ReConcile agents identify a Nabothian cyst from the small, firm, non-tender yellow mass at the exocervical os and choose option”

Probably not a substantial claim: may not pass S1 (asserted), S3 (load-bearing), S4 (based). May combine two claims.

The final MAS answer remained E, although the ground truth was G, and the case received majority human failure labels.

“The final MAS answer remains”

May combine two claims. The quote only partly states this claim.

In the VQA-RAD case asking whether an enhancing lesion is present in the liver on a contrast-enhanced abdominal CT image, the Radiology, Internal Medicine, and Surgery agents in MedAgents with Gemini-3-Flash all answered B and relied on the same core finding: the liver parenchyma enhances homogeneously without a focal hyperenhancing mass.

“In Round 1, the Radiology, Internal Medicine, and Surgery agents in MedAgents with Gemini-3-Flash all answer”

Probably not a substantial claim: may not pass S3 (load-bearing).

The audit marked the response as a failure to activate specialist knowledge rather than as an isolated answer error.

“The audit therefore marks the response as a failure to activate specialist knowledge rather than as an isolated answer error.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

In Round 1, MedAgents with Gemini-3-Flash assigned Internal Medicine, General Medicine, and a factual-accuracy role, and all three agents answered C for projection.

“In Round 1, MedAgents with Gemini-3-Flash assigns Internal Medicine, General Medicine, and a factual-accuracy role, and all three agents answer”

Probably not a substantial claim: may not pass S1 (asserted), S3 (load-bearing), S4 (based). May combine two claims.

The final MAS answer remained B, although the ground truth was A, and the case received unanimous human failure labels.

“The final MAS answer remains”

Probably not a substantial claim: may not pass S3 (load-bearing).

The final MAS answer matched the ground truth.

“The final MAS answer matches the ground truth.”

May combine two claims. The quote only partly states this claim.

The final MAS answer remained A, the ground truth was B, and the case received unanimous human failure labels.

“The final MAS answer remains”

May combine two claims.

In the July effect case, the meta agent shifted its final answer between rounds without providing justification for the change.

“The agent exhibits an shift between rounds.”

• Also stated in: § Failure Mode 2.2.2: Unresolved Conflicts During Collaborative Discussion · plain

  “The agent does not provide any justification for why 'C' is now appropriate.”

Classified as probably not a substantial claim.

The response did not move beyond lay visual description into assessment of hemorrhagic change, vascularity, or gross specimen appearance.

“The response does not move beyond lay visual description into assessment of hemorrhagic change, vascularity, or gross specimen appearance.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims. The quote only partly states this claim.

The final system answer stayed C, which matched the ground truth.

“The final system answer stays”

Classified as probably not a substantial claim. The quote only partly states this claim.

In the PubMedQA case about ultrasound before puncture for internal jugular vein cannulation, all three MedAgents specialists already answered A by emphasizing the subgroup without visible respiratory jugular venodilation, where ultrasound improves access and success rates.

“In Round 1, all three MedAgents specialists already answer”

May combine two claims.

The auditor achieved a macro-average sensitivity of 0.965 and specificity of 0.808.

“The auditor achieves a macro-average sensitivity of 0.965, producing only six false negatives across all 400 evaluated cases.”

• Also stated in: § 2.3 Clinical Validation of the Automated Auditor · plain

  “The macro-average specificity reaches 0.808, with 48 false positives”

May combine two claims.

Studies on general LLM self-correction, debate, voting, and multi-agent failure analysis show that interaction alone does not reliably correct reasoning errors and often reinforces them across turns and agents.

“Studies on general LLM self-correction, debate, voting, and multi-agent failure analysis demonstrate that interaction alone does not reliably correct reasoning errors, frequently reinforcing them across turns and agents [22][23][24][25].”

• cites [22]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [23]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [24]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Classified as probably not a substantial claim. May combine two claims.

Large language models show physician-level performance on medical benchmarks and are being integrated into clinical workflows for triage, diagnostic assistance, and documentation.

“Large language models (LLMs) are rapidly extending their general-purpose capabilities into clinical-facing software, demonstrating physician-level performance on medical benchmarks and entering workflows for triage, diagnostic assistance, and documentation [1][2][3][4][5][6][7][8].”

• cites [1]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [2]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [3]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

May combine two claims.

Medical multi-agent systems implement team-based structure within LLM frameworks by assigning clinical roles, eliciting independent opinions, and synthesizing collaborative decisions.

“Medical multi-agent systems (MAS) operationalize this team-based structure within LLM frameworks by assigning clinical roles, eliciting independent opinions, and synthesizing collaborative decisions [11][12][13][14][15][16][17][18][19].”

• cites [11]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [12]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [13]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S2 (checkable).

A critical gap remains for medical multi-agent systems: evaluating their clinical viability requires a structured, clinically grounded audit that localizes failures at the collaborative interaction.

“Despite these advances, a critical gap remains for medical MAS.Evaluating their clinical viability requires a structured, clinically grounded audit to localize failures at the collaborative interaction, measuring whether false observations are critically evaluated, retained, or blindly repeated as they propagate across diverse architectures, clinical tasks, and modalities [35,[37][38][39][40].”

• cites [35]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [37]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [38]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Classified as probably not a substantial claim.

Clinical diagnosis is a staged, coordinated process in which clinicians synthesize patient history, multimodal findings, and biomedical knowledge to determine which explanations remain plausible and which uncertainties require action.

“Clinical diagnosis, however, inherently exceeds single-turn question answering; it is a staged, coordinated process where clinicians synthesize patient history, multimodal findings, and biomedical knowledge to decide which explanations remain plausible and which uncertainties require action [9,10].”

• cites [9]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [10]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Classified as probably not a substantial claim. May combine two claims.

Benchmark QA and VQA datasets provide controlled comparisons but simplify clinical presentation, remove longitudinal patient context, and exclude many pressures of care delivery.

“Benchmark QA and VQA datasets provide controlled comparisons across systems; they simplify clinical presentation, remove longitudinal patient context, and exclude many pressures of care delivery, including evolving test results, local workflow constraints, patient preferences, and clinician-team communication.”

Classified as probably not a substantial claim. May combine two claims.

MAC operates without explicit role assignment, prompting all domain agents to act as general medical experts, with a supervisor agent determining consensus and triggering new debate rounds if no consensus.

“The MAC framework [16] operates without explicit role assignment, prompting all domain agents to act as general medical experts.”

• cites [16]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S3 (load-bearing).

PathVQA consists of visual question answering pairs based on pathology images, including histological slices, microscopic cellular images, and stained microorganisms.

“The PathVQA dataset [44] consists of visual question answering pairs based on pathology images, primarily encompassing histological slices, microscopic cellular images, and stained microorganisms sourced from textbooks and digital libraries.”

• cites [44]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

May combine two claims.

Once that occurs, synthesis and decision-making steps can drop correct minority views, combine incompatible clinical reasoning processes, and produce different final answers from the same case information across rounds.

“Once that occurs, synthesis and decision-making steps can drop correct minority views, combine incompatible clinical reasoning processes, and produce different final answers from the same case information across rounds.”

May combine two claims.

Sharing the same answer does not by itself trigger authority bias; the failure occurs when a speaker's role is treated as sufficient justification instead of checking the image or case text.

“Sharing the same answer does not itself trigger authority bias; the failure appears when the speaker's role is treated as sufficient justification rather than checking the image or case text.”

May combine two claims.

This creates evidentiary drift: unsupported findings can gain plausibility through reuse, unresolved disagreements can be softened by summarization, and answer agreement can obscure incompatible clinical rationales.

“This creates a subtle form of evidentiary drift: unsupported findings can gain plausibility through reuse, unresolved disagreements can be softened by summarization, and answer agreement can obscure incompatible clinical rationales.”

Classified as probably not a substantial claim. May combine two claims.

An independent Auditor Agent powered by Gemini-3-Flash read the interaction state at predefined steps without altering the underlying execution pathway or the framework's combination of prior outputs.

“we deploy an independent Auditor Agent powered by Gemini-3-Flash.This agent reads the interaction state available at predefined interaction steps, extracting the required context without altering the underlying execution pathway or the way the framework combines prior agent outputs into a final answer.”

Classified as probably not a substantial claim. May combine two claims.

MedAgents combines dynamic role assignment with an iterative synthesis-and-review loop, iterating until consensus or maximum round limit.

“MedAgents [15] combines dynamic role assignment with an iterative synthesis-and-review loop.”

• cites [15]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Classified as probably not a substantial claim. May combine two claims.

The auditor tracks intermediate agent states in real time without changing the primary diagnostic workflow.

“The auditor tracks intermediate agent states in real time without changing the primary diagnostic workflow.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The taxonomy was developed from the six MAS architectures and execution logs in this study, so other system types may reveal additional failure modes.

“The taxonomy was developed from the six MAS architectures and execution logs included in this study, so retrieval-augmented systems, tool-using agents, autonomous test-ordering workflows, EHR-integrated deployments, and clinician-interactive settings may reveal additional failure modes.”

Probably not a substantial claim: may not pass S3 (load-bearing).

MedQA consists of USMLE-style questions for medical knowledge and clinical decision-making.

“The MedQA dataset [41] consists of United States Medical Licensing Examination (USMLE)-style questions to evaluate medical knowledge and clinical decision-making.”

• cites [41]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

SLAKE consists of bilingual (English and Chinese) visual question answering pairs based on semantically labeled clinical radiology images, assessing visual content retrieval and external medical knowledge.

“The SLAKE dataset [46] consists of bilingual (English and Chinese) visual question answering pairs based on semantically labeled clinical radiology images, assessing both visual content retrieval and external medical knowledge.”

• cites [46]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Classified as probably not a substantial claim.

Each of the 400 instances was independently reviewed by three different clinical experts who answered binary questions about the occurrence of the specified failure mode.

“Each of the 400 instances is independently reviewed by three different clinical experts, who answer binary (yes/no) questions regarding the occurrence of the specified failure mode.”

Classified as probably not a substantial claim.

The human ground truth for each instance was established via a majority vote mechanism requiring consensus from at least two of the three evaluating clinical experts.

“The human ground truth for each instance is established via a majority vote mechanism, requiring a consensus from at least two of the three evaluating clinical experts.”

Classified as probably not a substantial claim.

To ensure an unbiased comparison and prevent data leakage, the study randomly sampled 100 distinct instances from each dataset's test set.

“To ensure an unbiased comparison and prevent data leakage, we randomly sample 100 distinct instances from the test set of each dataset.”

Classified as probably not a substantial claim.

Six clinical experts were recruited to serve as independent evaluators.

“We recruit six clinical experts to serve as independent evaluators.”

Classified as probably not a substantial claim. May combine two claims.

ColaCare implements a predefined sequential collaboration pipeline with role assignment, meta-agent synthesis, review, and a final decision-maker, iterating until consensus or maximum round limit.

“The ColaCare architecture [17] implements a predefined sequential collaboration pipeline.”

• cites [17]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S3 (load-bearing).

PubMedQA consists of biomedical questions derived from research publication titles and abstracts, requiring answers from the provided abstract context.

“The PubMedQA dataset [42] consists of biomedical questions derived from the titles and abstracts of research publications, requiring models to answer using the provided abstract context.”

• cites [42]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

May combine two claims.

The evaluators identified no additional failure patterns outside the established codebook, supporting the coverage of the observed cases.

“The evaluators have identified no additional failure patterns outside the established codebook, supporting the coverage of the observed cases.”

May combine two claims.

The absence of new failure patterns verifies that the predefined classification is comprehensive and confirms the reliability of the proposed taxonomy.

“This finding verifies that the predefined classification is comprehensive, thereby confirming the reliability of the proposed taxonomy.”

Classified as probably not a substantial claim.

The annotation platform managed deterministic case assignments to ensure independent, blinded reviews.

“while managing deterministic case assignments to ensure independent, blinded reviews.”

May combine two claims.

The study conducted a two-phase human evaluation to establish the reliability of the failure mode taxonomy and to verify the consistency of the automated auditing system with human expert annotations.

“To establish the reliability of the failure mode taxonomy and verify the consistency of the automated auditing system with human expert annotations, we have conducted a two-phase human evaluation.”

Classified as probably not a substantial claim.

All recruited annotators underwent a standardized training procedure based on the evaluation guidelines before the evaluation.

“Before commencing the evaluation, all recruited annotators have undergone a standardized training procedure based on our evaluation guidelines (Supplementary Section E).”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

MDAgents utilizes dynamic routing based on task complexity, with basic tasks bypassing MAS, intermediate tasks recruiting specific experts, and advanced tasks orchestrating a multi-disciplinary team.

“MDAgents [14] utilizes a dynamic routing mechanism based on task complexity.”

• cites [14]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Classified as probably not a substantial claim. May combine two claims.

The automated open-coding outputs were manually reviewed by two clinical experts with interdisciplinary experience in medicine and AI, who consolidated labels into a unified codebook.

“The automated open-coding outputs are manually reviewed by two clinical experts with interdisciplinary experience in medicine and artificial intelligence, who consolidate the generated labels into a unified codebook.”

Classified as probably not a substantial claim.

Multimodal cases make this problem more acute because the first textual description of an image can shape the rest of the workflow even when the image itself is no longer inspected.

“Multimodal cases make this problem more acute, because the first textual description of an image can shape the rest of the workflow even when the image itself is no longer inspected.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Cases that could not produce a valid response after retries due to provider-side safety or content filtering were excluded from the denominator and reported with sample size; this occurred for five GLM-4.6V cases in PathVQA.

“Cases that cannot produce a valid response after the allowed retries because of provider-side safety or content filtering are excluded from the denominator for that model-dataset combination and reported with the resulting sample size; this occurred for five GLM-4.6Vcases in PathVQA.”

Classified as probably not a substantial claim. May combine two claims.

ReConcile replaces central meta-agent synthesis with a round-table consensus mechanism, using confidence-weighted voting for final decision.

“The ReConcile architecture [47] replaces central meta-agent synthesis with a round-table consensus mechanism.”

• cites [47]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Classified as probably not a substantial claim. May combine two claims.

Although expert validation supports scalable automated auditing, assessments of specialist adequacy, visual grounding, and contradiction resolution still involve clinical judgment.

“Although expert validation supports scalable automated auditing, assessments of specialist adequacy, visual grounding, and contradiction resolution still involve clinical judgment.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

VQA-RAD consists of visual question answering pairs based on clinical radiology images (CT, MRI, X-ray) from the MedPix database, addressing spatial reasoning, counting, and abnormality detection.

“The VQA-RAD dataset [45] consists of visual question answering pairs based on clinical radiology images (CT, MRI, and X-ray) sourced from the MedPix database, addressing tasks such as spatial reasoning, counting, and abnormality detection.”

• cites [45]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Classified as probably not a substantial claim.

The study randomly sampled 20 logs for each of the 36 dataset-framework combinations, yielding 720 logs for open-coding.

“We randomly sample 20 logs for each of the 36 combinations of datasets and multi-agent frameworks, yielding a subset of 720 logs.”

May combine two claims.

The study used a matched evaluation protocol to establish single-LLM baselines and measure multi-agent collaboration effects on the same clinical cases.

“We use a matched evaluation protocol to establish single-LLM diagnostic baselines and then measure the effect of multi-agent collaboration on the same clinical cases before auditing the collaboration process.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

GPT-5.2 was used as the open-coding agent, distinct from the Gemini-3-Flash auditor used for automated auditing.

“We use GPT-5.2 as the open-coding agent, distinct from the Gemini-3-Flash auditor used for the subsequent automated auditing system.”

Classified as probably not a substantial claim. May combine two claims.

The training ensured that annotators understood the definitions and evaluation criteria for the ten predefined failure modes and focused on the collaborative process rather than only the final diagnostic outcome.

“The training ensures that annotators comprehend the definitions and evaluation criteria for the ten predefined failure modes and focus on the collaborative process rather than only the final diagnostic outcome.”

Classified as probably not a substantial claim.

The 360 instances were independently annotated by two clinical experts with interdisciplinary experience in medicine and artificial intelligence using a back-to-back double-blind labeling protocol.

“These 360 instances are independently annotated by two clinical experts with interdisciplinary experience in medicine and artificial intelligence using a back-to-back double-blind labeling protocol on the annotation platform.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

HealthcareAgent integrates a safety mechanism with execution planning, general doctor agents, specialized safety supervisors, and a final decision-maker.

“HealthcareAgent [18] integrates a rigorous safety mechanism into the clinical diagnostic process.”

• cites [18]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Classified as probably not a substantial claim.

The samples used for performance evaluation were strictly distinct from those reserved for open-coding to prevent data leakage.

“These samples are strictly distinct from those reserved for the open-coding phase to prevent data leakage.”

Probably not a substantial claim: may not pass S3 (load-bearing).

MedXpertQA consists of medical questions from examinations and clinical resources to assess advanced clinical reasoning.

“The MedXpertQA dataset [43] consists of medical questions sourced from examinations and clinical resources to assess advanced clinical reasoning.”

• cites [43]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Classified as probably not a substantial claim.

Evaluators remained completely blind to the predictions generated by the automated audit system.

“Evaluators remain completely blind to the predictions generated by the automated audit system.”

Probably not a substantial claim: may not pass S3 (load-bearing).

ReConcile was omitted from the per-case analysis of neglect of contradictions in reasoning during decision because its final aggregation simply counts votes and does not produce a synthesis or decision explanation that weighs supporting claims from different agents.

“ReConcile is omitted because its final aggregation directly counts votes, leaving no synthesis or decision explanation that compares supporting claims across agents.”

Probably not a substantial claim: may not pass S3 (load-bearing).

No domain mismatch exists in that case.

“No domain mismatch exists.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

This case contrasts with Supplementary Figure 17: when evidence is explicit in text, the system can remain close to the input and avoid the input-level fabrications that dominate visual datasets.

“This case contrasts with Supplementary Figure 17: when the evidence is explicit in text, the system can remain close to the input and avoid the input-level fabrications that dominate the visual datasets.”

• cites Supplementary Figure 17Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

May combine two claims.

The research exclusively utilized publicly available and anonymized datasets, ensuring no direct involvement of human subjects or access to private patient information.

“This research exclusively utilizes publicly available and anonymized datasets, ensuring no direct involvement of human subjects or access to private patient information.”

Probably not a substantial claim: may not pass S3 (load-bearing). The quote only partly states this claim.

MDAgents was not audited for F-2.2.2 because its workflow lacks a review step where agents revisit conflicting claims after peer discussion.

“MDAgents is therefore not audited for F-2.2.2, and MDAgents cases labeled”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The MDAgents row was dashed because its workflow lacks a review step where an agent revisits a previous view after peer discussion, so F-2.2.1 was not audited for MDAgents.

“The MDAgents row is dashed because its workflow does not include a review step in which an agent revisits a previous view after peer discussion; therefore F-2.2.1 is not audited for MDAgents.”

Classified as probably not a substantial claim. May combine two claims.

In the MedQA case about a psychiatric defense mechanism in an intoxicated ER patient with behavioral issues, while Psychiatry is most specific, Internal Medicine/General Medicine are appropriate because these practitioners handle initial assessment of substance use disorders and behavioral manifestations in the ER, and psychiatric defense mechanisms are standard content in their board exams.

“While Psychiatry is most specific, Internal Medicine/General Medicine are appropriate: these practitioners handle initial assessment of substance use disorders and behavioral manifestations in the ER, and psychiatric defense mechanisms are standard content in their board exams.”

May combine two claims.

The task depended on microscopic tissue morphology rather than radiologic imaging or general clinical management.

“The task depends on microscopic tissue morphology rather than radiologic imaging or general clinical management.”

Probably not a substantial claim: may not pass S3 (load-bearing).

ReConcile was omitted from the F-3.1.2 evaluation because its final aggregation directly counts votes, leaving no synthesis or decision explanation in which role-label justification can be evaluated.

“ReConcile is omitted because its final aggregation directly counts votes, leaving no synthesis or decision explanation in which role-label justification can be evaluated.”

Classified as probably not a substantial claim.

Process failures can occur even when the final answer is correct.

“Process failures can coexist with correct final answers.”

Probably not a substantial claim: may not pass S3 (load-bearing).

ReConcile was omitted from the F-3.1.1 evaluation because its final aggregation directly counts votes, leaving no synthesis or decision explanation in which suppression of a correct minority rationale can be evaluated.

“ReConcile is omitted because its final aggregation directly counts votes, leaving no synthesis or decision explanation in which suppression of a correct minority rationale can be evaluated.”

May combine two claims.

The observed inter-annotator agreement supports the clarity and reliability of the proposed codebook.

“The macro-average kappa of 0.76 indicates substantial overall agreement, validating the clarity and reliability of the proposed codebook.”

May combine two claims.

In phase 1, inputs are misread or missed, and the resulting interpretation deviation builds up.

“Phase 1 summary: input is misperceived or missed, and the resulting interpretation deviation accumulates.”

Probably not a substantial claim: may not pass S3 (load-bearing).

HealthcareAgent, MAC, and ReConcile were omitted from the per-audit evaluation of mismatch between assigned roles and clinical tasks because they do not use dynamic role assignment.

“HealthcareAgent, MAC, and ReConcile are omitted because they do not use dynamic role assignment.”

Probably not a substantial claim: may not pass S3 (load-bearing).

MDAgents cases labeled 'basic' were excluded from auditing because they do not involve multi-agent collaboration.

“MDAgents cases labeled “basic” are excluded because they do not involve multi-agent collaboration.”

Probably not a substantial claim: may not pass S3 (load-bearing).

In the corresponding non-failure PubMedQA case about MRI substituting for diagnostic arthroscopy in occult post-traumatic knee lesions, because it is a QA task, modality adherence means using the abstract and no visual evidence is expected.

“This PubMedQA case asks whether magnetic resonance can substitute for diagnostic arthroscopy in occult post-traumatic knee lesions.Because this is a QA task, modality adherence means using the abstract; no visual evidence is expected.”

Classified as probably not a substantial claim.

ReConcile was omitted from evaluation of suppression of correct minority views by incorrect consensus because its final aggregation directly counts votes and provides no synthesis or decision explanation that could reveal suppression of a correct minority rationale.

“ReConcile is omitted because its final aggregation directly counts votes, leaving no synthesis or decision explanation in which suppression of a correct minority rationale can be evaluated.”

Probably not a substantial claim: may not pass S3 (load-bearing).

For F-3.2.1, HealthcareAgent, MDAgents, and ReConcile were not audited because their workflows do not include cross-round meta-agent steps for this mode.

“The HealthcareAgent, MDAgents, and ReConcile rows are dashed because their audited workflows do not include cross-round meta-agent steps for this mode”

Probably not a substantial claim: may not pass S3 (load-bearing).

ReConcile was omitted from evaluation of reasoning distorted by authority bias because its final aggregation directly counts votes and provides no synthesis or decision explanation that could assess role-label justification.

“ReConcile is omitted because its final aggregation directly counts votes, leaving no synthesis or decision explanation in which role-label justification can be evaluated.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The HealthcareAgent, MDAgents, and ReConcile rows were dashed because their audited workflows lack cross-round meta-agent steps for self-contradiction in viewpoints across rounds.

“The HealthcareAgent, MDAgents, and ReConcile rows are dashed because their audited workflows do not include cross-round meta-agent steps for this mode”

Probably not a substantial claim: may not pass S3 (load-bearing).

This case contrasts with Supplementary Figure 19: when the required evidence source is text, the agent can stay aligned with that modality and avoid the text-only fallback seen in the visual failure case.

“This case contrasts with Supplementary Figure 19: when the required evidence source is text, the agent can stay aligned with that modality and avoid the text-only fallback seen in the visual failure case.”

• cites Supplementary Figure 19Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

May combine two claims.

In phase 2, discussion frequently repeats earlier judgments, and unresolved conflicts can persist into later discussion steps.

“Phase 2 summary: discussion often repeats prior judgments, and unresolved conflicts can remain in subsequent discussion steps.”

May combine two claims.

The PathVQA case illustrates F-1.2.1 as a text-only fallback: the reply sounds medically plausible but does not inspect the image evidence needed to decide whether the displayed p-component itself is the fibrillar amyloid pattern.

“This case shows F-1.2.1 as a text-only fallback: the reply sounds medically plausible but does not inspect the image evidence needed to decide whether the displayed p-component itself is the fibrillar amyloid pattern.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

For GLM-4.6V on PathVQA, 95 cases were evaluated, and five were blocked by internal safety and content filters so no valid response was generated.

“Notably, the evaluation of GLM-4.6V on the PathVQA dataset comprises 95 cases; five cases trigger its internal safety and content filters, preventing the generation of valid responses.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

Each framework requires a tailored probe integration strategy.

“Each framework requires a tailored probe integration strategy.”

• AssumptionRests on an assumption: The ColaCare architecture employs a structured pipeline encompassing role assignment, discussion, review, and decision phases.
• AssumptionRests on an assumption: The Multi-Agent Conversation (MAC) framework relies on open iterative debate and a central supervisor agent.
• AssumptionRests on an assumption: The HealthcareAgent framework separates preliminary analysis from multi-faceted safety supervision.
• AssumptionRests on an assumption: MDAgents dynamically adjusts its collaboration topology based on task complexity.
• AssumptionRests on an assumption: MedAgents utilizes dynamic role assignment combined with a meta-agent synthesis and peer review loop.
• AssumptionRests on an assumption: ReConcile employs independent analysis followed by round-table discussions and confidence-weighted voting, eliminating central meta-agent synthesis.

Classified as probably not a substantial claim.

This case shows F-2.2.1 as a review step that preserves an existing interpretation without testing whether the repeated claim actually matches the study's scope.

“This case shows F-2.2.1 as a review step that preserves an existing interpretation without testing whether the repeated claim actually matches the study's scope.”

May combine two claims.

This case contrasts with Supplementary Figure 33: consensus alone does not create F-3.1.3; the failure requires that a synthesizer present incompatible supporting rationales as if they were mutually compatible.

“This case contrasts with Supplementary Figure 33: consensus alone does not create F-3.1.3;the failure requires that a synthesizer present incompatible supporting rationales as if they were mutually compatible.”

• cites Supplementary Figure 33Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Classified as probably not a substantial claim. May combine two claims.

This case contrasts with Supplementary Figure 31: agreement among specialists does not itself constitute F-3.1.2; the failure requires that synthesis privilege the source of a claim over direct verification of the clinical or visual evidence.

“This case contrasts with Supplementary Figure 31: agreement among specialists does not itself constitute F-3.1.2;the failure requires that synthesis privilege the source of a claim over direct verification of the clinical or visual evidence.”

• cites Supplementary Figure 31Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S3 (load-bearing).

This case shows F-2.1.2 as role execution without the diagnostic depth expected from the assigned clinician.

“This case shows F-2.1.2as role execution without the diagnostic depth expected from the assigned clinician.”

Classified as probably not a substantial claim. May combine two claims.

This case contrasts with Supplementary Figure 29: an incorrect final answer can arise from shared or unanimous reasoning, but F-3.1.1 requires that a correct dissenting view first appear and then be overridden by the subsequent consensus mechanism.

“This case contrasts with Supplementary Figure 29: an incorrect final answer can arise from shared or unanimous reasoning, but F-3.1.1 requires that a correct dissenting view first appear and then be overridden by the subsequent consensus mechanism.”

• cites Supplementary Figure 29Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Although Psychiatry would be more specialized, the task is QA rather than image or slide interpretation, so the recruited roles remained within scope and the case was not flagged as a failure by the auditor.

“Although Psychiatry would be more specialized, the task is QA rather than image or slide interpretation, so the recruited roles remain within scope and the case is not flagged as a failure by the auditor.”

Classified as probably not a substantial claim. May combine two claims.

This case contrasts with Supplementary Figure 25: repeating the same conclusion is not required for consensus; a review step can change the trajectory when it adds reasoning about the small sample, the two esophageal intubations, and the broader meaning of impaired airway management in the helicopter environment.

“This case contrasts with Supplementary Figure 25: repeating the same conclusion is not required for consensus; a review step can change the trajectory when it adds reasoning about the small sample, the two esophageal intubations, and the broader meaning of impaired airway management in the helicopter environment.”

• cites Supplementary Figure 25Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

May combine two claims.

This case shows F-3.1.1 as count-based synthesis: the correct minority observation is preserved in the interaction history but discarded when the system reduces consensus to majority voting rather than re-checking the image evidence.

“This case shows F-3.1.1 as count-based synthesis: the correct minority observation is preserved in the interaction history but discarded when the system reduces consensus to majority voting rather than re-checking the image evidence.”

May combine two claims.

This case contrasts with Supplementary Figure 21: not every non-psychiatrist assignment constitutes F-2.1.1; the failure depends on whether the recruited roles miss the expertise required by the case.

“This case contrasts with Supplementary Figure 21: not every non-psychiatrist assignment constitutes F-2.1.1;the failure depends on whether the recruited roles miss the expertise required by the case.”

• cites Supplementary Figure 21Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Classified as probably not a substantial claim.

This case shows F-3.1.3 as false coherence: the synthesis preserves agreement on option E while ignoring contradictions in the anatomical and staging claims used to support that option.

“This case shows F-3.1.3as false coherence: the synthesis preserves agreement on option”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

This case contrasts with Supplementary Figure 23: the agent adapted its reasoning to the assigned role, used role-appropriate clinical terminology, and made the management decision from key synovial-fluid findings rather than generic advice.

“This case contrasts with Supplementary Figure 23: the agent adapts its reasoning to the assigned role, uses role-appropriate clinical terminology, and makes the management decision from the key synovial-fluid findings rather than from generic advice.”

• cites Supplementary Figure 23Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

May combine two claims.

This case contrasts with Supplementary Figure 27: agreement alone does not create F-2.2.2; the failure requires a direct contradiction that subsequent discussion leaves unresolved.

“This case contrasts with Supplementary Figure 27: agreement alone does not create F-2.2.2; the failure requires a direct contradiction that subsequent discussion leaves unresolved.”

• cites Supplementary Figure 27Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

May combine two claims.

Disagreement within the collaboration does not itself create F-3.2.1; the failure requires that the synthesizer or decision-maker reverse its own cross-round position without new evidence.

“disagreement within the collaboration does not itself create F-3.2.1; the failure requires that the synthesizer or decision-maker reverse its own cross-round position without new evidence.”