IBISAgent: Reinforcing Pixel-Level Visual Reasoning in MLLMs for Universal Biomedical Object Referring and Segmentation

May combine two claims.

UniBiomed produces incorrect textual reasoning and responses, describing objects that do not exist in the image and generating low-quality segmentation masks.

“UniBiomed [41] produces incorrect textual reasoning and responses, describing objects that do not exist in the image and generating low-quality segmentation masks.”

• cites [41] [41] Linshan Wu, Yuxiang Nie, Sunan He, Jiaxin Zhuang, Luyang Luo, Neeraj Mahboobani, Varut Vardhanabhuti, Ronald Cheong Kin Chan, Yifan Peng, Pranav Rajpurkar, et al. Unibiomed: A universal foundation model for grounded biomedical image interpretation. arXiv preprint arXiv:2504.21336 , 2025. 2, 6, 7 - sourceDisputed

  The Method section describes UniBiomed as generating precise segmentation masks and detailed diagnostic descriptions, rather than the low-quality masks and incorrect responses attributed to it. The Discussion acknowledges limits in rare-disease coverage and clinical validation, but does not report descriptions of nonexistent objects or incorrect textual reasoning. (LLM, conf=0.77)

  Checked by a reasoning model — may be wrong.

  Experimental — accuracy not yet measured

Classified as probably not a substantial claim.

Recent studies have explored activating the intrinsic pixel-level understanding capabilities of MLLMs via RL, enabling the models to generate bounding boxes or point prompts that precisely localize target regions.

“Recent studies [11, 20, 24, 25] have explored activating the intrinsic pixel-level understanding capabilities of MLLMs via RL, enabling the models to generate bounding boxes or point prompts that precisely localize target regions.”

• cites [11] [11] Jiaqi Huang, Zunnan Xu, Jun Zhou, Ting Liu, Yicheng Xiao, Mingwen Ou, Bowen Ji, Xiu Li, and Kehong Yuan. Sam-r1: Leveraging sam for reward feedback in multimodal segmentation via reinforcement learning. arXiv preprint arXiv:2505.22596 , 2025. 3, 5 - sourceSupported
• cites [20] [20] Binxu Li, Tiankai Yan, Yuanting Pan, Jie Luo, Ruiyang Ji, Jiayuan Ding, Zhe Xu, Shilong Liu, Haoyu Dong, Zihao Lin, et al. Mmedagent: Learning to use medical tools with multimodal agent. arXiv preprint arXiv:2407.02483 , 2024. 3, 6, 7 - sourceNot checked yet

  Checked before — its earlier verdict is held back until a new check runs.

• cites [24] [24] Yuqi Liu, Bohao Peng, Zhisheng Zhong, Zihao Yue, Fanbin Lu, Bei Yu, and Jiaya Jia. Seg-zero: Reasoning-chain guided segmentation via cognitive reinforcement. arXiv preprint arXiv:2503.06520 , 2025. 3, 5 - sourceSupported
• cites [25] [25] Yuqi Liu, Tianyuan Qu, Zhisheng Zhong, Bohao Peng, Shu Liu, Bei Yu, and Jiaya Jia. Visionreasoner: Unified visual perception and reasoning via reinforcement learning. arXiv preprint arXiv:2505.12081 , 2025. 3, 5, 6 - sourceSupported

May combine two claims.

Existing medical MLLM approaches introduce implicit segmentation tokens and require joint fine-tuning of the MLLM and external pixel decoders, increasing the risk of catastrophic forgetting and limiting out-of-domain generalization.

“First, they introduce implicit segmentation tokens and require joint fine-tuning of the MLLM and external pixel decoders, increasing the risk of catastrophic forgetting and limiting out-of-domain generalization.”

• cites [1] [1] Fan Bai, Yuxin Du, Tiejun Huang, Max Q-H Meng, and Bo Zhao. M3d: Advancing 3d medical image analysis with multi-modal large language models. arXiv preprint arXiv:2404.00578 , 2024. 2 - sourcePartly supported

  Section 4.1.4 (p. 6) describes M3D-LaMed using a [SEG] token to prompt a segmentation module, and Section 4.3 (p. 7) says the LLM and segmentation module are fine-tuned together. These passages describe this paper’s model, not existing medical MLLM approaches generally. They do not say that joint fine-tuning increases catastrophic forgetting or limits out-of-domain generalization. (LLM, conf=0.83)

  Experimental — accuracy not yet measured
• cites [13] [13] Xiaoshuang Huang, Lingdong Shen, Jia Liu, Fangxin Shang, Hongxiang Li, Haifeng Huang, and Yehui Yang. Towards a multimodal large language model with pixel-level insight for biomedicine. In Proceedings of the AAAI Conference on Artificial Intelligence , pages 3779-3787, 2025. 2, 6, 7 - sourcePartly supported

  The Introduction (page unavailable) says MedPLIB uses a <SEG> token with a mask decoder, and Multi-stage Training (page unavailable) describes fine-tuning the model and pixel decoder. These passages support the architectural description as an example, but do not establish that existing medical MLLM approaches generally require joint fine-tuning. The paper discusses generalization challenges but does not link them—or catastrophic forgetting—to that fine-tuning approach. (LLM, conf=0.84)

  Experimental — accuracy not yet measured
• cites [38] [38] Qinyue Tong, Ziqian Lu, Jun Liu, Yangming Zheng, and Zheming Lu. Medisee: Reasoning-based pixel-level perception in medical images. arXiv preprint arXiv:2504.11008 , 2025. 2 - sourcePartly supported

  Section 4.2 says existing MLLM-based methods generate a special segmentation token whose hidden representation is used by a downstream decoder to produce a mask. Section 4.3 describes fine-tuning MediSee’s own language model and mask decoder, but does not establish that existing approaches require joint fine-tuning or that this causes catastrophic forgetting or limits out-of-domain generalization. Page numbers are unavailable in our copy. (LLM, conf=0.86)

  Experimental — accuracy not yet measured
• cites [41] [41] Linshan Wu, Yuxiang Nie, Sunan He, Jiaxin Zhuang, Luyang Luo, Neeraj Mahboobani, Varut Vardhanabhuti, Ronald Cheong Kin Chan, Yifan Peng, Pranav Rajpurkar, et al. Unibiomed: A universal foundation model for grounded biomedical image interpretation. arXiv preprint arXiv:2504.21336 , 2025. 2, 6, 7 - sourceNot checked yet

  Not checked in the free run.

May combine two claims.

The introduction of implicit segmentation tokens disrupts the MLLM's natural text output space, weakening its reasoning ability and failing to reflect the model's intrinsic pixel-level understanding.

“Moreover, the introduction of implicit segmentation tokens disrupts the MLLM's natural text output space, thereby weakening its reasoning ability and failing to reflect the model's intrinsic pixel-level understanding [35].”

• cites [35] [35] Mennatullah Siam. Pixfoundation: Are we heading in the right direction with pixel-level vision foundation models? arXiv preprint arXiv:2502.04192 , 2025. 2 - sourcePartly supported

  Section 3.2 (p. 5) reports degraded VQA and instruction-following in pixel-level MLLMs trained with mask supervision. The Introduction (p. 2) and C.2 (p. 22) also report that visual grounding can emerge at output tokens other than the expected referring phrase. The paper does not establish that implicit segmentation tokens disrupt the natural text output space or cause the reasoning decline, nor does it say that such tokens fail to reflect intrinsic pixel-level understanding. (LLM, conf=0.85)

  Experimental — accuracy not yet measured

May combine two claims.

This process disrupts the MLLM's original text output space and increases the risk of catastrophic forgetting, thereby compromising semantic generalization by deviating from language-based outputs.

“This process disrupts the MLLM's original text output space and increases the risk of catastrophic forgetting, thereby compromising semantic generalization by deviating from languagebased outputs.”

• cites [12] [12] Xiaoshuang Huang, Lingdong Shen, Jia Liu, Fangxin Shang, Hongxiang Li, Haifeng Huang, and Yehui Yang. Towards a multimodal large language model with pixel-level insight for biomedicine, 2025. 2, 6, 7 - sourceCould not verify

  The Introduction describes the challenge of balancing visual question answering and pixel-level analysis, while Multi-stage Training says expert mixing preserves each expert’s prior knowledge. Neither passage says that the process disrupts the MLLM’s text output space or increases catastrophic forgetting. Searches for those terms and for impaired semantic generalization found no passage addressing that claim. (LLM, conf=0.73)

  Experimental — accuracy not yet measured
• cites [14] [14] Yu Huang, Zelin Peng, Yichen Zhao, Piao Yang, Xiaokang Yang, and Wei Shen. Medseg-r: Reasoning segmentation in medical images with multimodal large language models, 2025. 2 - sourceCould not verify

  Section 2.2 describes generating text responses with special segmentation tokens and training for both text quality and segmentation accuracy. Our copy does not address disruption of the original text output space, catastrophic forgetting, or a resulting loss of semantic generalization (searched those terms and deviation from language-based outputs). No page number is available in the indexed passages. (LLM, conf=0.72)

  Experimental — accuracy not yet measured
• cites [39] [39] Quoc-Huy Trinh, Minh-Van Nguyen, Jung Zeng, Ulas Bagci, and Debesh Jha. Prs-med: Position reasoning segmentation with vision-language model in medical imaging, 2025. 2 -Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [40] [40] Guoxin Wang, Jun Zhao, Xinyi Liu, Yanbo Liu, Xuyang Cao, Chao Li, Zhuoyun Liu, Qintian Sun, Fangru Zhou, Haoqiang Xing, and Zhenhong Yang. Citrus-v: Advancing medical foundation models with unified medical image grounding for clinical reasoning, 2025. 2, 6, 7 - sourcePartly supported

  Section 4.2 (p. 17) reports that directly combining segmentation loss with other tasks can significantly damage VQA performance, supporting a risk of interference with an existing capability. It does not say that the process disrupts the MLLM’s text output space, causes catastrophic forgetting, or compromises semantic generalization by departing from language-based outputs. (LLM, conf=0.83)

  Experimental — accuracy not yet measured

May combine two claims.

Existing medical MLLMs focus on visual question answering and are largely confined to image-level understanding tasks, failing to achieve fine-grained pixel-level comprehension.

“Prevailing medical MLLMsfocus on visual question answering (VQA) and are largely confined to image-level understanding tasks, thus failing to achieve fine-grained pixel-level comprehension.”

• cites [4] [4] Junying Chen, Chi Gui, Ruyi Ouyang, Anningzhe Gao, Shunian Chen, Guiming Hardy Chen, Xidong Wang, Ruifei Zhang, Zhenyang Cai, Ke Ji, et al. Huatuogpt-vision, towards injecting medical visual knowledge into multimodal llms at scale. arXiv preprint arXiv:2406.19280 , 2024. 1, 7, 8 - sourcePartly supported

  The Medical MLLMs and §2.1 Existing Medical VQA Data passages describe substantial use of VQA datasets to develop medical MLLMs. They do not establish that prevailing models are confined to image-level tasks or fail at pixel-level comprehension; a search for fine-grained and pixel-level understanding found no supporting passage. Our copy provides no page numbers for these passages. (LLM, conf=0.84)

  Experimental — accuracy not yet measured
• cites [21] [21] Chunyuan Li, Cliff Wong, Sheng Zhang, Naoto Usuyama, Haotian Liu, Jianwei Yang, Tristan Naumann, Hoifung Poon, and Jianfeng Gao. Llava-med: Training a large languageand-vision assistant for biomedicine in one day. Advances in Neural Information Processing Systems , 36:28541-28564, 2023. 1, 7, 8 -Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [33] [33] Andrew Sellergren, Sahar Kazemzadeh, Tiam Jaroensri, Atilla Kiraly, Madeleine Traverse, Timo Kohlberger, Shawn Xu, Fayaz Jamil, C´ ıan Hughes, Charles Lau, et al. Medgemma technical report. arXiv preprint arXiv:2507.05201 , 2025. 1 - sourcePartly supported

  The Introduction (p. not available) says recent multimodal models have demonstrated utility in standard 2D imaging, while models addressing more complex imaging tasks are limited. Section 3.1 (p. not available) includes VQA among existing MedGemma benchmarks, but also describes report generation and image-based multiple-choice tasks. The report does not establish that prevailing medical MLLMs are largely confined to VQA or image-level tasks, or that they fail at pixel-level comprehension. (LLM, conf=0.82)

  Experimental — accuracy not yet measured
• cites [43] [43] Weiwen Xu, Hou Pong Chan, Long Li, Mahani Aljunied, Ruifeng Yuan, Jianyu Wang, Chenghao Xiao, Guizhen Chen, Chaoqun Liu, Zhaodonghui Li, et al. Lingshu: A generalist foundation model for unified multimodal medical understanding and reasoning. arXiv preprint arXiv:2506.07044 , 2025. 1, 7, 8 - sourcePartly supported

  The Abstract/Introduction (page unavailable) says existing medical MLLMs have a narrow focus on visual modalities and reports their results on medical VQA. It also describes their use in report generation and diagnostic support, so it does not establish that they are confined to image-level tasks. The searched passages do not state that prevailing models fail at pixel-level comprehension; that more specific limitation is unsupported. (LLM, conf=0.83)

  Experimental — accuracy not yet measured

May combine two claims.

LISA-style MLLMs require MLLMs to learn task-specific implicit tokens and undergo additional joint fine-tuning with a segmentation decoder.

“While promising, these methods require MLLMs to learn task-specific implicit tokens and undergo additional joint fine-tuning with a segmentation decoder.”

• cites [18] [18] Xin Lai, Zhuotao Tian, Yukang Chen, Yanwei Li, Yuhui Yuan, Shu Liu, and Jiaya Jia. Lisa: Reasoning segmentation via large language model. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 9579-9589, 2024. 2, 6 - sourceSupported

May combine two claims.

Dependence on joint fine-tuning of the MLLM and external pixel decoders heightens the risk of catastrophic forgetting, resulting in strong in-domain performance but weak cross-domain generalization.

“Their dependence on joint fine-tuning of the MLLM and external pixel decoders heightens the risk of catastrophic forgetting , resulting in strong in-domain performance but weak cross-domain generalization.”

• cites [1] [1] Fan Bai, Yuxin Du, Tiejun Huang, Max Q-H Meng, and Bo Zhao. M3d: Advancing 3d medical image analysis with multi-modal large language models. arXiv preprint arXiv:2404.00578 , 2024. 2 - sourcePartly supported

  The MLLM Training section (p. 7) confirms that the vision encoder, LLM, and segmentation module are fine-tuned together. It does not attribute catastrophic forgetting or weak cross-domain generalization to that training; instead, the OOD case study (pp. 13–14) reports strong generalization to questions outside the training data and says LoRA fine-tuning preserves the LLM’s original knowledge. OOD-question performance does not, by itself, establish cross-domain performance. (LLM, conf=0.87)

  Experimental — accuracy not yet measured
• cites [13] [13] Xiaoshuang Huang, Lingdong Shen, Jia Liu, Fangxin Shang, Hongxiang Li, Haifeng Huang, and Yehui Yang. Towards a multimodal large language model with pixel-level insight for biomedicine. In Proceedings of the AAAI Conference on Artificial Intelligence , pages 3779-3787, 2025. 2, 6, 7 - sourcePartly supported

  The Zero-shot to Pixel Grounding section reports that smaller medical image-grounding models face generalization challenges on unseen imaging modalities, while the evaluation table shows substantially higher scores on the MeCoVQA-G test set than on zero-shot modalities for several such models. The paper does not attribute that gap to joint fine-tuning of an MLLM and external pixel decoder or to catastrophic forgetting. (LLM, conf=0.79)

  Experimental — accuracy not yet measured
• cites [38] [38] Qinyue Tong, Ziqian Lu, Jun Liu, Yangming Zheng, and Zheming Lu. Medisee: Reasoning-based pixel-level perception in medical images. arXiv preprint arXiv:2504.11008 , 2025. 2 - sourcePartly supported

  Section 4.3 (page unavailable) describes fine-tuning the MLLM with LoRA while fully fine-tuning the mask decoder and training the detection decoder. It does not say that this creates a risk of catastrophic forgetting or produces strong in-domain but weak cross-domain performance; searches for those outcomes found no supporting passage. (LLM, conf=0.83)

  Experimental — accuracy not yet measured
• cites [41] [41] Linshan Wu, Yuxiang Nie, Sunan He, Jiaxin Zhuang, Luyang Luo, Neeraj Mahboobani, Varut Vardhanabhuti, Ronald Cheong Kin Chan, Yifan Peng, Pranav Rajpurkar, et al. Unibiomed: A universal foundation model for grounded biomedical image interpretation. arXiv preprint arXiv:2504.21336 , 2025. 2, 6, 7 - sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

These spatial coordinates are subsequently passed to SAM as prompts to generate the corresponding segmentation masks.

“These spatial coordinates are subsequently passed to SAM as prompts to generate the corresponding segmentation masks.”

• cites [11] [11] Jiaqi Huang, Zunnan Xu, Jun Zhou, Ting Liu, Yicheng Xiao, Mingwen Ou, Bowen Ji, Xiu Li, and Kehong Yuan. Sam-r1: Leveraging sam for reward feedback in multimodal segmentation via reinforcement learning. arXiv preprint arXiv:2505.22596 , 2025. 3, 5 - sourceSupported
• cites [20] [20] Binxu Li, Tiankai Yan, Yuanting Pan, Jie Luo, Ruiyang Ji, Jiayuan Ding, Zhe Xu, Shilong Liu, Haoyu Dong, Zihao Lin, et al. Mmedagent: Learning to use medical tools with multimodal agent. arXiv preprint arXiv:2407.02483 , 2024. 3, 6, 7 - sourceSupported
• cites [24] [24] Yuqi Liu, Bohao Peng, Zhisheng Zhong, Zihao Yue, Fanbin Lu, Bei Yu, and Jiaya Jia. Seg-zero: Reasoning-chain guided segmentation via cognitive reinforcement. arXiv preprint arXiv:2503.06520 , 2025. 3, 5 - sourceSupported
• cites [25] [25] Yuqi Liu, Tianyuan Qu, Zhisheng Zhong, Bohao Peng, Shu Liu, Bei Yu, and Jiaya Jia. Visionreasoner: Unified visual perception and reasoning via reinforcement learning. arXiv preprint arXiv:2505.12081 , 2025. 3, 5, 6 - sourcePartly supported

  Section 3.3 (p. 4) says VisionReasoner produces bounding boxes and center points that connect to a segmentation module to produce masks. Sections 4.1 (p. 6) and 4.2 (p. 7) identify that module as SAM2 and describe sending bounding-box predictions to it. The claimed workflow is supported, but the paper specifies SAM2 rather than SAM. (LLM, conf=0.86)

  Experimental — accuracy not yet measured

May combine two claims.

GPT-4o outputs seemingly correct textual reasoning but produces an incorrect localization bounding box for a liver tumor case requiring reasoning.

“For a case requiring reasoning to identify a liver tumor, the powerful closed-source GPT-4o [15] outputs seemingly correct textual reasoning but produces an incorrect localization bounding box.”

• cites [15] [15] Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card. arXiv preprint arXiv:2410.21276 , 2024. 7, 8 -Supported

May combine two claims.

MedPLIB provides coarse localization that roughly captures the lesion region but suffers from false-positive predictions.

“MedPLIB [13] provides coarse localization that roughly captures the lesion region but suffers from false-positive predictions.”

• cites [13] [13] Xiaoshuang Huang, Lingdong Shen, Jia Liu, Fangxin Shang, Hongxiang Li, Haifeng Huang, and Yehui Yang. Towards a multimodal large language model with pixel-level insight for biomedicine. In Proceedings of the AAAI Conference on Artificial Intelligence , pages 3779-3787, 2025. 2, 6, 7 - sourcePartly supported

  The Conclusion shows MedPLIB being prompted to segment lesions, and Additional information (page not provided) acknowledges hallucinations and substantial pixel-grounding errors for small targets. The paper does not characterize its lesion localization as coarse or specifically report false-positive predictions. (LLM, conf=0.75)

  Experimental — accuracy not yet measured

May combine two claims.

BiomedParseData contains 3.4 million image-mask-label tuples that cover 82 biomedical object types across 9 imaging modalities.

“To this end, we utilize the large-scale BiomedParseData [46], containing 3.4 million image-mask-label tuples that cover 82 biomedical object types across 9 imaging modalities.”

• cites [46] [46] Theodore Zhao, Yu Gu, Jianwei Yang, Naoto Usuyama, Ho Hin Lee, Tristan Naumann, Jianfeng Gao, Angela Crabtree, Jacob Abel, Christine Moung-Wen, et al. Biomedparse: a biomedical foundation model for image parsing of everything everywhere all at once. arXiv preprint arXiv:2405.12971 , 2024. 1, 4, 5, 6, 9 sourceSupported

May combine two claims.

BiomedParseData provides diverse, high-quality masks that supply rich pixel-level details for fine-grained analysis and serve as reliable supervision for synthesizing click-based annotation trajectories.

“The dataset provides diverse, high-quality masks that supply rich pixel-level details for fine-grained analysis and serve as reliable supervision for synthesizing click-based annotation trajectories.”

• cites [46] [46] Theodore Zhao, Yu Gu, Jianwei Yang, Naoto Usuyama, Ho Hin Lee, Tristan Naumann, Jianfeng Gao, Angela Crabtree, Jacob Abel, Christine Moung-Wen, et al. Biomedparse: a biomedical foundation model for image parsing of everything everywhere all at once. arXiv preprint arXiv:2405.12971 , 2024. 1, 4, 5, 6, 9 sourcePartly supported

  Details of BiomedParseData (p. 8) describes a diverse dataset of pixel-level segmentation masks and stringent criteria for including them, supporting the claim about detailed, high-quality masks. The paper does not describe using those masks to synthesize click-based annotation trajectories; the Overview (p. 3) instead describes text-based training. (LLM, conf=0.86)

  Experimental — accuracy not yet measured

Probably not a substantial claim: may not pass S3 (load-bearing).

The released split of MeCoVQA-G is 100K training pairs and 2,344 test pairs.

“The released split is 100 K training pairs and 2 , 344 test pairs.”

• cites [13] [13] Xiaoshuang Huang, Lingdong Shen, Jia Liu, Fangxin Shang, Hongxiang Li, Haifeng Huang, and Yehui Yang. Towards a multimodal large language model with pixel-level insight for biomedicine. In Proceedings of the AAAI Conference on Artificial Intelligence , pages 3779-3787, 2025. 2, 6, 7 - sourceSupported

May combine two claims.

On average, IBISAgent surpasses existing medical-specific MLLMs by at least 35.13% in IoU, 37.58% in DSC, and 29.79% in F1 score.

“On average, it surpasses these models by at least 35 . 13% in IoU, 37 . 58% in DSC, and 29 . 79% in F1 score.”

Probably not a substantial claim: may not pass S3 (load-bearing).

In total, 888K VQA samples are used for RL training, forming dataset D_rl.

“In total, 888 K VQA samples are used for RL training, forming the dataset D rl.”

May combine two claims.

IBISAgent generates correct and coherent textual reasoning and adaptively produces high-quality masks through step-by-step, precise pixel grounding.

“In contrast, IBISAgent not only generates correct and coherent textual reasoning but also adaptively produces high-quality masks through step-by-step, precise pixel grounding.”

May combine two claims.

S_pseg and S_len improve interaction efficiency by discouraging redundant or uninformative clicks and preventing unnecessarily long trajectories, enabling the model to learn when to stop.

“Meanwhile, S pseg and S len improve interaction efficiency by discouraging redundant or uninformative clicks and preventing unnecessarily long trajectories, enabling the model to learn when to stop.”

May combine two claims.

The prompt-only approach is neither adaptable nor robust.

“The prompt-only approach is neither adaptable nor robust, whereas progressively incorporating our proposed training stages yields substantial improvements over the base model, highlighting the necessity of each component.”

May combine two claims.

Progressively incorporating the proposed training stages yields substantial improvements over the base model, highlighting the necessity of each component.

“The prompt-only approach is neither adaptable nor robust, whereas progressively incorporating our proposed training stages yields substantial improvements over the base model, highlighting the necessity of each component.”

May combine two claims.

S_click and S_pseg provide the greatest improvements in mask quality, as they ensure accurate click localization and encourage each interaction step to make a positive contribution to the segmentation outcome.

“The rewards S click and S pseg provide the greatest improvements in mask quality, as they ensure accurate click localization and encourage each interaction step to make a positive contribution to the segmentation outcome.”

Classified as probably not a substantial claim. May combine two claims.

In the held-out set, CT images include 100 liver tumor cases, 100 gallbladder tumor cases, 100 pancreatic cancer cases, and 100 kidney tumor cases; MRI images include 100 colon tumor cases and 100 brain cancer cases; histopathology images include 400 breast cancer cases.

“In this held-out set, CT images include 100 liver tumor cases, 100 gallbladder tumor cases, 100 pancreatic cancer cases, and 100 kidney tumor cases. MRI images include 100 colon tumor cases and 100 brain cancer cases. Histopathology images include 400 breast cancer cases.”

Probably not a substantial claim: may not pass S3 (load-bearing).

D_cold has an average trajectory length per sample of 8.69 steps.

“Furthermore, the average trajectory length per sample is 8 . 69 steps, indicating a rich capture of the multi-step refinement processes required for complex segmentation tasks.”

May combine two claims.

The SFT dataset D_cold comprises 47,146 individual samples and 456,795 VQA pairs.

“It comprises a total of 47 , 146 individual samples (e.g., slices or images), which collectively contain 456 , 795 visual question-answer (VQA) pairs that capture a wide range of diverse scenarios.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The RL corpus comprises 60,826 samples and 564,385 QA pairs.

“This RL corpus comprises a total of 60 , 826 samples, which collectively contain 564 , 385 QA pairs.”

May combine two claims.

D_cold has an overall average IoU of 94.27 (median 95.07) and an average Dice score of 0.9703 (median 97.47).

“achieving an overall average Intersection over Union (IoU) of 94 . 27 (median: 95 . 07 ) and an average Dice score of 0 . 9703 (median: 97 . 47 ).”

Probably not a substantial claim: may not pass S3 (load-bearing).

D_test spans 32 distinct task groups.

“Consequently, the test set spans 32 distinct task groups.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The in-domain test set D_test comprises 9,902 samples and 156,289 VQA pairs.

“The test set comprises a total of 9 , 902 samples, containing 156 , 289 VQA pairs.”

Probably not a substantial claim: may not pass S3 (load-bearing).

For each case in the held-out set, human annotators construct a VQA pair along with a corresponding reasoning trajectory.

“For each case, human annotators construct a VQA pair along with a corresponding reasoning trajectory.”

May combine two claims.

During mask refinement, if the user supplies an initial mask that does not match the described segmentation target, IBISAgent detects the inconsistency, corrects the erroneous mask, and generates the appropriate segmentation result.

“During mask refinement, if the user supplies an initial mask that does not match the described segmentation target, IBISAgent detects the inconsistency, corrects the erroneous mask, and generates the appropriate segmentation result.”

May combine two claims.

IBISAgent successfully initiates correct segmentation and employs its unique reasoning-driven refinement mechanism to correct errors, ultimately achieving high-quality segmentation and accurate diagnostic descriptions.

“In contrast, IBISAgent successfully initiates correct segmentation and employs its unique reasoning-driven refinement mechanism to correct errors-such as retracting masks from adjacent healthy tissues-ultimately achieving high-quality segmentation and accurate diagnostic descriptions.”

May combine two claims.

The approach generates dense, anatomically grounded reasoning traces without requiring manual templates for every scenario.

“As shown in Fig. 8, this approach generates dense, anatomically grounded reasoning traces without requiring manual templates for every scenario (like we did before using smaller models).”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The SFT dataset D_cold contains 47,146 samples and 456,795 QAs.

“Total | 47,146 | 8.69 | 456,795 | 0.9427 | 0.9507 | 0.9703 | 0.9747”

May combine two claims.

IBISAgent preserves the LLM's inherent internal language representations and extends segmentation beyond mere pixel prediction to encompass fine-grained visual reasoning and action planning.

“Compared with previous medical MLLMs that rely on implicit tokens and additional pixel decoders, IBISAgent preserves the LLM's inherent internal language representations and extends segmentation beyond mere pixel prediction to encompass finegrained visual reasoning and action planning.”

May combine two claims.

IBISAgent supports flexible usage scenarios, including both from-scratch segmentation and refinement of pre-existing masks.

“Based on this formulation, IBISAgent supports flexible usage scenarios, including both from-scratch segmentation and refinement of pre-existing masks.”

May combine two claims.

IBISAgent iteratively generates interleaved reasoning and text-based click commands, invokes segmentation tools, and refines the current segmentation based on evolving visual features.

“It iteratively generates interleaved reasoning and text-based click commands, invokes segmentation tools, and refines the current segmentation based on evolving visual features.”

Probably not a substantial claim: may not pass S4 (based). May combine two claims.

This design facilitates vision-centric, multi-step decision-making and enables genuine reasoning to support advanced tasks such as automatic mask refinement.

“This design facilitates vision-centric, multi-step decision-making and enables genuine reasoning to support advanced tasks such as automatic mask refinement (Fig. 1).”

Probably not a substantial claim: may not pass S4 (based). May combine two claims.

Our SFT strategy endows the model with strong pixel-level reasoning and self-reflection abilities, providing a solid foundation for subsequent RL.

“Our SFT strategy endows the model with strong pixel-level reasoning and self-reflection abilities, providing a solid foundation for subsequent RL.”

Probably not a substantial claim: may not pass S4 (based). May combine two claims.

The ability is inherited from the model's native capability of visual grounding and action planning, and further incentivized and enhanced via end-to-end SFT and RL training.

“The ability is inherited from the model's native capability of visual grounding and action planning, and further incentivized and enhanced via end-to-end SFT (Sec. 3.2) and RL training (Sec. 3.3).”

Probably not a substantial claim: may not pass S4 (based).

This design encourages the model to autonomously explore and strengthen its pixel-level reasoning ability during RL.

“This design encourages the model to autonomously explore and strengthen its pixel-level reasoning ability during RL.”

Probably not a substantial claim: may not pass S3 (load-bearing).

All generated reasoning traces are post-filtered for format and correctness by human annotations.

“All generated reasoning traces are post-filtered for format and correctness by human annotations.”

Classified as probably not a substantial claim.

The study evaluates on a private held-out dataset collected from three medical centers to avoid unintentional data leakage.

“To ensure fair evaluation and avoid unintentional data leakage, we additionally evaluate on a private held-out dataset collected from three medical centers, comprising 1K CT, MRI, and pathology images across seven cancer types.”

May combine two claims.

GPT-4o fails to align textual reasoning with visual features, possibly hallucinating plausible answers rather than truly understanding the medical image, leading to incorrect localization.

“This indicates that the model fails to align textual reasoning with visual features-it may be hallucinating plausible answers rather than truly understanding the medical image, ultimately leading to incorrect localization.”

Probably not a substantial claim: may not pass S4 (based).

Our fine-grained reward scheme better reflects the complexity of iterative segmentation, guiding the model to produce actions that are both spatially valid and semantically accurate.

“Our finegrained reward scheme better reflects the complexity of iterative segmentation, guiding the model to produce actions that are both spatially valid and semantically accurate.”

Classified as probably not a substantial claim.

This hybrid data composition enables IBISAgent to selectively activate pixel-space reasoning only when necessary.

“This hybrid data composition enables IBISAgent to selectively activate pixel-space reasoning only when necessary.”

May combine two claims.

Through RL training, the agent learns to adaptively reason over pixel features and plan click actions when necessary, achieving superior autonomous interactive segmentation.

“Through RL training, the agent learns to adaptively reason over pixel features and plan click actions when necessary, achieving superior autonomous interactive segmentation.”

Classified as probably not a substantial claim.

The study applies loss masks to tokens corresponding to segmentation outputs from executed actions as well as to designated erroneous actions within self-correction trajectories.

“Specifically, we apply loss masks to tokens corresponding to segmentation outputs from executed actions as well as to designated erroneous actions within the self-correction trajectories.”

May combine two claims.

IBISAgent reformulates segmentation as a multi-step Markov Decision Process and enhances MLLMs' pixel-level understanding without introducing additional model components.

“IBISAgent reformulates segmentation as a multi-step Markov Decision Process and enhances MLLMs' pixel-level understanding without introducing additional model components.”

May combine two claims.

Current methods that rely on implicit '<SEG>' tokens for segmentation disrupt the MLLM's native text output space, compromising both its language capability and semantic generalization.

“This observation reinforces our motivation: current methods that rely on implicit ' &lt;SEG&gt; ' tokens for segmentation disrupt the MLLM's native text output space, compromising both its language capability and semantic generalization.”

May combine two claims.

These approaches exhibit fundamentally limited pixel-level visual reasoning, preventing the model from truly understanding fine-grained visual features.

“Moreover, these approaches exhibit fundamentally limited pixel-level visual reasoning, preventing the model from truly understanding fine-grained visual features.”

May combine two claims.

IBISAgent's capability arises from a design that decouples reasoning from segmentation, thereby preserving the MLLM's inherent language reasoning ability while simultaneously enabling adaptive, multi-round refinement.

“This capability arises from our design that decouples reasoning from segmentation, thereby preserving the MLLM's inherent language reasoning ability while simultaneously enabling adaptive, multi-round refinement.”

May combine two claims.

IBISAgent is developed through a two-stage training protocol and fine-grained rewards that incentivize continuous self-improvement reasoning.

“We develop IBISAgent through a two-stage training protocol and design fine-grained rewards that incentivize continuous self-improvement reasoning.”

May combine two claims.

IBISAgent's method effectively activates the model's intrinsic pixel-level reasoning ability, enabling advanced segmentation performance that cannot be achieved through simple workflow-style tool calling.

“IBISAgent consistently outperforms all competing agent systems, demonstrating that our method effectively activates the model's intrinsic pixel-level reasoning ability, enabling advanced segmentation performance that cannot be achieved through simple workflow-style tool calling.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Multi-step interactions between the agent and the environment are intended to trade additional computation time for improved performance, an inherent limitation of multi-round agent systems.

“Fundamentally, such multi-step interactions between the agent and the environment are intended to trade additional computation time for improved performance-an inherent limitation of multi-round agent systems.”

May combine two claims.

RL's exploration-exploitation dynamics and reward feedback are crucial for acquiring context-aware decision-making policies and enabling genuine vision-centric multi-step reasoning.

“The RL stage provides the largest performance gain, indicating that RL's exploration-exploitation dynamics and reward feedback are crucial for acquiring context-aware decision-making policies and enabling genuine vision-centric multi-step reasoning.”

The quote only partly states this claim.

Conditioning generation on ground-truth evidence prevents the LLM from hallucinating non-existent anatomical features or pathologies.

“To mitigate this, we strictly conditioned the generation process on ground-truth evidence.”

Probably not a substantial claim: may not pass S4 (based).

The optimal reasoning for the agent's next action requires an 'oracle' or 'ground-truth' perspective, knowing the precise False Positive and False Negative regions.

“the optimal reasoning for its next action (e.g., 'correcting an over-segmentation') requires an 'oracle' or 'ground-truth' perspective (knowing the precise False Positive and False Negative regions).”

Classified as probably not a substantial claim.

The agent (M1) operates from a limited, first-person perspective, seeing only the current segmentation mask.

“our agent (M1) operates from a limited, first-person perspective (seeing only the current segmentation mask)”

Classified as probably not a substantial claim.

The system prompt forces the teacher model to reverse-engineer the logic: instead of simply stating the error, it must explain why visual features suggest an error exists.

“This forces the teacher model to reverse-engineer the logic: instead of simply stating the error (which it knows), it must explain why the visual features (e.g., texture differences, anatomical landmarks) suggest an error exists.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The final-answer reward encompasses multiple task types, providing diverse feedback.

“The final-answer reward S ans encompasses multiple task types, thereby providing the agent with diverse feedback.”

Probably not a substantial claim: may not pass S2 (checkable).

IBISAgent substantially pushes the boundary of MLLM-based biomedical image reasoning and segmentation.

“Together with our novel behavioral formulation and training framework, IBISAgent substantially pushes the boundary of MLLM-based biomedical image reasoning and segmentation.”

Classified as probably not a substantial claim.

The progressive improvement reward incentivizes the agent to strictly ascend the gradient of segmentation quality.

“This incentivizes the agent to strictly ascend the gradient of segmentation quality.”

Probably not a substantial claim: may not pass S4 (based). May combine two claims.

The segmentation reward supplies strong positive feedback only when the predicted region closely matches the ground truth, while at lower IoU levels it provides guidance that encourages gradual improvement.

“This segmentation reward supplies strong positive feedback only when the predicted region closely matches the ground truth, while at lower IoU levels it provides guidance that encourages gradual improvement.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

A primary risk in the pipeline is 'prompt leakage,' where the teacher accidentally reveals its privileged knowledge.

“A primary risk in this pipeline is 'prompt leakage,' where the teacher accidentally reveals its privileged knowledge (e.g., mentioning 'the red mask').”

Classified as probably not a substantial claim.

IBISAgent endows MLLMs with powerful pixel-level visual reasoning capabilities.

“IBISAgent endows MLLMs with powerful pixel-level visual reasoning capabilities.”

May combine two claims.

Under our formulation, the agent performs precise pixel-level visual reasoning to accurately localize target regions and iteratively refine masks, enabling segmentation performance that exceeds the inherent upper bound of the underlying segmentation tools themselves.

“Under our formulation, the agent performs precise pixel-level visual reasoning to accurately localize target regions and iteratively refine masks, enabling segmentation performance that exceeds the inherent upper bound of the underlying segmentation tools themselves.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

The trajectory length reward formulation balances the trade-off between exhaustive refinement and interaction efficiency.

“This formulation balances the trade-off between exhaustive refinement and interaction efficiency.”

• AssumptionRests on an assumption: Efficiency is a critical metric for clinical assistants.

Probably not a substantial claim: may not pass S4 (based).

The click reward effectively guides the policy to minimize the symmetric difference between the predicted and ground-truth masks step-by-step.

“This reward effectively guides the policy to minimize the symmetric difference between the predicted and ground-truth masks step-by-step.”

Probably not a substantial claim: may not pass S4 (based).

The framework enforces semantically valid interactions through S_click.

“we enforce semantically valid interactions through S click.”

• AssumptionRests on an assumption: Generic RL agents may learn to click arbitrarily to trigger tool usage.

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

The current agent operates primarily in 2D settings.

“the current agent operates primarily in 2D settings”