Multimodal fusion and vision–language models: A survey for robot vision
Open accessAbstract
Robot vision has greatly benefited from advancements in multimodal fusion techniques and vision-language models (VLMs). We adopt a task-oriented perspective to systematically review the applications and advancements of multimodal fusion methods and VLMs in the field of robot vision. For semantic scene understanding tasks, we categorize fusion approaches into encoder-decoder frameworks, attention-based architectures, and graph neural networks. Meanwhile, we also analyze the architectural characteristics and practical implementations of these fusion strategies in key tasks such as simultaneous localization and mapping (SLAM), 3D object detection, navigation, and manipulation. We compare the evolutionary paths and applicability of VLMs based on large language models (LLMs) with traditional multimodal fusion methods.Additionally, we conduct an in-depth analysis of commonly used datasets, evaluating their applicability and challenges in real-world robotic scenarios. Building on this analysis, we identify key challenges in current research, including cross-modal alignment, efficient fusion, real-time deployment, and domain adaptation. We propose future directions such as self-supervised learning for robust multimodal representations, structured spatial memory and environment modeling to enhance spatial intelligence, and the integration of adversarial robustness and human feedback mechanisms to enable ethically aligned system deployment. Through a comprehensive review, comparative analysis, and forward-looking discussion, we provide a valuable reference for advancing multimodal perception and interaction in robotic vision. A comprehensive list of studies in this survey is available at https://github.com/Xiaofeng-Han-Res/MF-RV.
AI snapshot
Written from the paper’s full text.
TL;DR
This survey systematically reviews multimodal fusion and vision–language models for robot vision across semantic scene understanding, SLAM, 3D object detection, navigation/localization, and manipulation. It compares traditional fusion pipelines with emerging VLMs, summarizes datasets and benchmarks, and identifies cross-modal alignment, efficient fusion, real-time deployment, and domain adaptation as key challenges.
Key findings
- The survey organizes multimodal fusion into early, mid, and late fusion, and reviews encoder–decoder, attention-based, and GNN-based methods for semantic scene understanding; it also covers LiDAR–camera, radar–camera, and radar–LiDAR fusion for 3D detection.
- On nuScenes 3D object detection, MV2DFusion-e reports mAP 77.9 and NDS 78.8, IS-Fusion reports mAP 76.5 and NDS 77.4, LiDAR-only FocalFormer3D reports mAP 68.7 and NDS 72.6, camera-only SparseBEV reports mAP 60.3 and NDS 67.5, and radar–camera RCBEVDet++ reports mAP 67.3 and NDS 72.7.
- On R2R navigation, BEVBert reports Val Unseen SR 75 and SPL 64, and Test Unseen SR 73 and SPL 62; DUET reports Val Unseen SR 72 and SPL 60.
- On ScanQA, LEO reports CIDEr 101.4, BLEU-4 13.2, METEOR 20.0, and ROUGE 49.2; LLaVA-3D reports CIDEr 91.7, BLEU-4 14.5, METEOR 20.7, and ROUGE 50.1; Qwen2-VL-7B reports CIDEr 53.9, BLEU-4 3.0, METEOR 11.4, and ROUGE 29.3.
Methods
- Design: a systematic survey and comparative analysis of multimodal fusion and VLMs for robot vision, covering semantic scene understanding, SLAM, 3D object detection, navigation and localization, and robot manipulation.
- Data/sample: literature and benchmark datasets including nuScenes, SemanticKITTI, Matterport3D, R2R, ScanQA, DROID, RT-1, and Open X-Embodiment; evaluation metrics include IoU/mIoU, AP/mAP/NDS, ATE/RPE, SR/SPL, and CIDEr/BLEU-4/METEOR/ROUGE.
- Analysis: categorizes fusion strategies and architectures, compares traditional multimodal fusion with LLM-based VLMs, summarizes datasets and benchmarks, and reports performance comparisons from cited works.
Limitations
- The paper states key challenges including low-quality data, heterogeneity, efficient training and inference, and scarcity of high-quality datasets; it also identifies cross-modal alignment, efficient fusion strategies, real-time deployment, and domain adaptation as critical research challenges.
- The paper notes that many robot manipulation benchmarks remain simulation-only, with only COLOSSEUM and GenSim2 supporting real-world reproducibility.
- Evident from the text: as a survey, it does not present new empirical experiments; performance comparisons rely on results reported in original papers or reproduced under aligned experimental settings.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
167 of 357 cited sources checked · 16 supported · 15 flagged · accuracy not yet measured
- Premisecitation
Point-Fusion proposed a dense fusion network that fuses point cloud features extracted by PointNet and image features extracted by ResNet.
“Point-Fusion [78] proposed a dense fusion network that fuses point cloud features extracted by PointNet [105] and image features extracted by ResNet [106].”
Quote in the paperNot a substantial claim1 of 3 sources checked- cites [78] M. Ye, S. Xu, T. Cao, Hvnet: Hybrid voxel network for lidar based 3d object detection, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 1631–1640. source
- cites [105] Kim, 2024, Rcm-fusion: Radar-camera multi-level fusion for 3d object detection source
The Proposed Method section describes this paper’s model as RCM-Fusion, which fuses radar point clouds with camera images. The Radar and Camera Backbones section says it uses PillarNet for radar features and ResNet for image features—not a Point-Fusion network using PointNet for point-cloud features. No page numbers are available in our copy. (LLM, conf=0.88)
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measured - cites [106] K. Qian, S. Zhu, X. Zhang, L.E. Li, Robust multimodal vehicle detection in foggy weather using complementary lidar and radar signals, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021, pp. 444–453. source
- Premisecitation
IS-Fusion captures instance-level and scene-level context using Hierarchical Scene Fusion and Instance-Guided Fusion, integrating multimodal information at different granularities.
“IS-Fusion [90] captures both instance-level and scene-level context using Hierarchical Scene Fusion and Instance-Guided Fusion, effectively integrating multimodal information at different granularities.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [90] Huang, 2020, Epnet: Enhancing point features with image semantics for 3d object detection source
The paper identifies its framework as EPNet and its fusion method as LI-Fusion (contributions, p. 3), not IS-Fusion. Its Two-stream RPN section (p. 6) describes LI-Fusion as point-wise correspondence generation and LiDAR-guided fusion, rather than Hierarchical Scene Fusion and Instance-Guided Fusion. (LLM, conf=0.88)
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measured
- Premisecitation
RD-VIO incorporates radar data to leverage radar robustness under adverse weather for object tracking and localization.
“RD-VIO [6] takes this further by incorporating radar data, leveraging radar's robustness under adverse weather conditions to enhance object tracking and localization.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [6] Li, 2024, RD-VIO: Robust visual-inertial odometry for mobile augmented reality in dynamic environments, IEEE Trans. Vis. Comput. Graphics source
The paper describes RD-VIO as a visual-inertial system: its opening description (p. 1) says it matches landmarks using visual and IMU measurements, and the Introduction (p. 2) describes its handling of moving objects and pure rotation. It does not describe incorporating radar or using radar’s adverse-weather robustness; the claimed sensing method is contrary to the one the paper presents. (LLM, conf=0.90)
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measured
- Premisecitation
LL3DA improves instruction-following accuracy through visual interaction fine-tuning.
“LL3DA [283] enhances instruction-following accuracy through visual interaction fine-tuning.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [283] Qi, 2020, Object-and-action aware model for visual language navigation source
The Introduction (p. 3) identifies the paper’s proposed model as OAAM, not LL3DA. The Training Loss section (p. 8) describes imitation learning, reinforcement learning, and a path loss rather than visual interaction fine-tuning. The cited paper therefore describes a different model and method from those attributed to it. (LLM, conf=0.84)
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measured
- Premisecitation
Contrastive learning and cross-modal attention mechanisms are applied to improve alignment performance.
“contrastive learning and cross-modal attention mechanisms are applied to improve alignment performance [14];”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [14] Xu, 2024, DefFusion: Deformable multimodal representation fusion for 3D semantic segmentation source
The AAF Module and the discussion of semantic fusion describe attention scores used to combine 2D image and 3D point-cloud semantics. They do not describe contrastive learning, and the reported improvement is in 3D object detection rather than alignment performance (pages unavailable in our copy). (LLM, conf=0.86)
Experimental — accuracy not yet measured
- Premisecitation
Traditional approaches that use only RGB images have perceptual limitations in complex real-world tasks, including occlusions, lighting changes, sparse textures, and insufficient semantic information.
“Traditional unimodal approaches (such as relying solely on RGB images) often encounter perceptual limitations when dealing with complex tasks in real-world environments-such as occlusions, lighting variations, sparse textures, and insufficient semantic information [1,2,3].”
Quote in the paperSubstantial claim1 of 3 sources checked- cites [1] Z. Xu, W. Zhang, X. Ye, X. Tan, W. Yang, S. Wen, E. Ding, A. Meng, L. Huang, Zoomnet: Part-aware adaptive zooming neural network for 3d object detection, in: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, 2020, pp. 12557–12564, 07. source
The Introduction (p. 2) says occlusions make image-based localization and stereo disparity estimation difficult, supporting that example of a perceptual limitation. The paper focuses on stereo-image 3D detection, however, rather than establishing limitations of RGB-only approaches generally; the passages do not support lighting variations, sparse textures, or insufficient semantic information as the stated limitations. (LLM, conf=0.87)
Experimental — accuracy not yet measured - cites [2] X. Shi, Q. Ye, X. Chen, C. Chen, Z. Chen, T.-K. Kim, Geometry-based distance decomposition for monocular 3d object detection, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021, pp. 15172–15181. source
- cites [3] L. Wang, L. Du, X. Ye, Y. Fu, G. Guo, X. Xue, J. Feng, L. Zhang, Depth-conditioned dynamic message propagation for monocular 3d object detection, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021, pp. 454–463. source
- Premisecitation
Some recent surveys focus primarily on traditional perception tasks such as semantic segmentation or 3D object detection, lacking a systematic review of deep learning-driven cross-modal modeling.
“Some surveys primarily focus on traditional perception tasks such as semantic segmentation [27] or 3D object detection [28], lacking a systematic review of deep learning-driven cross-modal modeling approaches.”
Quote in the paperSubstantial claim2 of 2 sources checked- cites [27] Zhang, 2021, Deep multimodal fusion for semantic image segmentation: A survey, Image Vis. Comput. source
The Introduction identifies semantic image segmentation as this survey’s focus. But the Conclusion says it categorizes 20 deep multimodal fusion methods and discusses their architectural design, so the claim that it lacks a systematic review of deep-learning-driven cross-modal approaches is too broad. Its review is limited to semantic image segmentation. (LLM, conf=0.86)
Experimental — accuracy not yet measured - cites [28] Alaba, 2024, Emerging trends in autonomous vehicle perception: Multimodal fusion for 3D object detection, World Electr. Veh. J. source
The Abstract and Related Work sections identify this paper as a survey focused on 3D object detection, supporting the description attached to [28]. But the Abstract says it provides an exhaustive review of multimodal fusion methods focused on CNN- and Transformer-based models, and Section 4.2 reviews such methods in detail. The claim that it lacks a systematic review of deep-learning-driven cross-modal approaches is therefore not supported without a narrower qualification about scope. (LLM, conf=0.86)
Experimental — accuracy not yet measured
- Premisecitation
A common attention mechanism similar to transformer structure has been proposed to improve applicability of different modal data and capture local feature correlations.
“A common attention mechanism similar to transformer structure has been proposed to improve the applicability of different modal data and capture local feature correlations [43].”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [43] Xiao, 2019, Multimodal end-to-end autonomous driving, IEEE Trans. Intell. Transp. Syst. source
The Fusion schemes section (p. 4) describes CNN-based early, mid and late fusion of RGB and depth data. Searches for a transformer-like attention mechanism and local feature correlations found no passage addressing those parts of the claim in our copy. (LLM, conf=0.76)
Experimental — accuracy not yet measured
- Premisecitation
Attention-based fusion aligns key features and dynamically assigns weights between modalities for deeper semantic capture.
“Attention-based fusion [49,50,51] aligns key features and dynamically assigns weights between modalities for deeper semantic capture.”
Quote in the paperNot a substantial claim1 of 3 sources checked- cites [49] Y. Wu, E.Y. Chang, K.C.-C. Chang, J.R. Smith, Optimal multimodal fusion for multimedia data analysis, in: Proceedings of the 12th Annual ACM International Conference on Multimedia, 2004, pp. 572–579. source
- cites [50] A. Roitberg, K. Peng, Z. Marinov, C. Seibold, D. Schneider, R. Stiefelhagen, A Comparative Analysis of Decision-Level Fusion for Multimodal Driver Behaviour Understanding, in: 2022 IEEE Intelligent Vehicles Symposium, IV, 2022, pp. 1438–1444. source
- cites [51] Wang, 2024, Efficient neural decoding based on multimodal training, Brain Sci. source
Section 3.2 describes a mid-fusion image–fMRI model with attention heads, and Section 5 reports improved reconstruction of semantic content. The paper does not say that this fusion aligns key features or dynamically assigns weights between modalities; Section 3.3 describes cross-attention as conditioning the diffusion model on fMRI input instead. Page numbers are not available in our copy. (LLM, conf=0.83)
Experimental — accuracy not yet measured
- Premisecitation
Relying on traditional single-modal approaches such as RGB images can encounter limitations in complex scenarios such as occlusion, lighting variations, and multi-object ambiguities.
“Relying on traditional single-modal approaches (e.g., solely using RGB images) can encounter various limitations in complex scenarios, such as occlusion, lighting variations, and multi-object ambiguities [32,33,34].”
Quote in the paperSubstantial claim1 of 3 sources checked- cites [32] Lin, 2023
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites [33] Chen, 2025, A review of research on slam technology based on the fusion of lidar and vision, Sensors source
The Introduction (page unavailable) says low light and moving objects can impair visual SLAM, and §5.1 (page unavailable) identifies occlusion and lighting changes as problems for SLAM sensor data. The paper does not specifically identify multi-object ambiguity as a limitation of RGB-only approaches; its discussion is about visual SLAM and SLAM systems more broadly. (LLM, conf=0.85)
Experimental — accuracy not yet measured - cites [34] Abdulmaksoud, 2025, Transformer-based sensor fusion for autonomous vehicles: A comprehensive review, IEEE Access source
- Premisecitation
Unlike ROI-level fusion of MV3D and AVOD, MVX-Net's VoxelFusion is voxel-level.
“Unlike the ROI-level fusion of MV3D [5] and AVOD [71], the Vox-elFusion fusion strategy proposed in MVX-Net [79] is voxellevel.”
Quote in the paperNot a substantial claim1 of 3 sources checked- cites [5] X. Chen, H. Ma, J. Wan, B. Li, T. Xia, Multi-view 3d object detection network for autonomous driving, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017, pp. 1907–1915. source
- cites [71] Liu, 2023, Muse-gnn: Learning unified gene representation from multimodal biological graph data, Adv. Neural Inf. Process. Syst. source
- cites [79] J. Deng, S. Shi, P. Li, W. Zhou, Y. Zhang, H. Li, Voxel r-cnn: Towards high performance voxel-based 3d object detection, in: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, 2021, pp. 1201–1209, 2. source
The comparison table (4.1 Datasets, p. 5) lists MV3D and AVOD-FPN as RGB+LiDAR detectors, but does not describe their fusion level. The discussion of Voxel RoI pooling (3.1, pp. 3–4) concerns Voxel R-CNN, not MVX-Net; searches for MVX-Net and VoxelFusion found no passage addressing the claimed voxel-level fusion. (LLM, conf=0.78)
Experimental — accuracy not yet measured
- Premisecitation
MGA and MGNNS demonstrate that heterogeneous GNNs and multi-channel architectures can effectively bridge structural and semantic modality gaps by integrating multimodal graph representations.
“In addition, models such as MGA [69] and MGNNS [70] demonstrate that heterogeneous GNNs and multi-channel architectures can effectively bridge structural and semantic modality gaps by integrating multimodal graph representations.”
Quote in the paperNot a substantial claim1 of 2 sources checked- cites [69] Yun, 2024 source
The Introduction and §C, “Hierarchical Multimodal GNN” (page unavailable), describe MISSIONGNN using a multimodal GNN and knowledge graphs to reason over image and semantic information. They do not establish that a model called MGNNS uses a heterogeneous GNN or multi-channel architecture, or that it bridges structural and semantic modality gaps as claimed. (LLM, conf=0.82)
Experimental — accuracy not yet measured - cites [70] Y. Wang, M. Yasunaga, H. Ren, S. Wada, J. Leskovec, Vqa-gnn: Reasoning with multimodal knowledge via graph neural networks for visual question answering, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 21582–21592. source
- Premisecitation
VoxelFusion can fuse at voxel feature encoding layer, aggregating image features into corresponding voxels and concatenating with voxel point cloud features.
“VoxelFusion can fuse at the voxel feature encoding layer, aggregating image features into corresponding voxels, and concatenating them with voxel point cloud features.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [79] J. Deng, S. Shi, P. Li, W. Zhou, Y. Zhang, H. Li, Voxel r-cnn: Towards high performance voxel-based 3d object detection, in: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, 2021, pp. 1201–1209, 2. source
The Introduction (p. 2) describes Voxel R-CNN, and §3.1 (p. 4) describes concatenating voxel features pooled from different stages and scales. Searches for VoxelFusion and image-feature concatenation found no passage about fusing image features into corresponding voxels at the voxel feature encoding layer. (LLM, conf=0.76)
Experimental — accuracy not yet measured
- Interpretationcitation
InstructNav, NaVid, BEVBert, and CVDN demonstrate transformative potential of multimodal fusion in embodied navigation.
“These methods, such as InstructNav [9], NaVid [114], BEVBert [116], and CVDN [124], demonstrate the transformative potential of multimodal, fusion in embodied navigation.”
Quote in the paperNot a substantial claim2 of 3 sources checked- cites [9] Long, 2024 source
- cites [114] H. Huang, V. Jain, H. Mehta, A. Ku, G. Magalhaes, J. Baldridge, E. Ie, Transferable representation learning in vision-and-language navigation, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, 2019, pp. 7404–7413. source
- cites [116] Yan, 2024 source
The Vision-and-Language Navigation section (p. 2) describes multimodal alignment as a challenge and says navigation approaches aim to improve agents’ comprehension of multimodal information. The References (p. 7) include a vision-and-dialog navigation work, but the paper does not say that CVDN—or the four named methods collectively—demonstrates transformative potential through multimodal fusion. (LLM, conf=0.76)
Experimental — accuracy not yet measured
- Premisecitation
DeepInteraction addresses limitation of unilateral fusion by introducing multimodal interaction encoder and decoder.
“DeepInteraction [86] addresses the limitation of unilateral fusion by introducing a multimodal interaction encoder and decoder.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [86] S. Vora, A.H. Lang, B. Helou, O. Beijbom, Pointpainting: Sequential fusion for 3d object detection, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 4604–4612. source
The Introduction and Contributions describe PointPainting, a sequential fusion method that augments lidar points with image semantics. Our copy does not discuss DeepInteraction or a multimodal interaction encoder and decoder; searches for those terms found no supporting passage. (LLM, conf=0.76)
Experimental — accuracy not yet measured
- Premisecitation
InstructNav performed outstandingly in R2R-CE and Habitat ObjectNav tasks.
“It has demonstrated outstanding performance in R2R-CE and Habitat ObjectNav tasks.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [9] Long, 2024 source
- Premisecitation
InstructNav uses DCoN and Multi-sourced Value Maps for zero-shot cross-task planning, generating executable trajectories from language instructions.
“InstructNav [9] employs a Dynamic Chain-of-Navigation (DCoN) framework, combined with Multi-sourced Value Maps, to achieve efficient zeroshot cross-task planning, enabling robots to directly generate executable trajectories from language instructions.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [9] Long, 2024 source
- Premisecitation
Goal-oriented navigation capability has been validated in Habitat's ObjectNav task, showing effective goal localisation in unknown environments.
“This capability has been extensively validated in Habitat's ObjectNav task [108,109,17,110], demonstrating the effectiveness of achieving goal localisation in unknown environments.”
Quote in the paperNot a substantial claim1 of 4 sources checked- cites [108] C.R. Qi, H. Su, K. Mo, L.J. Guibas, Pointnet: Deep learning on point sets for 3d classification and segmentation, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017, pp. 652–660.
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites [109] K. He, X. Zhang, S. Ren, J. Sun, Deep residual learning for image recognition, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 770–778. source
- cites [17] Shah, 2021, Ving: Learning open-world navigation with visual goals source
Section IV (p. 3) describes localizing the robot and its visual goal in a graph, and “Generalization and Adaptation” (pp. 5–6) reports successful goal-reaching after adaptation in previously unseen outdoor environments. But the evaluation is on a real robot, not Habitat’s ObjectNav task; the paper therefore does not support the claim of validation in that task. (LLM, conf=0.90)
Experimental — accuracy not yet measured - cites [110] Y. Chae, H. Kim, K.-J. Yoon, Towards Robust 3D Object Detection with LiDAR and 4D Radar Fusion in Various Weather Conditions, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 15162–15172. source
- Premisecitation
BEVFusion aligns image and point cloud features in BEV space using convolutional encoder.
“BEVFusion [88] aligns image and point cloud features in BEV space using a convolutional encoder.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [88] Yoo, 2020, 3D-cvf: Generating joint camera and lidar features using cross-view spatial feature fusion for 3d object detection source
Section 3.1 (p. 6) says the paper projects camera features into BEV and then combines them with LiDAR features, supporting the broad alignment-and-fusion description. But the paper names its method 3D-CVF, not BEVFusion, and §3.2 (p. 6) describes auto-calibrated projection and interpolation for the mapping, not a convolutional encoder. The additional convolutional layers in §3.1 enhance the projected feature map. (LLM, conf=0.88)
Experimental — accuracy not yet measured
- Interpretationcitation
Integrating visual input, language processing, and spatial reasoning lets robots follow natural language instructions, navigate to visual goals, or perform goal-directed tasks with greater flexibility and adaptability.
“By integrating visual input, language processing, and spatial reasoning, robots can now follow natural language instructions, navigate toward visual goals, or perform goal-directed navigation tasks with greater flexibility and adaptability”
Quote in the paperSubstantial claim3 of 4 sources checked- cites [114] H. Huang, V. Jain, H. Mehta, A. Ku, G. Magalhaes, J. Baldridge, E. Ie, Transferable representation learning in vision-and-language navigation, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, 2019, pp. 7404–7413. source
- cites [126] Nguyen, 2019 source
The task description in §2 describes a simulated navigation agent using camera images, natural-language instructions, and destination images to reach a requested object. The results report that language-based assistance improves task completion in novel environments. The paper does not establish the broader claim about physical robots or identify spatial reasoning as a distinct capability responsible for greater flexibility and adaptability; page numbers are unavailable in our copy. (LLM, conf=0.84)
Experimental — accuracy not yet measured - cites [127] Thomason, 2020, Vision-and-dialog navigation source
The Introduction (p. 1) and Conclusions (p. 8) describe agents using visual context and dialog to navigate toward a goal in simulated environments; the Experiments (p. 8) report better navigation performance when agents use more dialog history. The paper does not establish that physical robots can now perform the full range of claimed tasks, or demonstrate greater flexibility and adaptability beyond the specific navigation improvements tested. (LLM, conf=0.86)
Experimental — accuracy not yet measured - cites [128] Gao, 2022, Dialfred: Dialogue-enabled agents for embodied instruction following, IEEE Robot. Autom. Lett. source
The Baselines (p. 6) describe an agent using language instructions and visual observations to predict actions, and the Related Work (p. 2) identifies tasks requiring navigation and manipulation. The Results (p. 7) report improved task performance when question-answer dialogue is added. The paper does not establish the broader claim that integrating spatial reasoning enables navigation toward visual goals or makes robots generally more flexible and adaptable. (LLM, conf=0.84)
Experimental — accuracy not yet measured
- Premisecitation
Recent deep-learning multimodal fusion has significantly enhanced localization accuracy and robustness.
“vancements in multimodal fusion, particularly deep-learningbased approaches, have significantly enhanced localization accuracy and robustness.”
Quote in the paperNot a substantial claim1 of 3 sources checked- cites [129] Liang, 2025 source
Section 2.1 (p. 2) discusses advances in vision-language models, and Section 2.3 (p. 2) describes multimodal preference optimization for task planning. Neither addresses localization accuracy or robustness; searches for “localization” and “localization accuracy robustness multimodal fusion” found no supporting passage in our copy. (LLM, conf=0.76)
Experimental — accuracy not yet measured - cites [130] Lin, 2023
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites [131] Wang, 2024, Data-efficient learning-based iterative optimization method with time-varying prediction horizon for multi-agent collaboration, IEEE Internet Things J. source
- Premisecitation
These localization methods use RGB, depth, and inertial measurements to address sensor noise, dynamic scenes, and varying lighting.
“These methods leverage complementary sensory inputs such as RGB images, depth data, and inertial measurements to address challenges like sensor noise, dynamic scenes, and varying lighting conditions [129,130,131].”
Quote in the paperNot a substantial claim1 of 3 sources checked- cites [129] Liang, 2025 source
Section B.8 (p. 16) describes a task-planning agent that receives image observations and textual instructions, not a localization method combining RGB, depth, and inertial measurements. Searches for those inputs and for sensor noise, dynamic scenes, and varying lighting found no passage in our copy addressing the attributed claim. (LLM, conf=0.76)
Experimental — accuracy not yet measured - cites [130] Lin, 2023
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites [131] Wang, 2024, Data-efficient learning-based iterative optimization method with time-varying prediction horizon for multi-agent collaboration, IEEE Internet Things J. source
- Premisecitation
DeepVO and D3VO integrate convolutional and recurrent neural networks for end-to-end ego-motion and depth estimation.
“Model,s like DeepVO [20] and D3VO [132] exemplify the integration of convolutional and recurrent neural networks, enabling end-to-end estimation of ego-motion and depth.”
Quote in the paperNot a substantial claim1 of 2 sources checked- cites [20] Mohanty, 2016 source
The Introduction (p. 2) says DeepVO uses CNN features to estimate the transformation between successive scenes and reconstruct the vehicle’s trajectory, supporting the ego-motion portion. But Future Work (p. 7) proposes recurrent connections as a possible improvement rather than part of the model, and the paper does not report depth estimation by DeepVO. This verdict concerns the claim attributed to DeepVO, not D3VO. (LLM, conf=0.91)
Experimental — accuracy not yet measured - cites [132] Lin, 2025
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Premisecitation
V-LOAM integrated visual odometry with LiDAR-based mapping, leveraging complementary geometric and texture features.
“Early multimodal fusion approaches, such as V-LOAM [143], integrated visual odometry with LiDAR-based mapping, effectively leveraging the complementary nature of geometric and texture-based features.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [143] C. Wang, S. Chen, Y. Song, R. Xu, Z. Zhang, J. Zhang, H. Yang, Y. Zhang, K. Fu, S. Du, et al., Focus on Local: Finding Reliable Discriminative Regions for Visual Place Recognition, in: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 39, 2025, pp. 7536–7544, 7. source
The Related Works and Two-Stage VPR sections discuss image-based visual place recognition, not V-LOAM. Searches for V-LOAM, LiDAR-based mapping, and visual odometry found no passage in our copy addressing the attributed integration or its geometric and texture features. The returned passages have no page numbers. (LLM, conf=0.76)
Experimental — accuracy not yet measured
- Premisecitation
Scale differences and noise in large-scale unlabeled data can affect alignment accuracy.
“Other challenges include scale differences and noise in large-scale unlabeled data, which can affect alignment accuracy [178].”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [178] Zhang, 2023, Bimodal fusion network with multi-head attention for multimodal sentiment analysis, Appl. Sci. source
The Introduction discusses misalignment between text and audio, and the Results discuss aligning their representations, but neither links alignment accuracy to scale differences or noise in large-scale unlabeled data. The Dataset section describes manually annotated data. Searches for scale differences, noise, and unlabeled data found no passage addressing the claimed causes. (LLM, conf=0.74)
Experimental — accuracy not yet measured
- Premisecitation
Cross-modal pre-training typically creates a shared embedding space where data from various modalities are projected, allowing the model to learn relationships.
“This is typically achieved by creating a shared embedding space where data from various modalities (such as images, text, audio, video, etc.) are projected into the same space, allowing the model to learn the relationships between these modalities [169,170].”
Quote in the paperNot a substantial claim1 of 2 sources checked- cites [169] Guzey, 2024, See to touch: Learning tactile dexterity through visual incentives source
Section IV-B describes an image encoder that maps visual observations to a lower-dimensional latent state. It does not describe projecting multiple modalities into a shared embedding space or establish that this is how cross-modal pre-training typically works; its contrastive loss compares nearby visual observations. (LLM, conf=0.79)
Experimental — accuracy not yet measured - cites [170] Dou, 2022, Coarse-to-fine vision-language pre-training with fusion in the backbone, Adv. Neural Inf. Process. Syst. source
- Premisecitation
EfficientNet achieves state-of-the-art performance with fewer parameters and operations than traditional networks by scaling CNN depth, width, and resolution.
“By proposing a new method for scaling the depth, width, and resolution of CNNs, EfficientNet achieves state-of-the-art performance with fewer parameters and operations compared to traditional networks.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [230] Lan, 2019 source
The Introduction (p. 2) describes ALBERT’s parameter-reduction techniques for language models, while Appendix A.1 (p. 15) examines ALBERT’s depth and width. Searches for EfficientNet and CNN scaling found no passage addressing the claimed depth-width-resolution method or its performance and efficiency. (LLM, conf=0.78)
Experimental — accuracy not yet measured
- Premisereasoning
LLaVA's flexible task-switching makes it advantageous in complex environments, especially for cross-modal and visual reasoning.
“This flexible task-switching capability makes LLaVA [211] particularly advantageous in complex environments, excelling in tasks such as cross-modal and visual reasoning.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [211] Wang, 2020 source
The Model section (p. 4) describes Linformer's linear self-attention mechanism, and the Introduction (p. 2) discusses language-model benchmarks. Searches for LLaVA, task-switching, and cross-modal or visual reasoning found no passage in our copy addressing the cited claim. (LLM, conf=0.78)
Experimental — accuracy not yet measured
- Premisecitation
Llama's strong reasoning abilities, especially in commonsense reasoning and text generation, make it effective for complex cross-modal tasks.
“Its strong reasoning abilities, especially in commonsense reasoning and text generation, make it effective for complex cross-modal tasks.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [209] Kitaev, 2020 source
The Introduction (p. 2) describes Reformer experiments on text and image-generation tasks, and the Conclusion (p. 10) discusses its potential for generative tasks. Searches for Llama, commonsense reasoning, and cross-modal abilities found no passage addressing the claim about Llama. (LLM, conf=0.78)
Experimental — accuracy not yet measured
- Premisecitation
HAMT uses history-aware transformers for long-horizon reasoning.
“HAMT [120] leverages history-aware transformers for long- horizon reasoning”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [120] Anderson, 2019, Chasing ghosts: Instruction following as bayesian state tracking, Adv. Neural Inf. Process. Syst. source
The Introduction (p. 2) describes this paper’s Bayesian state-tracking model, and §4.2 (p. 5) describes an LSTM encoder and decoder. Our copy does not discuss HAMT or whether it uses history-aware transformers for long-horizon reasoning (searched: HAMT, history-aware transformers, long-horizon reasoning). (LLM, conf=0.74)
Experimental — accuracy not yet measured
- Premisecitation
LEO, as an embodied intelligence model, exhibits superior adaptability in long-term task planning and interactive tasks.
“LEO [284], as an embodied intelligence model, exhibits superior adaptability in long-term task planning and interactive tasks.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [284] Zhu, 2020, Vision-language navigation with self-supervised auxiliary reasoning tasks source
The Introduction and Conclusion describe AuxRN and its performance on vision-language navigation, not LEO. Searches for LEO and for long-term planning and interactive tasks found no passage evaluating the claimed adaptability. (LLM, conf=0.76)
Experimental — accuracy not yet measured
- Premisecitation
Unlike VLMbench and GemBench, VIMA integrates multi-modal task execution and precise instruction following.
“Unlike VLMbench [290] and Gem-Bench [289], which primarily evaluate vision-language models (VLMs) in robotic control, VIMA [291] further integrates multi-modal task execution and precise instruction following, making it distinct in its benchmarking approach.”
Quote in the paperNot a substantial claim1 of 3 sources checked- cites [290] Y. Hong, Q. Wu, Y. Qi, C. Rodriguez-Opazo, S. Gould, Vln bert: A recurrent vision-and-language bert for navigation, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021, pp. 1643–1653. source
- cites [289] P.-L. Guhur, M. Tapaswi, S. Chen, I. Laptev, C. Schmid, Airbert: In-domain pretraining for vision-and-language navigation, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021, pp. 1634–1643. source
The Introduction describes Airbert as a model for vision-and-language navigation, and Figure 3 describes its path-instruction pretraining. Searches for VIMA, VLMbench, Gem-Bench, and the claimed benchmarking distinction found no passage in our copy addressing them. (LLM, conf=0.76)
Experimental — accuracy not yet measured - cites [291] C. Liu, F. Zhu, X. Chang, X. Liang, Z. Ge, Y.-D. Shen, Vision-language navigation with random environmental mixup, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021, pp. 1644–1654. source
- Premisecitation
Existing denoising methods mainly follow modality-specific denoising and cross-modality denoising.
“Existing denoising methods mainly have two directions: modality-specific denoising [297,298] and cross-modality denoising [299,300,301].”
Quote in the paperNot a substantial claim2 of 5 sources checked- cites [297] Huang, 2025
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites [298] W. Mo, Y. Liu, Bridging the gap between 2d and 3d visual question answering: A fusion approach for 3d vqa, in: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 38, 2024, pp. 4261–4268, 5. source
The 2D-3D VQA Model section describes fusion and cross-attention between visual modalities, not denoising. Searches for “modality-specific denoising,” “cross-modality denoising,” and noise-related terms found no passage in our copy classifying existing denoising methods into those two directions. The available passages have no page numbers. (LLM, conf=0.77)
Experimental — accuracy not yet measured - cites [299] Fu, 2024 source
Section 5.2 discusses comparisons among input modalities, and Section 6 reports their performance on 3D question-answering benchmarks. Neither addresses a division of denoising methods into modality-specific and cross-modality approaches; searches for those terms and for noise removal found no relevant passage. (LLM, conf=0.76)
Experimental — accuracy not yet measured - cites [300] Huang, 2023
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites [301] Z. Jin, M. Hayat, Y. Yang, Y. Guo, Y. Lei, Context-aware alignment and mutual masking for 3d-language pre-training, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 10984–10994. source
- Premisecitation
Lightweight visual language fusion architectures such as CrossViT, MobileViT, and MoCoViT reduce computational complexity.
“On the one hand, the computational complexity is effectively reduced by introducing lightweight visual language fusion architectures, such as CrossViT [308], MobileViT [231],MoCoViT [309]and other multi-scale Transformer architectures;”
Quote in the paperSubstantial claim1 of 3 sources checked- cites [308] Zheng, 2022, Vlmbench: A compositional benchmark for vision-and-language manipulation, Adv. Neural Inf. Process. Syst. source
The Vision-and-Language Manipulation Agent section (p. 8) describes feature fusion in 6D-CLIPort, but does not say it reduces computational complexity. Searches for CrossViT, MobileViT, MoCoViT, lightweight architectures, and computational complexity found no passage addressing the attributed claim. (LLM, conf=0.76)
Experimental — accuracy not yet measured - cites [231] A. Akbik, D. Blythe, R. Vollgraf, Contextual string embeddings for sequence labeling, in: Proceedings of the 27th International Conference on Computational Linguistics, 2018, pp. 1638–1649.
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites [309] E. Xing, A. Gupta, S. Powers, V. Dean, Kitchenshift: Evaluating zero-shot generalization of imitation-based policy learning under domain shifts, in: NeurIPS 2021 Workshop on Distribution Shifts: Connecting Methods and Applications, 2021.
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Premisecitation
CLIP and ALIGN use large-scale weakly annotated or natural language supervised learning to learn generalised visual language representations.
“models such as CLIP [181], ALIGN [182], etc., which use large-scale weakly annotated or natural language supervised learning of generalised visual language representations”
Quote in the paperNot a substantial claim1 of 2 sources checked- cites [181] Berg, 2024, Multimodal supervised contrastive learning in remote sensing downstream tasks, IEEE Geosci. Remote. Sens. Lett. source
- cites [182] Li, 2024, Joint self-supervised and supervised contrastive learning for multimodal MRI data: Towards predicting abnormal neurodevelopment, Artif. Intell. Med. source
The Introduction describes CLIP-based methods as learning a shared image–text representation through contrastive learning, and reference 24 identifies natural-language supervision. Our copy does not discuss ALIGN or establish the claim about large-scale weakly annotated training. Page numbers are unavailable in the indexed passages. (LLM, conf=0.85)
Experimental — accuracy not yet measured
- Findingreasoning
Multimodal fusion methods combining LiDAR, camera, and radar significantly improve detection accuracy, robustness, and environmental adaptability compared with single-modality methods.
“In contrast, multimodal fusion methods effectively combine LiDAR, camera, and radar, significantly improving detection accuracy, robustness, and environmental adaptability.”
Quote in the paperSubstantial claim - Premisereasoning
Most recent surveys do not cover key topics such as SLAM, embodied navigation, multimodal 3D detection, or visual localization.
“Most of them do not cover key topics such as SLAM, embodied navigation, multimodal 3D detection, or visual localization.”
Quote in the paperSubstantial claim - Premisereasoning
Several recent surveys on multimodal fusion and VLMs have significant limitations.
“Although several recent surveys have addressed multimodal fusion and VLMs, there remain significant limitations.”
Quote in the paperSubstantial claim - Premisecitation
Graph neural networks are used to enhance modeling of relational structures within scenes.
“and graph neural networks are used to enhance the modeling of relational structures within scenes [15].”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [15] A. Aflalo, S. Bagon, T. Kashti, Y. Eldar, DeepCut: Unsupervised Segmentation Using Graph Neural Networks Clustering, in: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops, 2023, pp. 32–41. source
- Interpretationreasoning
The rise of VLMs marks a shift in robotic vision systems from passive perception to proactive intelligent systems capable of semantic understanding and natural language interaction.
“This evolution marks a shift in robotic vision systems from passive perception to proactive intelligent systems capable of semantic understanding and natural language interaction.”
Quote in the paperSubstantial claim - Premisecitation
CONet within the Occ-Net framework combines vision with LiDAR and Radar to overcome environmental challenges.
“For example, the CONet approach within the Occ-Net framework [62] combines vision with LiDAR and Radar to overcome environmental challenges.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [62] Daneshfar, 2025, Elastic deep multi-view autoencoder with diversity embedding, Inform. Sci. source
- Premisecitation
CLFT fuses camera and LiDAR data by projecting LiDAR point clouds onto the camera plane, upsampling them into 2D feature maps, and inputting them with camera images into a Transformer-based network.
“The CLFT [61] model achieves the fusion of camera and LiDAR data by projecting Li-DAR point clouds onto the camera plane, upsampling them into 2D feature maps, and then inputting them together with camera images into a Transformer-based network.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [61] B. Cheng, I. Misra, A.G. Schwing, A. Kirillov, R. Girdhar, Masked-Attention Mask Transformer for Universal Image Segmentation, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR, 2022, pp. 1290–1299. source
- Premisereasoning
Integrating additional sensors like LiDAR and Radar improves perception in complex environments.
“To address this, researchers integrate additional sensors like Li-DAR and Radar to improve perception in complex environments.”
Quote in the paperSubstantial claim - Premisecitation
Mid-term fusion typically employs attention mechanisms, feature weighting, connectivity, or GNNs to combine modality-specific features.
“This method typically employs attention mechanisms, feature weighting, connectivity, or GNNs to combine modality specific features [41,42].”
Quote in the paperNot a substantial claim0 of 2 sources checked - Premisecitation
Encoder-decoder methods efficiently represent scene semantics through encoding, interaction, and decoding.
“The encoder-decoder method [48] efficiently represents scene semantics through encoding, interaction, and decoding.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [48] Atrey, 2010, Multimodal fusion for multimedia analysis: a survey, Multimedia Syst. source
- Premisecitation
VQA-GNN introduces a shared context node to bridge visual and conceptual subgraphs, facilitating bidirectional message passing to mitigate modality-level representational gaps.
“VQA-GNN [66] exemplifies this approach by introducing a shared context node to bridge visual and conceptual subgraphs, facilitating bidirectional message passing to mitigate modality-level representational gaps.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [66] X. Wang, Z. Zhu, W. Xu, Y. Zhang, Y. Wei, X. Chi, Y. Ye, D. Du, J. Lu, X. Wang, Openoccupancy: A large scale benchmark for surrounding semantic occupancy perception, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 17850–17859. source
- Premisenone given
MV3D has lower detection accuracy for small objects.
“MV3D [5] has lower detection accuracy for small objects.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [5] X. Chen, H. Ma, J. Wan, B. Li, T. Xia, Multi-view 3d object detection network for autonomous driving, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017, pp. 1907–1915. source
- Premisecitation
TransFusion introduces a soft association fusion framework using a Transformer decoder for high-precision 3D detection.
“TransFusion [13] introduces a soft association fusion framework using a Trans-”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [13] X. Bai, Z. Hu, X. Zhu, Q. Huang, Y. Chen, H. Fu, C.-L. Tai, Transfusion: Robust lidar-camera fusion for 3d object detection with transformers, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 1090–1099. source
- Premisenone given
Early LiDAR and camera feature fusion methods were non-attention-based, such as element-wise mean fusion, weighted fusion, MLP.
“Early LiDAR and camera feature fusion methods were nonattention-based, such as element-wise mean fusion operations, weighted fusion, MLP, etc.”
Quote in the paperNot a substantial claim - Premisecitation
Early fusion methods such as MV3D and AVOD are generally attention-agnostic, relying on direct concatenation or feature pooling.
“Early fusion methods, such as those used in MV3D [5] and AVOD [71], are generally attention-agnostic, relying on direct concatenation or feature pooling to combine multimodal data.”
Quote in the paperNot a substantial claim0 of 2 sources checked- cites [5] X. Chen, H. Ma, J. Wan, B. Li, T. Xia, Multi-view 3d object detection network for autonomous driving, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017, pp. 1907–1915. source
- cites [71] Liu, 2023, Muse-gnn: Learning unified gene representation from multimodal biological graph data, Adv. Neural Inf. Process. Syst. source
- Premisecitation
MV3D utilizes multi-view (BV, FV, RGB) feature fusion of LiDAR and cameras.
“MV3D [5] is a representative work that utilizes multi-view (BV, FV, RGB) feature fusion of Li-DAR and cameras.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [5] X. Chen, H. Ma, J. Wan, B. Li, T. Xia, Multi-view 3d object detection network for autonomous driving, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017, pp. 1907–1915. source
- Premisecitation
NaVid uses a video-driven VLM to navigate from RGB video and human instructions.
“Similarly, NaVid [114] leverages a videodriven Vision-Language Model (VLM), allowing robots to navigate using RGB video and human-issued instructions.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [114] H. Huang, V. Jain, H. Mehta, A. Ku, G. Magalhaes, J. Baldridge, E. Ie, Transferable representation learning in vision-and-language navigation, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, 2019, pp. 7404–7413. source
- Interpretationreasoning
Dialogue-based navigation lets agents interact with an oracle or human to acquire information, clarify ambiguous instructions, or correct errors, enhancing adaptability and generalization.
“Dialogue-based navigation enables intelligent agents to actively interact with an oracle or a human during navigation to acquire additional environmental information, clarify ambiguous instructions, or correct navigation errors, thereby enhancing adaptability and generalization.”
Quote in the paperNot a substantial claim - Interpretationreasoning
In LiDAR-Camera fusion, there has been evolution from early simple feature-level fusion to complex attention mechanisms and Transformer architectures.
“As shown in Figure 6, in the field of LiDAR-Camera fusion, we have witnessed the evolution from early simple feature level fusion to today's complex attention mechanisms and Transformer architectures.”
Quote in the paperNot a substantial claim - Interpretationreasoning
Topological maps play a crucial role in enhancing robotic navigation.
“The construction of topological maps plays a crucial role in enhancing robotic navigation capabilities.”
Quote in the paperNot a substantial claim - Premisereasoning
Navigation agents can dynamically update environmental models by integrating current observations with existing topological information to expand navigable regions.
“As shown in Figure 8, navigation agents can dynamically update environmental models, integrating current observations with existing topological information to progressively expand navigable regions.”
Quote in the paperNot a substantial claim - Interpretationreasoning
Single-modal SLAM approaches are inherently limited by their sensors.
“While these single-modal approaches have achieved significant progress, they are inherently limited by the constraints of their respective sensors.”
Quote in the paperSubstantial claim - Interpretationreasoning
Combining these modalities has significantly improved SLAM robustness, accuracy, and environmental adaptability.
“By combining these modalities, SLAM systems have significantly improved robustness, accuracy, and environmental adaptability.”
Quote in the paperSubstantial claim - Interpretationnone given
Relying only on vision limits robot task understanding.
“However, relying only on vision limits the robot's task understanding.”
Quote in the paperSubstantial claim - Interpretationreasoning
Hybrid frameworks combining classical geometric models with deep neural networks address scale ambiguity and drift.
“Hybrid frameworks combining classical geometric models with deep neural networks offer a promising solution to challenges like scale ambiguity and drift.”
Quote in the paperNot a substantial claim - Premisecitation
OpenVLA uses LoRA for parameter-efficient fusion.
“Efficient alignment of multimodal features is critical, with OpenVLA [19] using Low-Rank Adaptation (LoRA) for parameter-efficient fusion, while DeeR-VLA [155] optimizes reasoning with a dynamic early-exit mechanism to reduce computational overhead.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [19] Kim, 2024
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Interpretationnone given
Fusion of vision, language, and spatial cues enables more robust robot navigation in diverse dynamic environments.
“The fusion of different sensory modalities such as vision, language, and spatial cues enables robots to navigate more robustly in diverse and dynamic environments, pushing the boundaries of autonomous navigation systems.”
Quote in the paperSubstantial claim - Premisecitation
D3VO incorporates uncertainty estimation into visual odometry for robust performance in dynamic and occluded environments.
“For example, D3VO incorporates uncertainty estimation into the visual odometry pipeline for robust performance in dynamic and occluded environments.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [132] Lin, 2025
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Premisecitation
LIMO incorporated IMU data and sparse visual features to enhance stability in dynamic environments.
“LIMO [144] further incorporated IMU data, utilizing sparse visual features to enhance system stability in dynamic environments.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [144] Zhang, 2014, LOAM : Lidar odometry and mapping in real-time, Robot.: Sci. Syst. Conf. (RSS) source
- Premisecitation
ConceptFusion introduces open-set multimodal 3D mapping combining vision, language, and audio for multimodal querying and semantic understanding.
“Meanwhile, ConceptFusion [148] introduces open-set multimodal 3D mapping, combining vision, language, and audio data to enable multimodal querying of the environment, thereby enhancing semantic understanding.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [148] Zhang, 2015, Visual-lidar odometry and mapping: Low-drift, robust, and fast source
- Premisecitation
DeeR-VLA optimizes reasoning with dynamic early-exit mechanism to reduce computational overhead.
“Efficient alignment of multimodal features is critical, with OpenVLA [19] using Low-Rank Adaptation (LoRA) for parameter-efficient fusion, while DeeR-VLA [155] optimizes reasoning with a dynamic early-exit mechanism to reduce computational overhead.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [155] H. Wang, J. Wang, L. Agapito, Co-slam: Joint coordinate and sparse parametric encodings for neural real-time slam, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 13293–13302. source
- Premisecitation
MM3DGS SLAM integrates vision, depth, and IMU with 3D Gaussian mapping to improve real-time mapping quality and pose estimation accuracy.
“Additionally, MM3DGS SLAM [147] integrates vision, depth, and IMU data, utilizing a 3D Gaussian mapping approach to improve real-time mapping quality and pose estimation accuracy.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [147] Engel, 2014, LSD-SLAM: Large-scale direct monocular SLAM source
- Interpretationreasoning
These studies highlight the potential of combining vision and touch, from improving grasp path planning precision to dynamically adjusting grasping forces.
“From improving the precision of grasp path planning to dynamically adjusting grasping forces, these studies highlight the immense potential of combining vision and touch.”
Quote in the paperSubstantial claim - Premisecitation
Pre-training plays a crucial role in multimodal fusion.
“Pre-training plays a crucial role in multimodal fusion.”
Quote in the paperSubstantial claim - Premisecitation
RotateIt integrates visual, tactile, and proprioceptive data to optimize force closure in multi-axis rotation tasks.
“RotateIt [158] integrates visual, tactile, and proprioceptive data to optimize force closure in multi-axis rotation tasks.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [158] Zhang, 2025
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Interpretationreasoning
VLA-based multimodal fusion leverages visual, language, and action modalities to provide robust perception and decision-making for manipulation.
“In summary, multimodal feature fusion methods based on VLA models leverage the complementary strengths of visual, language, and action modalities to provide robots with robust perception and decision-making capabilities for manipulation tasks.”
Quote in the paperSubstantial claim - Premisecitation
Octopi uses a GelSight sensor to collect surface data, aligning tactile and visual info with CLIP to enhance grasping strategies by learning object properties.
“Octopi [162] uses a GelSight sensor to collect surface data, aligning tactile and visual information with the CLIP model to enhance grasping strategies by learning object properties.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [162] Huang, 2023
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Premisereasoning
Challenges arise from differences in structure, representation, and data features between modalities.
“However, challenges arise from differences in structure, representation, and data features between modalities.”
Quote in the paperSubstantial claim - Premisereasoning
Vision-touch multimodal fusion is crucial for precision and stability in robotic grasping.
“Vision-touch multimodal fusion is crucial for precision and stability in robotic grasping tasks.”
Quote in the paperSubstantial claim - Premisecitation
TAVI combines visual signals for spatial reasoning and tactile feedback for multi-finger tasks like flipping objects, enabling more precise robot operations.
“TAVI [163] combines visual signals for spatial reasoning and tactile feedback for multi-finger tasks like flipping objects, enabling more precise robot operations.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [163] Li, 2023, VITO-transformer: a visual-tactile fusion network for object recognition, IEEE Trans. Instrum. Meas. source
- Premisecitation
Cross-modal pre-training is crucial for VLMs because it integrates and understands information from different modalities.
“Cross-modal pre-training is crucial for applications such as vision-language models because it effectively integrates and understands information from different modalities [? ].”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [? ]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Premisecitation
ViT demonstrated Transformer architectures could be directly applied to image classification, achieving performance surpassing CNNs on large-scale datasets.
“Vision Transformer (ViT) [198] demonstrated that Transformer architectures could be directly applied to image classification tasks, achieving performance surpassing CNNs on large-scale datasets.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [198] Vaswani, 2017, Attention is all you need, Adv. Neural Inf. Process. Syst.
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Interpretationreasoning
Following success in NLP, Transformer was rapidly adopted in computer vision, challenging CNN dominance in image representation learning.
“Following its success in NLP, Transformer was rapidly adopted in computer vision, challenging the dominance of convolutional neural networks (CNNs) in image representation learning.”
Quote in the paperNot a substantial claim - Interpretationreasoning
These studies demonstrated Transformer could serve as a general-purpose deep learning architecture applicable to both text and visual modalities.
“These studies demonstrated that Transformer could serve as a general-purpose deep learning architecture applicable to both text and visual modalities.”
Quote in the paperNot a substantial claim - Premisecitation
Flamingo integrates few-shot learning for rapid adaptation with minimal examples.
“while Flamingo [203] integrates few-shot learning for rapid adaptation with minimal examples.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [203] Zhu, 2023
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Premisenone given
In visual reasoning, instruction tuning strengthens complex logical reasoning capabilities, including causal, spatial, and attribute reasoning.
“In visual reasoning, instruction tuning strengthens complex logical reasoning capabilities, including causal, spatial, and attribute reasoning.”
Quote in the paperNot a substantial claim - Premisecitation
PaLI-X uses large-scale multimodal instruction training to improve vision-language task generalization.
“PaLI-X [216] uses large-scale multimodal instruction training to improve vision-language task generalization, while Flamingo [203] integrates few-shot learning for rapid adaptation with minimal examples.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [216] Du, 2022, Glam: Efficient scaling of language models with mixture-of-experts
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Premisecitation
AutoAlignV2 enhances cross-modal feature fusion through deformable feature alignment, making it more effective in small-object detection such as pedestrians and bicycles.
“AutoAlignV2 [259] enhances cross-modal feature fusion through deformable feature alignment, making it more effective in small-object detection (e.g., pedestrians and bicycles).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [259] Zhao, 2022, A real 3d embodied dataset for robotic active visual learning, IEEE Robot. Autom. Lett. source
- Premisenone given
LiDAR-camera fusion methods are susceptible to low-light conditions and adverse weather, particularly when LiDAR sampling density decreases, leading to potential information loss.
“However, LiDAR-camera fusion methods are susceptible to low-light conditions and adverse weather, particularly when LiDAR sampling density decreases, leading to potential information loss.”
Quote in the paperNot a substantial claim - Premisecitation
RCM and Chasing Ghosts improved navigation efficiency by using reinforcement learning and topological reasoning to enable adaptive decision-making.
“Reinforcement learning and topological reasoning, as seen in RCM [267] and Chasing Ghosts [117], improved navigation efficiency by enabling adaptive decision-making.”
Quote in the paperNot a substantial claim0 of 2 sources checked- cites [267] Chen, 2022, Deformable feature aggregation for dynamic multi-modal 3D object detection source
- cites [117] Zhang, 2024
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Premisecitation
VIMA, built on Ravens, is a multimodal manipulation benchmark for evaluating robot performance across vision, language, and physics constraints.
“Meanwhile, VIMA [291], built on Ravens, is a multimodal manipulation benchmark designed for evaluating robot performance across multi-modal inputs (vision, language, and physics constraints).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [291] C. Liu, F. Zhu, X. Chang, X. Liang, Z. Ge, Y.-D. Shen, Vision-language navigation with random environmental mixup, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021, pp. 1644–1654. source
- Premisenone given
Cross-modal denoising mainly uses model-based rectifying and noise robustness regularization.
“As for crossmodal denoising, the main approaches are model-based rectifying and noise robustness regularization.”
Quote in the paperNot a substantial claim - Interpretationnone given
Incomplete multimodal learning is challenged by missing modalities due to device issues, data loss, or user preference, which affect fusion performance.
“Incomplete multimodal learning presents challenges in fusion tasks, as missing modalities due to device issues, data loss, or user preference affect performance.”
Quote in the paperNot a substantial claim - Interpretationnone given
Annotation biases such as semantic homogeneity, cultural bias, or linguistic representation bias in datasets significantly affect reliability of trained VLMs in real-world applications.
“In addition, the reliability of the trained VLMs in realworld applications is significantly affected due to the annotation biases (e.g., semantic homogeneity, cultural bias, or linguistic representation bias) present in the dataset.”
Quote in the paperSubstantial claim - Interpretationreasoning
VLM performance and generalisation depend heavily on the availability, diversity, and quality of training datasets.
“The performance and generalisation capabilities of VLMs are highly dependent on the availability, diversity and quality of training datasets.”
Quote in the paperSubstantial claim - Interpretationnone given
Modality imbalance affects performance through property and quality discrepancies, causing models to over-rely on certain modalities.
“Modality imbalance also affects performance, including property and quality discrepancies, causing models to over-rely on certain modalities.”
Quote in the paperNot a substantial claim - Interpretationreasoning
Efficient training and inference is an important challenge for practical deployment of multimodal large visual-language models.
“Efficient training and inference is an important challenge in the practical deployment of multimodal large models of visual language.”
Quote in the paperSubstantial claim - Interpretationreasoning
Heterogeneity is a central challenge in multimodal fusion due to differences in data structure, feature distribution, and information representation.
“Heterogeneity is a central challenge in multimodal fusion tasks, mainly due to the inherent differences in data structure, feature distribution and information representation.”
Quote in the paperSubstantial claim - Premisenone given
Synthetic datasets can be generated using rules or third-party tools such as Stable Diffusion and DALL-E to create diverse and semantically rich image-text pairs.
“using rules or third-party tools (e.g., Stable Diffusion, DALL-E) to generate image-text pairs with diversity and semantic richness”
Quote in the paperNot a substantial claim - Interpretationnone given
Multimodal fusion and VLMs have significantly advanced robot vision capabilities, enabling enhanced perception, reasoning, and interaction in complex environments.
“Multimodal fusion and vision-language models have significantly advanced the capabilities of robot vision, enabling enhanced perception, reasoning, and interaction in complex environments.”
Quote in the paperSubstantial claim
- May combine two claims. 258
- Classified as probably not a substantial claim. 185
- The quote only partly states this claim. 1
May combine two claims.
PremisecitationDefFusion introduces a deformable transformer mechanism that dynamically adjusts to multimodal data, mitigating over-attention and noise, and improving 3D semantic segmentation on NuScenes.
“DefFusion [14] introduces a deformable transformer mechanism that dynamically adjusts to multimodal data, mitigating over-attention and noise, and significantly improving 3D semantic segmentation, particularly on large-scale benchmarks like NuScenes.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [14] Xu, 2024, DefFusion: Deformable multimodal representation fusion for 3D semantic segmentation source
Section III describes the proposed method as Multi-Sem Fusion, using adaptive-attention and deep-feature-fusion modules, not a deformable transformer called DefFusion. The contributions passage and Table III report improvements in 3D object detection on nuScenes; semantic segmentation supplies inputs rather than being the reported outcome. Our copy does not substantiate the claimed mitigation of over-attention and noise. (LLM, conf=0.88)
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measured
May combine two claims.
PremisecitationWu et al. provide an in-depth analysis of multimodal techniques for embodied navigation but do not include emerging VLM architectures or place SLAM and manipulation in a unified analytical framework.
“In addition, although Wu et al. [31] provide an in-depth analysis of multimodal techniques for embodied navigation, their work does not include emerging vision-language model architectures, nor does it place tasks such as SLAM and manipulation within a unified analytical framework.”
Quote in the paperSubstantial claim1 of 1 sources checked- cites [31] Bordes, 2024 source
The cited paper is Bordes et al.’s introduction to vision-language models, not the Wu et al. work described in the claim. Its Introduction explicitly says it presents VLM training paradigms and models using pretrained backbones, contrary to the claim that the cited work omits VLM architectures. Our copy does not establish what Wu et al. cover. (LLM, conf=0.84)
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measured
May combine two claims.
PremisecitationMulti-Sem Fusion integrates LiDAR and camera data for 3D object detection, employing adaptive attention and deep feature fusion to enhance detection accuracy, but relies on offline processing limiting real-time applicability.
“Lastly, Multi-Sem Fusion [60] integrates LiDAR and camera data for 3D object detection, employing adaptive attention and deep feature fusion to enhance detection accuracy, although its reliance on offline processing limits real-time applicability.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [60] Hong, 2021 source
The paper’s opening summary (p. 1) describes DDRNets for real-time semantic segmentation, not Multi-Sem Fusion for LiDAR–camera 3D object detection, and reports 102 FPS on Cityscapes. The Introduction (p. 2) says the method achieves its results without an attention mechanism. Its real-time performance directly contradicts the claimed reliance on offline processing. (LLM, conf=0.90)
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measured
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationMuSe-GNN integrates modality-specific entities into a unified semantic space by leveraging shared GNN encoders and jointly optimizing similarity- and contrastive-based objectives.
“MuSe-GNN [67] further integrates modality-specific entities into a unified semantic space by leveraging shared GNN encoders and jointly optimizing similarity-and contrastive-based objectives.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [67] J. Hu, L. Shen, G. Sun, Squeeze-and-excitation networks, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 7132–7141. source
The conclusion (p. 11) identifies this paper’s contribution as the squeeze-and-excitation (SE) block for channel-wise feature recalibration, not MuSe-GNN. The introduction (p. 1) likewise describes a convolutional-network architecture rather than shared GNN encoders or similarity- and contrastive-based objectives. (LLM, conf=0.86)
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measured
May combine two claims.
PremisecitationCenterFusion uses staccato-based method to correlate radar data with camera centroids, enhancing 3D object detection accuracy, especially in dynamic environments by integrating velocity and depth.
“CenterFusion [94] uses a staccato-based method to correlate radar data with camera cen-troids, enhancing 3D object detection accuracy, especially in dynamic environments by integrating velocity and depth information.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [94] Nabati, 2019, Rrpn: Radar region proposal network for object detection in autonomous vehicles source
The Introduction and Section 3 identify the paper’s method as RRPN, which uses radar detections to generate regions of interest—not CenterFusion or a staccato-based method correlating radar data with camera centroids. Section 3 does say radar range and range-rate can supply depth and velocity information, but it does not substantiate the claimed CenterFusion method or its 3D detection result. (LLM, conf=0.88)
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measured
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationCRAFT adapts radar-camera correlation in polar coordinates, utilizing cross-attention for robust fusion.
“The CRAFT [100] model adapts radar-camera correlation in polar coordinates, utilizing cross-attention for robust fusion.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [100] R. Nabati, H. Qi, Centerfusion: Center-based radar and camera fusion for 3d object detection, in: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 2021, pp. 1527–1536. source
Section 4 (p. 4) identifies the paper’s model as CenterFusion and describes associating radar detections with object centers for middle fusion. The Introduction (p. 2) specifies frustum-based association and radar feature maps projected onto the image plane, rather than the claim’s CRAFT model with polar-coordinate correlation and cross-attention. (LLM, conf=0.88)
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measured
May combine two claims.
PremisecitationWang et al. used hybrid neural representations and multi-resolution hash grids for high-fidelity scene reconstruction, improving RGB-D SLAM accuracy and robustness.
“Wang et al. [149] adopted hybrid neural representations and multi-resolution hash grids to achieve high-fidelity scene reconstruction, significantly improving accuracy and robustness in RGB-D SLAM tasks.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [149] Shan, 2021, Lvi-sam: Tightly-coupled lidar-visual-inertial odometry via smoothing and mapping source
The Abstract (page not provided) identifies this as Shan et al.’s LVI-SAM, a factor-graph system coupling lidar, visual, and inertial odometry—not a Wang et al. method using hybrid neural representations or multi-resolution hash grids. The Experiments section describes tests of LVI-SAM, not high-fidelity reconstruction for RGB-D SLAM. (LLM, conf=0.89)
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measured
May combine two claims.
Premisecitation3D-VLA enhances visual representation by generating 3D point clouds and depth maps, improving task precision and robustness with language-guided action generation in dynamic environments.
“The 3D-VLA [154] model enhances visual representation by generating 3D point clouds and depth maps, improving task precision and robustness with language-guided action generation in dynamic environments.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [154] Jatavallabhula, 2023 source
The map-representation section (p. 4) identifies the system as ConceptFusion and says it takes color and depth images as input to build a 3D map; it does not describe generating depth maps. Related Work (p. 3) explicitly distinguishes its perception system from approaches focused on planning and control, contrary to the claim that this model generates language-guided actions. (LLM, conf=0.88)
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measured
May combine two claims.
PremisecitationBLIP jointly trains images and text through a generative model, enabling mutual generation and understanding and enhancing vision-language generation tasks.
“BLIP [190], through a generative model, jointly trains images and text, enabling mutual generation and understanding between the two modalities, thus enhancing the performance of vision-language generation tasks.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [190] Alwassel, 2020, Self-supervised learning by cross-modal audio-video clustering, Adv. Neural Inf. Process. Syst. source
The Introduction describes the paper's method as self-supervised learning from audio and video, and Figure 1 depicts clustering audio and visual features to train encoders. That is contrary to identifying this paper as BLIP or describing its method as generative image–text training for vision-language generation. (LLM, conf=0.88)
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measured
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationViT introduced patch embedding, dividing images into patches and converting them into tokens for Transformer, enabling effective feature extraction through self-attention.
“ViT [198] introduced patch embedding, dividing images into small patches and converting them into tokens for input into the Transformer, enabling effective feature extraction through self-attention.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [198] Vaswani, 2017, Attention is all you need, Adv. Neural Inf. Process. Syst. source
The Introduction (p. 2) says this paper introduces the Transformer for sequence transduction, not ViT or image patch embedding. The Conclusion (p. 10) describes applying the Transformer to images as future work, contrary to attributing an image-patch method to this paper. (LLM, conf=0.88)
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measured
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationDuring training, the model learns the joint distribution between two modalities and optimizes via adversarial training or maximum likelihood, enabling accurate target modality generation.
“During training,the model learns the joint distribution between the two modalities and optimizes the model through methods such as adversarial training or maximizing likelihood, enabling it to generate the target modality”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [188] Radford, 2021, Learning transferable visual models from natural language supervision source
Section 2.3 (p. 4) says CLIP learns to identify which images and texts are paired, using a contrastive objective and symmetric cross-entropy loss. It explicitly distinguishes this from predicting the accompanying text; it does not describe adversarial training, maximizing likelihood to learn a joint distribution, or generating a target modality. (LLM, conf=0.90)
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measured
May combine two claims.
PremisecitationMultimodal fusion combines complementary information from sensor inputs such as vision, language, depth, LiDAR, and tactile data, improving perception, reasoning, and decision-making in robot vision systems.
“In contrast, multimodal fusion integrates complementary information from various sensor inputs (e.g., vision, language, depth, LiDAR, and tactile data), thereby enhancing the perception, reasoning, and decision-making capabilities of robot vision systems [4,5,6,7].”
Quote in the paperSubstantial claim1 of 4 sources checked- cites [4] Xu, 2024, MRFTrans: Multimodal representation fusion transformer for monocular 3D semantic scene completion, Inf. Fusion source
- cites [5] X. Chen, H. Ma, J. Wan, B. Li, T. Xia, Multi-view 3d object detection network for autonomous driving, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017, pp. 1907–1915. source
- cites [6] Li, 2024, RD-VIO: Robust visual-inertial odometry for mobile augmented reality in dynamic environments, IEEE Trans. Vis. Comput. Graphics source
The Introduction (p. 1) says fusing visual and inertial information improves robustness in complex environments, and §2.1 (p. 2) says that fusion helps recover metric scale. The paper does not support the broader examples of language, depth, LiDAR, or tactile inputs, or the claimed improvements in reasoning and decision-making. (LLM, conf=0.88)
Experimental — accuracy not yet measured - cites [7] Babadian, 2023, Fusion of tactile and visual information in deep learning models for object recognition, Inf. Fusion source
May combine two claims.
PremisecitationLarge-scale pretrained VLMs have cross-modal alignment and generalization abilities and show strong potential in zero-shot understanding, instruction following, and visual question answering.
“Large-scale pretrained VLMs not only possess cross-modal alignment and generalization abilities, but also demonstrate strong potential in tasks such as zero-shot understanding [8], instruction following [9,10], and visual question answering [11].”
Quote in the paperSubstantial claim3 of 4 sources checked- cites [8] Radford, 2019, Language models are unsupervised multitask learners, OpenAI Blog
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites [9] Long, 2024 source
Section 2.2, “Large Models in Robotics Navigation” (page not provided), describes multimodal large models as capable of instruction following and visual perception, with strong generalization. Section 4.3 reports zero-shot visual-language navigation. The paper does not establish the broader claim of zero-shot understanding or explicitly discuss cross-modal alignment; visual question answering is attached to a separate citation in the citing sentence. (LLM, conf=0.82)
Experimental — accuracy not yet measured - cites [10] Zhou, 2024, Navgpt-2: Unleashing navigational reasoning capability for large vision-language models source
The citation is attached to the claim about VLMs’ zero-shot understanding. The Introduction (p. 3) discusses zero-shot vision-and-language navigation using LLMs, but does not establish strong zero-shot understanding by pretrained VLMs in general. A search for zero-shot VLM understanding found no passage addressing that claim. (LLM, conf=0.70)
Experimental — accuracy not yet measured - cites [11] Team, 2023 source
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationRoitberg et al. compared and analyzed seven decision-level fusion strategies for driver behavior understanding.
“Roitberg et al. [47] compared and analyzed seven decision-level fusion strategies for driver behavior understanding, offering valuable insights for selecting appropriate fusion schemes.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [47] Valada, 2018, Self-supervised model adaptation for multimodal semantic segmentation, Int. J. Comput. Vis. source
The Multimodal Fusion Ablation Studies (p. 34) and Fusion Stage and Reliance on Modalities (p. 35) compare configurations of this paper’s semantic-segmentation model, not seven decision-level strategies for driver behavior understanding. Searches for Roitberg and the seven-strategy claim found no passage in our copy addressing it. (LLM, conf=0.75)
Experimental — accuracy not yet measured
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationMultimodal fusion strategies can be categorized into early, mid, and late fusion based on fusion strategy.
“As illustrated in Figure 3, multimodal fusion strategies can be categorized into early fusion, mid fusion, and late fusion based on the fusion strategy [38,39].”
Quote in the paperNot a substantial claim1 of 2 sources checkedMay combine two claims.
PremisecitationDMA focuses on open-vocabulary 3D scene understanding, aligning dense correspondences between 3D points, 2D images, and text, using VLMs to generalize across indoor and outdoor scenarios.
“On the other hand ,DMA [59] focuses on open-vocabulary 3D scene understanding, aligning dense correspondences between 3D points, 2D images, and text, utilizing vision-language models to generalize across diverse indoor and outdoor scenarios.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [59] Wang, 2021, Deep high-resolution representation learning for visual recognition, IEEE Trans. Pattern Anal. Mach. Intell. source
The Semantic Segmentation section (p. 7) describes HRNetV2 assigning class labels to image pixels, not DMA. Searches for DMA, open-vocabulary 3D scene understanding, 3D–2D–text correspondences, and vision-language models found no passage addressing the attributed claim in our copy. (LLM, conf=0.77)
Experimental — accuracy not yet measured
May combine two claims.
PremisecitationAdversarial representation learning has been used to create modality-invariant embedding spaces, reduce modal gaps, and improve cross-modal fusion.
“In addition, adversarial representation learning has been used to create modality invariant embedding spaces, reduce modal gaps, and improve cross modal fusion.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [44] Mai, 2019 source
Classified as probably not a substantial claim.
PremisecitationMultimodal fusion combines sources such as vision, language, depth, LiDAR, and speech to enhance scene semantic understanding.
“It combines various sources of information, such as vision, language, depth data, lidar, and speech, to enhance the capability of scene semantic understanding [35,36,37].”
Quote in the paperSubstantial claim1 of 3 sources checked- cites [35] Mikolajczyk, 2005, A performance evaluation of local descriptors, IEEE Trans. Pattern Anal. Mach. Intell. source
- cites [36] Everingham, 2010, The pascal visual object classes (VOC) challenge, Int. J. Comput. Vis. source
Section 5.1 describes fusion of multiple visual representations for image classification, and Section 6.1.1 says high-performing methods combined visual feature channels. The paper does not support the claimed combination of vision with language, depth, LiDAR, and speech for scene semantic understanding. Page numbers were not available in our copy. (LLM, conf=0.84)
Experimental — accuracy not yet measured - cites [37] Geiger, 2012, Are we ready for autonomous driving? The KITTI vision benchmark suite source
Probably not a substantial claim: may not pass S3 (load-bearing).
Premisecitation3D-CVF employs adaptive gating fusion with attention maps for feature weighting.
“3D-CVF [81] employs adaptive gating fusion with attention maps for feature weighting.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [81] Li, 2022, Unifying voxel-based representation with transformer for 3d object detection, Adv. Neural Inf. Process. Syst. source
The Related Work section (p. 3) cites 3D-CVF as an example of instance-level fusion, and the References (p. 12) identify that work. Neither passage describes adaptive gating or attention-map feature weighting; searches for those terms found only descriptions of this paper’s own fusion and transformer attention. Our copy therefore does not verify the attributed detail about 3D-CVF. (LLM, conf=0.77)
Experimental — accuracy not yet measured
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationPoint-Painting projects LiDAR point clouds onto output of image semantic segmentation network and concatenates channel activation values with LiDAR intensity values, utilizing image semantic information.
“Point-Painting [80] projects LiDAR point clouds onto the output of the image semantic segmentation network and concatenates the channel activation values with the LiDAR point cloud intensity values, effectively utilizing image semantic information.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [80] J. Mao, Y. Xue, M. Niu, H. Bai, J. Feng, X. Liang, H. Xu, C. Xu, Voxel transformer for 3d object detection, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021, pp. 3164–3173. source
The Related Work section (p. 2) discusses point- and voxel-based detectors, but does not describe Point-Painting. Searches for PointPainting and for LiDAR–image semantic fusion found no passage explaining the claimed projection or concatenation of values. (LLM, conf=0.76)
Experimental — accuracy not yet measured
May combine two claims.
PremisecitationCCA-GNN proposes temporal positional encoding for graph nodes and aligns audio-visual representations through canonical correlation analysis.
“To enhance temporal consistency and modality alignment, CCA-GNN [68] proposes a temporal positional encoding scheme for graph nodes and aligns audio-visual representations through canonical correlation analysis.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [68] Ming, 2024, OccFusion: Multi-sensor fusion framework for 3D semantic occupancy prediction, IEEE Trans. Intell. Veh. source
The Implementation Details section describes OccFusion’s camera, lidar, and radar framework, not CCA-GNN. Searches for CCA-GNN, temporal positional encoding, and audio-visual alignment by canonical correlation analysis found no passage addressing the claim. The References section identifies [68] as VoxFormer, but that does not establish what CCA-GNN proposes. (LLM, conf=0.78)
Experimental — accuracy not yet measured
May combine two claims.
PremisecitationLiDAR-based detection processes 3D point cloud data, offering precise depth estimation and spatial structure representation.
“In contrast, LiDAR-based detection [73,74,75,76,77] processes 3D point cloud data, offering precise depth estimation and spatial structure representation.”
Quote in the paperNot a substantial claim4 of 5 sources checked- cites [73] A. Dutta, J. Alcaraz, A. TehraniJamsaz, E. Cesar, A. Sikora, A. Jannesari, Performance optimization using multimodal modeling and heterogeneous GNN, in: Proceedings of the 32nd International Symposium on High-Performance Parallel and Distributed Computing, 2023, pp. 45–57. source
Section 2.1 discusses representations of program code, and Section 4.2.2 discusses mapping code to CPUs and GPUs. Searches for LiDAR, 3D point clouds, depth sensing, and object detection found no passage addressing the attributed claim. The retrieved passages have no page numbers. (LLM, conf=0.78)
Experimental — accuracy not yet measured - cites [74] X. Yang, S. Feng, Y. Zhang, D. Wang, Multimodal sentiment detection based on multi-channel graph neural networks, in: Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), 2021, pp. 328–339. source
- cites [75] Ku, 2018, Joint 3d proposal generation and object detection from view aggregation source
The abstract (p. 1) says AVOD processes LiDAR point clouds alongside RGB images to predict objects’ extents and orientation in 3D space. The feature-map method (p. 3) describes encoding point heights and density, supporting spatial-structure representation. The paper reports accurate 3D localization, but does not establish the claim of precise depth estimation by LiDAR-based detection specifically. (LLM, conf=0.84)
Experimental — accuracy not yet measured - cites [76] D. Park, R. Ambrus, V. Guizilini, J. Li, A. Gaidon, Is pseudo-lidar needed for monocular 3d object detection?, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021, pp. 3142–3152. source
Section 4 (page unavailable) says that a LiDAR-based 3D detector processes a point cloud and that ground-truth depth measurements come from LiDAR point clouds. It also describes point-cloud data as containing 3D coordinates, supporting the spatial-representation point. The copy does not establish the claimed precision of LiDAR-based depth estimation; its statement about high precision refers to a monocular depth network. (LLM, conf=0.78)
Experimental — accuracy not yet measured - cites [77] Yang, 2018 source
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationDeepFusion ensures precise alignment of LiDAR and image features using InverseAug and cross-attention.
“DeepFusion [85] ensures precise alignment of LiDAR and image features using InverseAug and cross-attention mechanisms.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [85] Yang, 2022, Deepinteraction: 3d object detection via modality interaction, Adv. Neural Inf. Process. Syst. source
The Introduction (p. 1) includes DeepFusion among methods that merge image and LiDAR features, and the References (p. 12) identify it as a separate paper. Our copy does not say that DeepFusion uses InverseAug or cross-attention to ensure precise alignment; searches for those terms found no supporting passage. (LLM, conf=0.76)
Experimental — accuracy not yet measured
May combine two claims.
PremisecitationCamera-based single-modal 3D object detection infers 3D information from 2D images; RGB images provide rich visual data but lack direct depth perception, requiring complex estimation, making detection vulnerable to occlusions, poor angles, and adverse weather.
“Camera-based single-modal 3D object detection [1, 72, 2, 3] infers 3D information from 2D images.While RGB images provide rich visual data, the lack of direct depth perception requires complex estimation techniques, making detection vulnerable to occlusions, poor angles, and adverse weather conditions.”
Quote in the paperSubstantial claim2 of 4 sources checked- cites [1] Z. Xu, W. Zhang, X. Ye, X. Tan, W. Yang, S. Wen, E. Ding, A. Meng, L. Huang, Zoomnet: Part-aware adaptive zooming neural network for 3d object detection, in: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, 2020, pp. 12557–12564, 07. source
Split into 2 atoms — 1 confirmed, 1 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.
Experimental — accuracy not yet measured - cites [72] Passos, 2023, Multimodal audio-visual information fusion using canonical-correlated graph neural network for energy-efficient speech enhancement, Inf. Fusion source
Split into 2 atoms — 0 confirmed, 0 partially supported, 0 disputed, 0 misattributed, 2 unverifiable. Expand the claim to see per-atom verdicts.
Experimental — accuracy not yet measured - cites [2] X. Shi, Q. Ye, X. Chen, C. Chen, Z. Chen, T.-K. Kim, Geometry-based distance decomposition for monocular 3d object detection, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021, pp. 15172–15181. source
- cites [3] L. Wang, L. Du, X. Ye, Y. Fu, G. Guo, X. Xue, J. Feng, L. Zhang, Depth-conditioned dynamic message propagation for monocular 3d object detection, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021, pp. 454–463. source
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationFusion-Net uses dense 2D radar maps and spatial transformation to align sensor feature maps.
“Fusion-Net [92] uses dense 2D radar maps and spatial transformation to align sensor feature maps.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [92] Xu, 2021, Fusionpainting: Multimodal fusion with adaptive attention for 3d object detection source
The Multi-modal Fusion and Adaptive Attention Module sections describe FusionPainting’s LiDAR–camera fusion, not Fusion-Net. Searches for “Fusion-Net,” “dense 2D radar maps,” and “spatial transformation” found no passage addressing the attributed method; the available passages therefore neither support nor contradict the claim about Fusion-Net. (LLM, conf=0.76)
Experimental — accuracy not yet measured
May combine two claims.
PremisecitationBEVFusion's camera features lack spatial awareness, which GAFusion improves using Sparse Depth Guidance and Laser Occupancy Guidance.
“However, its camera features lack spatial awareness, which GAFusion [89] improves using Sparse Depth Guidance and Laser Occupancy Guidance.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [89] Liu, 2023, Bevfusion: Multi-task multi-sensor fusion with unified bird’s-eye view representation source
Section C, “Fully-Convolutional Fusion” (page unavailable), says BEVFusion’s camera and LiDAR features can be spatially misaligned because of inaccurate depth, supporting a narrower version of the spatial-awareness concern. The paper describes its own convolution-based BEV encoder as a response; our copy does not identify GAFusion or describe Sparse Depth Guidance or Laser Occupancy Guidance. (LLM, conf=0.85)
Experimental — accuracy not yet measured
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationSimple-BEV combines camera and radar features through rasterised radar information and bilinear sampling for 3D bird's-eye view representation.
“Simple-BEV [99] combines camera and radar features through rasterised radar information and bilinear sampling for 3D bird's-eye view representation.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [99] Y. Kim, S. Kim, J.W. Choi, D. Kum, Craft: Camera-radar 3d object detection with spatio-contextual fusion transformer, in: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 37, 2023, pp. 1160–1168, 1. source
The Camera-Radar Fusion section discusses other fusion methods but does not mention Simple-BEV. Searches for Simple-BEV, rasterised radar information, and bilinear sampling found no passage in our copy that addresses the claimed method. (LLM, conf=0.75)
Experimental — accuracy not yet measured
May combine two claims.
PremisecitationInstruction-following navigation relies on R2R and REVERIE tasks, enhancing natural language interaction and environment-specific reasoning, and highlighting multimodal perception in complex scenarios.
“Instructionfollowing Navigation relies on the development of Room-to-Room (R2R) and REVERIE tasks, which enhances natural language interaction and environment-specific reasoning, and further highlights the role of multimodal perception in complex scenarios [111,31,112,113].”
Quote in the paperNot a substantial claim2 of 4 sources checked- cites [111] M. Savva, A. Kadian, O. Maksymets, Y. Zhao, E. Wijmans, B. Jain, J. Straub, J. Liu, V. Koltun, J. Malik, et al., Habitat: A platform for embodied ai research, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, 2019, pp. 9339–9347. source
- cites [31] Bordes, 2024 source
The passages in §3.5 discuss instruction tuning and multimodal instruction following, but not instruction-following navigation. Searches for “R2R,” “Room-to-Room,” and “REVERIE” found no passage in our copy connecting those tasks to natural-language interaction, environment-specific reasoning, or multimodal perception in navigation. (LLM, conf=0.76)
Experimental — accuracy not yet measured - cites [112] Zhu, 2017, Target-driven visual navigation in indoor scenes using deep reinforcement learning source
- cites [113] Ren, 2024 source
Section 4.1 (page unavailable) describes navigation to an object given language instructions, and the Introduction discusses environmental perception and task reasoning. But our copy does not identify R2R or REVERIE as tasks underlying instruction-following navigation or attribute the claimed advances to them; searches for both names found no relevant passage. (LLM, conf=0.86)
Experimental — accuracy not yet measured
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationViNG enables visual goal navigation via a topological graph without pre-built maps or explicit spatial priors.
“Meanwhile, ViNG [17] enables visual goal navigation by constructing a topological graph, allowing robots to operate without pre-built maps or explicit spatial priors.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [17] Shah, 2021, Ving: Learning open-world navigation with visual goals source
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationUVTR converts images into voxel representations, enabling cross-modal learning.
“UVTR [84] converts images into voxel representations, enabling cross-modal learning.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [84] Sindagi, 2019, Mvx-net: Multimodal voxelnet for 3d object detection source
The Introduction and C. Multimodal Fusion (page not provided) describe MVX-Net combining image features with LiDAR features at 3D points or voxels, supporting the cross-modal learning aspect. They do not describe UVTR or say that images are converted into voxel representations; VoxelFusion instead appends image features to features of voxels formed from point clouds. (LLM, conf=0.86)
Experimental — accuracy not yet measured
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationGAFusion employs Laser-Guided Adaptive Fusion Transformer and temporal fusion module for multi-frame aggregation.
“Additionally, it employs a Laser-Guided Adaptive Fusion Transformer and a temporal fusion module for multi-frame aggregation.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [89] Liu, 2023, Bevfusion: Multi-task multi-sensor fusion with unified bird’s-eye view representation source
The Method overview and Fully-Convolutional Fusion section describe BEVFusion’s shared BEV representation and convolution-based encoder, not GAFusion. Searches for GAFusion, the Laser-Guided Adaptive Fusion Transformer, and a temporal fusion module found no passage in our copy addressing those components or their use for multi-frame aggregation. (LLM, conf=0.76)
Experimental — accuracy not yet measured
May combine two claims.
PremisecitationRT-2 aligns visual and language representations via pretraining, allowing robot to generate control commands from linguistic input using Transformer to map high-level semantics to actions.
“For example, the RT-2 [152] model aligns visual and language representations via pretraining, allowing the robot to generate control commands from linguistic input using a Transformer architecture to map high-level semantics to actions.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [152] Delama, 2023, UVIO: An UWB-aided visual-inertial odometry framework with bias-compensated anchors initialization source
The Introduction and Conclusion describe UVIO as a UWB-aided visual-inertial localization framework, not an RT-2 model. Searches for RT-2, vision-language pretraining, and Transformer-based language-to-action control found no passage addressing the claim in our copy. (LLM, conf=0.78)
Experimental — accuracy not yet measured
May combine two claims.
PremisecitationLVI-SAM uses factor graph optimization to couple visual and LiDAR data, excelling in low-texture and dynamic environments.
“Furthermore, LVI-SAM [150] represents a landmark multimodal SLAM framework, utilizing factor graph optimization to deeply couple visual and LiDAR data, particularly excelling in low-texture and dynamic environments.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [150] Zuo, 2020, Lic-fusion 2.0: Lidar-inertial-camera odometry with sliding-window plane-feature tracking source
The Introduction and Related Work describes LIC-Fusion as a filtering approach and discusses graph optimization in other methods; it does not identify or describe LVI-SAM. The Vicon Room Sequences section reports LIC-Fusion 2.0 results in cluttered environments, not LVI-SAM performance in low-texture or dynamic environments. Searches for LVI-SAM, factor-graph coupling, and those environments found no passage addressing the attributed claim. (LLM, conf=0.76)
Experimental — accuracy not yet measured
May combine two claims.
PremisecitationSupervised localization methods excel in constrained settings with labeled datasets, while self-supervised methods use geometric and photometric consistency to adapt to unseen environments.
“Supervised methods excel in constrained settings by utilizing labeled datasets, while self-supervised approaches leverage geometric and photometric consistency to adapt effectively to unseen environments [133].”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [133] Chen, 2023, Deep learning for visual localization and mapping: A survey, IEEE Trans. Neural Netw. Learn. Syst. source
May combine two claims.
PremisecitationLIC-Fusion and LIC-Fusion 2.0 used tightly coupled optimization to integrate LiDAR, IMU, and visual data, balancing high accuracy and real-time performance in large-scale environments.
“More advanced systems, such as LIC-Fusion [18] and its successor LIC-Fusion 2.0 [145], adopted tightly coupled optimization frameworks to deeply integrate LiDAR, IMU, and visual data, achieving a balance between high accuracy and real-time performance in large-scale environments.”
Quote in the paperNot a substantial claim1 of 2 sources checked- cites [18] Zuo, 2019, Lic-fusion: Lidar-inertial-camera odometry source
The Abstract and Introduction describe LIC-Fusion as a tightly coupled, efficient fusion of LiDAR, IMU, and camera measurements, and the Outdoor Tests report improved accuracy. But the paper describes an MSCKF-based filter, not a tightly coupled optimization framework, and its experiments do not establish performance in large-scale environments. LIC-Fusion 2.0 is separately cited in the citing sentence, so this verdict concerns LIC-Fusion only. (LLM, conf=0.88)
Experimental — accuracy not yet measured - cites [145] Shan, 2018, Lego-loam: Lightweight and ground-optimized lidar odometry and mapping on variable terrain source
May combine two claims.
PremisecitationNeRF and Semantic-NeRF encode scene geometry and semantics into compact neural fields, enabling robust localization in new environments and showing multimodal fusion potential for perception and mapping.
“Neural implicit representations, like NeRF [135] and Semantic-NeRF [136], encode scene geometry and semantics into compact neural fields, enabling robust localization in new environments and showcasing the potential of multimodal fusion for bridging perception and mapping.”
Quote in the paperNot a substantial claim1 of 2 sources checked- cites [135] Wang, 2023, Attention weighted local descriptors, IEEE Trans. Pattern Anal. Mach. Intell. source
- cites [136] C. Wang, R. Xu, Y. Zhang, S. Xu, W. Meng, B. Fan, X. Zhang, MTLDesc: Looking wider to describe better, in: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 36, 2022, pp. 2388–2396, 2. source
The Method (p. 2) describes MTLDesc as a convolutional network for keypoint detection and local descriptors, not a neural field. Searches for NeRF, Semantic-NeRF, implicit representations, and multimodal fusion found no passage in our copy addressing the attributed claim. (LLM, conf=0.76)
Experimental — accuracy not yet measured
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationVINS-Fusion has become a mainstream solution for large-scale, high-precision localization and is widely used in UAVs and mobile robots.
“In terms of vision-IMU-GPS fusion, VINS-Fusion has become a mainstream solution for large-scale, high-precision localization and is widely used in UAVs and mobile robots.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [151] Qin, 2018, Vins-mono: A robust and versatile monocular visual-inertial state estimator, IEEE Trans. Robot. source
The Introduction (p. 1) describes VINS-Mono as a high-accuracy system used in drone navigation and large-scale state-estimation tasks, and the Experimental Results (p. 13) report a large-scale test. The cited paper concerns VINS-Mono, however, not VINS-Fusion; it does not establish that VINS-Fusion fuses GPS, is a mainstream solution, or is widely used in UAVs and mobile robots. (LLM, conf=0.87)
Experimental — accuracy not yet measured
May combine two claims.
PremisecitationRoboMamba incorporates action dynamics into multimodal fusion, optimizing vision, language, and action for task planning.
“Additionally, the RoboMamba [153] model incorporates action dynamics into the multimodal fusion framework, optimizing vision, language, and action for task planning.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [153] Sun, 2024, Mm3dgs slam: Multi-modal 3d gaussian splatting for slam using vision, depth, and inertial measurements source
The Conclusions describe MM3DGS as a SLAM framework combining visual, depth, and inertial measurements for rendering and trajectory tracking. A search for RoboMamba, action dynamics, and vision-language-action task planning found no passage addressing the attributed claim. (LLM, conf=0.75)
Experimental — accuracy not yet measured
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationIn cross-modal self-supervised learning, tasks such as image inpainting or text mask prediction are typically designed.
“In cross-modal self-supervised learning, tasks such as image inpainting or text mask prediction are typically designed.[183]”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [183] Kim, 2021, Vilt: Vision-and-language transformer without convolution or region supervision source
The Introduction (p. 1) and Pre-training Objectives (p. 5) describe masked language modeling as common in vision-and-language pre-training, including prediction of masked text tokens. They identify image–text matching—not image inpainting—as the other common objective. The Conclusion (p. 9) calls for future work on visual masking objectives, so the paper does not support presenting image inpainting as a typical task. (LLM, conf=0.87)
Experimental — accuracy not yet measured
May combine two claims.
PremisecitationMimic-Touch uses a strategy learning framework mimicking human tactile-guided behaviors, combining self-supervised representation learning with RL to optimize grasp stability.
“For example, Mimic-Touch [161] uses a strategy learning framework that mimics human tactile-guided behaviors, collecting multimodal tactile data and combining self-supervised representation learning with reinforcement learning to optimize grasp stability.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [161] Zhen, 2024 source
The Abstract (p. 1) describes 3D-VLA as a 3D vision-language-action generative world model, not a tactile-guided grasping framework. Searches for Mimic-Touch, tactile-guided behavior, self-supervised representation learning, and reinforcement learning found no passage in our copy addressing the attributed claim. (LLM, conf=0.76)
Experimental — accuracy not yet measured
May combine two claims.
PremisecitationVision and language are fundamentally distinct, and cross-modal pre-training enables models to learn deep correlations, making them more accurate in vision-language tasks.
“Vision and language are fundamentally distinct forms of information, and cross-modal pre-training enables models to learn the deep correlations between them, making them more accurate in performing vision-language tasks [168].”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [168] Yu, 2024 source
The “Large Vision-Language Models” section (page not provided) says these models effectively use image-text data for image-based reasoning. It does not say that vision and language are fundamentally distinct, or that cross-modal pre-training learns deep correlations that make vision-language tasks more accurate. (LLM, conf=0.82)
Experimental — accuracy not yet measured
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationA contrastive loss brings related modality pairs closer and distances unrelated pairs in a shared embedding space.
“The model is trained using a contrastive loss function to bring related modality pairs, like images and descriptions, closer in a shared embedding space while distancing unrelated pairs [179,180].”
Quote in the paperNot a substantial claim1 of 2 sources checked- cites [179] Alatkar, 2023, CMOT: cross-modality optimal transport for multimodal inference, Genome Biol. source
The Methods section (page unavailable) says CMOT projects modalities into a common manifold and minimizes distances between corresponding samples. It describes manifold alignment and optimal transport, not a contrastive loss that distances unrelated pairs; its examples concern single-cell modalities, not images and descriptions. (LLM, conf=0.88)
Experimental — accuracy not yet measured - cites [180] Qian, 2022, Integrating multi-label contrastive learning with dual adversarial graph neural networks for cross-modal retrieval, IEEE Trans. Pattern Anal. Mach. Intell. source
May combine two claims.
PremisecitationVITO-Transformer integrates tactile inputs through self-attention, enabling precise grasp point estimation and dynamic adaptability.
“Similarly, VITO-Transformer [157], based on Vision Transformer (ViT), integrates tactile inputs through selfattention, enabling precise grasp point estimation and dynamic adaptability.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [157] Xu, 2025 source
The Multi-Sem Fusion Framework and AAF Module sections describe camera images, point clouds, and attention-based fusion for 3-D object detection. Searches for VITO-Transformer, tactile inputs, and grasp-point estimation found no passage addressing the attributed method or its claimed capabilities. (LLM, conf=0.76)
Experimental — accuracy not yet measured
May combine two claims.
PremisecitationDuring training, specialized encoders extract features from different modalities, then cross-modal alignment mechanisms integrate them.
“During training, the model employs specialized encoders (e.g., convolutional neural networks for visual data, transformer architectures for text and audio data) [171,172] to extract features from different modalities, and then interacts and integrates these features through cross-modal alignment mechanisms [173,174].”
Quote in the paperNot a substantial claim2 of 4 sources checked- cites [171] Du, 2022
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites [172] Gan, 2022, Vision-language pre-training: Basics, recent advances, and future trends, Found. Trends® Comput. Graph. Vis. source
- cites [173] Sun, 2023, Fusion or defusion? Flexible vision-and-language pre-training source
- cites [174] Zhang, 2023, Multi-task paired masking with alignment modeling for medical vision-language pre-training, IEEE Trans. Multimed. source
May combine two claims.
PremisecitationPre-training on large-scale multimodal data lets models learn deep vision-language correlations, enhancing cross-modal understanding and representation.
“By pre-training on large-scale multimodal data, the model learns deep correlations between vision and language, significantly enhancing its ability to understand and represent information across different modalities [164,165].”
Quote in the paperSubstantial claim2 of 2 sources checked- cites [164] Qi, 2023, General in-hand object rotation with vision and touch source
Section 3.2 describes a transformer trained on depth images, tactile contact points, joint positions, and action history for in-hand object rotation. Searches for vision-language correlations, large-scale multimodal pre-training, and cross-modal language understanding found no passage addressing the attributed claim; the copy does not provide evidence for or against it. (LLM, conf=0.76)
Experimental — accuracy not yet measured - cites [165] Higuera, 2024 source
The Broader related work section (p. 17) describes methods that use natural language for video-language alignment and a model that learns visual representations with language supervision. It does not establish that pre-training on large-scale multimodal data produces deep vision-language correlations or significantly improves cross-modal understanding, and it describes other models rather than Sparsh on this point. (LLM, conf=0.78)
Experimental — accuracy not yet measured
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationALIGN trains cross-modal image and text data to enhance matching performance.
“ALIGN [182] follows a similar approach, training cross-modal image and text data to enhance matching performance.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [182] Li, 2024, Joint self-supervised and supervised contrastive learning for multimodal MRI data: Towards predicting abnormal neurodevelopment, Artif. Intell. Med. source
The Introduction (page unavailable) describes CLIP-based methods that create a shared representation space for images and text, but it does not identify ALIGN or discuss its matching performance. Searches for “ALIGN” and “ALIGN image text matching performance” found no passage about ALIGN; passages about aligning features concern this paper’s MRI method instead. (LLM, conf=0.75)
Experimental — accuracy not yet measured
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationTransformer was initially applied in NLP, with BERT and GPT showcasing advantages in NLU and NLG.
“Transformer was initially applied in NLP tasks, with BERT [192] and GPT [193] as key models showcasing its advantages in natural language understanding (NLU) and natural language generation (NLG), respectively.”
Quote in the paperNot a substantial claim1 of 2 sources checked- cites [192] M. Caron, H. Touvron, I. Misra, H. Jégou, J. Mairal, P. Bojanowski, A. Joulin, Emerging properties in self-supervised vision transformers, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021, pp. 9650–9660. source
The Introduction (p. 1) describes BERT and GPT as examples of successful Transformer use in NLP, and the References (p. 10) identify BERT with language understanding. The paper calls GPT a language-modeling example, but does not explicitly describe its advantages in natural language generation or establish that NLP was the first application of Transformers. (LLM, conf=0.84)
Experimental — accuracy not yet measured - cites [193] R. Hu, A. Singh, Unit: Multimodal multitask learning with a unified transformer, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021, pp. 1439–1449. source
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationVQ-VAE-2 improves upon traditional VAE methods and excels in multimodal generation, generating high-quality images or other modality content.
“VQ-VAE-2 [189] improves upon traditional VAE methods and excels in multimodal generation tasks, capable of generating high-quality images or other modality content.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [189] Jia, 2021, Scaling up visual and vision-language representation learning with noisy text supervision source
The Introduction (p. 1) and Conclusion (p. 9) describe ALIGN, an image–text representation model used for retrieval, not VQ-VAE-2. Searches for VQ-VAE-2 and variational-autoencoder-based multimodal generation found no passage addressing the claimed improvement or generation capabilities. (LLM, conf=0.78)
Experimental — accuracy not yet measured
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationLongformer and Linformer used sparse attention and low-rank projections, respectively, to decrease computational complexity, making Transformer more efficient for long-text sequences.
“Reformer [195] employed locality-sensitive hashing (LSH Attention) to reduce memory requirements for attention computation, while Longformer [196] and Linformer [197] utilized sparse attention and low-rank projections, respectively, to decrease computational complexity, making Transformer more efficient for handling long-text sequences.”
Quote in the paperNot a substantial claim2 of 2 sources checked- cites [196] Razavi, 2019, Generating diverse high-fidelity images with vq-vae-2, Adv. Neural Inf. Process. Syst. source
Section 3.2 (p. 5) discusses attention in a VQ-VAE-2 image model and memory constraints for its latent-code priors. Searches for Longformer, Linformer, sparse attention, and low-rank projections found no passage addressing the claimed methods for long-text sequences. (LLM, conf=0.76)
Experimental — accuracy not yet measured - cites [197] Li, 2022, Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation source
The Model Architecture and Pre-training Objectives sections (p. 3) discuss BLIP’s visual and text transformers, not Longformer or Linformer. Searches for those models and for long-text computational complexity found no passage in our copy addressing the attributed claim. (LLM, conf=0.76)
Experimental — accuracy not yet measured
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationBERT uses a Transformer encoder-only architecture and pretrains with MLM and NSP, improving text understanding tasks.
“Proposed by Google in 2018, BERT uses a Transformer encoder-only architecture and pretrains with masked language modeling (MLM) and next sentence prediction (NSP), improving text understanding tasks.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [192] M. Caron, H. Touvron, I. Misra, H. Jégou, J. Mairal, P. Bojanowski, A. Joulin, Emerging properties in self-supervised vision transformers, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021, pp. 9650–9660. source
The Introduction (p. 1) identifies BERT as an example of self-supervised pretraining and says such pretraining contributed to Transformers’ success in NLP. It does not specify that BERT is encoder-only, name MLM or NSP, or describe results on text-understanding tasks. The implementation discussion (p. 4) describes a Vision Transformer, not BERT’s architecture. (LLM, conf=0.86)
Experimental — accuracy not yet measured
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationTransformer has become a core architecture in deep learning, achieving major advancements in NLP, CV, and multimodal learning since 2017.
“Since its introduction by Vaswani et al. [191] in 2017, Transformer has become a core architecture in deep learning, achieving major advancements in natural language processing (NLP), computer vision (CV), and multimodal learning.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [191] Sayed, 2019, Cross and learn: Cross-modal self-supervision source
The Introduction (p. 1) and Conclusion (p. 13) discuss CNNs and cross-modal self-supervision for visual recognition, not Transformer. Searches for Transformer, Vaswani, self-attention, and NLP found no passage in our copy addressing the claimed architecture or its advances across those fields. (LLM, conf=0.76)
Experimental — accuracy not yet measured
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationUniT is a self-supervised framework for cross-modal tasks; it jointly pre-trains different modalities through self-supervised tasks, enabling universal features.
“UniT [186], on the other hand, is a self-supervised learning framework for cross-modal tasks.It jointly pre-trains different modalities (e.g., vision and text) through self-supervised tasks, enabling the model to learn universal features.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [186] Ferraro, 2021, Enriched music representations with multiple cross-modal contrastive learning, IEEE Signal Process. Lett. source
The Abstract (p. 1) describes a cross-modal contrastive approach for learning music-audio features from playlists, genre metadata, and audio. Searches for UniT and for joint self-supervised pre-training of vision and text found no passage addressing the claimed framework or its universal features. (LLM, conf=0.77)
Experimental — accuracy not yet measured
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationDeepSeek-MoE, using 64 experts and dynamic activation, drastically reduces computational redundancy while maintaining high performance, and improves expert scheduling for better generalization in multilingual and multi-task settings.
“DeepSeek-MoE [21], using 64 experts and dynamic activation, drastically reduces computational redundancy while maintaining high performance, and improves expert scheduling for better generalization in multilingual and multi-task settings.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [21] Dai, 2024 source
Section 4.2 reports 64 experts (one shared and 63 routed), and Section 3.2 describes routing that activates selected experts. The Conclusion reports performance comparable to two dense models with about 40% of their computation, while Section 3.2 addresses redundancy among expert parameters. The paper does not establish improved expert scheduling or better generalization specifically in multilingual and multi-task settings; it also does not describe the measured computation saving as a reduction in computational redundancy. (LLM, conf=0.88)
Experimental — accuracy not yet measured
May combine two claims.
PremisecitationMoE has been integrated into ultra-large-scale models like GPT-4 and DeepSeek-MoE, advancing multi-task and multimodal applications.
“MoE has been integrated into ultra-large-scale models like GPT-4 [22] and DeepSeek-MoE [21], advancing multi-task and multimodal applications.”
Quote in the paperNot a substantial claim1 of 2 sources checked- cites [22] Achiam, 2023
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites [21] Dai, 2024 source
The Introduction and Section 5 identify DeepSeekMoE as a MoE language model scaled to 16B parameters. Section 6 mentions multimodal models built with other MoE architectures, but the paper does not present DeepSeekMoE as advancing multimodal applications or establish the claimed multi-task applications. The GPT-4 portion has a separate citation and is not assessed here. (LLM, conf=0.85)
Experimental — accuracy not yet measured
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationGLaM refined MoE by introducing adaptive expert selection, dynamically routing inputs to the most relevant experts, balancing efficiency and performance.
“GLaM [202] refined MoE by introducing adaptive expert selection, dynamically routing inputs to the most relevant experts, balancing efficiency and performance.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [202] Driess, 2023 source
The Related Work section (p. 3) discusses language models used for embodied planning, while the Summary of Experiments (p. 9) discusses PaLM-E’s transfer and data efficiency. Neither addresses GLaM or mixture-of-experts routing; searches for adaptive expert selection and sparse expert routing found no relevant passage in our copy. (LLM, conf=0.75)
Experimental — accuracy not yet measured
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationThe FLAN dataset covers thousands of task instructions across multiple domains, allowing models to generalize effectively.
“For instance, the FLAN dataset [214] covers thousands of task instructions across multiple domains, allowing models to generalize effectively.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [214] Carion, 2020, End-to-end object detection with transformers source
The Experiments section (p. 8) describes DETR experiments on the COCO object-detection and segmentation datasets, not the FLAN dataset. Searches for FLAN, task instructions, and instruction tuning found no passage addressing the claimed dataset size, domain coverage, or model generalization. (LLM, conf=0.78)
Experimental — accuracy not yet measured
May combine two claims.
PremisecitationLLaVA can adjust to different tasks using natural language instructions, responding to VQA and image captioning and dynamically changing its strategy for new tasks.
“LLaVA [211]'s ability to adjust to different tasks using natural language instructions enables it to not only respond rapidly to traditional visuallanguage tasks like VQA and image captioning, but also to adjust its execution strategy dynamically for new tasks.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [211] Wang, 2020 source
The Introduction (p. 2) describes Linformer’s pretraining and evaluation on language tasks, and the Model section (p. 4) describes its linear self-attention mechanism. Searches for LLaVA, natural-language instructions, VQA, image captioning, and dynamic task strategies found no passage in our copy addressing the attributed claim. (LLM, conf=0.76)
Experimental — accuracy not yet measured
May combine two claims.
PremisecitationInternVL optimizes efficiency through expert scheduling and distributed computing, ensuring dynamic resource allocation for large-scale multimodal data.
“InternVL [212] further optimizes efficiency through expert scheduling and distributed computing, ensuring dynamic resource allocation for large-scale multimodal data.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [212] Dosovitskiy, 2020 source
The scaling study (Section 4.4, p. 8) and computational-cost analysis (Appendix D.5, p. 18) discuss Vision Transformer efficiency, not InternVL. Searches for InternVL, expert scheduling, distributed computing, and dynamic resource allocation found no passage addressing the attributed claim. (LLM, conf=0.78)
Experimental — accuracy not yet measured
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationLLaVA uses strategies similar to RLHF to generate more natural, user-friendly responses.
“LLaVA [211] also uses similar strategies to generate more natural, user-friendly responses.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [211] Wang, 2020 source
The Introduction (p. 1) discusses efficient self-attention, and Downstream Results (p. 7) reports Linformer evaluations. Searches for LLaVA, RLHF, instruction tuning, and user-friendly responses found no passage in our copy addressing the claim. (LLM, conf=0.75)
Experimental — accuracy not yet measured
May combine two claims.
PremisecitationThrough instruction tuning, LLaVA can generate precise image descriptions or answer visual questions according to task demands.
“Through instruction tuning, LLaVA [211] can generate precise image descriptions or answer visual-related questions based on different task demands.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [211] Wang, 2020 source
The Introduction (p. 2) and Downstream Results (p. 7) discuss Linformer’s performance on language tasks, not LLaVA. Searches for LLaVA, instruction tuning, image descriptions, and visual question answering found no passage in our copy addressing the claim. (LLM, conf=0.77)
Experimental — accuracy not yet measured
May combine two claims.
PremisecitationMoE-LLaVA uses a Mixture of Experts architecture that activates only relevant expert networks, reducing computational burden and improving reasoning efficiency on large datasets.
“MoE-LLaVA [208] uses a Mixture of Experts (MoE) architecture, activating only the relevant expert networks to reduce computational burden and improve reasoning efficiency, especially on large datasets.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [208] Brown, 2020, Language models are few-shot learners, Adv. Neural Inf. Process. Syst. source
The Introduction (pp. 3–4) discusses scaling transformer language models and in-context learning, not MoE-LLaVA. A mixture-of-experts paper appears in the References (p. 74), but our copy does not discuss MoE-LLaVA’s architecture, expert activation, computational burden, or reasoning efficiency (searched: MoE-LLaVA and mixture-of-experts efficiency). (LLM, conf=0.78)
Experimental — accuracy not yet measured
May combine two claims.
PremisecitationContext-aware modeling faces challenges such as handling dynamically changing contexts, balancing the significance of different modality data, and preventing overfitting.
“Context-aware modeling faces several challenges, such as handling dynamically changing contexts [223], balancing the significance of different modality data [224], and preventing overfitting.”
Quote in the paperNot a substantial claim2 of 2 sources checked- cites [223] Wei, 2021 source
The Discussion (p. 9) concerns instruction tuning and generalization to unseen tasks, not dynamically changing contexts or the relative importance of different modalities. Appendix B.1 (p. 24) mentions overfitting to instruction templates, but does not address it as a challenge of context-aware modeling. Searches for the context and modality claims found no supporting passage in our copy. (LLM, conf=0.75)
Experimental — accuracy not yet measured - cites [224] Sanh, 2021 source
Citation [223] is attached to the challenge of handling dynamically changing contexts. The Related Work section discusses multitask NLP prompting, and the retrieved QA prompt templates use “context” as an input field; neither addresses contexts changing dynamically. Searches for dynamically changing and adapting to changing contexts found no supporting passage. (LLM, conf=0.72)
Experimental — accuracy not yet measured
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationEdge computing emerged to process data directly at the network edge, handling both downstream cloud data and upstream IoT data.
“To address these, edge computing has emerged to process data directly at the network edge [229], handling both downstream cloud data and upstream IoT data.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [229] Malikireddy, 2024 source
Section 3.1 (System Architecture; page unavailable) describes training user-specific models locally on edge devices, which supports local processing at the edge. It does not describe edge computing’s emergence or say that it handles downstream cloud data and upstream IoT data; searches for those terms found no supporting passage. (LLM, conf=0.78)
Experimental — accuracy not yet measured
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationIn VQA, models like GPT-4V provide accurate responses in diverse domains, and LLaVA enhances visual understanding for detailed answers.
“In VQA, models like GPT-4V provide accurate responses in diverse domains, and LLaVA [211] enhances visual understanding for detailed answers.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [211] Wang, 2020 source
The citation is attached to the claim about LLaVA. The Introduction (p. 1) discusses NLP transformers and question answering, while the downstream results (p. 7) concern text-based tasks. Searches for LLaVA, visual understanding, and visual question answering found no passage addressing LLaVA’s ability to provide detailed answers. (LLM, conf=0.76)
Experimental — accuracy not yet measured
May combine two claims.
PremisecitationBLIP-2 employs the Query Transformer to optimize vision-language information extraction and adapts to instructions for image captioning and VQA.
“For example, BLIP-2 employs the Query Transformer to optimize vision-language information extraction and adapts to instructions for image captioning and VQA.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [218] Liu, 2024, Visual instruction tuning, Adv. Neural Inf. Process. Syst. source
The Vision Encoder section identifies the Q-former as BLIP-2’s method for connecting image and language representations, and Table 3 shows BLIP-2 describing an image. But Table 3 says BLIP-2 focuses on describing the image rather than following the user’s question instruction; the copy does not support the claimed adaptation to instructions for VQA. (LLM, conf=0.88)
Experimental — accuracy not yet measured
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationT0 trains language models using a large-scale instruction dataset, giving them zero-shot and few-shot learning capabilities.
“Similarly, T0 [215] trains language models using a large-scale instruction dataset, endowing them with zero-shot and few-shot learning capabilities.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [215] Fedus, 2022, Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity, J. Mach. Learn. Res. source
The Abstract (p. 1) and Downstream Results (p. 14) describe Switch Transformers, not T0. Searches for T0, instruction datasets, and zero-shot or few-shot learning found no passage in our copy addressing the attributed claim. (LLM, conf=0.77)
Experimental — accuracy not yet measured
May combine two claims.
PremisecitationPaLM-E integrates language, vision, and robotic control signals, optimizing reasoning performance in physical-world tasks.
“PaLM-E [204] integrates language, vision, and robotic control signals, optimizing reasoning performance in physical-world tasks.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [204] Bai, 2023 source
The Introduction and Related Work sections discuss Qwen-VL and other vision-language models, but do not describe PaLM-E. Searches for PaLM-E, robotic control signals, and embodied robotic reasoning found no passage addressing its claimed integration or physical-world reasoning performance; page numbers were not available in our copy. (LLM, conf=0.76)
Experimental — accuracy not yet measured
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationContext-aware modeling is widely used in natural language processing, computer vision, and recommendation systems, where it can substantially boost system performance and user experience.
“Context-aware modeling is widely used across various domains, including natural language processing [219], computer vision [220], and recommendation systems [221], where it can substantially boost both system performance and user experience.”
Quote in the paperNot a substantial claim2 of 3 sources checked- cites [219] Z. Chen, J. Wu, W. Wang, W. Su, G. Chen, S. Xing, M. Zhong, Q. Zhang, X. Zhu, L. Lu, et al., Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 24185–24198. source
- cites [220] Lu, 2024 source
The Related Work section says large language models have advanced natural language processing, but it does not say that context-aware modeling is widely used there or that it boosts performance and user experience. Searches for “context-aware modeling” and “context-aware” found no passage addressing that point. The other domains in the sentence have separate citations. (LLM, conf=0.72)
Experimental — accuracy not yet measured - cites [221] Li, 2023, Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models source
Section 4.3 (page not available in our copy) describes conditioning a vision-language model on a question to focus on informative image regions, and Section 4.1 reports strong visual-question-answering performance. This supports an example of context-dependent modeling in computer vision, the domain attached to this citation, but does not establish that it is widely used or that it improves user experience. (LLM, conf=0.82)
Experimental — accuracy not yet measured
May combine two claims.
PremisecitationKosmos-2 strengthens joint modeling of images and language through self-supervised learning, allowing it to handle more complex tasks such as deep reasoning and generation of image and text.
“Kosmos-2 [210] strengthens the joint modeling of images and language through self-supervised learning, allowing it to handle more complex tasks, such as deep reasoning and generation of image and text.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [210] Beltagy, 2020 source
The Introduction (p. 1) describes Longformer as a model for processing long text sequences, and Related Work (p. 2) discusses long-document transformers. Searches for Kosmos-2, joint image–language modeling, and image-and-text generation found no passage addressing the attributed claim. (LLM, conf=0.77)
Experimental — accuracy not yet measured
May combine two claims.
PremisecitationLLaVA employs instruction tuning, which gives it high flexibility across many tasks.
“LLaVA [211] (Language and Vision Assistant) is an innovative visual-language model that employs instruction tuning, giving the model a high degree of flexibility in handling various tasks.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [211] Wang, 2020 source
The Introduction (p. 2) describes pretraining and fine-tuning Linformer on language tasks, not LLaVA or visual-language instruction tuning. Searches for LLaVA and for visual-language instruction tuning found no passage addressing the claimed task flexibility. (LLM, conf=0.78)
Experimental — accuracy not yet measured
May combine two claims.
PremisecitationIn robotic navigation, PaLM-E predicts optimal paths, avoids obstacles, and provides action recommendations.
“In robotic navigation, PaLM-E [204] predicts optimal paths, avoids obstacles, and provides action recommendations.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [204] Bai, 2023 source
The Introduction and Related Work discuss Qwen-VL and other vision-language models, but do not describe PaLM-E performing robotic navigation. Searches for PaLM-E, path planning, obstacle avoidance, and robot actions found no passage supporting the attributed capabilities. (LLM, conf=0.76)
Experimental — accuracy not yet measured
May combine two claims.
PremisecitationLlama uses large-scale pretraining to improve text understanding and generation, and when combined with visual data it improves visual reasoning and image-to-text generation.
“Llama [209] uses large-scale pretraining to boost text understanding and generation, improving performance in tasks like visual reasoning and image-to-text generation when combined with visual data.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [209] Kitaev, 2020 source
The Introduction (p. 2) reports Reformer experiments on text and image generation separately, while Related Work (p. 7) discusses large language models generally. A search for Llama, visual reasoning, and image-to-text generation found no passage addressing the attributed claim. (LLM, conf=0.76)
Experimental — accuracy not yet measured
May combine two claims.
PremisecitationGemini extends this approach by jointly training large-scale data from images, text, and audio, enhancing cross-modal reasoning, particularly excelling in visual reasoning tasks.
“Gemini [11] further extends this approach by jointly training largescale data from images, text, and audio, enhancing the model's cross-modal reasoning ability, particularly excelling in visual reasoning tasks.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [11] Team, 2023 source
May combine two claims.
PremisecitationPaLM-E combines improvements in multimodal learning and reasoning capabilities, enabling it to process multimodal data such as images, text, and audio, performing well in cross-modal reasoning and generation tasks.
“Additionally, PaLM-E [204] combines improvements in multimodal learning and reasoning capabilities, enabling it to process multimodal data such as images, text, and audio, performing well in cross-modal reasoning and generation tasks.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [204] Bai, 2023 source
The Introduction and Related Work discuss Qwen-VL and other vision-language models, but do not describe PaLM-E. Searches for “PaLM-E” and for image, text, audio, and cross-modal reasoning capabilities found no passage supporting the attributed claim. (LLM, conf=0.78)
Experimental — accuracy not yet measured
May combine two claims.
PremisecitationOvis1.6 balances computational efficiency and reasoning accuracy, making it suitable for real-time visual-language tasks.
“Ovis1.6 [213] balances computational efficiency and reasoning accuracy, making it ideal for real-time visual-language tasks.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [213] Z. Liu, Y. Lin, Y. Cao, H. Hu, Y. Wei, Z. Zhang, S. Lin, B. Guo, Swin transformer: Hierarchical vision transformer using shifted windows, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021, pp. 10012–10022. source
The Introduction describes Swin Transformer as a computer-vision backbone, and the Ablation Study reports speed improvements for Swin models. Searches for Ovis1.6 and real-time visual-language reasoning found no passage addressing that model or its claimed balance of efficiency and accuracy. (LLM, conf=0.78)
Experimental — accuracy not yet measured
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationnuScenes offers data from six cameras, five radars, and one LiDAR, covering 360° views and supporting 3D detection and tracking across 23 categories.
“nuScenes [23] offers data from six cameras, five radars, and one LiDAR, covering 360° views and supporting 3D detection and tracking tasks across 23 categories.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [23] H. Caesar, V. Bankiti, A.H. Lang, S. Vora, V.E. Liong, Q. Xu, A. Krishnan, Y. Pan, G. Baldan, O. Beijbom, nuscenes: A multimodal dataset for autonomous driving, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 11621–11631. source
May combine two claims.
PremisecitationDROID covers 564 scenes and 86 tasks, combining multi-view camera streams and language annotations to enhance robots' adaptability to complex scenes.
“The former covers 564 scenes and 86 tasks, combining multi-view camera streams and language annotations to enhance robots' adaptability to complex scenes.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [25] Khazatsky, 2024 source
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationNYU Depth V2 offers 1,449 images from 464 indoor scenes with detailed semantic segmentation and object relationship annotations, instrumental for physical support reasoning and cluttered environments.
“NYU Depth V2 [239], as one of the earliest RGB-D datasets, offers 1,449 images from 464 diverse indoor scenes with detailed semantic segmentation and object relationship annotations, making it instrumental for research on physical support reasoning and cluttered environments.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [239] Mehta, 2021 source
The Mobile Semantic Segmentation section (p. 8) describes experiments on PASCAL VOC 2012, and the unseen-dataset section (p. 23) discusses MS-COCO. Searches for NYU Depth V2, the 1,449-image and 464-scene figures, and object-relationship annotations found no passage in our copy addressing the claim. (LLM, conf=0.78)
Experimental — accuracy not yet measured
May combine two claims.
PremisecitationR3ED provides 5,800 point clouds and 22,400 object annotations collected using IntelRealsense D455 sensors, supporting active visual learning tasks.
“Unlike the aforementioned datasets, R3ED [251] focuses on real-world data, providing 5,800 point clouds and 22,400 object annotations collected using IntelRealsense D455 sensors.It supports active visual learning tasks, such as next-best-view planning and 3D object detection.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [251] Cai, 2024 source
Section 4 and Appendix A.3 describe R2Sim, a simulated grasp dataset, not R3ED. Section 5.3 mentions an Intel RealSense D455 camera for grasping experiments, but our copy does not address R3ED, its claimed counts, or its use for active visual learning (searched: R3ED, 5,800, 22,400, and next-best-view planning). (LLM, conf=0.77)
Experimental — accuracy not yet measured
May combine two claims.
PremisecitationR2R was the first dataset to integrate natural language modalities, introducing the Vision-and-Language Navigation task.
“Building upon Matterport3D [26], Room-to-Room (R2R) [251] was the first dataset to integrate natural language modalities, introducing the Vision-and-Language Navigation (VLN) task.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [251] Cai, 2024 source
The cited paper’s Section 4 describes R2Sim, a simulated dataset for grasp detection, not Room-to-Room. Searches for Room-to-Room, Matterport3D, natural-language instructions, and Vision-and-Language Navigation found no passage addressing the claim. The copy is marked complete, but its silence does not establish a contrary finding. (LLM, conf=0.78)
Experimental — accuracy not yet measured
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationTinyBERT reduces the size of transformer-based models via knowledge distillation, making them suitable for mobile deployment.
“On the other hand, lightweight natural language processing (NLP) models such as TinyBERT [232] focus on reducing the size of transformer-based models via knowledge distillation, making them suitable for mobile deployment.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [232] Y.-H.H. Tsai, S. Bai, P.P. Liang, J.Z. Kolter, L.-P. Morency, R. Salakhutdinov, Multimodal transformer for unaligned multimodal language sequences, in: Proceedings of the Conference. Association for Computational Linguistics. Meeting, vol. 2019, 2019, p. 6558. source
The Introduction and Related Works sections describe MulT, a transformer for unaligned multimodal sequences, rather than TinyBERT. Searches for TinyBERT, knowledge distillation, model compression, and mobile deployment found no passage addressing the attributed claim; page numbers were not available in our copy. (LLM, conf=0.76)
Experimental — accuracy not yet measured
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationZhang et al. proposed a non-intrusive appliance state detection system that uses a single sensor to monitor multiple appliances via a cloud-edge-device collaborative framework.
“Zhang et al. proposed a non-intrusive appliance state detection system [235] that uses a single sensor to monitor multiple appliances via a cloud-edgedevice collaborative framework.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [235] R. Vacareanu, M.A. Valenzuela-Escarcega, R. Sharp, M. Surdeanu, An unsupervised method for learning representations of multi-word expressions for semantic classification, in: Proceedings of the 28th International Conference on Computational Linguistics, 2020, pp. 3346–3356. source
The Abstract and Introduction describe a method for learning representations of multi-word expressions, not an appliance-monitoring system. Searches for non-intrusive appliance detection, a single sensor, and a cloud-edge-device framework found no passage addressing the claim. (LLM, conf=0.78)
Experimental — accuracy not yet measured
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationHypersim generates 77,400 indoor scene images with pixel-level annotations, light decomposition, and surface normals, serving as a tool for Sim-to-Real transfer research.
“In addition, the Hypersim dataset [240], utilizing high-fidelity synthetic techniques, generates 77,400 indoor scene images with detailed pixel-level annotations, light decomposition, and surface normals, serving as an important tool for simulation-to-reality (Sim-to-Real) transfer research.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [240] Jiao, 2019 source
The TinyBERT Settings and GLUE Details sections discuss a distilled language model and natural-language benchmarks, not the Hypersim dataset. Searches for Hypersim, 77,400 images, synthetic indoor scenes, surface normals, and simulation-to-reality transfer found no passage addressing the claim. (LLM, conf=0.78)
Experimental — accuracy not yet measured
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationScanNet provides 2.5 million RGB-D frames across 1,513 scenes with crowdsourced annotations for semantic segmentation and 3D modeling.
“ScanNet [238], through its 2.5 million RGB-D frames across 1,513 scenes, utilizes crowdsourced annotations to provide a large-scale, high-quality dataset for semantic segmentation and 3D modeling.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [238] Tan, 2019, Efficientnet: Rethinking model scaling for convolutional neural networks source
The transfer-learning dataset table (Section 5.2, p. 8) lists image-classification datasets, not ScanNet. Searches for ScanNet, 2.5 million RGB-D frames, 1,513 scenes, and semantic segmentation found no passage in our copy addressing the claimed dataset or its annotations. (LLM, conf=0.78)
Experimental — accuracy not yet measured
May combine two claims.
PremisecitationOn the R2R navigation benchmark, early sequence-based models Seq2Seq and SF had poor global path optimization and performed poorly in unseen environments.
“Early sequence-based models like Seq2Seq [265] and SF [266] struggled with global path optimization, leading to poor performance in unseen environments.”
Quote in the paperNot a substantial claim1 of 2 sources checked- cites [265] H. Liu, Y. Teng, T. Lu, H. Wang, L. Wang, Sparsebev: High-performance sparse 3d object detection from multi-camera videos, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 18580–18590. source
- cites [266] Liu, 2024, Ray denoising: Depth-aware hard negative sampling for multi-view 3d object detection source
Section 4.1 (Dataset and Metrics; page unavailable) identifies nuScenes and Argoverse 2 as the paper’s evaluation datasets, not R2R. Searches for Seq2Seq, SF, and global path optimization found no passage addressing navigation performance in unseen environments. (LLM, conf=0.78)
Experimental — accuracy not yet measured
May combine two claims.
PremisecitationR2R provides 21,567 crowd-sourced navigation instructions paired with panoramic RGB-D data, enabling agents to complete path navigation following language commands.
“R2R [251] provides 21,567 crowd-sourced navigation instructions paired with panoramic RGB-D data, enabling agents to complete path navigation following language commands.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [251] Cai, 2024 source
Section 4, “R2Sim Dataset,” describes simulated RGB-D images and grasp annotations, not the R2R navigation dataset. Searches for “21,567 crowd-sourced navigation instructions” and “R2R Room-to-Room navigation” found no passage addressing the claimed instructions or language-guided navigation. Our copy therefore cannot verify what the citing sentence attributes to this source. (LLM, conf=0.77)
Experimental — accuracy not yet measured
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationLu et al. introduced CLONE, a framework for cooperative learning across edge devices, using real-world data from an electric vehicle company to focus on battery and component failures.
“Lu et al. introduced CLONE [234], a framework for cooperative learning across edge devices, using real-world data from an electric vehicle company to focus on battery and component failures.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [234] Ahn, 2024 source
The cited paper’s discussion of multimodal models and reinforcement learning from AI feedback, and its References section, do not address CLONE. Searches of our full-text copy for cooperative learning across edge devices and for electric-vehicle battery or component failures found no relevant passage. (LLM, conf=0.77)
Experimental — accuracy not yet measured
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationZhang et al. proposed a cloud-edge collaborative system that uses vehicle position, speed, and acceleration to model driving behavior and detect abnormal actions like speeding or drunk driving.
“To address this, Zhang et al. proposed a cloud-edge collaborative system [233], which uses data like vehicle position, speed, and acceleration to model driving behavior and detect abnormal actions like speeding or drunk driving.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [233] Dar, 2025, A social context-aware graph-based multimodal attentive learning framework for disaster content classification during emergencies, Expert Syst. Appl. source
Our copy’s Conclusion describes CrisisSpot as a method for classifying disaster-related social media content, not a vehicle-behavior system. Searches for cloud-edge collaboration, vehicle position, speed, acceleration, speeding, and drunk driving found no passage addressing the attributed claim. (LLM, conf=0.76)
Experimental — accuracy not yet measured
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationSemanticKITTI focuses on semantic segmentation of LiDAR point clouds and provides an important benchmark for sequence-level semantic scene completion.
“Additionally, SemanticKITTI [24] focuses on semantic segmentation of LiDAR point clouds and provides an important benchmark for sequence-level semantic scene completion.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [24] J. Behley, M. Garbade, A. Milioto, J. Quenzel, S. Behnke, C. Stachniss, J. Gall, Semantickitti: A dataset for semantic scene understanding of lidar sequences, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, 2019, pp. 9297–9307. source
May combine two claims.
PremisecitationRLBench is restricted to simulated tasks because of CoppeliaSim physics limitations, creating challenges for sim-to-real transfer.
“However, due to the limitations of Cop-peliaSim's physics engine, RLBench [288] remains restricted to purely simulated tasks, posing challenges for sim-to-real transfer.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [288] H. Wang, W. Wang, W. Liang, C. Xiong, J. Shen, Structured scene memory for vision-language navigation, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021, pp. 8455–8464. source
Section 4.1 describes experiments on the R2R vision-language navigation dataset, and Section 4.2 describes R4R; neither addresses RLBench. Searches for RLBench, CoppeliaSim physics limitations, and sim-to-real transfer found no passage in our copy that speaks to the claim. (LLM, conf=0.76)
Experimental — accuracy not yet measured
May combine two claims.
PremisecitationEnvEdit reduces overfitting and improves robustness in unseen scenarios through environment augmentation.
“EnvEdit [279], through environment augmentation, reduces overfitting and enhances robustness in unseen scenarios.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [279] Fried, 2018, Speaker-follower models for vision-and-language navigation, Adv. Neural Inf. Process. Syst. source
Section 4.2.1 (p. 7) reports that speaker-driven data augmentation improved success on unseen validation environments, supporting the generalization part of the claim. But Section 4.1 (p. 6) describes augmenting routes sampled from training environments, not environment augmentation; our copy does not identify the method as EnvEdit or report a reduction in overfitting. (LLM, conf=0.86)
Experimental — accuracy not yet measured
May combine two claims.
PremisecitationGemBench has advantages in multimodal fusion and generalization but remains simulation-constrained and lacks real-world validation.
“While GemBench [289] presents unique advantages in multimodal fusion and generalization, it also remains constrained to simulation and lacks real-world validation.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [289] P.-L. Guhur, M. Tapaswi, S. Chen, I. Laptev, C. Schmid, Airbert: In-domain pretraining for vision-and-language navigation, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021, pp. 1634–1643. source
The Experimental Results (Section 5) and Conclusion (Section 6) discuss Airbert’s performance on vision-and-language navigation tasks, not GemBench. A search for GemBench and for simulation or real-world validation found no passage in our copy assessing GemBench’s advantages or limitations. (LLM, conf=0.78)
Experimental — accuracy not yet measured
May combine two claims.
PremisecitationRLBench is one of the most widely used manipulation benchmarks, offering over 100 tasks from basic grasping to tool use and assembly, with task diversity, scalability, and demonstration datasets suitable for reinforcement learning, imitation learning, and traditional control.
“RLBench [288] is one of the most widely used robotic manipulation benchmarks, offering 100+ tasks ranging from basic object grasping to complex tool use and assembly tasks.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [288] H. Wang, W. Wang, W. Liang, C. Xiong, J. Shen, Structured scene memory for vision-language navigation, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021, pp. 8455–8464. source
Split into 2 atoms — 0 confirmed, 0 partially supported, 0 disputed, 0 misattributed, 2 unverifiable. Expand the claim to see per-atom verdicts.
Experimental — accuracy not yet measured
May combine two claims.
PremisecitationKitchenShift is useful for benchmarking imitation learning and reinforcement learning but does not support real-world reproducibility; its domain shift testing provides insight into real-world uncertainty modeling.
“KitchenShift [292] is particularly useful for benchmarking imitation learning (IL) and reinforcement learn-ing (RL) strategies, but like RLBench [288], it does not support real-world reproducibility, though its domain shift testing provides insight into real-world uncertainty modeling.”
Quote in the paperNot a substantial claim1 of 2 sources checked- cites [292] J. Li, H. Tan, M. Bansal, Envedit: Environment editing for vision-and-language navigation, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 15407–15417. source
- cites [288] H. Wang, W. Wang, W. Liang, C. Xiong, J. Shen, Structured scene memory for vision-language navigation, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021, pp. 8455–8464. source
Sections 4.1 and 4.2 evaluate vision-language navigation on R2R and R4R, not KitchenShift. Searches for KitchenShift, domain shift, and real-world reproducibility found no passage addressing the claimed benchmark or its properties; the paper’s mention of a kitchen is a navigation example. (LLM, conf=0.78)
Experimental — accuracy not yet measured
May combine two claims.
PremisecitationChat-Scene and Scene-LLM optimize structured embeddings and object-based feature representations, excelling in object-related queries and scene reasoning.
“Chat-Scene [286] and Scene-LLM [285] optimize structured embeddings and object-based feature representations, excelling in object-related queries and scene reasoning tasks.”
Quote in the paperSubstantial claim1 of 2 sources checked- cites [286] D. An, Y. Qi, Y. Huang, Q. Wu, L. Wang, T. Tan, Neighbor-view enhanced model for vision and language navigation, in: Proceedings of the 29th ACM International Conference on Multimedia, 2021, pp. 5101–5109. source
- cites [285] Wang, 2020, Active visual information gathering for vision-language navigation source
The Methodology (p. 3) describes a vision-language navigation agent, and the Experimental Setup (p. 11) describes viewpoint and instruction embeddings. Searches for Chat-Scene, Scene-LLM, object-based features, and object-related queries found no passage in our copy addressing the attributed claim. (LLM, conf=0.77)
Experimental — accuracy not yet measured
May combine two claims.
PremisecitationRLBench lacks real-world reproducibility because it is simulation-only, restricting its applicability in sim-to-real transfer.
“However, as RLBench is limited to simulation, it lacks real-world reproducibility, restricting its applicability in sim-to-real transfer.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [288] H. Wang, W. Wang, W. Liang, C. Xiong, J. Shen, Structured scene memory for vision-language navigation, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021, pp. 8455–8464. source
The Results sections (§§4.1–4.2; page numbers unavailable) evaluate vision-language navigation on R2R and R4R, not RLBench. Searches for RLBench, simulation-only limitations, real-world reproducibility, and sim-to-real transfer found no passage addressing the claim. (LLM, conf=0.76)
Experimental — accuracy not yet measured
May combine two claims.
Premisecitation3D-VLP uses a context-aware alignment strategy and performs well in dense scene description.
“3D-VLP [281] employs a context-aware alignment strategy, performing well”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [281] Ma, 2019 source
The Introduction (p. 1) and Visual and Textual Co-Grounding section (p. 3) describe a self-monitoring vision-and-language navigation agent, not 3D-VLP. A search for 3D-VLP, context-aware alignment, and dense scene description found no passage in our copy addressing the attributed claim. (LLM, conf=0.78)
Experimental — accuracy not yet measured
May combine two claims.
PremisecitationBEVBert, which integrates hybrid topometric pretraining, achieves state-of-the-art SR and SPL and shows superior generalization in seen and unseen environments.
“BEVBert [116], integrating hybrid topometric pretraining, achieves state-of-the-art SR and SPL, demonstrating superior generalization across seen and unseen environments.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [116] Yan, 2024 source
The Comparison with State-of-the-Arts section (pp. 5–6) reports SR and SPL for InstruGen and other named methods, but does not discuss BEVBert. Searches for BEVBert, hybrid topometric pretraining, and its seen/unseen results found no passage addressing the attributed claim. (LLM, conf=0.76)
Experimental — accuracy not yet measured
May combine two claims.
PremisecitationVision-language pretraining methods improve cross-modal alignment, and PRET (DINOv2) achieves higher SR and SPL through self-supervised learning.
“PRET (DINOv2) [119] and other vision-language pretraining (VLP) methods improve cross-modal alignment, with PRET (DINOv2) achieving higher SR and SPL through self-supervised learning.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [119] An, 2022 source
The Visual Representation in Vision-Language Pre-training discussion says grid-based pre-training enables multi-grained vision-language alignments, and the Conclusion (p. 8) says BEVBert enhances spatial-aware cross-modal reasoning. The paper reports higher SR and SPL for BEVBert, not PRET (DINOv2); the searched passages do not support the claim that PRET achieves those results through self-supervised learning. (LLM, conf=0.88)
Experimental — accuracy not yet measured
May combine two claims.
PremisecitationRLBench is built on PyRep (CoppeliaSim), provides diverse manipulation tasks with high-dimensional visual inputs, and is suited for end-to-end policy learning.
“RLBench [288] is built on PyRep (CoppeliaSim), a simulator that provides a diverse set of robotic manipulation tasks with high-dimensional visual inputs, making it well-suited for endto-end policy learning.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [288] H. Wang, W. Wang, W. Liang, C. Xiong, J. Shen, Structured scene memory for vision-language navigation, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021, pp. 8455–8464. source
The paper’s §§4.1–4.2 discuss vision-language navigation on R2R and R4R, while §3.4 describes training a navigation agent. Searches for RLBench, PyRep, CoppeliaSim, manipulation tasks, and end-to-end policy learning found no passage addressing the attributed description. (LLM, conf=0.78)
Experimental — accuracy not yet measured
May combine two claims.
PremisereasoningOn ScanQA, LLaVA-3D and 3D-LLM show strong generalization and excel in question-answering accuracy and spatial reasoning, mainly due to optimized multi-view feature extraction and 3D spatial embedding.
“Among them, LLaVA-3D [280] and 3D-LLM [282] demonstrate strong generalization capabilities, excelling in question-answering accuracy and spatial reasoning, primarily due to their optimized multi-view feature extraction and 3D spatial embedding mechanisms.”
Quote in the paperNot a substantial claim1 of 2 sources checked- cites [280] X. Wang, Q. Huang, A. Celikyilmaz, J. Gao, D. Shen, Y.-F. Wang, W.Y. Wang, L. Zhang, Reinforced cross-modal matching and self-supervised imitation learning for vision-language navigation, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, pp. 6629–6638. source
- cites [282] Tan, 2019 source
The supplied passages contain bibliographic metadata for a paper on navigation in unseen environments, not findings about ScanQA, LLaVA-3D, or 3D-LLM. Searches of our indexed copy for those names and for question-answering accuracy, spatial reasoning, multi-view features, and 3D embeddings returned no passage addressing the claim. (LLM, conf=0.76)
Experimental — accuracy not yet measured
May combine two claims.
PremisecitationGemBench, built on RLBench, focuses on task generalization with zero-shot setups and vision-language extensions to assess performance on previously unseen tasks and objects.
“In contrast, GemBench [289], built upon RLBench, focuses on task generalization, incorporating zero-shot task setups and vision-language extensions to assess robotic performance in previously unseen tasks and objects.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [289] P.-L. Guhur, M. Tapaswi, S. Chen, I. Laptev, C. Schmid, Airbert: In-domain pretraining for vision-and-language navigation, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021, pp. 1634–1643. source
The Introduction and Section 3 describe Airbert and its BnB dataset for vision-and-language navigation. Section 5 evaluates navigation on R2R and REVERIE, not GemBench. Searches for GemBench, RLBench, zero-shot task setups, and unseen robotic tasks or objects found no passage addressing the attributed description. (LLM, conf=0.78)
Experimental — accuracy not yet measured
May combine two claims.
PremisecitationCurrent mainstream VLMs are pre-trained on large-scale Transformer architectures with billions or tens of billions of parameters, requiring large computational resources and time, limiting training efficiency and scale-up and causing deployment bottlenecks.
“Current mainstream VLMs [209,206,204,208] are usually pre-trained based on large-scale Transformer architectures, which usually have billions or even tens of billions of parameters, and pre-training on large-scale datasets requires a large amount of computational resources and time costs, which limits the training efficiency and scale-up ability of the models, and makes the model deployment face”
Quote in the paperSubstantial claim3 of 4 sources checked- cites [209] Kitaev, 2020 source
Split into 2 atoms — 0 confirmed, 2 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.
Experimental — accuracy not yet measured - cites [206] Lin, 2024
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites [204] Bai, 2023 source
Split into 2 atoms — 0 confirmed, 2 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.
Experimental — accuracy not yet measured - cites [208] Brown, 2020, Language models are few-shot learners, Adv. Neural Inf. Process. Syst. source
Split into 2 atoms — 0 confirmed, 2 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.
Experimental — accuracy not yet measured
May combine two claims.
InterpretationcitationLow-quality data challenges include noise pollution, missing modalities, modal quality imbalance, and dynamic changes in modal quality, and are widely found in autonomous driving, medical image analysis, and speech recognition.
“The challenges of low-quality data are mainly reflected in various aspects such as noise pollution, missing modalities, modal quality imbalance, and dynamic changes in modal quality, etc., which are widely found in many application scenarios such as autonomous driving [81,88,257], medical image analysis, speech recognition [296], etc., and pose great challenges to the validity and reliability of”
Quote in the paperNot a substantial claim3 of 4 sources checked- cites [81] Li, 2022, Unifying voxel-based representation with transformer for 3d object detection, Adv. Neural Inf. Process. Syst. source
- cites [88] Yoo, 2020, 3D-cvf: Generating joint camera and lidar features using cross-view spatial feature fusion for 3d object detection source
The Introduction (pp. 1–2) identifies autonomous driving as an application of camera–LiDAR detection and describes limitations of each sensor. It also reports that camera data compensate for lower-resolution LiDAR data (p. 4). This supports an autonomous-driving example of uneven sensor quality, but the paper does not establish that the full list of low-quality-data challenges occurs in that setting. (LLM, conf=0.82)
Experimental — accuracy not yet measured - cites [257] A.D. Vuong, M.N. Vu, B. Huang, N. Nguyen, H. Le, T. Vo, A. Nguyen, Language-driven Grasp Detection, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 17902–17912. source
- cites [296] Wang, 2025, PanoGen++: Domain-adapted text-guided panoramic environment generation for vision-and-language navigation, Neural Netw. source
The Limitation and Discussion section (p. 11) describes blurriness and geometric inconsistencies in generated panoramas, while the Comparisons section (p. 8) mentions Cityscapes only as a dataset not used for training. Searches for the listed low-quality-data challenges and their occurrence in autonomous driving, medical image analysis, or speech recognition found no passage addressing that claim. (LLM, conf=0.76)
Experimental — accuracy not yet measured
May combine two claims.
PremisecitationExisting large-scale visual language datasets such as Visual Genome, Conceptual Captions, and LAION-400m generally have limited size, high annotation noise, insufficient semantic diversity, and expensive annotation.
“However, existing large-scale visual language datasets (e.g., Visual Genome [311], Conceptual Captions [312], and LAION-400m [313]) generally suffer from limited size, high annotation noise, insufficient semantic diversity, and expensive annotation.”
Quote in the paperSubstantial claim3 of 3 sources checked- cites [311] Pumacay, 2024 source
The References (p. 12) mention Visual Genome, but do not assess its size, annotation noise, semantic diversity, or annotation cost. The Baselines (p. 6) discuss robotic-manipulation models rather than these datasets; searches for the named datasets and their alleged limitations found no passage addressing the claim. (LLM, conf=0.78)
Experimental — accuracy not yet measured - cites [312] Jiang, 2022 source
The Introduction (p. 2) describes VIMA’s robot-manipulation model, and the Evaluation Results (pp. 6–7) discuss its performance on VIMA-BENCH. Searches for the named visual-language datasets and for annotation noise, semantic diversity, and annotation cost found no passage in our copy assessing those datasets or stating the claimed limitations. (LLM, conf=0.75)
Experimental — accuracy not yet measured - cites [313] Hua, 2024 source
The Introduction (pp. 2–3) discusses generating robotic simulation tasks and reducing labeling effort for simulated demonstrations, not the properties of visual-language datasets. Searches for Visual Genome, Conceptual Captions, LAION-400m, annotation noise, and semantic diversity found no passage assessing the named datasets or the claimed limitations. (LLM, conf=0.75)
Experimental — accuracy not yet measured
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationIn multispectral image fusion, noise can be removed by jointly optimising different modal data through a variational model.
“e.g., in multispectral image fusion tasks, noise is removed by jointly optimising different modal data through a variational model [298].”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [298] W. Mo, Y. Liu, Bridging the gap between 2d and 3d visual question answering: A fusion approach for 3d vqa, in: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 38, 2024, pp. 4261–4268, 5. source
The 2D-3D VQA Model and Question Answering sections describe fusing 2D images with 3D scene features and jointly optimizing answer predictions, not removing noise from multispectral images. Searches for multispectral fusion, denoising, and variational optimization found no passage addressing the claim; page numbers were not available in our copy. (LLM, conf=0.76)
Experimental — accuracy not yet measured
May combine two claims.
PremisecitationModality-specific encoders use Transformer for text, CNN for image, and RNN for audio, then align features through cross-modal attention.
“using Transformer for text, CNN for image, RNN for audio, etc., and then performs feature alignment through Cross-modal Attention [306,307].”
Quote in the paperNot a substantial claim1 of 2 sources checked- cites [306] James, 2020, Rlbench: The robot learning benchmark & learning environment, IEEE Robot. Autom. Lett. source
- cites [307] Garcia, 2024 source
Section B, “3D-LOTUS policy” (page not available), describes a CLIP text encoder and cross-attention that lets point-cloud features attend to word embeddings. It does not describe the claimed combination of a Transformer text encoder, CNN image encoder, and RNN audio encoder; its visual representation is a point cloud encoded by a point-cloud transformer. (LLM, conf=0.88)
Experimental — accuracy not yet measured
May combine two claims.
PremisecitationSolutions to missing modalities are imputation-based and imputation-free; imputation-based methods recover missing data via matrix decomposition, graph learning, or GANs, while imputation-free methods use available modalities through latent representation learning and deep learning.
“Solutions fall into imputation-based [302] and imputation-free methods [303].Imputation-based methods recover missing data via matrix decomposition, graph learning, or GANs, while imputation-free approaches leverage available modalities through latent representation learning and deep learning.”
Quote in the paperNot a substantial claim2 of 2 sources checked- cites [302] H. Huang, Y. Chen, Z. Wang, R. Huang, R. Xu, T. Wang, L. Liu, X. Cheng, Y. Zhao, J. Pang, et al., Chat-scene: Bridging 3d scene and large language models with object identifiers, in: The Thirty-Eighth Annual Conference on Neural Information Processing Systems, 2024. source
Split into 2 atoms — 0 confirmed, 0 partially supported, 0 disputed, 0 misattributed, 2 unverifiable. Expand the claim to see per-atom verdicts.
Experimental — accuracy not yet measured - cites [303] Hong, 2023, 3d-llm: Injecting the 3d world into large language models, Adv. Neural Inf. Process. Syst. source
Split into 2 atoms — 0 confirmed, 0 partially supported, 0 disputed, 0 misattributed, 2 unverifiable. Expand the claim to see per-atom verdicts.
Experimental — accuracy not yet measured
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationEncoder-decoder architectures and Transformer-based structures are widely used to model dependencies between modalities.
“For example, encoder-decoder architectures and Transformer-based structures are widely used to model dependencies between modalities [12,13];”
Quote in the paperNot a substantial claim0 of 2 sources checked- cites [12] J. Xu, Z. Xiong, S.P. Bhattacharyya, PIDNet: A Real-Time Semantic Segmentation Network Inspired by PID Controllers, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR, 2023, pp. 19529–19539. source
- cites [13] X. Bai, Z. Hu, X. Zhu, Q. Huang, Y. Chen, H. Fu, C.-L. Tai, Transfusion: Robust lidar-camera fusion for 3d object detection with transformers, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 1090–1099. source
May combine two claims.
InterpretationreasoningIntegrating diverse modalities allows robots to achieve more robust scene understanding, stronger task generalization, and more natural human-robot interaction in complex environments.
“The integration of diverse modalities enables robots to achieve more robust scene understanding, stronger task generalization, and more natural human-robot interaction in complex environments.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.
Interpretationnone givenFoundation model-driven multimodal pretraining paradigms are gradually being integrated into robot vision systems and show potential for scalability and generalization.
“At the same time, foundation modeldriven multimodal pretraining paradigms are gradually being integrated into robot vision systems, showing great potential for scalability and generalization.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.
Premisenone givenTraditional multimodal fusion methods offer a basic theoretical framework but struggle with complex data.
“Traditional multimodal fusion methods offer a basic theoretical framework but struggle with complex data.”
Quote in the paperNot a substantial claimMay combine two claims.
PremisecitationLi et al. presented an evolutionary overview of mainstream VLM architectures but did not explore their use in navigation or robotic manipulation.
“For example, Li et al. [29] presented an evolutionary overview of mainstream VLM architectures, but did not explore their use in tasks like navigation or robotic manipulation.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [29] Li, 2025
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.
Premisenone givenWith deep neural networks, feature extraction, modality interaction, and decision-making have become deeply integrated, making fusion stages less distinct.
“With deep neural networks, feature extraction, modality interaction, and decision-making have become deeply integrated, making fusion stages less distinct.”
Quote in the paperNot a substantial claimMay combine two claims.
PremisecitationThis has driven a shift from explicit to implicit fusion, where network design inherently captures modality relationships, enhancing deep semantic extraction.
“This has driven a shift from explicit to implicit fusion [37], where network design inherently captures modality relationships, enhancing deep semantic extraction.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [37] Geiger, 2012, Are we ready for autonomous driving? The KITTI vision benchmark suite source
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Premisenone givenOccupancy prediction is vital for multimodal scene understanding, predicting semantic labels for occupied spatial regions, crucial for autonomous navigation, obstacle avoidance, and decision-making.
“Occupancy Prediction is vital for multimodal scene understanding, predicting semantic labels for occupied spatial regions, which is crucial for autonomous navigation, obstacle avoidance, and decision-making.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing), S4 (based).
PremisereasoningMore recent approaches have integrated attention mechanisms with the encoder-decoder framework to enhance flexibility and accuracy of information fusion.
“However, with the advancement of research, more recent approaches have integrated various attention mechanisms with the encoder-decoder framework to enhance the flexibility and accuracy of information fusion.”
Quote in the paperNot a substantial claimMay combine two claims.
Premisenone givenMany occupancy prediction methods rely mainly on visual data, leading to performance drops in adverse conditions like rain or fog.
“However, many occupancy prediction methods rely mainly on visual data, leading to performance drops in adverse conditions like rain or fog.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing), S4 (based).
Premisenone givenEarly multimodal fusion methods primarily relied on the encoder-decoder framework to extract and combine information from different modalities.
“Early multimodal fusion methods primarily relied on the encoder-decoder framework to extract and combine information from different modalities.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Premisenone givenIn the encoder-decoder framework, the encoder independently processes each modality to generate high-dimensional representations, and the decoder fuses these features to produce the final output.
“In multimodal fusion, the encoder-decoder framework first extracts features from different modalities (e.g., image, text, etc.), with the encoder independently processing each modality to generate high-dimensional representations.The decoder then fuses these features to produce the final output.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisereasoningAttention mechanisms' parallel computing capability accelerates processing of large-scale datasets, making them suitable for real-time scene understanding.
“Its parallel computing capability also accelerates the processing of large-scale datasets, making it particularly suitable for real-time scene understanding tasks.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim. May combine two claims.
Premisenone givenBordes et al.'s introductory survey covers training paradigms and evaluation methods but is mainly a beginner-friendly guide and fails to address core challenges of multimodal fusion in robotic vision.
“The introductory survey by Bordes et al. [30] covers training paradigms and evaluation methods, but is mainly intended as a beginnerfriendly guide and fails to address the core challenges of multimodal fusion in robotic vision.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [30] Wu, 2024, Embodied navigation with multi-modal information: A survey from tasks to methodology, Inf. Fusion source
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisereasoningAttention mechanisms guide models to focus on key features across modalities, improving multimodal learning performance, especially in complex tasks.
“These attention mechanisms effectively guide the model to focus on key features across modalities, thereby improving the performance of multimodal learning, especially in handling complex tasks.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.
PremisereasoningAttention mechanisms reduce complexity of data preprocessing and strengthen fusion of multimodal data, becoming mainstream in multimodal fusion.
“By utilizing a unified model architecture, the attention mechanism reduces the complexity of data preprocessing and strengthens the fusion of multimodal data, becoming the mainstream method in the field of multimodal fusion.”
Quote in the paperNot a substantial claimMay combine two claims.
PremisecitationMRFTrans fuses semantic, geometric, and depth information for monocular 3D scene completion, achieving state-of-the-art on SemanticKITTI and NYUv2.
“MRFTrans [4], a transformer-based framework, fuses semantic, geometric, and depth information to address the challenge of monocular 3D scene completion, achieving state-of-the-art performance on SemanticKITTI and NYUv2 datasets by effectively modeling long-range dependencies and refining depth estimates.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [4] Xu, 2024, MRFTrans: Multimodal representation fusion transformer for monocular 3D semantic scene completion, Inf. Fusion source
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationPost fusion combines decision-level independent processing of modalities.
“Post fusion is a key method in multimodal analysis, which combines the results of decision level indepen-dent processing of modalities [45].”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [45] P. Li, X. Li, Multimodal Fusion with Co-attention Mechanism, in: 2020 IEEE 23rd International Conference on Information Fusion, FUSION, 2020, pp. 1–8. source
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationEarly fusion directly integrates multimodal inputs at the data level, such as concatenating RGB images and depth maps before model input.
“Early fusion directly integrates multimodal inputs at the data level, such as concatenating RGB [40] images and depth maps before feeding them into the model.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [40] Zhao, 2024, Deep multimodal data fusion, ACM Comput. Surv. source
Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).
Premisenone givenCross-modal feature fusion in encoder-decoder frameworks is typically achieved through early or deep fusion, often optimized by attention mechanisms.
“During this process, cross-modal feature fusion is typically achieved through early or deep fusion, often optimized by incorporating attention mechanisms to enhance the flow of information and focus on key features.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
InterpretationreasoningThese approaches collectively advance multimodal fusion for semantic scene understanding.
“These approaches collectively advance multimodal fusion for semantic scene understanding.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Premisenone givenPost fusion techniques include weighted averaging, voting mechanisms, and logical rules.
“Common techniques include weighted averaging, voting mechanisms, and logical rules .”
Quote in the paperNot a substantial claimMay combine two claims.
PremisecitationOccuFusion uses SENet modules to adaptively weight modality features, improving generalization and reliability.
“Methods like OccuFusion [62] use SENet modules [63] to adaptively weight modality features, improving generalization and reliability.”
Quote in the paperNot a substantial claim0 of 2 sources checked- cites [62] Daneshfar, 2025, Elastic deep multi-view autoencoder with diversity embedding, Inform. Sci. source
- cites [63] Li, 2024
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
May combine two claims.
Premisenone givenEarly fusion has a simple structure and can directly use raw data characteristics, but features between modalities may not match, making high-dimensional data difficult to handle.
“Its advantage lies in its simple structure and the ability to directly utilize the raw data characteristics, while its disadvantage is that the features between modalities may not match, making it difficult to handle high-dimensional data.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Premisenone givenMid-term fusion occurs in the middle layer and is a common method for balancing modal independence and capturing semantic relationships.
“Mid term fusion occurring in the middle layer is a common method for balancing modal independence and capturing semantic relationships.”
Quote in the paperNot a substantial claimMay combine two claims.
PremisecitationGNN-based methods model multimodal data through graph structures, enhancing global and robust semantic understanding.
“GNN-based methods [52,53] model multimodal data through graph structures, enhancing global and robust semantic understanding.”
Quote in the paperNot a substantial claim0 of 2 sources checked- cites [52] Liu, 2022, Focus your attention: A focal attention for multimodal learning, IEEE Trans. Multimed. source
- cites [53] R. Kuga, A. Kanezaki, M. Samejima, Y. Sugano, Y. Matsushita, Multi-task Learning Using Multi-modal Encoder-Decoder Networks with Shared Skip Connections, in: 2017 IEEE International Conference on Computer Vision Workshops, ICCVW, 2017, pp. 403–411. source
May combine two claims.
PremisecitationPost fusion has advantages such as strong modal independence, ease of individual optimization, and scalability.
“This method has multiple advantages, such as strong modal independence, ease of individual optimization, and scalability of multimodal systems [46] .”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [46] Zhang, 2024, Multimodal remote sensing image matching via learning features and attention mechanism, IEEE Trans. Geosci. Remote Sens. source
May combine two claims.
InterpretationreasoningMultimodal fusion reduces sensor limitations such as occlusions and noise and enhances robustness of scene interpretation.
“This fusion reduces sensor limitations (e.g., occlusions, noise) and enhances scene interpretation robustness.”
Quote in the paperSubstantial claimMay combine two claims.
PremisecitationCLFT uses a bidirectional network and cross-fusion strategy to integrate data at the Transformer decoder layer, enhancing accuracy and robustness of semantic segmentation in autonomous driving.
“The model utilizes a bidirectional network and a cross-fusion strategy to integrate data from both sensors at the Transformer decoder layer, enhancing the accuracy and robustness of semantic segmentation in autonomous driving.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [61] B. Cheng, I. Misra, A.G. Schwing, A. Kirillov, R. Girdhar, Masked-Attention Mask Transformer for Universal Image Segmentation, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR, 2022, pp. 1290–1299. source
Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.
Premisenone givenAttention mechanisms capture long-range dependencies between cross-modal features, enhancing association between modal information and improving expressive power and generalization.
“In multimodal fusion for semantic scene understanding, the attention mechanism, with its powerful adaptive weighting ability, effectively captures long-range dependencies between cross-modal features, enhancing the association between different modal information and thereby improving the model's expressive power and generalization ability.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Premisenone givenPoint-level fusion usually occurs at an earlier stage, facilitating ample interaction between point cloud and image features.
“Point-level fusion usually occurs at an earlier stage, facilitating ample interaction between point cloud features and image features.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing), S4 (based).
Premisenone givenIn multimodal fusion of robotic vision systems, effectiveness and adaptability of fusion strategies are primarily determined by three key design considerations: when to fuse, what to fuse, and how to fuse.
“In the multimodal fusion of robotic vision systems, the effectiveness and adaptability of fusion strategies are primarily determined by three key design considerations: when to fuse, what to fuse, and how to fuse.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Premisenone givenFor LiDAR, typical inputs consist of raw point clouds, voxelized representations, and multi-view projections such as BEV, FV, and RV.
“For LiDAR, typical inputs consist of raw point clouds, voxelized representations, and multi-view projections such as BEV (Bird's Eye View), FV (Front View), and RV (Rear View).”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing), S4 (based).
Premisenone givenAttention-based methods enhance object detection by dynamically focusing on key multimodal information.
“Attention-based methods enhance object detection by dynamically focusing on key multimodal information.”
Quote in the paperNot a substantial claimMay combine two claims.
PremisereasoningGNN approaches leverage relational modeling, flexible multimodal fusion, high scalability, and generalizability, making them suitable for complex scene understanding, while maintaining interpretability.
“This approach leverages the powerful relational modeling capabilities of GNNs, their flexible multimodal fusion strategies, high scalability, and generalizability across tasks, making them particularly suitable for complex scene understanding tasks, while also maintaining good interpretability.”
Quote in the paperSubstantial claimMay combine two claims.
PremisereasoningGNNs construct graph structures by mapping multimodal data onto nodes and edges, then extract and fuse high-level semantic representations through message passing and feature aggregation.
“In multimodal semantic scene understanding tasks, Graph Neural Networks (GNNs) initially construct a graph structure by mapping multimodal data (such as images and text) onto nodes and edges of a graph.Through message passing and feature aggregation, GNNs then extract and fuse high-level semantic representations from different modalities.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationAVOD designed a high-resolution feature extractor based on encoder-decoder structure.
“To address this issue, AVOD [71] designed a high-resolution feature extractor based on an encoder-decoder structure.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [71] Liu, 2023, Muse-gnn: Learning unified gene representation from multimodal biological graph data, Adv. Neural Inf. Process. Syst. source
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Premisenone given3D object detection is one of the crucial perception tasks in autonomous driving, aiming to obtain 3D information, identify and locate targets, and support path planning and decision-making.
“3D object detection is one of the crucial perception tasks in autonomous driving systems, whose primary goal is to obtain three-dimensional information about the environment through various sensors, identify and locate targets such as vehicles, pedestrians, and obstacles, and then provide support for path planning and decision-making.”
Quote in the paperNot a substantial claimMay combine two claims.
InterpretationreasoningAttention mechanisms, especially cross-modal and self-attention, help emphasize useful features while reducing redundancy, improving occupancy prediction model robustness and generalization.
“Overall, attention mechanisms, especially cross-modal and selfattention, help emphasize useful features while reducing redundancy, improving occupancy prediction model robustness and generalization.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisereasoningMore and more multi-modal fusion strategies are adopted in autonomous driving, among which fusion of cameras and LiDAR point clouds is a main research direction.
“To solve this problem, more and more multi-modal fusion strategies are adopted in autonomous driving systems, among which the fusion of Cameras and LiDAR point clouds is a main research direction.”
Quote in the paperNot a substantial claimMay combine two claims.
PremisecitationHyDRa models uncertainty and enforces depth consistency constraints, enhancing cross-modal depth estimation.
“Additionally, HyDRa [16] models uncertainty and enforces depth consistency constraints, enhancing cross-modal depth estimation.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [16] Wolters, 2024
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).
PremisereasoningWith the rise of attention mechanisms and transformer architectures, attention-based fusion techniques have become increasingly important in 3D object detection, enabling more dynamic and context-aware integration of features.
“With the rise of attention mechanisms and transformer architectures, attention-based fusion techniques have become increasingly important in 3D object detection, enabling more dynamic and context-aware integration of features.”
Quote in the paperNot a substantial claimMay combine two claims.
InterpretationreasoningSingle-modal 3D object detection methods each have advantages but all have limitations.
“Therefore, although single-modal 3D object detection methods each have their own advantages, they all have certain limitations.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationEP-Net introduces Li-Fusion module that adaptively fuses LiDAR and image features at point level.
“EP-Net [82] introduces a Li-Fusion module that adaptively fuses LiDAR and image features at the point level.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [82] D. Xu, D. Anguelov, A. Jain, Pointfusion: Deep sensor fusion for 3d bounding box estimation, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 244–253. source
Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).
Premisenone givenThese factors play a crucial role in determining performance of multimodal perception systems, especially in autonomous driving.
“These factors play a crucial role in determining the performance of multimodal perception systems, especially in complex applications such as autonomous driving.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Premisenone givenFusion techniques can be classified as attention-agnostic or attention-based.
“Fusion techniques can be further classified as attentionagnostic or attention-based.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationFusionPainting integrates 2D and 3D segmentation to address boundary blurring.
“FusionPainting [83] integrates 2D and 3D segmentation to address boundary blurring.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [83] Y. Li, A.W. Yu, T. Meng, B. Caine, J. Ngiam, D. Peng, J. Shen, Y. Lu, D. Zhou, Q.V. Le, et al., Deepfusion: Lidar-camera deep fusion for multi-modal 3d object detection, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 17182–17191. source
Probably not a substantial claim: may not pass S3 (load-bearing).
Premisenone givenIntermediate fusion takes place after feature extraction, combining features from different modalities.
“Intermediate fusion takes place after feature extraction, where features from different modalities are combined.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationMV3D first maps point cloud to bird's eye view to generate 3D candidate boxes, projects candidates into multiple views, employs deep fusion where intermediate layers interact, and uses element-wise mean.
“It first maps point cloud data into a bird's eye view to generate highly accurate 3D candidate boxes, then projects these candidates into multiple views to extract features from different perspectives.It then employs a deep fusion approach where intermediate layers from different views interact with each other, and feature fusion is performed using an element-wise mean method.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [5] X. Chen, H. Ma, J. Wan, B. Li, T. Xia, Multi-view 3d object detection network for autonomous driving, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017, pp. 1907–1915. source
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Premisenone givenLate fusion happens after each modality makes independent decisions, and these decisions are merged.
“Late fusion happens after each modality makes independent decisions, and these decisions are subsequently merged.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Premisenone givenIn LiDAR-Camera systems, granularity of fusion can be divided into ROI-wise, voxel-wise, and point-wise approaches.
“• How to Fuse: As shown in Figure 5, in LiDAR-Camera systems, the granularity of fusion can be divided into ROI-wise, voxel-wise, and point-wise approaches.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing), S4 (based).
Premisenone givenFor camera inputs, common data elements include feature maps, attention maps, and pseudo-LiDAR point clouds.
“• What to Fuse: For camera inputs, common data elements include feature maps, attention maps, and pseudo-LiDAR point clouds.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing), S4 (based).
Premisenone givenEarly fusion occurs at the raw data stage, integrating sensor inputs.
“Early fusion occurs at the raw data stage, integrating sensor inputs.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Premisenone givenTiming of fusion can be categorized into early, intermediate, and late fusion.
“• When to Fuse: The timing of fusion can be categorized into early fusion, intermediate fusion, and late fusion.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationBIRANet improves target detection using Attentive FPN with spatial and channel scSE modules.
“BIRANet [93] improves target detection using Attentive FPN with spatial and channel scSE modules.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [93] J. Yin, J. Shen, R. Chen, W. Li, R. Yang, P. Frossard, W. Wang, Is-fusion: Instance-scene collaborative fusion for multimodal 3d object detection, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 14905–14915. source
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationRCBEVDet dynamically aligns radar and camera BEV features with Cross Attention Multi-Layer Fusion Module.
“RCBEVDet [102] dynamically aligns radar and camera BEV features with the Cross Attention Multi-Layer Fusion Module (CAMF).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [102] X. Dong, B. Zhuang, Y. Mao, L. Liu, Radar camera fusion via representation learning in autonomous driving, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021, pp. 1672–1681. source
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationRISFNet integrates multi-frame radar data with RGB images using temporal encoding and self-attention.
“RISFNet [96] integrates multi-frame radar data with RGB images using temporal encoding and selfattention.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [96] T.-Y. Lim, A. Ansari, B. Major, D. Fontijne, M. Hamilton, R. Gowaikar, S. Subramanian, Radar and camera early fusion for vehicle detection in advanced driver assistance systems, in: Machine Learning for Autonomous Driving Workshop At the 33rd Conference on Neural Information Processing Systems, vol. 2, 2019, 7.
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Interpretationreasoning4D radar's adverse-weather robustness and high spatial resolution make it promising for autonomous driving and ADAS.
“Its robustness in adverse weather and high spatial resolution make it promising for autonomous driving and advanced driver assistance systems (ADAS).”
Quote in the paperNot a substantial claim- Rests on an assumption: 4D radar provides 3D positional data and velocity measurements, improving dynamic object tracking and identification.
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationCRN uses Radar-Assisted View Transformation to align camera and radar features in BEV and applies multimodal deformable cross-attention for precise fusion.
“CRN [101] uses Radar-Assisted View Transformation (RVT) to align camera and radar features in BEV and applies multimodal deformable cross-attention (MDCA) for precise fusion.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [101] Y. Kim, J. Shin, S. Kim, I.-J. Lee, J.W. Choi, D. Kum, Crn: Camera radar net for accurate, robust, efficient 3d perception, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 17615–17626. source
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationRCM-Fusion combines radar and camera data at both feature and instance layers, refining 3D schemes using radar points.
“RCM-Fusion [103] combines radar and camera data at both the feature and instance layers, refining 3D schemes using radar points.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [103] Z. Lin, Z. Liu, Z. Xia, X. Wang, Y. Wang, S. Qi, Y. Dong, N. Dong, L. Zhang, C. Zhu, RCBEVDet: Radar-camera Fusion in Bird’s Eye View for 3D Object Detection, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 14928–14937. source
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationAssociationNet introduces deep association learning module to tackle data association challenges and uses loss sampling and ordinal loss to refine spatial relationships.
“Asso-ciationNet [95] introduces a deep association learning module to tackle data association challenges and uses loss sampling and ordinal loss to refine spatial relationships.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [95] Wang, 2022, InterFusion: Interaction-based 4D radar and LiDAR fusion for 3D object detection source
Probably not a substantial claim: may not pass S3 (load-bearing).
Interpretationnone givenSDG, LOG, and self-attention mechanisms continue to drive progress in sensor fusion.
“Innovative fusion techniques, including Sparse Depth Guidance (SDG), Laser Occupancy Guidance (LOG), and selfattention mechanisms, continue to drive progress in this field.”
Quote in the paperNot a substantial claimMay combine two claims.
Premiseown dataEmbodied navigation research focuses on goal-oriented, instruction-following, and dialogue-based navigation.
“As shown in Figure 7, in recent years, the research in this field mainly focuses on the three directions of Goal-oriented Navigation, Instruction-following Navigation and Dialogue-Based Navigation.”
Quote in the paperNot a substantial claimMay combine two claims.
PremisecitationNaVid achieved SOTA on R2R and R2R-RxR and showed Sim2Real transferability.
“It has achieved state-of-the-art (SOTA) performance on R2R and R2R-RxR tasks while exhibiting remarkable Sim2Real transferability.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [114] H. Huang, V. Jain, H. Mehta, A. Ku, G. Magalhaes, J. Baldridge, E. Ie, Transferable representation learning in vision-and-language navigation, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, 2019, pp. 7404–7413. source
May combine two claims.
InterpretationreasoningThese sensor combinations enhance environmental perception by leveraging complementary strengths, such as radar velocity for dynamic object tracking and 4D radar stability in harsh conditions.
“These approaches enhance environmental perception by leveraging complementary strengths, such as using radar's velocity data for improved dynamic object tracking and 4D radar's stability in harsh conditions.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).
Interpretationnone givenLong-horizon planning is essential for embodied navigation.
“Additionally, long-horizon planning is essential for embodied navigation.”
Quote in the paperNot a substantial claimMay combine two claims.
InterpretationreasoningIncorporating topological maps lets robots optimize paths using global information and mitigates random exploration-induced disorientation.
“Compared to relying solely on local decision-making, incorporating topological maps allows robots to optimize navigation paths based on global information, effectively mitigating random exploration-induced disorientation.”
Quote in the paperNot a substantial claimMay combine two claims.
PremisecitationCMT enhances object localization by encoding 3D coordinates and using Transformer decoder for multimodal interaction.
“CMT [87] enhances object localization by encoding 3D coordinates and using a Transformer decoder for multimodal interaction.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [87] J. Yan, Y. Liu, J. Sun, F. Jia, S. Li, T. Wang, X. Zhang, Cross modal transformer: Towards fast and robust 3d object detection, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 18268–18278. source
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
InterpretationreasoningRadar-Camera and Radar-Lidar sensor combinations are gaining traction.
“Beyond LiDAR-Camera fusion, other sensor combinations like Radar-Camera and Radar-Lidar are also gaining traction.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationCRKD proposes cross-modal knowledge distillation framework for improving camera-radar fusion through BEV representations.
“CRKD [104] proposes a cross-modal knowledge distillation framework for improving camera-radar fusion through BEV representations.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [104] Y. Cheng, H. Xu, Y. Liu, Robust small object detection on the water surface through fusion of camera and millimeter wave radar, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021, pp. 15263–15272. source
May combine two claims.
Premisenone givenIn addition to Lidar-Camera fusion, there are studies based on Radar-Camera and Lidar-Radar.
“In addition to Lidar-Camera fusion, there are studies based on Radar-Camera and Lidar-Radar.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationRRPN maps radar detections to image coordinate system, generating anchor boxes for quick ROI suggestions.
“RRPN [91] maps radar detections to the image coordinate system, generating anchor boxes for quick ROI suggestions.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [91] X. Li, B. Fan, J. Tian, H. Fan, GAFusion: Adaptive Fusing LiDAR and Camera with Multiple Guidance for 3D Object Detection, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 21209–21218. source
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationDRL applies RL for goal-driven navigation, enabling autonomous learning of generalized strategies and quick adaptation.
“Additionally, DRL [109] applies reinforcement learning (RL) for goal-driven navigation, enabling robots to autonomously learn generalized navigation strategies and adapt quickly to diverse environments.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [109] K. He, X. Zhang, S. Ren, J. Sun, Deep residual learning for image recognition, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 770–778. source
May combine two claims.
Premisenone givenPure visual SLAM struggles in low-light or textureless environments, while LiDAR SLAM lacks semantic understanding and suffers data sparsity in long-range or low-reflectivity areas.
“For instance, pure visual SLAM struggles in low-light or textureless environments, whereas LiDAR-based methods lack semantic understanding and suffer from data sparsity in long-range or low-reflectivity areas.”
Quote in the paperSubstantial claimMay combine two claims.
Interpretationnone givenChallenges remain in visual localization: large-scale datasets, high computational costs, and limited generalization to real-world conditions.
“Despite progress, challenges remain, such as the need for large-scale datasets, high computational costs, and limited generalization to real-world conditions.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing), S4 (based).
Premisenone givenMultimodal feature fusion is core technology enabling transition from perception to action in robot manipulation.
“Multimodal feature fusion is the core technology enabling the seamless transition from perception to action in robot manipulation tasks.”
Quote in the paperNot a substantial claimMay combine two claims.
PremisecitationUVIO uses a transformer backbone to jointly learn from visual and IMU inputs, improving pose estimation in dynamic environments.
“For example, UVIO [146] employs a transformer backbone to jointly learn from visual and IMU inputs, significantly improving pose estimation in dynamic environments.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [146] Davison, 2007, MonoSLAM: Real-time single camera SLAM, IEEE Trans. Pattern Anal. Mach. Intell. source
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Premisenone givenScene coordinate regression eliminates explicit 3D maps by learning direct image-pixel-to-3D-coordinate transformations, simplifying localization.
“Scene coordinate regression techniques eliminate the need for explicit 3D maps by learning direct transformations from image pixels to 3D coordinates, simplifying localization.”
Quote in the paperNot a substantial claimMay combine two claims.
InterpretationreasoningMultimodal SLAM has evolved from loosely coupled to deeply integrated frameworks, achieving stronger cross-modal alignment and environmental understanding.
“In summary, multimodal SLAM systems have evolved from loosely coupled architectures to deeply integrated frameworks, achieving stronger cross-modal alignment and environmental understanding.”
Quote in the paperNot a substantial claimMay combine two claims.
PremisecitationIntegrating learned depth and optical flow predictions yields consistent pose estimations and improves monocular system accuracy.
“By integrating learned depth and optical flow predictions [134], these systems achieve consistent pose estimations, improving monocular system accuracy.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [134] Wang, 2023, Triple robustness augmentation local features for multi-source image registration, ISPRS J. Photogramm. Remote Sens. source
Probably not a substantial claim: may not pass S4 (based). May combine two claims.
Interpretationnone givenVLA models extract complementary features from different modalities and integrate them into a unified representation, enabling precise action planning and execution.
“The key innovation in VLA models lies in their ability to extract complementary features from different modalities and integrate them into a unified representation, enabling precise action planning and execution.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Premisenone givenLiDAR provides precise spatial structure, vision offers rich semantics, and IMU compensates for short-term motion estimation uncertainties.
“LiDAR provides precise spatial structural information, vision offers rich semantic details, and IMU compensates for short-term motion estimation uncertainties.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing), S4 (based).
InterpretationreasoningMultimodal fusion-based SLAM has emerged as a research hotspot.
“To overcome these limitations, multimodal fusion-based SLAM has emerged as a research hotspot.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S4 (based).
Premisenone givenVisual modality is crucial for manipulation, providing object position, shape, and layout.
“The visual modality is crucial for manipulation tasks, providing information about object position, shape, and layout.”
Quote in the paperNot a substantial claimMay combine two claims.
InterpretationreasoningVLA models integrate language to process natural language instructions, bridging gaps in visual perception.
“To address this, VLA models integrate language to process natural language instructions, bridging gaps in visual perception.”
Quote in the paperSubstantial claimMay combine two claims.
InterpretationreasoningThese SLAM approaches enhance adaptability in complex environments and pave the way for semantic SLAM.
“These approaches not only enhance system adaptability in complex environments but also pave the way for semantic SLAM, enabling robots to perceive and understand their surroundings at a higher cognitive level.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.
Premisenone givenVLA models integrate visual perception, language understanding, and action planning, offering an efficient framework for complex manipulation tasks.
“Vision-Language-Action (VLA) models, which integrate visual perception, language understanding, and action planning, offer an efficient framework for addressing the”
Quote in the paperNot a substantial claimMay combine two claims.
PremisecitationPre-training facilitates joint learning across multiple modalities, strengthening generalization on new tasks and data.
“Additionally, pretraining facilitates joint learning across multiple modalities, strengthening the model's generalization ability on new tasks and data [167].”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [167] K. Yu, Y. Han, Q. Wang, V. Saxena, D. Xu, Y. Zhao, MimicTouch: Leveraging Multi-modal Human Tactile Demonstrations for Contact-rich Manipulation, in: 8th Annual Conference on Robot Learning, 2024.
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
May combine two claims.
PremisecitationThis approach adjusts multimodal feature weights based on sensor signal quality, improving grasp robustness and prediction accuracy in dynamic environments.
“This approach adjusts the weight of multimodal features based on sensor signal quality, improving grasp robustness and prediction accuracy in dynamic environments.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [160] Liu, 2024
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisereasoningThe model learns representations by generating and inferring missing modality information.
“The model learns representations by generating and inferring missing modality information.”
Quote in the paperNot a substantial claimMay combine two claims.
PremisecitationSparsh uses a universal tactile representation pre-trained on a large tactile dataset, enhancing slip detection, grasp stability, and planning accuracy in dynamic environments.
“Sparsh [159] uses a universal tactile representation method pre-trained on a large tactile dataset, enhancing slip detection, grasp stability, and planning accuracy in dynamic environments.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [159] Brohan, 2023
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
May combine two claims.
InterpretationreasoningPre-training is a key step in multimodal fusion and essential for improving VLM performance.
“Therefore, pre-training is a key step in multimodal fusion technologies and plays an essential role in improving the performance of vision-language models.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationCross-modal alignment maps information from modalities into a shared space to understand relationships.
“Crossmodal alignment maps information from modalities, such as images and text, into a shared space to understand their relationships [177].”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [177] Zhang, 2023, Dynamic alignment and fusion of multimodal physiological patterns for stress recognition, IEEE Trans. Affect. Comput. source
May combine two claims.
PremisereasoningCross-modal alignment and representation learning are crucial for VLMs, enabling different modalities to interact in a shared embedding space.
“Cross-modal alignment and representation learning are crucial for vision-language models, enabling different modalities (e.g., vision, text, audio) to interact in a shared embedding space.”
Quote in the paperSubstantial claimMay combine two claims.
PremisecitationFusionNet-A combines tactile and visual features using 3D pooling and global average pooling for classification, improving object recognition and planning.
“FusionNet-A [7] combines tactile and visual features using 3D pooling and global average pooling for classification, improving object recognition and planning.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [7] Babadian, 2023, Fusion of tactile and visual information in deep learning models for object recognition, Inf. Fusion source
May combine two claims.
PremisecitationSuch pre-training improves multimodal task performance and overall performance in complex applications.
“Such pre-training not only enables better handling of multimodal data but also enhances the model's overall performance in complex applications [176].”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [176] Agrawal, 2022, Tie your embeddings down: Cross-modal latent spaces for end-to-end spoken language understanding source
May combine two claims.
InterpretationreasoningVision-touch fusion in robotic grasping compensates for single-modality perception shortcomings and provides robust perception and decision-making for complex tasks.
“In summary, the application of vision-touch multimodal fusion in robotic grasping tasks compensates for the shortcomings of single-modality perception and provides robust perception and decision-making capabilities for complex tasks.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Premisenone givenImages are high-dimensional while text consists of discrete symbols, making alignment difficult.
“Images are high-dimensional, while text consists of discrete symbols, making alignment difficult.”
Quote in the paperNot a substantial claimMay combine two claims.
InterpretationreasoningThese VLA approaches enhance task semantics understanding and action planning, opening possibilities for deploying robots in dynamic complex environments.
“These approaches not only significantly enhance the model's ability to understand task semantics and plan actions but also open up new possibilities for deploying robots in dynamic and complex environments.”
Quote in the paperSubstantial claimMay combine two claims.
PremisecitationThis improves performance on vision-language tasks like VQA and image captioning and enables knowledge transfer across modalities, reducing reliance on labeled data.
“This not only improves the model's performance on vision-language tasks (such as visual question answering and image captioning) but also enables knowledge transfer across modalities, reducing the reliance on large amounts of labeled data [166].”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [166] Li, 2024, Grasp stability assessment through spatio-temporal attention mechanism and adaptive gate fusion, IEEE Sens. J. source
May combine two claims.
PremisecitationVoxPoser introduces language-driven affordance inference to generate 3D value maps for complex manipulation, dynamically adjusting strategies through semantic reasoning for real-time adaptation.
“Furthermore, VoxPoser [156] introduces a language-driven affordance inference mechanism to generate 3D value maps for complex manipulation tasks, dynamically adjusting manipulation strategies through semantic reasoning for”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [156] Graeter, 2018, Limo: Lidar-monocular visual odometry source
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Premisenone givenContrastive learning minimizes distance between positive samples and maximizes it for negative samples.
“Contrastive learning aims to minimize the distance between positive samples (same-category) and maximize it for negative samples (different-category).”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationCLIP trains vision and language models with contrastive learning, aligning images and text in a shared space for consistency.
“The popular method CLIP [181] trains both vision and language models using contrastive learning, aligning images and text in a shared space for consistency.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [181] Berg, 2024, Multimodal supervised contrastive learning in remote sensing downstream tasks, IEEE Geosci. Remote. Sens. Lett. source
Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).
Premisenone givenRepresentation learning extracts useful features from each modality for various tasks.
“Representation learning extracts useful features from each modality for various tasks.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S4 (based). May combine two claims.
InterpretationreasoningThese VLA methods enhance task success rates and enable efficient deployment on resource-constrained platforms.
“These methods enhance task success rates and enable efficient deployment on resource-constrained platforms.”
Quote in the paperNot a substantial claimMay combine two claims.
PremisecitationContrastive and self-supervised learning optimize alignment between modalities, enhancing cross-modal information flow and interaction.
“By utilizing methods such as contrastive learning and self-supervised learning, the model can optimize the alignment between modalities, thereby enhancing the flow and interaction of information across them [175].”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [175] Xu, 2020, Joint feature synthesis and embedding: Adversarial cross-modal retrieval revisited, IEEE Trans. Pattern Anal. Mach. Intell. source
Probably not a substantial claim: may not pass S3 (load-bearing).
Premisenone givenSelf-supervised learning is unsupervised learning where the model generates labels from the data itself.
“Self-supervised learning is a type of unsupervised learning where the model is trained by generating labels from the data itself.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim. May combine two claims.
Premisenone givenVision provides global object position, posture, and geometry; touch offers local force, surface, and slip feedback.
“Vision provides global information on object position, posture, and geometry, while touch offers local feedback on force, surface properties, and slip detection.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisereasoningFor cross-modal tasks, positive samples are visual-language pairs from the same context, while negative samples come from different contexts.
“For crossmodal tasks, positive samples are visual and language pairs from the same context, while negative samples come from different contexts.”
Quote in the paperNot a substantial claimMay combine two claims.
InterpretationreasoningCombining vision and touch lets robots overcome single-modality limits and adapt to dynamic environments.
“By combining these modalities, robots can overcome single modality limitations and adapt to dynamic environments.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Premisenone givenCommon representation learning methods include contrastive learning, self-supervised learning, and cross-modal generative models.
“Common methods for representation learning include contrastive learning, self-supervised learning, and cross-modal generative models.”
Quote in the paperNot a substantial claimMay combine two claims.
PremisecitationLi et al. introduced a model using spatial and temporal attention, where spatial attention focuses on key object regions and temporal attention tracks dynamic grasp changes.
“LI et al. [160] introduced a model using spatial and temporal attention, where spatial attention focuses on key object regions and temporal attention tracks dynamic grasp changes.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [160] Liu, 2024
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
May combine two claims.
Interpretationnone givenThis combination improves performance in image captioning, VQA, and speech recognition, advancing multimodal capabilities.
“This combination improves performance in tasks like image captioning, visual question answering, and speech recognition, advancing multimodal capabilities.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Premisenone givenAlignment maps relationships between modalities, such as images to text or images to audio.
“Alignment helps models map relationships between modalities, such as images to text or images to audio.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
InterpretationreasoningThese early models helped establish large-scale pre-trained language models as a standard for NLP tasks.
“These early models helped establish large-scale pre-trained language models as a standard for NLP tasks.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim.
InterpretationreasoningThese methods help models learn richer and deeper representations from multimodal data.
“Through different training mechanisms and strategies, they help models learn richer and deeper representations from multimodal data.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Premisenone givenCross-modal generative models aim to generate one modality based on another.
“The goal of cross-modal generative models is to generate one modality (e.g., an image) based on another modality (e.g., text).”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim. May combine two claims.
InterpretationreasoningThese methods improve alignment and interaction between modalities and enhance multimodal task performance, supporting more accurate vision-language understanding and generation.
“These methods not only improve the alignment and interaction between different modalities but also enhance the overall performance of multimodal tasks, providing strong support for more accurate vision-language understanding and generation.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationSelf-supervised learning does not rely on large amounts of manually labeled data and can be trained on unlabeled data.
“Self-supervised learning does not rely on large amounts of manually labeled data and can be trained using unlabeled data [184].”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [184] M. Ni, H. Huang, L. Su, E. Cui, T. Bharti, L. Wang, D. Zhang, N. Duan, M3p: Learning universal representations via multitask multilingual multimodal pre-training, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021, pp. 3977–3986. source
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationReformer employed locality-sensitive hashing attention to reduce memory requirements for attention computation.
“Reformer [195] employed locality-sensitive hashing (LSH Attention) to reduce memory requirements for attention computation, while Longformer [196] and Linformer [197] utilized sparse attention and low-rank projections, respectively, to decrease computational complexity, making Transformer more efficient for handling long-text sequences.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [195] Zhang, 2020, Multi-pathway generative adversarial hashing for unsupervised cross-modal retrieval, IEEE Trans. Multimed. source
Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.
Interpretationnone givenTransformer has been scaled and optimized with more computational power, becoming the foundation for large-scale AI systems.
“With more computational power, Transformer has been scaled and optimized, becoming the foundation for large-scale AI systems.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Interpretationnone givenAs computational power increased, Transformer models rapidly expanded in scale to enhance generalization across multiple tasks and complex reasoning.
“As computational power increased, the scale of Transformer models rapidly expanded to enhance their generalization ability across multiple tasks and complex reasoning.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Premisenone givenVisual-language pretraining is the foundational technique for VLMs, allowing models to learn deep relationships between images and text through joint training of large-scale image-text pairs, improving performance on cross-modal tasks.
“Visual-language pretraining is the foundational technique for VLMs, allowing models to learn deep relationships between images and text through the joint training of large-scale imagetext pairs, which improves the model's performance on crossmodal tasks.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationDINO is a self-supervised contrastive method without negative samples; it learns visual representations by maximizing similarity between an image and its transformed versions.
“A recent mainstream approach is DINO [185], a self-supervised method based on contrastive learning that does not rely on negative samples.Instead, it learns visual representations by maximizing the similarity between an image and its transformed versions.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [185] Daneshfar, 2024, Image captioning by diffusion models: a survey, Eng. Appl. Artif. Intell. source
Probably not a substantial claim: may not pass S3 (load-bearing).
Premisenone givenThe core idea of MoE is to activate only a fraction of model parameters during inference, preventing linear growth in computational cost.
“The core idea is to activate only a fraction of the model parameters during inference, preventing linear growth in computational cost.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Interpretationnone givenAs models scaled, MoE architecture emerged to optimize computational resource usage.
“As models scaled, the Mixture of Experts (MoE) architecture emerged to optimize computational resource usage.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationGPT-3 scaled to 175 billion parameters and introduced in-context learning, allowing adaptation to new tasks with minimal fine-tuning.
“GPT-3 [194] further scaled up to 175 billion parameters and introduced incontext learning (ICL), allowing models to adapt to new tasks with minimal fine-tuning.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [194] Kang, 2023, A cross-modal generative adversarial network for scenarios generation of renewable energy, IEEE Trans. Power Syst. source
Probably not a substantial claim: may not pass S3 (load-bearing).
InterpretationreasoningAs Transformer models grew, computational complexity became a pressing issue.
“However, as Transformer models continued to grow in size, computational complexity became a pressing issue.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationGPT-2 increased parameter count to 1.5 billion, showcasing zero-shot and few-shot learning and establishing a foundation for general-purpose text generation.
“GPT-2 [8] increased the parameter count to 1.5 billion, showcasing zeroshot and few-shot learning capabilities and establishing a foundation for general-purpose text generation.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [8] Radford, 2019, Language models are unsupervised multitask learners, OpenAI Blog
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
May combine two claims.
PremisereasoningMoE dynamically activates a subset of expert networks, allowing models to scale parameters without significantly increasing computational cost, improving efficiency while maintaining generalization.
“MoE dynamically activates a subset of expert networks, allowing models to scale parameters without significantly increasing computational cost, improving efficiency while maintaining strong generalization.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationSwin Transformer introduced shifted window attention, significantly improving computational efficiency and performance in dense prediction tasks such as object detection and semantic segmentation.
“Subsequently, Swin Transformer [199] introduced shifted window attention, significantly improving computational efficiency and performance in dense prediction tasks such as object detection and semantic segmentation.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [199] J.D.M.-W.C. Kenton, L.K. Toutanova, Bert: Pre-training of deep bidirectional transformers for language understanding, in: Proceedings of NaacL-HLT, vol. 1, Minneapolis, Minnesota, 2019, 2.
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationT2F uses GANs to convert text descriptions into facial images, combining text and image generation tasks.
“Recent mainstream methods include T2F (Text-to-Face), which uses Generative Adversarial Networks (GANs) to convert text descriptions into facial images, successfully combining text and image generation tasks.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationSwitch Transformer was one of the first MoE-based Transformers, activating experts selectively in each layer, reducing computational overhead while outperforming dense Transformers in NLP tasks.
“The Switch Transformer [201] was one of the first MoE-based Transformers, activating experts selectively in each layer, reducing computational overhead while outperforming dense Transformers in NLP tasks.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [201] Alayrac, 2022, Flamingo: a visual language model for few-shot learning, Adv. Neural Inf. Process. Syst. source
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisereasoningUnlike traditional RNNs, Transformer removes sequential computation constraints, enhancing performance in large-scale data tasks.
“Unlike traditional recurrent neural networks (RNNs), Transformer removes sequential computation constraints, enhancing performance in large-scale data tasks.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Interpretationown dataThese models represent the evolution of vision-language systems, with advancements in visual processing, decision-making, and multimodal integration across time.
“These models represent the evolution of vision-language systems, with advancements in visual processing, decision-making, and multimodal integration across time.”
Quote in the paperNot a substantial claimMay combine two claims.
PremisecitationFlamingo combines few-shot learning and cross-modal reasoning to quickly adapt to new tasks with minimal examples, especially excelling in image captioning and VQA.
“Flamingo [203] is a classic example, combining few-shot learning and cross-modal reasoning to quickly adapt to new tasks with minimal examples, especially excelling in image captioning and visual question answering (VQA).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [203] Zhu, 2023
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisereasoningDue to the quadratic complexity O(N^2) of self-attention, processing long-sequence inputs led to an exponential increase in computational cost.
“Due to the quadratic complexity O(N 2 ) of the self-attention mechanism, processing long-sequence inputs led to an exponential increase in computational cost.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Premisenone givenTransformer's self-attention mechanism efficiently models long-range dependencies and improves training speed through parallel computation.
“Its key innovation, the self-attention mechanism, efficiently models long-range dependencies and improves training speed through parallel computation.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).
Premisenone givenVLMs have played a crucial role in advancing the deep integration of computer vision and natural language processing.
“With the widespread application of visual-language models (VLMs) in multimodal tasks, these models have played a crucial role in advancing the deep integration of computer vision and natural language processing.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationThis process uses generative models such as VAEs or GANs to map information from one modality into another modality's space.
“This process uses generative models (such as Variational Autoencoders or Generative Adversarial Networks) to map information from one modality into the space of another modality [187].”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [187] Hu, 2022, Unsupervised contrastive cross-modal hashing, IEEE Trans. Pattern Anal. Mach. Intell. source
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationGPT employs a decoder-only autoregressive approach, laying the foundation for generative AI.
“In contrast, OpenAI's GPT employs a decoder-only autoregressive approach, laying the foundation for generative AI.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [193] R. Hu, A. Singh, Unit: Multimodal multitask learning with a unified transformer, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021, pp. 1439–1449. source
Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.
Interpretationnone givenIn cross-modal representation learning, contrastive learning, self-supervised learning, and cross-modal generative models each play an important role.
“In cross modal representation learning, methods such as contrastive learning, self-supervised learning, and cross-modal generative models each play an important role.”
Quote in the paperNot a substantial claimMay combine two claims.
PremisecitationDETR leveraged an end-to-end Transformer-based approach to replace traditional RPNs in object detection, highlighting Transformer's potential in vision tasks.
“Additionally, DETR [200] leveraged an end-to-end Transformer-based approach to replace traditional region proposal networks (RPNs) in object detection, highlighting Transformer's potential in vision tasks.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [200] Radford, 2018
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
May combine two claims.
InterpretationreasoningMoE enables Transformers to balance parameter expansion with computational efficiency, addressing the need for better generalization and complex reasoning without increasing computational burden.
“MoE enables Transformers to balance parameter expansion with computational efficiency, addressing the need for better generalization and complex reasoning without increasing”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationGPT-4 incorporates MoE-style routing, optimizing efficiency and reducing inference costs while supporting multimodal inputs.
“GPT-4 [22] incorporates MoE-style routing, optimizing efficiency and reducing inference costs while supporting multimodal inputs.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [22] Achiam, 2023
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).
InterpretationreasoningInstruction tuning plays a pivotal role in optimizing large-scale multimodal models, significantly improving their adaptability and generalization across multiple tasks and domains.
“Instruction tuning plays a pivotal role in optimizing largescale multimodal models, significantly improving their adaptability and generalization across multiple tasks and domains.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisereasoningSolutions to context-aware challenges involve adaptive model architectures, attention mechanisms, and transfer learning to enhance robustness and generalization.
“Solutions to these challenges involve using adaptive model architectures, incorporating attention mechanisms, and leveraging techniques like transfer learning to enhance the robustness and generalization of the model.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing), S4 (based).
Premisenone givenRLHF improves instruction tuning by aligning models with human preferences for complex real-world tasks.
“Reinforcement Learning with Human Feedback (RLHF) improves instruction tuning by aligning models with human preferences for complex real-world tasks.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing), S4 (based).
PremisereasoningGPT-4V uses RLHF to improve controllability and reliability in VQA tasks, better aligning with user intent.
“Models like GPT-4V use RLHF to improve controllability and reliability in VQA tasks, better aligning with user intent.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Premisenone givenInstruction tuning guides models to learn different task modes through explicit task instructions, enabling efficient transfer and generalization across multiple tasks.
“Its core idea is to guide models in learning different task modes through explicit task instructions, enabling efficient transfer and generalization across multiple tasks.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
PremisereasoningThe increasing need for real-time inference in resource-constrained environments has driven the development of lightweight multimodal models.
“The increasing need for real-time inference in resource-constrained environments has driven the development of lightweight multimodal models.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing), S4 (based).
Premisenone givenInstruction tuning is a crucial technique for improving the generalization ability of large-scale pretrained models.
“Instruction tuning serves as a crucial technique for improving the generalization ability of large-scale pretrained models.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Premisenone givenMultimodal adaptation extends instruction tuning to tasks involving multiple modalities such as images, text, audio, and video.
“Multimodal Adaptation extends instruction tuning to tasks involving multiple modalities such as images, text, audio, and video.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Premisenone givenContext-aware modeling is designed to boost representational power and prediction accuracy by merging data from various modalities.
“Context-aware modeling is a cutting-edge approach in multimodal learning, designed to boost the representational power and prediction accuracy of models by merging data from various modalities.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S4 (based).
Premisenone givenIncluding contextual information during training allows the model to better grasp relationships between data points and their context.
“The key idea behind this method is to include contextual information during the training process, which allows the model to better grasp the relationships between data points and their underlying context.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisereasoningContext-aware modeling makes models more adaptable to diverse data distributions and real-world scenarios, enhancing generalization.
“By being context-aware, the model becomes more adaptable to diverse data distributions and real-world scenarios, enhancing its ability to generalize.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisereasoningLightweight multimodal models balance efficiency and accuracy and are ideal for mobile devices, embedded systems, and edge computing.
“These models, balancing efficiency and accuracy, are ideal for mobile devices, embedded systems, and edge computing.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.
Premisenone givenInstruction tuning enhances model controllability and adaptability across various multimodal tasks.
“Instruction tuning enhances model controllability and adaptability across various multimodal tasks.”
Quote in the paperNot a substantial claimMay combine two claims.
PremisecitationPaLI-X integrates multilingual and multimodal instructions, improving transfer learning in vision-language applications.
“In multimodal tasks, datasets like PaLI-X [216] integrate multilingual and multimodal instructions, improving transfer learning in vision-language applications.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [216] Du, 2022, Glam: Efficient scaling of language models with mixture-of-experts
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
May combine two claims.
PremisereasoningTraditional cloud computing models are inadequate for edge-generated data because of poor real-time capabilities, bandwidth limits, high energy consumption, and data security concerns.
“This growth has made traditional cloud computing models inadequate for handling edgegenerated data.Conventional cloud computing faces issues like poor real-time capabilities, bandwidth limitations, high energy consumption, and data security concerns.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing), S4 (based).
Premisenone givenIn multimodal tasks, instruction tuning typically involves instruction dataset construction, supervised fine-tuning, RLHF, and multimodal adaptation.
“In multimodal tasks, the implementation of instruction tuning typically involves key steps such as instruction dataset construction, supervised fine-tuning (SFT), reinforcement learning with human feedback (RLHF), and multimodal adaptation.”
Quote in the paperNot a substantial claimMay combine two claims.
PremisecitationQwen-VL improves computational efficiency with M-RoPE and dynamic resolution, increasing reasoning speed and accuracy in high-resolution image processing by better understanding spatial relationships.
“Qwen-VL [206] enhances computational efficiency with Multimodal Rotary Positional Embedding (M-RoPE) and dynamic resolution, improving reasoning speed and accuracy in high-resolution image processing by better understanding spatial relationships.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [206] Lin, 2024
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim: may not pass S3 (load-bearing).
Premisenone givenSupervised fine-tuning is the core step of instruction tuning, using the constructed instruction dataset to fine-tune models for precise task execution.
“Supervised Fine-Tuning (SFT) is the core step of instruction tuning, leveraging the constructed instruction dataset to fine-tune models for precise task execution.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationIn text classification, context-awareness helps the model capture the underlying meaning of sentences, and in image recognition it improves identification and understanding of complex scenes.
“For instance, in text classification tasks, contextawareness helps the model capture the underlying meaning of sentences [222], while in image recognition, it improves the model's ability to identify and understand complex scenes.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [222] Zhu, 2024
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
May combine two claims.
PremisecitationPaLI-X improves cross-language image description, helping in document processing and translation.
“PaLI-X [216] improves cross-language image description, helping in document processing and translation.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [216] Du, 2022, Glam: Efficient scaling of language models with mixture-of-experts
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationIDC predicts that by 2025 global data volume will exceed 180 zettabytes, with over 70% of IoT data processed at the network edge.
“IDC predicts that by 2025, global data volume will exceed 180 zettabytes (ZB), with over 70% of IoT data processed at the network edge [228].”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [228] Prakash, 2023, Adapt-FuseNet: Context-aware multimodal adaptive fusion of face and gait features using attention techniques for human identification source
Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.
Premisenone givenIn image captioning, models adjust style and detail based on the task, as seen in BLIP-2, which aids in medical imaging and social media captioning.
“In image captioning, models adjust style and detail based on the task, as seen in BLIP-2, which aids in medical imaging and social media captioning.”
Quote in the paperNot a substantial claimMay combine two claims.
PremisecitationRadar-camera fusion enhances motion detection by leveraging radar velocity information, with RCBEVDet++ demonstrating greater stability in dynamic scenes such as high-speed driving and urban intersections.
“In contrast, radarcamera fusion enhances motion detection by leveraging radar velocity information, with RCBEVDet++ [263] demonstrating greater stability in dynamic scenes, such as high-speed driving and urban intersections.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [263] Y. Chen, Z. Yu, Y. Chen, S. Lan, A. Anandkumar, J. Jia, J.M. Alvarez, Focalformer3d: focusing on hard instance for 3d object detection, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 8394–8405. source
May combine two claims.
PremisecitationR2SGrasp uses a Real-to-Sim framework to minimize domain gaps between simulation and reality, demonstrating outstanding performance in 6-DoF grasp detection.
“R2SGrasp [243] utilizes a”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [243] Zhang, 2017, IEHouse: A non-intrusive household appliance state recognition system source
May combine two claims.
PremisecitationScanQA combines natural language question answering with 3D scene understanding, exploring object localization and scene reasoning guided by language.
“ScanQA [242], on the other hand, takes a step forward in combining natural language question answering with 3D scene understanding.It explores object localization and scene reasoning guided by language through question-answering tasks that integrate linguistic and visual depth.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [242] S. Lu, Y. Yao, W. Shi, Collaborative learning on the edges: A case study on connected vehicles, in: 2nd USENIX Workshop on Hot Topics in Edge Computing, HotEdge 19, 2019.
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim: may not pass S4 (based). May combine two claims.
Premisenone givenIn edge AI for real-time fusion, development is predominantly built on existing cloud computing frameworks, which often fall short in edge-specific scenarios and struggle with heterogeneous cloud-edge-device systems.
“In edge AI for Real-time fusion scenarios, development is predominantly built on existing cloud computing frameworks.However, these frameworks often fall short in edge-specific scenarios and struggle to accommodate the diverse and heterogeneous nature of cloud-edge-device systems.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing), S4 (based).
Premisenone givenThe availability and diversity of multimodal datasets have played a foundational role in advancing robot vision across scene understanding, navigation, and manipulation.
“The availability and diversity of multimodal datasets have played a foundational role in advancing robot vision across a broad spectrum of tasks, including scene understanding, navigation, and manipulation.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationRT-1 employs a Transformer architecture to achieve efficient real-time control in multi-task learning.
“The latter, on the other hand, employs a Transformer architecture to achieve efficient real-time control in multi-task learning.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [244] H. Chen, Y. Hou, C. Qu, I. Testini, X. Hong, J. Jiao, 360+ x: A Panoptic Multi-modal Scene Understanding Dataset, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 19373–19382. source
May combine two claims.
PremisecitationSOON provides 4,000 natural language instructions and 40,000 trajectories, capturing rich scene descriptions including object attributes, spatial relationships, and contextual information.
“With 4,000 natural language instructions and 40,000 trajectories, it captures rich scene descriptions, including object attributes, spatial relationships, and contextual information.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [253] Yang, 2022
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
May combine two claims.
PremisecitationREVERIE extends Matterport3D by adding bounding box annotations for 4,140 target objects and 21,702 natural language instructions, introducing a task requiring agents to locate target objects while navigating.
“To address the lack of object interaction in R2R [251], REVERIE [252] extends the capabilities of Matterport3D [26] by adding bounding box annotations for 4,140 target objects and 21,702 natural language instructions.It introduces the Remote Embodied Visual Referring Expression (REVERIE [252]) task, requiring agents to locate target objects while navigating the environment.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [252] Brohan, 2022
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
May combine two claims.
PremisecitationSimpleBEV adopts a BEV-based multi-scale fusion strategy, strengthening depth perception and maintaining stable overall detection performance.
“SimpleBEV [260], adopting a BEV-based multi-scale fusion strategy, strengthens depth perception and maintains stable overall detection performance.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [260] Y. Qi, Q. Wu, P. Anderson, X. Wang, W.Y. Wang, C. Shen, A.v.d. Hengel, Reverie: Remote embodied visual referring expression in real indoor environments, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 9982–9991. source
Classified as probably not a substantial claim.
Study design and conductnone givenTo ensure evaluation consistency, the paper primarily refers to results reported in original papers or reproduced under aligned experimental settings by other studies.
“To ensure evaluation consistency, we primarily refer to results reported in the original papers or reproduced under aligned experimental settings by other studies.”
Quote in the paperSubstantial claimMay combine two claims.
PremisecitationOpen X-Embodiment integrates diverse robotic forms and environmental data from 21 institutions, emphasizing generalizability and transferability of multi-robot systems.
“Unlike these, Open X-Embodiment [250] adopts a broader taskagnostic perspective by integrating diverse robotic forms and environmental data from 21 institutions.It emphasizes the generalizability and transferability of multi-robot systems, serving as a valuable resource for advancing cross-platform research in robotics.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [250] Silberman, 2012, Indoor segmentation and support inference from rgbd images source
May combine two claims.
PremisecitationTouch and Go collects paired visual and tactile data from real-world environments and explores cross-modal relationships using self-supervised learning.
“Touch and Go [245] collects paired visual and tactile data from real-world environments and explores cross-modal relationships using self-supervised learning methods.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [245] D. Azuma, T. Miyanishi, S. Kurita, M. Kawanabe, Scanqa: 3d question answering for spatial scene understanding, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 19129–19139. source
May combine two claims.
PremisecitationGrasp-Anything++ extends to fine-grained grasping tasks, supporting language-based localized grasping.
“Its successor, Grasp-Anything++ [249], further extends to fine-grained grasping tasks, supporting language-based localized grasping, such as manipulating specific parts of an object.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [249] M. Cordts, M. Omran, S. Ramos, T. Rehfeld, M. Enzweiler, R. Benenson, U. Franke, S. Roth, B. Schiele, The cityscapes dataset for semantic urban scene understanding, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 3213–3223. source
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationMobileViT introduces a hybrid architecture leveraging CNNs and ViTs, reducing computational costs while maintaining high accuracy.
“For instance, MobileViT [231] introduces a novel hybrid architecture that leverages the strengths of both convolutional neural networks (CNNs) and vision transformers (ViTs), effectively reducing computational costs while maintaining high accuracy.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [231] A. Akbik, D. Blythe, R. Vollgraf, Contextual string embeddings for sequence labeling, in: Proceedings of the 27th International Conference on Computational Linguistics, 2018, pp. 1638–1649.
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationCVDN provides 2,050 human dialogs simulating scenarios where agents request assistance through natural language interactions during navigation.
“CVDN [124] provides 2,050 human dialogs simulating scenarios where agents request assistance through natural language interactions during navigation.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [124] Y. Zhao, J. Chen, C. Gao, W. Wang, L. Yang, H. Ren, H. Xia, S. Liu, Target-driven structured transformer planner for vision-language navigation, in: Proceedings of the 30th ACM International Conference on Multimedia, 2022, pp. 4194–4203. source
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationMatterport3D provides 10,800 panoramic RGB-D views, accurate 3D reconstructions, and semantic annotations, making it widely applicable for semantic segmentation, 3D reconstruction, and multi-task learning.
“Matterport3D [26] provides 10,800 panoramic RGB-D views, along with accurate 3D reconstructions and semantic annotations, making it widely applicable for semantic segmentation, 3D reconstruction, and multi-task learning.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [26] Chang, 2017 source
May combine two claims.
Premisecitation360+x combines 360° panoramic views, first-person and third-person perspectives, and integrates audio and textual descriptions, providing a foundation for multi-view scene understanding and multimodal fusion.
“For instance, 360+x [241] combines 360° panoramic views, firstperson, and third-person perspectives while integrating modalities such as audio and textual descriptions, providing a rich ex-perimental foundation for multi-view scene understanding and multimodal fusion research.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [241] X. Zhang, M. Qiao, L. Liu, Y. Xu, W. Shi, Collaborative cloud-edge computation for personalized driving behavior modeling, in: Proceedings of the 4th ACM/IEEE Symposium on Edge Computing, 2019, pp. 209–221. source
May combine two claims.
PremisecitationMatterport3D provides 90 building-scale indoor scenes, 10,800 panoramic views, and 194,400 RGB-D images with 2D and 3D semantic annotations.
“Matterport3D [26] is the foundational dataset, providing 90 building-scale indoor scenes, 10,800 panoramic views, and 194,400 RGB-D images with 2D and 3D semantic annotations.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [26] Chang, 2017 source
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Premisenone givenGrasp-Anything-6D combines point cloud and language modalities, employing a negative prompt strategy to optimize accuracy of 6-DoF grasp detection.
“Meanwhile, Grasp-Anything-6D combines point cloud and language modalities, employing an innovative negative prompt strategy to optimize the accuracy of 6-DoF grasp detection.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationThe Waymo Open Dataset contains 12 million 3D bounding box annotations and supports cross-domain generalization and multimodal sensor fusion research.
“The Waymo [236] dataset leads in terms of geographic coverage and data scale, containing 12 million 3D bounding box annotations and supporting cross-domain generalization and multimodal sensor fusion research.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [236] Zwolenski, 2020, The digital universe: Rich data and the increasing value of the internet of things, J. Telecommun. Digit. Econ. source
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisereasoningA lack of standardization hampers development efficiency and limits the potential benefits of collaboration.
“This lack of standardization hampers development efficiency and limits the potential benefits of collaboration.”
Quote in the paperNot a substantial claimMay combine two claims.
PremisecitationVisGel constructs bidirectional generation tasks between tactile and visual modalities, validating feasibility of inter-modal information transformation.
“VisGel [246], leveraging GelSight sensors, constructs bidirectional generation tasks between tactile and visual modalities, further validating the feasibility of inter-modal information transformation.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [246] M. Roberts, J. Ramapuram, A. Ranjan, A. Kumar, M.A. Bautista, N. Paczan, R. Webb, J.M. Susskind, Hypersim: A photorealistic synthetic dataset for holistic indoor scene understanding, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021, pp. 10912–10922. source
May combine two claims.
PremisecitationObjectFolder 2.0 achieves efficient rendering of visual, auditory, and tactile modalities with high-quality virtual object data and implicit neural representation, supporting virtual-to-reality transfer learning.
“ObjectFolder 2.0 [247], with its high-quality virtual object data and implicit neural representation technology, achieves efficient rendering of visual, auditory, and tactile modalities, supporting virtualto-reality transfer learning tasks.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [247] P. Sun, H. Kretzschmar, X. Dotiwalla, A. Chouard, V. Patnaik, P. Tsui, J. Guo, Y. Zhou, Y. Chai, B. Caine, et al., Scalability in perception for autonomous driving: Waymo open dataset, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 2446–2454. source
May combine two claims.
PremisereasoningLiDAR-camera fusion leverages precise geometric information from LiDAR and rich semantic features from cameras, excelling in object localization and classification.
“LiDAR-camera fusion leverages the precise geometric information of LiDAR and the rich semantic features of cameras, excelling in object localization and classification.”
Quote in the paperNot a substantial claimMay combine two claims.
PremisecitationCOLOSSEUM primarily assesses robustness and multi-task learning, making it more aligned with real-world robotic applications and valuable for sim-to-real transfer.
“COLOSSEUM [294] primarily assesses robot robustness and multi-task learning performance, making it more aligned with real-world robotic applications compared to other simulation-only benchmarks, making it highly valuable for sim-to-real transfer research.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [294] Hao, 2025
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisereasoningRavens uses Transporter Networks for visuomotor policy learning and PyBullet as its physics engine.
“It employs Transporter Networks for efficient visuomotor policy learning and uses PyBullet as the underlying physics engine.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Premisenone givenPyBullet is an open-source lightweight physics engine widely used for reinforcement learning training, with high computational efficiency and OpenAI Gym integration.
“PyBullet, an open-source lightweight physics engine, is widely adopted for reinforcement learning training, offering high computational efficiency and seamless integration with OpenAI Gym.”
Quote in the paperNot a substantial claimMay combine two claims.
PremisecitationTD-STP and DUET use structured transformer-based planning and achieve better global search efficiency.
“TD-STP [121] and DUET [115] employ structured transformer-based planning for better global search efficiency.”
Quote in the paperNot a substantial claim0 of 2 sources checked- cites [121] W. Hao, C. Li, X. Li, L. Carin, J. Gao, Towards learning a generic agent for vision-and-language navigation via pre-training, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 13137–13146. source
- cites [115] Chen, 2024
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
May combine two claims.
PremisecitationLarge-scale pretrained models such as Qwen2-VL achieve high accuracy in inference-intensive tasks through vision-language alignment, supporting the importance of cross-modal alignment for 3D question answering.
“Notably, large-scale pretrained models such as Qwen2-VL [287] achieve high accuracy in inference-intensive tasks through vision-language alignment, further validating the importance of cross-modal alignment for 3D question answering (3D-QA).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [287] J. Chen, C. Gao, E. Meng, Q. Zhang, S. Liu, Reinforced structured state-evolution for vision-language navigation, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 15450–15459. source
May combine two claims.
PremisecitationGenSim2 allows dynamic adjustment of task complexity and policy testing across environments, making it ideal for policy generalization and real-world deployment.
“It allows researchers to dynamically adjust task complexity and test policies across different environments, making it an ideal platform for policy generalization and realworld deployment.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [295] Zhang, 2024
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
May combine two claims.
InterpretationreasoningThe navigation results indicate that structured memory, environment adaptation, and multimodal pretraining are increasingly important for efficient and adaptable embodied navigation.
“These results highlight the growing importance of structured memory, environment adaptation, and multimodal pretraining in embodied navigation, pushing the field toward more efficient and adaptable navigation systems.”
Quote in the paperSubstantial claimMay combine two claims.
Premisenone givenRavens excels in vision-guided manipulation but has a narrow scope, mainly grasping and placing, with limited multi-step planning support.
“While Ravens excels in vision-guided manipulation, its scope is relatively narrow, primarily focusing on grasping and placing tasks, with limited support for multi-step planning.”
Quote in the paperNot a substantial claimMay combine two claims.
PremisecitationCALVIN, based on PyBullet, includes 34 tasks and emphasizes long-horizon planning and multi-task adaptability.
“CALVIN [293], based on PyBullet, includes 34 tasks and emphasizes long-horizon planning and multi-task adaptability.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [293] M. Hwang, J. Jeong, M. Kim, Y. Oh, S. Oh, Meta-explore: Exploratory hierarchical vision-and-language navigation using scene object spectrum grounding, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 6683–6693. source
May combine two claims.
PremisecitationSM and SSM improve trajectory reasoning, and SSM's structured memory leads to better long-term path efficiency.
“SM [268] and SSM [275] improve trajectory reasoning, with SSM's structured memory leading to better long-term path efficiency.”
Quote in the paperNot a substantial claim0 of 2 sources checked- cites [268] Zhao, 2024
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites [275] Wolters, 2024
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Premiseown dataRavens is a Google Robotics simulation platform for robotic grasping and object manipulation, specializing in vision-based manipulation.
“Additionally, Ravens, developed by Google Robotics, is a robotic grasping and object manipulation simulation platform specializing in vision-based manipulation tasks.”
Quote in the paperNot a substantial claimMay combine two claims.
PremisecitationIsaac Sim provides high-fidelity physics, is ideal for reinforcement learning, imitation learning, and robotic vision, and is used in KitchenShift and GenSim2.
“It provides high-fidelity physics simulation, making it ideal for reinforcement learning (RL), imitation learning (IL), and robotic vision tasks, and is widely used in KitchenShift [292] and GenSim2 [295] benchmarks.”
Quote in the paperNot a substantial claim0 of 2 sources checked- cites [292] J. Li, H. Tan, M. Bansal, Envedit: Environment editing for vision-and-language navigation, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 15407–15417. source
- cites [295] Zhang, 2024
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
May combine two claims.
PremisecitationCOLOSSEUM supports real-world reproducibility, includes 20 tasks, and introduces environmental perturbations to test robots under different physical conditions.
“COLOSSEUM [294], on the other hand, extends beyond simulated environments, supporting real-world reproducibility.It includes 20 tasks and introduces environmental perturbations, allowing robots to be tested under different physical conditions.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [294] Hao, 2025
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Premisenone givenIsaac Sim is a GPU-accelerated PhysX-based simulation platform that supports large-scale parallel simulation and direct sim-to-real deployment.
“In contrast, Isaac Sim, developed by NVIDIA, is a GPUaccelerated physics simulation platform based on the PhysX engine, supporting large-scale parallel simulations and enabling direct deployment in sim-to-real tasks.”
Quote in the paperNot a substantial claimMay combine two claims.
PremisecitationPyBullet powers CALVIN and Ravens, but its physics accuracy is lower than Isaac Sim and MuJoCo, especially in complex multi-body interactions.
“It powers benchmarks such as CALVIN [293] and Ravens, but its physics accuracy is lower than that of Isaac Sim and MuJoCo, particularly in complex multi-body interactions.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [293] M. Hwang, J. Jeong, M. Kim, Y. Oh, S. Oh, Meta-explore: Exploratory hierarchical vision-and-language navigation using scene object spectrum grounding, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 6683–6693. source
May combine two claims.
InterpretationreasoningOverall, LLaVA-3D and 3D-LLM lead in 3D scene reasoning, Chat-Scene and Scene-LLM perform best in object-based understanding, and LEO excels in long-term task execution, reflecting trade-offs among generalization, spatial alignment, and embodied interaction.
“Overall, LLaVA-3D and 3D-LLM lead in 3D scene reasoning, Chat-Scene and Scene-LLM perform best in object-based understanding tasks, and LEO excels in long-term task execution, reflecting the trade-offs between generalization capability, spatial alignment, and embodied interaction.”
Quote in the paperSubstantial claimMay combine two claims.
PremisecitationGenSim2 is real-world-capable, uses MuJoCo and Isaac Sim, supports variable task configurations, and focuses on large-scale data generation and sim-to-real transfer.
“GenSim2 [295], another real-world-capable benchmark, leverages MuJoCo and Isaac Sim to support variable task configurations, focusing on large-scale data generation and simto-real transfer.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [295] Zhang, 2024
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
May combine two claims.
PremisecitationVLMbench is suited for evaluating vision-language models such as CLIP and GPT-4V, but its 8-task set may limit comprehensiveness for benchmarking generalizable policies.
“While VLMbench [290] is well-suited for evaluating visionlanguage models (VLMs) such as CLIP and GPT-4V, its task set is relatively small (only 8 tasks), potentially limiting its comprehensiveness in benchmarking generalizable policies.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [290] Y. Hong, Q. Wu, Y. Qi, C. Rodriguez-Opazo, S. Gould, Vln bert: A recurrent vision-and-language bert for navigation, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021, pp. 1643–1653. source
May combine two claims.
PremisecitationUnlike COLOSSEUM, GenSim2 prioritizes data scalability and adaptability and supports traditional control methods such as dynamics modeling.
“Unlike COLOSSEUM [294], which focuses on environmental perturbation robustness, GenSim2 [295] prioritizes data scalability and adaptability, with additional support for traditional control methods such as dynamics modeling.”
Quote in the paperNot a substantial claim0 of 2 sources checked- cites [294] Hao, 2025
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites [295] Zhang, 2024
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationVIMA includes 17 tasks that test a robot's ability to understand complex task instructions and execute them.
“VIMA [291] includes 17 tasks that test a robot's ability to understand complex task instructions and execute them accordingly.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [291] C. Liu, F. Zhu, X. Chang, X. Liang, Z. Ge, Y.-D. Shen, Vision-language navigation with random environmental mixup, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021, pp. 1644–1654. source
May combine two claims.
PremisecitationKitchenShift focuses on robot generalization under environmental domain shifts in a simulated kitchen, introducing shifts in object materials, lighting, and camera angles to evaluate policy robustness.
“In contrast, KitchenShift [292] focuses on robot generalization under environmental domain shifts, simulating a kitchen environment where the robot interacts with various objects, such as opening an oven, moving utensils, and mixing liquids.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [292] J. Li, H. Tan, M. Bansal, Envedit: Environment editing for vision-and-language navigation, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 15407–15417. source
May combine two claims.
Interpretationnone givenMemory and environmental adaptation are important for generalization in embodied navigation.
“Memory and environmental adaptation have also played a key role in generalization.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.
Premisenone givenKnowledge distillation, contrastive learning, and dynamic data augmentation enhance lower-quality modalities, improving integration and model robustness.
“Additionally, knowledge distillation, contrastive learning, and dynamic data augmentation enhance lower-quality modalities, improving integration and model robustness.”
Quote in the paperNot a substantial claimMay combine two claims.
InterpretationreasoningComputational cost limits practical application scope and increases energy consumption and cost, especially on edge devices or in real-time applications.
“This computational cost not only limits the practical application scope of the model, but also increases energy consumption and cost, especially in edge devices or real-time application scenarios (e.g., autonomous driving).”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationIn autonomous driving, fusion weights of RGB images and LiDAR point clouds can be adaptively adjusted according to ambient lighting.
“For example, in autonomous driving, the fusion weights of RGB images and LiDAR point clouds can be adaptively adjusted according to the ambient lighting conditions [82].”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [82] D. Xu, D. Anguelov, A. Jain, Pointfusion: Deep sensor fusion for 3d bounding box estimation, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 244–253. source
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationCLIP uses Visual-Linguistic Contrast learning so that images and text share the same latent space.
“e.g., CLIP [181] employs Visual-Linguistic Contrast learning so that images and text share the same latent space.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [181] Berg, 2024, Multimodal supervised contrastive learning in remote sensing downstream tasks, IEEE Geosci. Remote. Sens. Lett. source
May combine two claims.
InterpretationreasoningThese dataset problems limit learning of rich and accurate cross-modal semantic alignment and lead to overfitting to specific contexts or tasks, hindering generalisation.
“These problems not only limit the models from learning rich and accurate cross-modal semantic alignment relations, but also easily lead to overfitting the models to specific contexts or tasks, making it difficult to effectively generalise them to wider and more realistic application scenarios.”
Quote in the paperSubstantial claimMay combine two claims.
InterpretationreasoningHeterogeneity makes direct multimodal fusion difficult, making uniform representation learning, feature alignment, and information interaction key issues.
“This heterogeneity makes it difficult to directly fuse multimodal information, making uniform represen-tation learning, feature alignment, and information interaction key issues.”
Quote in the paperSubstantial claimMay combine two claims.
PremisereasoningCurrent research addresses efficiency via lightweight network architecture design, model compression and knowledge distillation, and dynamic modal fusion.
“In order to alleviate the above problems, current research mainly proposes solutions from lightweight network architecture design, model compression and knowledge distillation, and dynamic modal”
Quote in the paperNot a substantial claimMay combine two claims.
PremisecitationKnowledge distillation migrates knowledge from large teacher models to lightweight student models, significantly improving training efficiency while maintaining performance.
“on the other hand, the knowledge from the large-scale teacher model is migrated to the lightweight student model through knowledge distillation [310], which significantly improves the training efficiency while maintaining the performance of the while significantly improving the training efficiency.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [310] Mees, 2022, Calvin: A benchmark for language-conditioned policy learning for long-horizon robot manipulation tasks, IEEE Robot. Autom. Lett. source
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
InterpretationreasoningAlthough heterogeneity methods alleviate the problem to some extent, many challenges remain.
“Although these methods alleviate the heterogeneity problem to some extent, there are still many challenges.”
Quote in the paperNot a substantial claimMay combine two claims.
PremisereasoningThree main approaches address heterogeneity: unified feature space learning, modality-specific encoders, and adaptive modality fusion.
“To address the heterogeneity challenge, researchers have proposed three main types of approaches:”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing), S4 (based).
Premisenone givenModality Dropout or Adaptive Modality Selection can reduce redundant modal computations.
“In addition, Modality Dropout or Adaptive Modality Selection methods can reduce redundant modal computations.”
Quote in the paperNot a substantial claimMay combine two claims.
InterpretationcitationOverall, RLBench and CALVIN suit large-scale multi-task reinforcement learning; GemBench, VLMbench, and VIMA focus on vision-language models in manipulation; KitchenShift and COLOSSEUM emphasize domain generalization and real-world adaptability; GenSim2 excels in data scalability and sim-to-real transfer.
“Overall, these benchmarks cater to different research objectives.RLBench [288] and CALVIN [293] are well-suited for large-scale multi-task RL research, while GemBench [289], VLMbench [290], and VIMA [291] focus on vision-language models in robotic manipulation.KitchenShift [292] and COLOSSEUM [294] emphasize domain generalization and real-world adaptability, whereas GenSim2 [295] excels in data sc”
Quote in the paperNot a substantial claimMay combine two claims. The quote only partly states this claim.
InterpretationreasoningThe survey identifies four main challenges: low-quality data, heterogeneity, efficient training and inference, and scarce high-quality datasets.
“As illustrated in Figure 11, these challenges primarily include managing uncertainties caused by low-quality data, addressing the computational complexity inherent in multimodal data processing, effectively modeling and fusing crossmodal features, and dealing with dataset limitations such as small dataset sizes and insufficient semantic diversity.”
Quote in the paperNot a substantial claimMay combine two claims.
Interpretationnone givenChallenges remain in assessment, uneven modality information, and incomplete annotations for incomplete multimodal learning.
“Challenges remain in assessment, uneven modality information, and incomplete annotations.”
Quote in the paperNot a substantial claimMay combine two claims.
Interpretationnone givenLow-quality data seriously affects multimodal learning performance and makes stable operation in complex environments difficult.
“However, in the real world, the presence of low-quality data seriously affects the performance of multimodal learning, making it difficult for models to work stably in complex environments.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Premisenone givenAdaptive modality fusion dynamically adjusts fusion according to data quality, information contribution, and task requirements.
“Adaptive Modality Fusion, which dynamically adjusts the fusion method according to the data quality, information contribution and task requirements of different modalities.”
Quote in the paperNot a substantial claimMay combine two claims.
InterpretationcitationMost current benchmarks remain simulation-dependent, while COLOSSEUM and GenSim2 are gradually bridging the gap toward real-world robotic deployment.
“While most current benchmarks remain simulation-dependent, COLOSSEUM [294] and GenSim2 [295] are gradually bridging the gap toward realworld robotic deployment, providing more valuable insights into real-world policy learning.”
Quote in the paperNot a substantial claim0 of 2 sources checked- cites [294] Hao, 2025
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites [295] Zhang, 2024
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Rests on an assumption: Performance on simulation-only benchmarks provides useful insight into real-world robotic deployment.
Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).
Premisenone givenIn medical diagnosis, key modalities such as PET scans may be unavailable.
“in medical diagnosis, key modalities like PET scans may be unavailable.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Premisenone givenMultimodal noise can be divided into modality-specific noise and cross-modal noise.
“Depending on the source, multimodal noise can be divided into two categories, modalityspecific noise and cross-modal noise.”
Quote in the paperNot a substantial claimMay combine two claims.
PremisecitationOptimization techniques, gradient modulation, and adaptive architectures help balance training under modality imbalance.
“To mitigate this, optimization techniques, gradient modulation, and adaptive architectures [304,305] help balance training.”
Quote in the paperNot a substantial claim0 of 2 sources checked- cites [304] S. Chen, X. Chen, C. Zhang, M. Li, G. Yu, H. Fei, H. Zhu, J. Fan, T. Chen, Ll3da: Visual interactive instruction tuning for omni-3d understanding reasoning and planning, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 26428–26438. source
- cites [305] Wang, 2024
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Classified as probably not a substantial claim. May combine two claims.
Premisenone givenImage data are high-dimensional continuous values, audio data are time-series signals, and text data are discrete symbol sequences.
“For example, image data are usually high-dimensional continuous values, audio data are time-series signals, and text data are discrete symbol sequences.”
Quote in the paperSubstantial claimMay combine two claims.
InterpretationreasoningDue to heterogeneity between visual and linguistic modalities, models may face low training efficiency, high inference latency, and high memory consumption during joint optimisation.
“In addition, due to the obvious heterogeneity of visual and linguistic modalities in terms of data dimensions and feature space, the models may face low training efficiency, high inference latency, and high memory consumption during joint optimisation.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S4 (based).
Interpretationnone givenDifferent modalities' distributions in high-dimensional space may be skewed, causing information loss or feature mismatch during direct fusion.
“For example, the distribution of different modalities in high-dimensional space may be skewed, leading to information loss or feature mismatch during direct fusion.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
PremisereasoningDomain-specific datasets closer to the physical world can be constructed using emulators or world models to interactively generate data and enhance VLM performance in the real physical world.
“constructing datasets closer to the physical world in specific domains (e.g., robotic manipulation, automated driving), using emulators or world models to interactively generate data to enhance the performance of VLM in the real physical world.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S4 (based). May combine two claims.
Interpretationnone givenDespite advances, challenges such as data heterogeneity, low-quality annotations, and real-time computational constraints persist.
“Despite these advancements, challenges such as data heterogeneity, low-quality annotations, and real-time computational constraints persist.”
Quote in the paperNot a substantial claimMay combine two claims.
PremisereasoningCurrent research explores three avenues: self-supervised or weakly supervised learning, synthetic dataset generation, and domain-specific datasets via simulators or world models.
“To alleviate this challenge, current research has explored three main avenues:”
Quote in the paperNot a substantial claim
The paper relies on this without support.
Efficiently integrating heterogeneous data from different modalities remains a core issue involving modality alignment, unified feature representation, and spatiotemporal synchronization.
“First, efficiently integrating heterogeneous data from different modalities remains a core issue, involving problems such as modality alignment, unified feature representation, and spatiotemporal synchronization.”
The paper relies on this without support.
Robotic systems impose stringent requirements on real-time performance and resource efficiency, necessitating a balance between computational cost and model accuracy when designing fusion architectures.
“Second, robotic systems impose stringent requirements on realtime performance and resource efficiency, necessitating a balance between computational cost and model accuracy when designing fusion architectures.”
The paper relies on this without support.
Pretrained VLMs have limited adaptability to specific robotic tasks, particularly in scenarios with limited annotated data or dynamic environments.
“Third, although pretrained VLMs have demonstrated strong general capabilities, their adaptability to specific robotic tasks is still limited-particularly in scenarios with limited annotated data or dynamic environments.”
The paper relies on this without support.
A single sensor often cannot meet the requirements for safe and reliable perception.
“A single sensor often cannot meet the requirements for safe and reliable perception.”
The paper relies on this without support.
LiDAR data is often sparse, especially at long distances or on low-reflectivity surfaces, making it challenging to detect small or distant objects.
“However, LiDAR data is often sparse, especially at long distances or on low-reflectivity surfaces, making it challenging to detect small or distant objects.”
The paper relies on this without support.
The absence of color and texture in LiDAR limits its ability to distinguish visually similar but categorically different objects.
“Additionally, the absence of color and texture limits its ability to distinguish visually similar but categorically different objects.”
The paper relies on this without support.
4D radar provides 3D positional data and velocity measurements, improving dynamic object tracking and identification.
“4D radar not only provides 3D positional data but also includes velocity measurements, improving dynamic object tracking and identification.”
The paper relies on this without support.
Embodied navigation depends on multimodal information such as visual, tactile, and auditory inputs to guide intelligent agents in dynamic unstructured environments.
“Embodied Navigation relies on multimodal information (e.g., visual, tactile, and auditory inputs) to guide intelligences to act efficiently in dynamic and unstructured environments.”
The paper relies on this without support.
The benchmark metrics used in the survey adequately measure the robot vision capabilities being compared.
“Table 7 evaluates different approaches based on TL, NE, OSR, SR, and SPL, providing insights into their effectiveness in seen and unseen environments.”
The paper relies on this without support.
The cited benchmark results are comparable across methods despite differences in implementations and evaluation protocols.
“The comparison of navigation models on the Room-to-Room (R2R) dataset highlights advancements in embodied navigation and the role of multimodal learning in improving generalization.”
The paper relies on this without support.
Performance on simulation-only benchmarks provides useful insight into real-world robotic deployment.
“While most current benchmarks remain simulation-dependent, COLOSSEUM [294] and GenSim2 [295] are gradually bridging the gap toward realworld robotic deployment, providing more valuable insights into real-world policy learning.”
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
OccFusion is cited with the OpenOccupancy reference number
MinorThe discussion cites OccuFusion as [62], whereas the paper's reference list identifies [62] as OpenOccupancy and [64] as Occfusion.
“Methods like OccuFusion [62] use SENet modules [63] to adaptively weight modality features, improving generalization and reliability.”
“64. Z. Ming, J. S. Berrio, M. Shan, S. Worrall, Occfusion: Multi-sensor fusion framework for 3d semantic occupancy prediction, IEEE Transac- tions on Intelligent Vehicles (2024).”
PaLM-E is assigned to 2022 despite its 2023 reference
MinorThe Figure 10 caption places PaLM-E among early-2022 models, while the cited PaLM-E reference is dated 2023.
“early 2022 (e.g., Flamingo [203], PaLM-E [204])”
“204. D. Driess, F. Xia, M. S. Sajjadi, C. Lynch, A. Chowdhery, B. Ichter, A. Wahid, J. Tompson, Q. Vuong, T. Yu, et al., Palm-e: An embodied multimodal language model, arXiv preprint arXiv:2303.03378 (2023).”
Quadratic attention cost is described as exponential
MinorThe stated O(N2) complexity implies quadratic growth with sequence length, not exponential growth.
“processing long-sequence inputs led to an exponential increase in computational cost.”
“Due to the quadratic complexity O(N 2 ) of the self-attention mechanism”
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · cited 1× · see citing works
“Point-Fusion [81] proposed a dense fusion network that fuses point cloud features extracted by PointNet [108] and image features extracted by ResNet [109].”
Self-citationVaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · see citing works
Jakubův, J., Chvalovský, K., Goertzel, Z., et al. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,648 · see citing works
Squeeze-and-Excitation Networks
Hu, J., Shen, L., & Sun, G. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00745 · Cited by 35,667 · cited 7× · see citing works
“MRFTrans [4] 2024 DefFusion [14] 2024 DMA [64] 2024 Multi-Sem Fusion [65] 2024 CLFT [66] 2024 OccFusion [69] 2024 HyDRa [16] 2024 CONet [67] 2023 SENET [68] 2018 Graph Neural Network MISSIONGNN [70] 2024 VQA-GNN [71] 2023 MuSe-GNN [72] 2023 CCA-GNN [73] 2023 MGA [74] 2023 MGNNS [75] 2021 ods favors…”
InfluentialSwin Transformer: Hierarchical Vision Transformer using Shifted Windows
Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., & Guo, B. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00986 · Cited by 35,534 · cited 1× · see citing works
“Subsequently, Swin Trans-former [208] introduced shifted window attention, significantly improving computational efficiency and performance in dense prediction tasks such as object detection and semantic segmentation.”
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372 · see citing works
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al. (2020) · arXiv · doi:10.48550/arxiv.2010.11929 · Cited by 21,793 · cited 2× · see citing works
“Vision Transformer (ViT) [207] demonstrated that Transformer architectures could be directly applied to image classification tasks, achieving performance surpassing CNNs on large-scale datasets.”
Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation
Chen, L.-C., Zhu, Y., Papandreou, G., Schroff, F., & Adam, H. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01234-2_49 · Cited by 20,054 · cited 2× · see citing works
“DeepLabv3+ [58] is a classic encoder-decoder framework for semantic segmentation tasks.”
The pascal visual object classes (VOC) challenge
Everingham, M., Van Gool, L., Williams, C. K. I., Winn, J., & Zisserman, A. (2010) · International Journal of Computer Vision · doi:10.1007/s11263-009-0275-4 · Cited by 19,599 · see citing works
End-to-End Object Detection with Transformers
Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., & Zagoruyko, S. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58452-8_13 · Cited by 18,285 · see citing works
Are we ready for autonomous driving? The KITTI vision benchmark suite
Geiger, A., Lenz, P., & Urtasun, R. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6248074 · Cited by 14,897 · see citing works
Kaugeranna, R., Kaugeranna, E., & Aion, (. S. 4. 6. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.giscience.2023.43 · Cited by 13,695 · see citing works
The Cityscapes Dataset for Semantic Urban Scene Understanding
Cordts, M., Omran, M., Ramos, S., Rehfeld, T., Enzweiler, M., Benenson, R., Franke, U., Roth, S., & Schiele, B. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.350 · Cited by 12,433 · cited 1× · see citing works
“Localization and navigation metrics are directly related to practical robot performance but require extensive ground-truth data.”
Edge computing: Vision and challenges
Shi, W., Cao, J., Zhang, Q., Li, Y., & Xu, L. (2016) · IEEE Internet of Things Journal · doi:10.1109/jiot.2016.2579198 · Cited by 8,733 · cited 1× · see citing works
“To address these, edge computing has emerged to process data directly at the network edge [239], handling both downstream cloud data and up-stream IoT data.”
nuScenes: A Multimodal Dataset for Autonomous Driving
Caesar, H., Bankiti, V., Lang, A. H., Vora, S., Liong, V. E., Xu, Q., Krishnan, A., Pan, Y., Baldan, G., & Beijbom, O. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01164 · Cited by 7,152 · cited 5× · see citing works
“For 3D object detection , which is crucial for autonomous driving and robotic manipulation, key metrics include Average Precision (AP), mean Average Precision (mAP), and the nuScenes Detection Score (NDS) [23].”
InfluentialNeRF: representing scenes as neural radiance fields for view synthesis
Mildenhall, B., Srinivasan, P. P., Tancik, M., Barron, J. T., Ramamoorthi, R., & Ng, R. (2022) · Communications of the ACM · doi:10.1145/3503250 · Cited by 6,852 · see citing works
A performance evaluation of local descriptors
Mikolajczyk, K., & Schmid, C. (2005) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2005.188 · Cited by 6,723 · cited 1× · see citing works
“Relying on traditional single-modal approaches (e.g., solely using RGB images) can encounter various limitations in complex scenarios, such as occlusion, lighting variations, and multi-object ambiguities [36, 37, 38].”
Indoor segmentation and support inference from rgbd images
Silberman, N., Hoiem, D., Kohli, P., & Fergus, R. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-33715-4_54 · Cited by 5,760 · see citing works
Emerging Properties in Self-Supervised Vision Transformers
Caron, M., Touvron, H., Misra, I., Jegou, H., Mairal, J., Bojanowski, P., & Joulin, A. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00951 · Cited by 5,638 · cited 1× · see citing works
“The modality combinations and task types presented here directly impact model performance and generalization, which are further analyzed in the following section.”
NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis
Mildenhall, B., Srinivasan, P. P., Tancik, M., Barron, J. T., Ramamoorthi, R., & Ng, R. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58452-8_24 · Cited by 5,351 · see citing works
Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations
Krishna, R., Zhu, Y., Groth, O., Johnson, J., Hata, K., Kravitz, J., Chen, S., Kalantidis, Y., Li, L.-J., Shamma, D. A., et al. (2017) · International Journal of Computer Vision · doi:10.1007/s11263-016-0981-7 · Cited by 5,107 · cited 1× · see citing works
“However, existing large-scale visual language datasets (e.g., Visual Genome [331], Conceptual Cap-tions [332], and LAION-400m [333]) generally suffer from limited size, high annotation noise, insufficient semantic diversity, and expensive annotation.”
Self-citationDeep High-Resolution Representation Learning for Visual Recognition
Wang, J., Sun, K., Cheng, T., Jiang, B., Deng, C., Zhao, Y., Liu, D., Mu, Y., Tan, M., Wang, X., et al. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.2983686 · Cited by 4,894 · cited 2× · see citing works
“HRNet [60] achieves precise semantic segmentation by preserving high-resolution feature maps.”
VINS-Mono: A Robust and Versatile Monocular Visual-Inertial State Estimator
Qin, T., Li, P., & Shen, S. (2018) · IEEE Transactions on Robotics · doi:10.1109/tro.2018.2853729 · Cited by 4,669 · cited 1× · see citing works
“Modality V-LOAM [150] LiDAR + RGB LVI-SAM [157] LiDAR + RGB LIC-Fusion [18] LiDAR + RGB + IMU LIC-Fusion 2.0 [152] LiDAR + RGB + IMU VINS-Fusion [158] RGB + IMU + GPS UVIO [153] RGB + IMU RD-VIO [6] Radar + RGB + IMU MM3DGS SLAM [154] RGB + IMU + Depth ConceptFusion [155] RGB + Audio Co-SLAM [156]…”
InfluentialScanNet: Richly-Annotated 3D Reconstructions of Indoor Scenes
Dai, A., Chang, A. X., Savva, M., Halber, M., Funkhouser, T., & Niessner, M. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.261 · Cited by 4,368 · cited 3× · see citing works
“Many widely used datasets are built upon Western-centric indoor layouts (e.g., ScanNet [245], Matterport3D [26]), contain only English-language instructions (e.g., Room-to-Room [258] for vision-language navigation), or are collected in urban driving environments (e.g., nuScenes [23], Waymo [243]).”
InfluentialALBERT: A Lite BERT for Self-supervised Learning of Language Representations
Lan, Z., Chen, M., Goodman, S., Gimpel, K., Sharma, P., & Soricut, R. (2019) · arXiv · doi:10.48550/arxiv.1909.11942 · Cited by 4,308 · cited 1× · see citing works
“To address the lack of object interaction in R2R [261], REVERIE [262] extends the capabilities of Matterport3D [26] by adding bounding box annotations for 4,140 target objects and 21,702 natural language instructions.”
MonoSLAM: Real-time single camera SLAM
Davison, A. J., Reid, I. D., Molton, N. D., & Stasse, O. (2007) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2007.1049 · Cited by 3,959 · cited 1× · see citing works
“Traditional SLAM systems include LiDAR-based[146, 147] and vision-based [148, 149] methods.”
LSD-SLAM: Large-scale direct monocular SLAM
Engel, J., Schöps, T., & Cremers, D. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10605-2_54 · Cited by 3,851 · see citing works
Masked-attention Mask Transformer for Universal Image Segmentation
Cheng, B., Misra, I., Schwing, A. G., Kirillov, A., & Girdhar, R. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.00135 · Cited by 3,565 · cited 2× · see citing works
“Representative Methods Year Encoder-Decoder Framework PIDNet [12] 2023 MAMT [62] 2022 DDRNet [61] 2021 PointRend [59] 2020 HRNet [60] 2019 DeepLabv3+ [58] 2018”
Multi-view 3D Object Detection Network for Autonomous Driving
Chen, X., Ma, H., Wan, J., Li, B., & Xia, T. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.691 · Cited by 3,450 · cited 8× · see citing works
“In 2017, MV3D [5] pioneered the fusion of Lidar’s projections in BEV (Bird’s Eye View) and FV (Front View) with RGB features, followed by AVOD [75], which uses FPN to extract high-resolution features for feature fusion; methods such as Pointfusion [81], MVX-Net [82], and Pointpainting [83] have…”
InfluentialScalability in Perception for Autonomous Driving: Waymo Open Dataset
Sun, P., Kretzschmar, H., Dotiwalla, X., Chouard, A., Patnaik, V., Tsui, P., Guo, J., Zhou, Y., Chai, Y., Caine, B., et al. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00252 · Cited by 3,437 · cited 4× · see citing works
“Many widely used datasets are built upon Western-centric indoor layouts (e.g., ScanNet [245], Matterport3D [26]), contain only English-language instructions (e.g., Room-to-Room [258] for vision-language navigation), or are collected in urban driving environments (e.g., nuScenes [23], Waymo [243]).”
InfluentialLOAM: Lidar Odometry and Mapping in Real-time
Zhang, J., & Singh, S. (2014) · Robotics: Science and Systems X · doi:10.15607/rss.2014.x.007 · Cited by 3,303 · cited 1× · see citing works
“Traditional SLAM systems include LiDAR-based[146, 147] and vision-based [148, 149] methods.”
PointRCNN: 3D Object Proposal Generation and Detection From Point Cloud
Shi, S., Wang, X., & Li, H. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00086 · Cited by 3,093 · see citing works
Unsupervised Learning of Depth and Ego-Motion from Video
Zhou, T., Brown, M., Snavely, N., & Lowe, D. G. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.700 · Cited by 2,910 · cited 1× · see citing works
“Super-vised methods excel in constrained settings by utilizing labeled datasets, while self-supervised approaches leverage geometric and photometric consistency to adapt effectively to unseen environments [139].”
PaLM: Scaling Language Modeling with Pathways
Chowdhery, A., Narang, S., Devlin, J., Bosma, M., Mishra, G., Roberts, A., Barham, P., Chung, H. W., Sutton, C., Gehrmann, S., et al. (2022) · arXiv · doi:10.48550/arxiv.2204.02311 · Cited by 2,590 · cited 1× · see citing works
“Compared to R2SGrasp [253], which focuses more on sim-to-real adaptation, the Grasp-Anything series exhibits superior semantic understanding in language-driven grasping tasks.”
Liu, H., Li, C., Wu, Q., & Lee, Y. J. (2023) · Advances in Neural Information Processing Systems 36 · doi:10.52202/075280-1516 · Cited by 2,428 · see citing works
Lego-loam: Lightweight and ground-optimized lidar odometry and mapping on variable terrain
Shan, T., & Englot, B. (2018) · 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2018.8594299 · Cited by 2,295 · see citing works
SemanticKITTI: A Dataset for Semantic Scene Understanding of LiDAR Sequences
Behley, J., Garbade, M., Milioto, A., Quenzel, J., Behnke, S., Stachniss, C., & Gall, J. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00939 · Cited by 2,294 · cited 2× · see citing works
“Based on the analysis of model architectures and training paradigms, we further explore the connections and differences between traditional multimodal methods and emerging VLMs [21, 22], revealing their complementarity and integration potential.”
Longformer: The Long-Document Transformer
Beltagy, I., Peters, M. E., & Cohan, A. (2020) · arXiv · doi:10.48550/arxiv.2004.05150 · Cited by 2,281 · cited 1× · see citing works
“However, the dataset is limited to monocular RGB images captured from a fixed onboard camera, and lacks 3D information and sensor diversity, making it less suitable for tasks that require depth reasoning, multimodal fusion, or adaptation to dynamic urban scenes.”
Multimodal Transformer for Unaligned Multimodal Language Sequences
Tsai, Y.-H. H., Bai, S., Liang, P. P., Kolter, J. Z., Morency, L.-P., & Salakhutdinov, R. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1656 · Cited by 2,259 · cited 1× · see citing works
“This dataset emphasizes semantic reasoning and object localization, but the increased task complexity poses significant challenges for existing models in handling semantic inference and object interaction.”
CrossViT: Cross-Attention Multi-Scale Vision Transformer for Image Classification
Chen, C.-F. R., Fan, Q., & Panda, R. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00041 · Cited by 2,242 · see citing works
2008 IEEE/RSJ International Conference on Intelligent Robots and Systems
(2008) · doi:10.1109/iros13647.2008 · Cited by 2,203 · see citing works
Center-based 3D Object Detection and Tracking
Yin, T., Zhou, X., & Krähenbühl, P. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01161 · Cited by 2,197 · see citing works
ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Tasks
Lu, J., Batra, D., Parikh, D., & Lee, S. (2019) · arXiv · doi:10.48550/arxiv.1908.02265 · Cited by 2,109 · cited 1× · see citing works
“(2) Modality-Specific Encoders, which designs special encoders for different modalities, using Transformer for text, CNN for image, RNN for audio, etc., and then performs feature alignment through Cross-modal Attention [325, 326].”
UNITER: UNiversal Image-TExt Representation Learning
Chen, Y.-C., Li, L., Yu, L., El Kholy, A., Ahmed, F., Gan, Z., Cheng, Y., & Liu, J. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58577-8_7 · Cited by 2,055 · see citing works
Conceptual Captions: A Cleaned, Hypernymed, Image Alt-text Dataset For Automatic Image Captioning
Sharma, P., Ding, N., Goodman, S., & Soricut, R. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p18-1238 · Cited by 1,879 · cited 1× · see citing works
“However, existing large-scale visual language datasets (e.g., Visual Genome [331], Conceptual Cap-tions [332], and LAION-400m [333]) generally suffer from limited size, high annotation noise, insufficient semantic diversity, and expensive annotation.”
Matterport3D: Learning from RGB-D Data in Indoor Environments
Chang, A., Dai, A., Funkhouser, T., Halber, M., Niebner, M., Savva, M., Song, S., Zeng, A., & Zhang, Y. (2017) · 2017 International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2017.00081 · Cited by 1,858 · cited 8× · see citing works
“Matterport3D [26] provides high-quality foundational data, while R2R [258] and CVDN introduce language and dialog interaction in path navigation tasks.”
InfluentialTinyBERT: Distilling BERT for natural language understanding
Jiao, X., Yin, Y., Shang, L., Jiang, X., Chen, X., Li, L., Wang, F., & Liu, Q. (2020) · Findings of the Association for Computational Linguistics: EMNLP 2020 · doi:10.18653/v1/2020.findings-emnlp.372 · Cited by 1,786 · cited 1× · see citing works
“By combining evaluation metrics with task-specific benchmarks, this chapter provides a structured overview of how multimodal methods perform under diverse robotic scenarios, laying the groundwork for the following analysis of strengths, limitations, and emerging trends.”
Li, J., Li, D., Xiong, C., & Hoi, S. (2022) · arXiv · doi:10.48550/arxiv.2201.12086 · Cited by 1,741 · see citing works
Joint 3D Proposal Generation and Object Detection from View Aggregation
Ku, J., Mozifian, M., Lee, J., Harakeh, A., & Waslander, S. L. (2018) · 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2018.8594049 · Cited by 1,697 · see citing works
Flamingo: a Visual Language Model for Few-Shot Learning
Alayrac, J.-B., Donahue, J., Luc, P., Miech, A., Barr, I., Hasson, Y., Lenc, K., Mensch, A., Millican, K., Reynolds, M., et al. (2022) · arXiv · doi:10.48550/arxiv.2204.14198 · Cited by 1,555 · cited 2× · see citing works
“As a result, there is a pressing need for a universal and cross-platform computational framework that can address these challenges.”
Reformer: The Efficient Transformer
Kitaev, N., Kaiser, Ł., & Levskaya, A. (2020) · arXiv · doi:10.48550/arxiv.2001.04451 · Cited by 1,435 · see citing works
Bevfusion: Multi-task multi-sensor fusion with unified bird’s-eye view representation
Liu, Z., Tang, H., Amini, A., Yang, X., Mao, H., Rus, D. L., & Han, S. (2023) · 2023 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra48891.2023.10160968 · Cited by 1,307 · see citing works
Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution
Wang, P., Bai, S., Tan, S., Wang, S., Fan, Z., Bai, J., Chen, K., Liu, X., Wang, J., Ge, W., et al. (2024) · arXiv · doi:10.48550/arxiv.2409.12191 · Cited by 1,302 · cited 2× · see citing works
“Scene Understanding [303] 69.4 12.0 14.5 35.7 20.5 LL3DA [304] 76.8 13.5 15.9 37.3 - Qwen2-VL-7B [305] 53.9 3.0 11.4 29.3 -”
InfluentialMultimodal fusion for multimedia analysis: a survey
Atrey, P. K., Hossain, M. A., El Saddik, A., & Kankanhalli, M. S. (2010) · Multimedia Systems · doi:10.1007/s00530-010-0182-0 · Cited by 1,279 · cited 1× · see citing works
“Post fusion is a key method in multimodal analysis, which combines the results of decision level independent processing of modalities [49].”
2012 IEEE Conference on Computer Vision and Pattern Recognition
(2012) · doi:10.1109/cvpr17489.2012 · Cited by 1,272 · see citing works
PointRend: Image Segmentation As Rendering
Kirillov, A., Wu, Y., He, K., & Girshick, R. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00982 · Cited by 1,244 · cited 2× · see citing works
“Representative Methods Year Encoder-Decoder Framework PIDNet [12] 2023 MAMT [62] 2022 DDRNet [61] 2021 PointRend [59] 2020 HRNet [60] 2019 DeepLabv3+ [58] 2018”
Habitat: A Platform for Embodied AI Research
Savva, M., Kadian, A., Maksymets, O., Zhao, Y., Wijmans, E., Jain, B., Straub, J., Liu, J., Koltun, V., Malik, J., et al. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00943 · Cited by 1,237 · cited 1× · see citing works
“In grasp planning, tactile feedback is crucial for path planning and posture adjustment.”
PointPainting: Sequential Fusion for 3D Object Detection
Vora, S., Lang, A. H., Helou, B., & Beijbom, O. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00466 · Cited by 1,236 · cited 2× · see citing works
“…and FV (Front View) with RGB features, followed by AVOD [75], which uses FPN to extract high-resolution features for feature fusion; methods such as Pointfusion [81], MVX-Net [82], and Pointpainting [83] have further explored the benefits of different fusion granularity feature fusion advantages.”
Anderson, P., Wu, Q., Teney, D., Bruce, J., Johnson, M., Sunderhauf, N., Reid, I., Gould, S., & van den Hengel, A. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00387 · Cited by 1,187 · cited 1× · see citing works
“This heterogeneity makes it difficult to directly fuse multimodal information, making uniform representation learning, feature alignment, and information interaction key issues.”
The Cityscapes Dataset for Semantic Urban Scene Understanding
Cordts, M., Omran, M., Ramos, S., Rehfeld, T., Enzweiler, M., Benenson, R., Franke, U., Roth, S., & Schiele, B. (2016) · arXiv · doi:10.48550/arxiv.1604.01685 · Cited by 1,174 · see citing works
2006 IEEE International Conference on Robotics and Automation
(2005) · IEEE Transactions on Robotics · doi:10.1109/tro.2005.847362 · Cited by 1,115 · see citing works
MobileViT: Light-weight, General-purpose, and Mobile-friendly Vision Transformer
Mehta, S., & Rastegari, M. (2021) · arXiv · doi:10.48550/arxiv.2110.02178 · Cited by 1,110 · see citing works
Finetuned Language Models Are Zero-Shot Learners
Wei, J., Bosma, M., Zhao, V. Y., Guu, K., Yu, A. W., Lester, B., Du, N., Dai, A. M., & Le, Q. V. (2021) · arXiv · doi:10.48550/arxiv.2109.01652 · Cited by 1,101 · see citing works
Voxel R-CNN: Towards High Performance Voxel-based 3D Object Detection
Deng, J., Shi, S., Li, P., Zhou, W., Zhang, Y., & Li, H. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i2.16207 · Cited by 1,048 · cited 1× · see citing works
“In contrast, LiDAR-based detection [78, 79, 80, 81] processes 3D point cloud data, offering precise depth estimation and spatial structure representation.”
The Dawn of LMMs: Preliminary Explorations with GPT-4V(ision)Preprint
Yang, Z.-Y., Li, L., Lin, K., Wang, J., Lin, C.-C., Liu, Z., & Wang, L.-J. (2023) · arXiv.org · doi:10.48550/arxiv.2309.17421 · Cited by 1,036 · cited 2× · see citing works
“In recent years, a new generation of VLMs has emerged, such as GPT-4V [213], BLIP-2 [220], and LLaVA-3D [221], demonstrating exceptional capabilities in open-vocabulary understanding, multimodal reasoning, and instruction following.”
Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity
Fedus, W., Zoph, B., & Shazeer, N. (2021) · arXiv · doi:10.48550/arxiv.2101.03961 · Cited by 1,005 · cited 1× · see citing works
“The Switch Trans-former [210] was one of the first MoE-based Transformers, activating experts selectively in each layer, reducing computational overhead while outperforming dense Transformers in NLP tasks.”
TransFusion: Robust LiDAR-Camera Fusion for 3D Object Detection with Transformers
Bai, X., Hu, Z., Zhu, X., Huang, Q., Chen, Y., Fu, H., & Tai, C.-L. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.00116 · Cited by 990 · cited 6× · see citing works
“Single-modality meth-ods, such as LiDAR-only [261, 13, 262] or camera-only [263, 264, 265], perform well in specific tasks but remain constrained in complex environments.”
InfluentialGemini: A Family of Highly Capable Multimodal Models
Gemini Team, Anil, R., Borgeaud, S., Alayrac, J.-B., Yu, J., Soricut, R., Schalkwyk, J., Dai, A. M., Hauth, A., Millican, K., et al. (2023) · arXiv · doi:10.48550/arxiv.2312.11805 · Cited by 990 · see citing works
Self-citationFlamingo: a visual language model for few-shot learning
Alayrac, J.-B., Donahue, J., Luc, P., Miech, A., Barr, I., Hasson, Y., Lenc, K., Mensch, A., Millican, K., Reynolds, M., et al. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-1723 · Cited by 966 · cited 2× · see citing works
“As a result, there is a pressing need for a universal and cross-platform computational framework that can address these challenges.”
(2022) · doi:10.15607/rss.2023.xix.025 · Cited by 916 · see citing works
Self-citationLi, J., Li, D., Savarese, S., & Hoi, S. (2023) · arXiv · doi:10.48550/arxiv.2301.12597 · Cited by 908 · see citing works
Linformer: Self-Attention with Linear Complexity
Wang, S., Li, B. Z., Khabsa, M., Fang, H., & Ma, H. (2020) · arXiv · doi:10.48550/arxiv.2006.04768 · Cited by 867 · cited 1× · see citing works
“…employed locality-sensitive hashing (LSH Attention) to reduce memory requirements for attention computation, while Long-former [205] and Linformer [206] utilized sparse attention and low-rank projections, respectively, to decrease computational complexity, making Transformer more efficient for…”
PointFusion: Deep Sensor Fusion for 3D Bounding Box Estimation
Xu, D., Anguelov, D., & Jain, A. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00033 · Cited by 856 · cited 2× · see citing works
“…and FV (Front View) with RGB features, followed by AVOD [75], which uses FPN to extract high-resolution features for feature fusion; methods such as Pointfusion [81], MVX-Net [82], and Pointpainting [83] have further explored the benefits of different fusion granularity feature fusion advantages.”
PIDNet: A Real-time Semantic Segmentation Network Inspired by PID Controllers
Xu, J., Xiong, Z., & Bhattacharyya, S. P. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.01871 · Cited by 817 · cited 3× · see citing works
“Representative Methods Year Encoder-Decoder Framework PIDNet [12] 2023 MAMT [62] 2022 DDRNet [61] 2021 PointRend [59] 2020 HRNet [60] 2019 DeepLabv3+ [58] 2018”
IEEE/RSJ International Conference on Intelligent Robots and Systems
(2022) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros47612.2022.9981344 · Cited by 810 · see citing works
2007 IEEE International Conference on Image Processing
(2007) · doi:10.1109/icip10338.2007 · Cited by 793 · see citing works
Gandhi, A., Adhvaryu, K., Poria, S., Cambria, E., & Hussain, A. (2023) · Information Fusion · doi:10.1016/j.inffus.2022.09.025 · Cited by 783 · see citing works
DataComp: In search of the next generation of multimodal datasetsPreprint
Gadre, S., Ilharco, G., Fang, A., et al. (2023) · Neural Information Processing Systems · doi:10.48550/arxiv.2304.14108 · Cited by 781 · cited 1× · see citing works
“Existing denoising methods mainly have two directions: modality-specific denoising [315, 316] and cross-modality denoising [317, 318, 319].”
Visual-lidar odometry and mapping: low-drift, robust, and fast
Zhang, J., & Singh, S. (2015) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2015.7139486 · Cited by 764 · see citing works
Intern VL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks
Chen, Z., Wu, J., Wang, W., Su, W., Chen, G., Xing, S., Zhong, M., Zhang, Q., Zhu, X., Lu, L., et al. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.02283 · Cited by 734 · cited 1× · see citing works
“By integrating visual, depth, audio, and linguistic data, these datasets provide abundant resources for comprehensive model training, evaluation, and optimization, while laying a solid foundation for future research in multimodal fusion and cross-domain transfer learning.”
RT-1: Robotics Transformer for Real-World Control at Scale
Brohan, A., Brown, N., Carbajal, J., Chebotar, Y., Dabis, J., Finn, C., Gopalakrishnan, K., Hausman, K., Herzog, A., Hsu, J., et al. (2022) · arXiv · doi:10.48550/arxiv.2212.06817 · Cited by 727 · see citing works
Self-citationGenerating diverse high-fidelity images with vq-vae-2
Razavi, A., Oord, A. V. D., & Vinyals, O. (2019) · arXiv · doi:10.48550/arxiv.1906.00446 · Cited by 709 · see citing works
Liu, H., Li, C., Wu, Q., & Lee, Y. J. (2023) · arXiv · doi:10.48550/arxiv.2304.08485 · Cited by 682 · cited 9× · see citing works
“LLaVA [220] also uses similar strategies to generate more natural, user-friendly responses.”
InfluentialConceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Concepts
Changpinyo, S., Sharma, P. K., Ding, N., & Soricut, R. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00356 · Cited by 626 · cited 1× · see citing works
“Existing denoising methods mainly have two directions: modality-specific denoising [315, 316] and cross-modality denoising [317, 318, 319].”
Lvi-sam: Tightly-coupled lidar-visual-inertial odometry via smoothing and mapping
Shan, T., Englot, B., Ratti, C., & Rus, D. (2021) · 2021 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra48506.2021.9561996 · Cited by 624 · see citing works
Multitask Prompted Training Enables Zero-Shot Task Generalization
Sanh, V., Webson, A., Raffel, C., Bach, S. H., Sutawika, L., Alyafeai, Z., Chaffin, A., Stiegler, A., Scao, T. L., Raja, A., et al. (2021) · arXiv · doi:10.48550/arxiv.2110.08207 · Cited by 612 · cited 1× · see citing works
“R2SGrasp [253] utilizes a "Real-to-Sim" framework to minimize domain gaps between simulation and reality, demonstrating outstanding performance in 6-DoF grasp detection.”
PaLM-E: An Embodied Multimodal Language Model
Driess, D., Xia, F., Sajjadi, M. S. M., Lynch, C., Chowdhery, A., Ichter, B., Wahid, A., Tompson, J., Vuong, Q., Yu, T., et al. (2023) · arXiv · doi:10.48550/arxiv.2303.03378 · Cited by 591 · see citing works
DeepFusion: Lidar-Camera Deep Fusion for Multi-Modal 3D Object Detection
Li, Y., Yu, A. W., Meng, T., Caine, B., Ngiam, J., Peng, D., Shen, J., Lu, Y., Zhou, D., Le, Q. V., et al. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01667 · Cited by 563 · cited 1× · see citing works
“Future research will likely focus on improving scalability, real-time capabilities, and developing unified frameworks that integrate localization with high-level perception and decision-making tasks [145].”
Mvx-net: Multimodal voxelnet for 3d object detection
Sindagi, V. A., Zhou, Y., & Tuzel, O. (2019) · 2019 International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2019.8794195 · Cited by 554 · see citing works
DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
DeepSeek-AI, Guo, D., Yang, D., Zhang, H., Song, J., Wang, P., Zhu, Q., Xu, R., Zhang, R., Ma, S., et al. (2025) · arXiv · doi:10.48550/arxiv.2501.12948 · Cited by 538 · see citing works
Self-citationWang, X., Huang, Q., Celikyilmaz, A., Gao, J., Shen, D., Wang, Y.-F., Wang, W. Y., & Zhang, L. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00679 · Cited by 522 · cited 1× · see citing works
“To address the heterogeneity challenge, researchers have proposed three main types of approaches:(1) Unified Feature Space Learning, which projects data from all modalities into the same feature space by constructing a common representation space, e.g., CLIP [190] employs Visual-Linguistic Contrast…”
Voxel Transformer for 3D Object Detection
Mao, J., Xue, Y., Niu, M., Bai, H., Feng, J., Liang, X., Xu, H., & Xu, C. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00315 · Cited by 520 · cited 1× · see citing works
“Despite progress, challenges remain, such as the need for large-scale datasets, high computational costs, and limited generalization to real-world conditions.”
Yoo, J. H., Kim, Y., Kim, J., & Choi, J. W. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58583-9_43 · Cited by 520 · see citing works
Epnet: Enhancing point features with image semantics for 3d object detection
Huang, T., Liu, Z., Chen, X., & Bai, X. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58555-6_3 · Cited by 518 · see citing works
Grattafiori, A., Dubey, A., Jauhri, A., Pandey, A., Kadian, A., Al-Dahle, A., Letman, A., Mathur, A., Schelten, A., Vaughan, A., et al. (2024) · arXiv · doi:10.48550/arxiv.2407.21783 · Cited by 505 · see citing works
Self-citationVoxelNeXt: Fully Sparse VoxelNet for 3D Object Detection and Tracking
Chen, Y., Liu, J., Zhang, X., Qi, X., & Jia, J. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.02076 · Cited by 490 · cited 1× · see citing works
“However, in the real world, the presence of low-quality data seriously affects the performance of multimodal learning, making it difficult for models to work stably in complex environments.”
Self-citationVoxelNeXt: Fully Sparse VoxelNet for 3D Object Detection and Tracking
Chen, Y., Liu, J., Zhang, X., Qi, X., & Jia, J. (2023) · arXiv · doi:10.48550/arxiv.2303.11301 · Cited by 486 · see citing works
Self-citationZhao, F., Zhang, C., & Geng, B. (2024) · ACM Computing Surveys · doi:10.1145/3649447 · Cited by 480 · see citing works
An Embodied Generalist Agent in 3D WorldPreprint
Huang, J., Yong, S.-L., Ma, X., Linghu, X., Li, P.-H., Wang, Y., Li, Q., Zhu, S.-C., Jia, B., & Huang, S.-Y. (2023) · International Conference on Machine Learning · doi:10.48550/arxiv.2311.12871 · Cited by 475 · cited 1× · see citing works
“Meanwhile, LEO [300], as an embodied intelligence model, exhibits superior adaptability in long-term task planning and interactive tasks.”
MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models
Zhu, D., Chen, J., Shen, X., Li, X., & Elhoseiny, M. (2023) · arXiv · doi:10.48550/arxiv.2304.10592 · Cited by 469 · cited 1× · see citing works
“By integrating visual, depth, audio, and linguistic data, these datasets provide abundant resources for comprehensive model training, evaluation, and optimization, while laying a solid foundation for future research in multimodal fusion and cross-domain transfer learning.”
COMPLETER: Incomplete Multi-view Clustering via Contrastive Prediction
Lin, Y., Gou, Y., Liu, Z., Li, B., Lv, J., & Peng, X. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01102 · Cited by 468 · cited 1× · see citing works
“Solutions fall into imputation-based [320] and imputation-free methods [321].”
Semi-Supervised DAS VSP Data Denoising Using Signal and Noise Distribution Difference
Zhang, M., Li, J., Zhao, Y., Wu, N., & Li, Y. (2024) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2024.3510781 · Cited by 463 · cited 1× · see citing works
“A common attention mechanism similar to Figure 3: The above diagram intuitively illustrates the basic processes of three main strategies for multimodal fusion: early fusion, mid fusion, and late fusion.”
CenterFusion: Center-based Radar and Camera Fusion for 3D Object Detection
Nabati, R., & Qi, H. (2021) · 2021 IEEE Winter Conference on Applications of Computer Vision (WACV) · doi:10.1109/wacv48630.2021.00157 · Cited by 461 · cited 1× · see citing works
“Traditional SLAM systems include LiDAR-based[146, 147] and vision-based [148, 149] methods.”
D3VO: Deep Depth, Deep Pose and Deep Uncertainty for Monocular Visual Odometry
Yang, N., von Stumberg, L., Wang, R., & Cremers, D. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00136 · Cited by 438 · cited 1× · see citing works
“In this section, we shift our focus to the evolution of vision-language models, examining their pretraining strategies, cross-modal alignment mechanisms, and emerging architectures that bridge visual and linguistic modalities for more generalizable robotic intelligence.”
RLBench: The Robot Learning Benchmark & Learning Environment
James, S., Ma, Z., Arrojo, D. R., & Davison, A. J. (2020) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2020.2974707 · Cited by 405 · cited 8× · see citing works
“RLBench [306] is one of the most widely used robotic manipulation benchmarks, offering 100+ tasks ranging from basic object grasping to complex tool use and assembly tasks.”
InfluentialIn-Place Scene Labelling and Understanding with Implicit Scene Representation
Zhi, S., Laidlow, T., Leutenegger, S., & Davison, A. J. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01554 · Cited by 402 · cited 1× · see citing works
“In this section, we shift our focus to the evolution of vision-language models, examining their pretraining strategies, cross-modal alignment mechanisms, and emerging architectures that bridge visual and linguistic modalities for more generalizable robotic intelligence.”
Hypersim: A Photorealistic Synthetic Dataset for Holistic Indoor Scene Understanding
Roberts, M., Ramapuram, J., Ranjan, A., Kumar, A., Bautista, M. A., Paczan, N., Webb, R., & Susskind, J. M. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01073 · Cited by 358 · cited 2× · see citing works
“For indoor scene understanding, datasets such as Matterport3D[26], ScanNet [248], NYU Depth V2 [249], and Hypersim [250] have been pivotal.”
LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairs
Schuhmann, C., Vencu, R., Beaumont, R., Kaczmarczyk, R., Mullis, C., Katta, A., Coombes, T., Jitsev, J., & Komatsuzaki, A. (2021) · arXiv · doi:10.48550/arxiv.2111.02114 · Cited by 358 · see citing works
Is Pseudo-Lidar needed for Monocular 3D Object detection?
Park, D., Ambrus, R., Guizilini, V., Li, J., & Gaidon, A. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00313 · Cited by 334 · cited 1× · see citing works
“Scene coordinate regression techniques eliminate the need for explicit 3D maps by learning direct transformations from image pixels to 3D coordinates, simplifying localization.”
Matterport3D: Learning from RGB-D Data in Indoor EnvironmentsPreprint
Chang, A., Dai, A., Funkhouser, T. A., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1709.06158 · Cited by 332 · see citing works
REVERIE: Remote Embodied Visual Referring Expression in Real Indoor Environments
Qi, Y., Wu, Q., Anderson, P., Wang, X., Wang, W. Y., Shen, C., & van den Hengel, A. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01000 · Cited by 331 · cited 22× · see citing works
“As shown in Table 3, Transformer architectures have gradually become mainstream, giving rise to a number of new models such as TransFusion, BEVFu-sion, and MV2DFusion.”
InfluentialMultimodal end-to-end autonomous driving
Xiao, Y., Codevilla, F., Gurram, A., Urfalioglu, O., & Lopez, A. M. (2022) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2020.3013234 · Cited by 319 · cited 1× · see citing works
“Early fusion directly integrates multimodal inputs at the data level, such as concatenating RGB [44] images and depth maps before feeding them into the model.”
UniT: Multimodal Multitask Learning with a Unified Transformer
Hu, R., & Singh, A. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00147 · Cited by 303 · cited 1× · see citing works
“The modality combinations and task types presented here directly impact model performance and generalization, which are further analyzed in the following section.”
Self-Supervised Learning by Cross-Modal Audio-Video Clustering
Alwassel, H., Mahajan, D., Korbar, B., Torresani, L., Ghanem, B., & Tran, D. (2019) · arXiv · doi:10.48550/arxiv.1911.12667 · Cited by 298 · cited 1× · see citing works
“[192]The model learns representations by generating and inferring missing modality information.”
Learning to Navigate Unseen Environments: Back Translation with Environmental Dropout
Tan, H., Yu, L., & Bansal, M. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1268 · Cited by 293 · see citing works
Self-supervised model adaptation for multimodal semantic segmentation
Valada, A., Mohan, R., & Burgard, W. (2020) · International Journal of Computer Vision · doi:10.1007/s11263-019-01188-y · Cited by 293 · cited 1× · see citing works
“These meth-ods demonstrate high flexibility, adaptability, and robustness in complex scenarios, effectively enhancing semantic understanding of the scene [48].”
Speaker-Follower Models for Vision-and-Language Navigation
Fried, D., Hu, R., Cirik, V., Rohrbach, A., Andreas, J., Morency, L.-P., Berg-Kirkpatrick, T., Saenko, K., Klein, D., & Darrell, T. (2018) · arXiv · doi:10.48550/arxiv.1806.02724 · Cited by 291 · cited 1× · see citing works
“Early sequence-based models like Seq2Seq [278] and SF [279] struggled with global path optimization, leading to poor performance in unseen environments.”
DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Models
Dai, D., Deng, C., Zhao, C., Xu, R. X., Gao, H., Chen, D., Li, J., Zeng, W., Yu, X., Wu, Y., et al. (2024) · Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2024.acl-long.70 · Cited by 291 · see citing works
Self-citationCo-SLAM: Joint Coordinate and Sparse Parametric Encodings for Neural Real-Time SLAM
Wang, H., Wang, J., & Agapito, L. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.01277 · Cited by 287 · cited 2× · see citing works
“Such pre-training not only enables better handling of multimodal data but also enhances the model’s overall performance in complex applications [183].”
RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control
Brohan, A., Brown, N., Carbajal, J., Chebotar, Y., Chen, X., Choromanski, K., Ding, T., Driess, D., Dubey, A., Finn, C., et al. (2023) · arXiv · doi:10.48550/arxiv.2307.15818 · Cited by 276 · cited 1× · see citing works
“Proposed by Google in 2018, BERT uses a Transformer encoder-only architecture and pretrains with masked language modeling (MLM) and next sentence prediction (NSP), improving text understanding tasks.”
Self-citationTowards Learning a Generic Agent for Vision-and-Language Navigation via Pre-Training
Hao, W., Li, C., Li, X., Carin, L., & Gao, J. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01315 · Cited by 264 · cited 1× · see citing works
“Future research will continue to focus on expanding the applicability of fusion models and improving their performance across diverse robotic platforms and complex scenarios.”
VLN↻BERT: A Recurrent Vision-and-Language BERT for Navigation
Hong, Y., Wu, Q., Qi, Y., Rodriguez-Opazo, C., & Gould, S. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00169 · Cited by 264 · see citing works
Deep multimodal fusion for semantic image segmentation: A survey
Zhang, Y., Sidibé, D., Morel, O., & Mériaudeau, F. (2021) · Image and Vision Computing · doi:10.1016/j.imavis.2020.104042 · Cited by 261 · cited 1× · see citing works
“It combines various sources of information, such as vision, language, depth data, lidar, and speech, to enhance the capability of scene semantic understanding [39, 40, 41].”
Deep Dual-resolution Networks for Real-time and Accurate Semantic Segmentation of Road Scenes
Hong, Y., Pan, H., Sun, W., & Jia, Y. (2021) · arXiv · doi:10.48550/arxiv.2101.06085 · Cited by 261 · cited 2× · see citing works
“Representative Methods Year Encoder-Decoder Framework PIDNet [12] 2023 MAMT [62] 2022 DDRNet [61] 2021 PointRend [59] 2020 HRNet [60] 2019 DeepLabv3+ [58] 2018”
Limo: Lidar-monocular visual odometry
Graeter, J., Wilczynski, A., & Lauer, M. (2018) · 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2018.8594394 · Cited by 261 · see citing works
HVNet: Hybrid Voxel Network for LiDAR Based 3D Object Detection
Ye, M., Xu, S., & Cao, T. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00170 · Cited by 242 · cited 1× · see citing works
“Despite progress, challenges remain, such as the need for large-scale datasets, high computational costs, and limited generalization to real-world conditions.”
Self-citationIEEE International Joint Conference on Biometrics
(2014) · doi:10.1109/ijcb32250.2014 · Cited by 240 · see citing works
Mai, S., Hu, H., & Xing, S. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i01.5347 · Cited by 237 · cited 1× · see citing works
“A common attention mechanism similar to Figure 3: The above diagram intuitively illustrates the basic processes of three main strategies for multimodal fusion: early fusion, mid fusion, and late fusion.”
Fusionpainting: Multimodal fusion with adaptive attention for 3d object detection
Xu, S., Zhou, D., Fang, J., Yin, J., Bin, Z., & Zhang, L. (2021) · 2021 IEEE International Intelligent Transportation Systems Conference (ITSC) · doi:10.1109/itsc48978.2021.9564951 · Cited by 237 · see citing works
Self-citationOptimal multimodal fusion for multimedia data analysis
Wu, Y., Chang, E. Y., Chang, K. C.-C., & Smith, J. R. (2004) · Proceedings of the 12th annual ACM international conference on Multimedia · doi:10.1145/1027527.1027665 · Cited by 229 · cited 1× · see citing works
“This method has multiple advantages, such as strong modal independence, ease of individual optimization, and scalability of multimodal systems [50] .”
ConceptFusion: Open-set multimodal 3D mapping
Jatavallabhula, K., Kuwajerwala, A., Gu, Q., Omama, M., Iyer, G., Saryazdi, S., Chen, T., Maalouf, A., Li, S., Keetha, N., et al. (2023) · Robotics: Science and Systems XIX · doi:10.15607/rss.2023.xix.066 · Cited by 228 · see citing works
ConceptFusion: Open-set Multimodal 3D Mapping
Jatavallabhula, K. M., Kuwajerwala, A., Gu, Q., Omama, M., Chen, T., Maalouf, A., Li, S., Iyer, G., Saryazdi, S., Keetha, N., et al. (2023) · arXiv · doi:10.48550/arxiv.2302.07241 · Cited by 228 · see citing works
A Survey of Vision-Language Pre-Trained Models
Du, Y., Liu, Z., Li, J., & Zhao, W. X. (2022) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2022/762 · Cited by 225 · cited 1× · see citing works
“IDC predicts that by 2025, global data volume will exceed 180 zettabytes (ZB), with over 70% of IoT data processed at the network edge [238].”
Vision-language navigation with self-supervised auxiliary reasoning tasks
Zhu, F., Zhu, Y., Chang, X., & Liang, X. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01003 · Cited by 223 · see citing works
Lic-fusion: Lidar-inertial-camera odometry
Zuo, X., Geneva, P., Lee, W., Liu, Y., & Huang, G. (2019) · 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros40897.2019.8967746 · Cited by 222 · see citing works
O’Neill, A., Rehman, A., Maddukuri, A., Gupta, A., Padalkar, A., Lee, A., Pooley, A., Gupta, A., Mandlekar, A., Jain, A., et al. (2024) · 2024 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra57147.2024.10611477 · Cited by 210 · see citing works
Self-citationRobust Multimodal Vehicle Detection in Foggy Weather Using Complementary Lidar and Radar Signals
Qian, K., Zhu, S., Zhang, X., & Li, L. E. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00051 · Cited by 206 · cited 1× · see citing works
“LIMO [151] further incorporated IMU data, utilizing sparse visual features to enhance system stability in dynamic environments.”
Self-citationSelf-Monitoring Navigation Agent via Auxiliary Progress Estimation
Ma, C.-Y., Lu, J., Wu, Z., AlRegib, G., Kira, Z., Socher, R., & Xiong, C. (2019) · arXiv · doi:10.48550/arxiv.1901.03035 · Cited by 200 · cited 1× · see citing works
“…three main types of approaches:(1) Unified Feature Space Learning, which projects data from all modalities into the same feature space by constructing a common representation space, e.g., CLIP [190] employs Visual-Linguistic Contrast learning so that images and text share the same latent space.”
Unsupervised Contrastive Cross-modal Hashing
Hu, P., Zhu, H., Lin, J., Peng, D., Zhao, Y.-P., & Peng, X. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3177356 · Cited by 199 · cited 1× · see citing works
“The model is trained using a contrastive loss function to bring related modality pairs, like images and descriptions, closer in a shared embedding space while distancing unrelated pairs [188, 189].”
ScanQA: 3D Question Answering for Spatial Scene Understanding
Azuma, D., Miyanishi, T., Kurita, S., & Kawanabe, M. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01854 · Cited by 197 · cited 1× · see citing works
“A robust evaluation strategy typically involves multiple metrics to account for these complexities, ensuring holistic assessments of multimodal fusion and vision-language models in robotic applications.”
Rrpn: Radar region proposal network for object detection in autonomous vehicles
Nabati, R., & Qi, H. (2019) · 2019 IEEE International Conference on Image Processing (ICIP) · doi:10.1109/icip.2019.8803392 · Cited by 197 · see citing works
Think Global, Act Local: Dual-scale Graph Transformer for Vision-and-Language Navigation
Chen, S., Guhur, P.-L., Tapaswi, M., Schmid, C., & Laptev, I. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01604 · Cited by 194 · cited 2× · see citing works
“(3) Adaptive Modality Fusion, which dynamically adjusts the fusion method according to the data quality, information contribution and task requirements of different modalities. and LiDAR point clouds can be adaptively adjusted according to the ambient lighting conditions [86].”
Self-citationApplications, Challenges, and Future Directions of Human-in-the-Loop Learning
Kumar, S., Datta, S., Singh, V., Datta, D., Singh, S. K., & Sharma, R. (2024) · IEEE Access · doi:10.1109/access.2024.3401547 · Cited by 194 · cited 1× · see citing works
“Recent advances have highlighted the potential of human-in-the-loop trust modeling [337] and ethical instruction tuning [19] to improve the controllability and societal acceptability of robot behavior.”
Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond
Bai, J., Bai, S., Yang, S., Wang, S., Tan, S., Wang, P., Lin, J., Zhou, C., & Zhou, J. (2023) · arXiv · doi:10.48550/arxiv.2308.12966 · Cited by 193 · see citing works
OpenOccupancy: A Large Scale Benchmark for Surrounding Semantic Occupancy Perception
Wang, X., Zhu, Z., Xu, W., Zhang, Y., Wei, Y., Chi, X., Ye, Y., Du, D., Lu, J., & Wang, X. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01636 · Cited by 192 · cited 2× · see citing works
“…to note that although the attention mechanism performs well in multi-modal alignment, the standard self-attention operation is computationally expensive, with its time and memory complexity growing quadratically with input length,as show in the standard self-attention operation is defined as:”
Mees, O., Hermann, L., Rosete-Beas, E., & Burgard, W. B. (2022) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2022.3180108 · Cited by 189 · cited 5× · see citing works
“CALVIN [311], based on PyBullet, includes 34 tasks and emphasizes long-horizon planning and multi-task adaptability.”
InfluentialCross Modal Transformer: Towards Fast and Robust 3D Object Detection
Yan, J., Liu, Y., Sun, J., Jia, F., Li, S., Wang, T., & Zhang, X. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01675 · Cited by 182 · cited 1× · see citing works
“Future research will likely focus on improving scalability, real-time capabilities, and developing unified frameworks that integrate localization with high-level perception and decision-making tasks [145].”
Self-citationA comprehensive survey on relation extraction: recent advances and new frontiers
Zhao, X., Deng, Y., Yang, M., Wang, L., Zhang, R., Cheng, H., Lam, W., Shen, Y., & Xu, R. (2024) · ACM Computing Surveys · doi:10.1145/3674501 · Cited by 168 · cited 2× · see citing works
“It combines various sources of information, such as vision, language, depth data, lidar, and speech, to enhance the capability of scene semantic understanding [39, 40, 41].”
Lic-fusion 2.0: Lidar-inertial-camera odometry with sliding-window plane-feature tracking
Zuo, X., Yang, Y., Geneva, P., Lv, J., Liu, Y., Huang, G., & Pollefeys, M. (2020) · 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros45743.2020.9340704 · Cited by 168
PMR: Prototypical Modal Rebalance for Multimodal Learning
Fan, Y., Xu, W., Wang, H., Wang, J., & Guo, S. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01918 · Cited by 166 · cited 1× · see citing works
“To mitigate this, optimization techniques, gradient modulation, and adaptive architectures [322, 323] help balance training.”
GCNet: Graph Completion Network for Incomplete Multimodal Learning in Conversation
Lian, Z., Chen, L., Sun, L., Liu, B., & Tao, J. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3234553 · Cited by 165 · cited 1× · see citing works
“Solutions fall into imputation-based [320] and imputation-free methods [321].”
DROID: A Large-Scale In-The-Wild Robot Manipulation Dataset
Khazatsky, A., Pertsch, K., Nair, S., Balakrishna, A., Dasari, S., Karamcheti, S., Nasiriany, S., Srirama, M., Chen, L., Ellis, K., et al. (2024) · Robotics: Science and Systems XX · doi:10.15607/rss.2024.xx.120 · Cited by 159 · see citing works
DROID: A Large-Scale In-The-Wild Robot Manipulation Dataset
Khazatsky, A., Pertsch, K., Nair, S., Balakrishna, A., Dasari, S., Karamcheti, S., Nasiriany, S., Srirama, M. K., Chen, L. Y., Ellis, K., et al. (2024) · arXiv · doi:10.48550/arxiv.2403.12945 · Cited by 159 · see citing works
Simple-BEV: What Really Matters for Multi-Sensor BEV Perception?
Harley, A. W., Fang, Z., Li, J., Ambruş, R., & Fragkiadaki, K. (2023) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48891.2023.10160831 · Cited by 159 · see citing works
Geometry-based Distance Decomposition for Monocular 3D Object Detection
Shi, X., Ye, Q., Chen, X., Chen, C., Chen, Z., & Kim, T.-K. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01489 · Cited by 158 · cited 2× · see citing works
“Traditional unimodal approaches (such as relying solely on RGB images) often encounter perceptual limitations when dealing with complex tasks in real-world environments—such as occlusions, lighting variations, sparse textures, and insufficient semantic information[1, 2, 3].”
DeeR-VLA: Dynamic Inference of Multimodal Large Language Models for Efficient Robot ExecutionPreprint
Yue, Y., Wang, Y., Kang, B.-Y., et al. (2024) · Neural Information Processing Systems · doi:10.48550/arxiv.2411.02359 · Cited by 157 · see citing works
Vision-Language Pre-Training: Basics, Recent Advances, and Future Trends
Gan, Z., Li, L., Li, C., Wang, L., Liu, Z., & Gao, J. (2022) · Foundations and Trends® in Computer Graphics and Vision · doi:10.1561/0600000105 · Cited by 155 · see citing works
CRN: Camera Radar Net for Accurate, Robust, Efficient 3D Perception
Kim, Y., Shin, J., Kim, S., Lee, I.-J., Choi, J. W., & Kum, D. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01615 · Cited by 155 · cited 1× · see citing works
“By combining these modalities, SLAM systems have significantly improved robustness, accuracy, and environmental adaptability.”
SparseBEV: High-Performance Sparse 3D Object Detection from Multi-Camera Videos
Liu, H., Teng, Y., Lu, T., Wang, H., & Wang, L. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01703 · Cited by 154 · cited 2× · see citing works
“However, in the real world, the presence of low-quality data seriously affects the performance of multimodal learning, making it difficult for models to work stably in complex environments.”
RCBEVDet: Radar-Camera Fusion in Bird's Eye View for 3D Object Detection
Lin, Z., Liu, Z., Xia, Z., Wang, X., Wang, Y., Qi, S., Dong, Y., Dong, N., Zhang, L., & Zhu, C. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.01414 · Cited by 153 · cited 1× · see citing works
“A generalized SLAM pipeline involves sensor data acquisition, feature extraction, data association, motion estimation (odometry), map initialization, loop closure detection, and map optimization, as illustrated in Figure 10.”
Depth-conditioned Dynamic Message Propagation for Monocular 3D Object Detection
Wang, L., Du, L., Ye, X., Fu, Y., Guo, G., Xue, X., Feng, J., & Zhang, L. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00052 · Cited by 146 · cited 2× · see citing works
“Traditional unimodal approaches (such as relying solely on RGB images) often encounter perceptual limitations when dealing with complex tasks in real-world environments—such as occlusions, lighting variations, sparse textures, and insufficient semantic information[1, 2, 3].”
Multimodal Sentiment Detection Based on Multi-channel Graph Neural Networks
Yang, X., Feng, S., Zhang, Y., & Wang, D. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.28 · Cited by 145 · cited 2× · see citing works
“In addition, models such as MGA [74] and MGNNS [75] demonstrate that heterogeneous GNNs and multi-channel architectures can effectively bridge structural and semantic modality gaps by integrating multimodal graph representations.”
3D-LLM: Injecting the 3D World into Large Language Models
Hong, Y., Zhen, H., Chen, P., Zheng, S., Du, Y., Chen, Z., & Gan, C. (2023) · neural information processing systems · doi:10.52202/075280-0900 · Cited by 143 · see citing works
Airbert: In-domain Pretraining for Vision-and-Language Navigation
Guhur, P.-L., Tapaswi, M., Chen, S., Laptev, I., & Schmid, C. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00166 · Cited by 139 · see citing works
Self-citationSOON: Scenario Oriented Object Navigation with Graph-based Exploration
Zhu, F., Liang, X., Zhu, Y., Yu, Q., Chang, X., & Liang, X. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01250 · Cited by 136 · cited 2× · see citing works
“This design significantly enhances cross-modal grounding and navigation robustness, making SUSA one of the most competitive and generalizable approaches to date.”
Deformable feature aggregation for dynamic multi-modal 3D object detection
Chen, Z., Li, Z., Zhang, S., Fang, L., Jiang, Q., & Zhao, F. (2022) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-20074-8_36 · Cited by 136 · see citing works
IS-Fusion: Instance-Scene Collaborative Fusion for Multimodal 3D Object Detection
Yin, J., Shen, J., Chen, R., Li, W., Yang, R., Frossard, P., & Wang, W. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.01412 · Cited by 133 · cited 1× · see citing works
“Future research will likely focus on improving scalability, real-time capabilities, and developing unified frameworks that integrate localization with high-level perception and decision-making tasks [145].”
TinyBERT: Distilling BERT for Natural Language UnderstandingPreprint
Jiao, X., Yin, Y., Shang, L., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.10351 · Cited by 133 · see citing works
IPOD: Intensive Point-based Object Detector for Point Cloud
Yang, Z., Sun, Y., Liu, S., Shen, X., & Jia, J. (2018) · arXiv · doi:10.48550/arxiv.1812.05276 · Cited by 132 · cited 1× · see citing works
“Despite progress, challenges remain, such as the need for large-scale datasets, high computational costs, and limited generalization to real-world conditions.”
Connecting Touch and Vision via Cross-Modal Prediction
Li, Y., Zhu, J.-Y., Tedrake, R., & Torralba, A. B. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.01086 · Cited by 132 · cited 3× · see citing works
“VisGel [253], leveraging GelSight sensors, con-20 structs bidirectional generation tasks between tactile and visual modalities, further validating the feasibility of inter-modal information transformation.”
CRAFT: Camera-Radar 3D Object Detection with Spatio-Contextual Fusion Transformer
Kim, Y., Kim, S., Choi, J. W., & Kum, D. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i1.25198 · Cited by 130 · see citing works
Kosmos-2: Grounding Multimodal Large Language Models to the WorldPreprint
Peng, Z., Wang, W., Dong, L., Hao, Y., Huang, S., Ma, S., & Wei, F. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2306.14824 · Cited by 129 · cited 1× · see citing works
“By integrating visual, depth, audio, and linguistic data, these datasets provide abundant resources for comprehensive model training, evaluation, and optimization, while laying a solid foundation for future research in multimodal fusion and cross-domain transfer learning.”
Unifying Voxel-based Representation with Transformer for 3D Object Detection
Li, Y., Chen, Y., Qi, X., Li, Z., Sun, J., & Jia, J. (2022) · arXiv · doi:10.48550/arxiv.2206.00630 · Cited by 125 · cited 2× · see citing works
“UVTR [88] converts images into voxel representations, enabling cross-modal learning.”
3D-LLM: Injecting the 3D World into Large Language Models
Hong, Y., Zhen, H., Chen, P., Zheng, S., Du, Y., Chen, Z., & Gan, C. (2023) · arXiv · doi:10.48550/arxiv.2307.12981 · Cited by 124 · cited 2× · see citing works
“Scene Understanding [303] 69.4 12.0 14.5 35.7 20.5 LL3DA [304] 76.8 13.5 15.9 37.3 - Qwen2-VL-7B [305] 53.9 3.0 11.4 29.3 -”
InfluentialStructured Scene Memory for Vision-Language Navigation
Wang, H., Wang, W., Liang, W., Xiong, C., & Shen, J. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00835 · Cited by 123 · cited 1× · see citing works
“…three main types of approaches:(1) Unified Feature Space Learning, which projects data from all modalities into the same feature space by constructing a common representation space, e.g., CLIP [190] employs Visual-Linguistic Contrast learning so that images and text share the same latent space.”
Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spaces
Yang, J., Yang, S., Gupta, A. W., Han, R., Fei-Fei, L., & Xie, S. (2025) · 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52734.2025.00994 · Cited by 120 · cited 1× · see citing works
“…requires models not only to possess geometric perception capabilities for 3D structures but also to perform memory, reasoning, and prediction over time-particularly maintaining persistent representations of spatial semantics, interaction history, and future actions in dynamic environments [329].”
DataComp: In search of the next generation of multimodal datasets
Gadre, S. Y., Ilharco, G., Fang, A., Hayase, J., Smyrnis, G., Nguyen, T., Marten, R., Wortsman, M., Ghosh, D., Zhang, J., et al. (2023) · Advances in Neural Information Processing Systems 36 · doi:10.52202/075280-1179 · Cited by 120 · see citing works
FocalFormer3D : Focusing on Hard Instance for 3D Object Detection
Chen, Y., Yu, Z., Chen, Y., Lan, S., Anandkumar, A., Jia, J., & Alvarez, J. M. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00771 · Cited by 119 · cited 1× · see citing works
“However, in the real world, the presence of low-quality data seriously affects the performance of multimodal learning, making it difficult for models to work stably in complex environments.”
Filtering, Distillation, and Hard Negatives for Vision-Language Pre-Training
Radenovic, F., Dubey, A., Kadian, A., Mihaylov, T., Vandenhende, S., Patel, Y., Wen, Y., Ramanathan, V., & Mahajan, D. (2023) · Computer Vision and Pattern Recognition · doi:10.1109/cvpr52729.2023.00673 · Cited by 117 · cited 1× · see citing works
“…Third, task-oriented and online adaptation fusion strategies are increasingly becoming a research focus, aiming to drive vision-language models to dynamically adjust their perception and decision-making paths through scene cognition, thereby supporting robot response capabilities under…”
Nguyen, K., & Daumé III, H. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1063 · Cited by 115 · cited 1× · see citing works
“Future research will continue to focus on expanding the applicability of fusion models and improving their performance across diverse robotic platforms and complex scenarios.”
Cheng, Y., Xu, H., & Liu, Y. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01498 · Cited by 109 · cited 1× · see citing works
“RISFNet [100] integrates multi-frame radar data with RGB images using temporal encoding and self-attention.”
DeepInteraction: 3D Object Detection via Modality Interaction
Yang, Z., Chen, J., Miao, Z., Li, W., Zhu, X., & Zhang, L. (2022) · arXiv · doi:10.48550/arxiv.2208.11112 · Cited by 108 · cited 1× · see citing works
“DeepInteraction [90] addresses the limitation of unilateral fusion by introducing a multimodal interaction encoder and decoder.”
Local feature matching using deep learning: A survey
Xu, S., Chen, S., Xu, R., Wang, C., Lu, P., & Guo, L. (2024) · Information Fusion · doi:10.1016/j.inffus.2024.102344 · Cited by 107 · cited 1× · see citing works
“Additionally, while LoRA ensures parameter efficiency, it may limit the model’s expressiveness in complex multimodal scenarios, such as indoor manipulation or dynamic path planning, where detailed spatial understanding is critical.”
Self-citationDeepInteraction: 3D Object Detection Via Modality Interaction
Yang, Z., Chen, J., Miao, Z., et al. (2022) · neural information processing systems · doi:10.52202/068431-0145 · Cited by 104 · see citing works
Multi-pathway generative adversarial hashing for unsupervised cross-modal retrieval
Zhang, J., & Peng, Y. (2020) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2019.2922128 · Cited by 103 · cited 1× · see citing works
“The modality combinations and task types presented here directly impact model performance and generalization, which are further analyzed in the following section.”
NaVid: Video-based VLM Plans the Next Step for Vision-and-Language Navigation
Zhang, J., Wang, K., Xu, R., Zhou, G., Hong, Y., Fang, X., Wu, Q., Zhang, Z., & Wang, H. (2024) · Robotics: Science and Systems XX · doi:10.15607/rss.2024.xx.079 · Cited by 103 · see citing works
Self-citationNguyen, K., Dey, D., Brockett, C., & Dolan, B. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.01281 · Cited by 101 · cited 1× · see citing works
“Future research will continue to focus on expanding the applicability of fusion models and improving their performance across diverse robotic platforms and complex scenarios.”
Multimodal data fusion for systems improvement: A review
Gaw, N., Yousefi, S., & Gahrooei, M. R. (2022) · IISE Transactions · doi:10.1080/24725854.2021.1987593 · Cited by 101 · cited 1× · see citing works
“As illustrated in Figure 3, multimodal fusion strategies can be categorized into early fusion, mid fusion, and late fusion based on the fusion strategy [42, 43].”
Object-and-action aware model for visual language navigation
Qi, Y., Pan, Z., Zhang, S., van den Hengel, A., & Wu, Q. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58607-2_18 · Cited by 100 · see citing works
Deep Learning for Visual Localization and Mapping: A Survey
Chen, C., Wang, B., Lu, C. X., Trigoni, N., & Markham, A. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3309809 · Cited by 97 · cited 1× · see citing works
“Recent advancements in multimodal fusion, particularly deep-learning-based approaches, have significantly enhanced localization accuracy and robustness [134].”
VIMA: General Robot Manipulation with Multimodal Prompts
Jiang, Y., Gupta, A., Zhang, Z., Wang, G., Dou, Y., Chen, Y., Fei-Fei, L., Anandkumar, A., Zhu, Y., & Fan, L. (2022) · arXiv · doi:10.48550/arxiv.2210.03094 · Cited by 95 · cited 2× · see citing works
“However, existing large-scale visual language datasets (e.g., Visual Genome [331], Conceptual Cap-tions [332], and LAION-400m [333]) generally suffer from limited size, high annotation noise, insufficient semantic diversity, and expensive annotation.”
Self-citationCoarse-to-fine vision-language pre-training with fusion in the backbone
Dou, Z.-Y., Kamath, A., Gan, Z., Zhang, P., Wang, J., Li, L., Liu, Z., Liu, C., Lecun, Y., Peng, N., et al. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-2387 · Cited by 93 · see citing works
LL3DA: Visual Interactive Instruction Tuning for Omni-3D Understanding, Reasoning, and Planning
Chen, S., Chen, X., Zhang, C., Li, M., Yu, G., Fei, H., Zhu, H., Fan, J., & Chen, T. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.02496 · Cited by 92 · cited 2× · see citing works
“Scene Understanding [303] 69.4 12.0 14.5 35.7 20.5 LL3DA [304] 76.8 13.5 15.9 37.3 - Qwen2-VL-7B [305] 53.9 3.0 11.4 29.3 -”
InfluentialSelf-citationVideoFlow: Exploiting Temporal Cues for Multi-frame Optical Flow Estimation
Shi, X., Huang, Z., Bian, W., Li, D., Zhang, M., Cheung, K. C., See, S. C. W., Qin, H., Dai, J., & Li, H. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01146 · Cited by 91 · cited 1× · see citing works
“In this section, we shift our focus to the evolution of vision-language models, examining their pretraining strategies, cross-modal alignment mechanisms, and emerging architectures that bridge visual and linguistic modalities for more generalizable robotic intelligence.”
A cross-modal generative adversarial network for scenarios generation of renewable energy
Kang, M., Zhu, R., Chen, D., Li, C., Gu, W., Qian, X., & Yu, W. (2024) · IEEE Transactions on Power Systems · doi:10.1109/tpwrs.2023.3277698 · Cited by 90 · cited 1× · see citing works
“This process uses generative models (such as Variational Autoencoders or Generative Adversarial Networks) to map information from one modality into the space of another modality [196].”
Cognitive neuroscience and robotics: Advancements and future research directions
Liu, S., Wang, L., & Gao, R. X. (2023) · Robotics and Computer-Integrated Manufacturing · doi:10.1016/j.rcim.2023.102610 · Cited by 90 · cited 1× · see citing works
“For example, recent advances in computational neuroscience have introduced biologically inspired attention and memory mechanisms that could inform the design of more interpretable and causally coherent vision-language architectures [332].”
Joint feature synthesis and embedding: Adversarial cross-modal retrieval revisited
Xu, X., Lin, K., Yang, Y., Hanjalic, A., & Shen, H. T. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.3045530 · Cited by 88 · cited 1× · see citing works
“This is typically achieved by creating a shared embedding space where data from various modalities (such as images, text, audio, video, etc.) are projected into the same space, allowing the model to learn the relationships be-tween these modalities [176, 177].”
Envedit: Environment Editing for Vision-and-Language Navigation
Li, J., Tan, H., & Bansal, M. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01497 · Cited by 87 · cited 1× · see citing works
“(2) Modality-Specific Encoders, which designs special encoders for different modalities, using Transformer for text, CNN for image, RNN for audio, etc., and then performs feature alignment through Cross-modal Attention [325, 326].”
Flagged first, then by citations
Research works printed but not linked: 11
- Radford, 2019, Language models are unsupervised multitask learners, OpenAI Blog, 1, 9
- Wolters, 2024
- Zhang, 2020, Deep multimodal fusion for semantic image segmentation: A survey, Image Vis. Comput., 105
- T.-Y. Lim, A. Ansari, B. Major, D. Fontijne, M. Hamilton, R. Gowaikar, S. Subramanian, Radar and camera early fusion for vehicle detection in advanced driver assistance systems, in: Machine Learning for Autonomous Driving Workshop At the 33rd Conference on Neural Information Processing Systems, vol. 2, 2019, 7.
- Chen, 2021, History aware multimodal transformer for vision-and-language navigation, Adv. Neural Inf. Process. Syst., 34, 5834
- K. Yu, Y. Han, Q. Wang, V. Saxena, D. Xu, Y. Zhao, MimicTouch: Leveraging Multi-modal Human Tactile Demonstrations for Contact-rich Manipulation, in: 8th Annual Conference on Robot Learning, 2024.
- J.D.M.-W.C. Kenton, L.K. Toutanova, Bert: Pre-training of deep bidirectional transformers for language understanding, in: Proceedings of NaacL-HLT, vol. 1, Minneapolis, Minnesota, 2019, 2.
- A. Akbik, D. Blythe, R. Vollgraf, Contextual string embeddings for sequence labeling, in: Proceedings of the 27th International Conference on Computational Linguistics, 2018, pp. 1638–1649.
- S. Lu, Y. Yao, W. Shi, Collaborative learning on the edges: A case study on connected vehicles, in: 2nd USENIX Workshop on Hot Topics in Edge Computing, HotEdge 19, 2019.
- Wolters, 2024
- E. Xing, A. Gupta, S. Powers, V. Dean, Kitchenshift: Evaluating zero-shot generalization of imitation-based policy learning under domain shifts, in: NeurIPS 2021 Workshop on Distribution Shifts: Connecting Methods and Applications, 2021.
Cited by
Sorted by citations
Nie, Q., Geng, J., & Liu, C. (2026) · Sensors · doi:10.3390/s26020702 · Cited by 22 · see citing works
“Multimodal data fusion represents the core technology of modern fault diagnosis systems, which integrates heterogeneous data from various sources to provide a more comprehensive and accurate assessment of equipment conditions [200].”
Large Language Models and 3D Vision for Intelligent Robotic Perception and Autonomy
Mehta, V., Sharma, C., & Thiyagarajan, K. (2025) · Sensors · doi:10.3390/s25206394 · Cited by 14 · see citing works
“…to deeply analyze varied 3D data types, including point clouds, voxels and meshes, and further distinguish our contribution by systematically examining the integration of non-visual sensory inputs such as tactile, thermal and auditory data, which are pivotal for robust embodied interaction [66].”
Akram, W., Din, M. U., Saoud, L. S., & Hussain, I. (2025) · Aquacultural Engineering · doi:10.1016/j.aquaeng.2025.102637 · Cited by 16 · see citing works
Elgohr, A. T., Rashad, M., El-Gendy, E. M., Shaaban, W., & Saafan, M. M. (2026) · Artificial Intelligence Review · doi:10.1007/s10462-026-11554-1 · Cited by 6 · see citing works
Thermal comfort assessment based on occupants’ action recognition using computer vision - A review
Wu, Z., Wang, Z., Wei, Z., & Calautit, J. K. (2025) · Building and Environment · doi:10.1016/j.buildenv.2025.113809 · Cited by 3 · see citing works
CoDrone: Autonomous Drone Navigation Assisted by Edge and Cloud Foundation Models
Chen, P., Ouyang, T., Ke, L., et al. (2025) · IEEE Internet of Things Journal · doi:10.1109/jiot.2025.3648721 · Cited by 2 · see citing works
“Several studies [27], [28], [29] explore the integration of VLMs into robotic systems, demonstrating their e ff ectiveness in complex perception tasks, such as semantic mapping, obstacle avoidance, and goal-oriented navigation.”
Ko, B. C. (2026) · Sensors · doi:10.3390/s26113541 · Cited by 2 · see citing works
“Han et al. [27] provide the most fusion-oriented analysis among prior surveys but cover robotic vision broadly rather than VLA-specific manipulation, and address sensor failure modes and calibration only peripherally.”
3D-MoRe: Unified Modal-Contextual Reasoning for Embodied Question Answering
Xu, R., Gao, H., Yu, M., An, D. U., Chen, S., Wang, C., Guo, L., Liang, X., & Xu, S. (2025) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros60139.2025.11247159 · Cited by 1 · see citing works
Self-citationLarge Language Models in Sensor-Driven Control Systems: Architectures, Challenges, and Opportunities
Aghaee, F., & Shaker, H. R. (2026) · Sensors · doi:10.3390/s26144350 · Cited by 3 · see citing works
Efficient semantic image communication for traffic monitoring at the edge
Assylbek, D., Aitymbetov, N., Ristin, M., & Zorbas, D. (2026) · Internet of Things · doi:10.1016/j.iot.2026.102029 · Cited by 1 · see citing works
Beyond Single Models: Enhancing LLM Detection of Ambiguity in Requests Through Debate
Davila, A., Colan, J., & Hasegawa, Y. (2025) · doi:10.23919/sicefes67750.2025.11236658
“Vision-Language Models (VLMs) like FreeGrasp resolve object ambiguities in cluttered scenes [13] and enhance spatial reasoning [14].”
Beyond Description: Cognitively Benchmarking Fine-Grained Action for Embodied Agents
Liu, D., Xu, C., Chen, W., Zhang, S., Wang, J., Deng, J., Sun, B., & Liu, Y. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37010-5_21 · Cited by 1 · see citing works
Fusing Events and Frames for Robust Gait Recognition
Chen, L., Zhang, Z., Li, C., et al. (2025) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2025.3642699
“Multimodal fusion methods [46], [47], [48], [49], [50], [51], [52], [53] have gained increasing attention in the field of computer vision.”
Descarten, R. O., & Treceñe, J. K. (2025) · Journal of Innovative Image Processing · doi:10.36548/jiip.2025.4.021
Wang, P., Sun, G., Jin, X. H., et al. (2025) · doi:10.1109/mcai66356.2025.11381955
“[18] This research is highly aligned with the requirements for generating robot actions, significantly addressing the following issues: a.”
Creative AI for Live Interactive Performances: Foundations, Challenges, and Future Directions
Woodward, K., Falcon-Caro, A., Ramchurn, R., & Benford, S. (2026) · Communications in computer and information science · doi:10.1007/978-3-032-16893-1_1
MAGIC: Multimodal Adaptive Graph Interaction Network for Conversation
Wang, L., Lin, J., Liao, S., et al. (2025) · International Conference on Computer, Internet of Things and Smart City (CIoTSC) · doi:10.1109/ciotsc67482.2025.11413098
“Subsequent approaches have developed more sophisticated fusion mechanisms [5].”
Anxu, W. (2025) · doi:10.1109/ccnis69465.2025.00079
“If the path design is improper, it can lead to cumulative errors exceeding 0.5m due to insufficient sensor observations[6].”
Self-Supervised Multimodal 3D Object Recognition for Embodied Robotic Manipulation
Liu, J., Li, Z., & Huang, W. (2025) · International Academic Exchange Conference on Science and Technology Innovation (IAECST) · doi:10.1109/iaecst68792.2025.11415249
Mobile robot inspection of building plenum spaces via vision-based multi-sensor fault detection
Lou, G., Liu, S., Shen, Y., Wong, W. Y., Lu, Y., & Duan, M. (2026) · Automation in Construction · doi:10.1016/j.autcon.2026.106889
Task-Aware Geometric Embeddings and Cross-Modal Interaction for 3D VQA
Gao, H., Xu, R., Wang, C., Guo, L., & Xu, S. (2026) · International Symposium on Computer Applications and Information Systems (ISCAIS) · doi:10.1109/iscait69154.2026.11477363
“Other works focus on compositional modules for visual reasoning [14, 15].”
Self-citationExplainable Multi-Modal Object Localization: Real-Time 2D–3D Perception for Autonomous Systems
Gouni, D. S., Kanyal, P., Bhatt, C., et al. (2026) · doi:10.1109/wccst67302.2026.11495622
“The coverage of intelligent systems from autonomous vehicles to self-driving cars is creating an ultimate demand for machines that can perceive reason and explain their understanding of complex environment it is a critical dependency on an environmental perception [1].”
Simulation-First Framework to Validate Assistive Robots in Real Scenarios
Martino, L. D., & Bochicchio, M. A. (2026) · Lecture notes in bioengineering · doi:10.1007/978-3-032-11050-3_20
A Low-Cost VLA-RL Framework for Dexterous Multi-Category Fruit Harvesting in Orchards
Tao, W., Luo, L., & Chen, M. (2026) · doi:10.1109/ipac69021.2026.11542950
“VLA model as a unified framework brings three essential capabilities: visual, semantic, and action[11].”
Multimodal Contrastive Learning for Zero-Shot Instruction-Following Robot with Synthetic Data
Kamadi, W., Njiri, J. G., Samuel, K., & Aoki, S. (2026) · Engineering Technology & Applied Science Research · doi:10.48084/etasr.18291
“However, existing multimodal and instruction-following approaches show several limitations that reduce their applicability to general-purpose robotic control [11, 12].”
InfluentialWM-DETR: Dual-branch wavelet-Mamba and sparse attention for robust underwater object detection
Han, X., Chen, S., Fu, Z., et al. (2026) · Expert Systems with Applications · doi:10.1016/j.eswa.2026.133148
Self-citationA Multimodal Data Acquisition and SemiAutomated Annotation Framework for Live-Line Working Robots
Zou, R., Zhang, K., Ding, L., et al. (2026) · IEEE International Conference on Control, Electronics and Computer Technology (ICCECT) · doi:10.1109/iccect68671.2026.11565353
“From a practical perspective, the limited availability of multimodal datasets, coupled with the high cost and effort required for annotation, has emerged as a persistent obstacle [2–4].”
H2R Link: A Multimodal Framework for Bridging Human Intent to Robot Execution
Dhungana, K., & Jiang, N. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-29842-3_21
From fuzzy intent to executable visual workflows: A multi-model orchestration approach
Huang, Y., Li, H., Lu, D., Li, F., Zhang, Z., & Xu, W. (2026) · Expert Systems with Applications · doi:10.1016/j.eswa.2026.133549
Negation in Vision-Language Models: A Survey
Pokhrel, A., Ghimire, B., Paudel, P. M., & Sheshappanavar, S. V. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-31927-2_17
Evolution of humanoid locomotion control
Gu, Y., Shi, G., Shi, F., Chang, I.-C., Wang, Y., Cheng, Q., Olkin, Z., López-Sánchez, I., Feng, Y., Zhang, J., et al. (2026) · Science Robotics · doi:10.1126/scirobotics.aed3973
Song, H., & Shao, L. (2026) · Information · doi:10.3390/info17090832
End–Edge–Cloud Collaborative Fast–Slow Semantic Planning for Agricultural Field Robots
Gao, B., Gong, L., Sun, Y., et al. (2026) · Agronomy · doi:10.3390/agronomy16171725
He, S., Guo, Y., Kou, J., Kang, J., Tao, S., & Feng, L. (2026) · IEEE International Conference on Smart Internet of Things (SmartIoT) · doi:10.1109/smartiot70864.2026.00024
“Surveys [18] also validate that visual-textual modeling and adaptive cross-modal alignment significantly improve video perception, especially in ambiguous scenes and long sequences.”
Zhu, W., Yu, J., Feng, C., Dong, Z., Song, Z., Gao, J., & Huang, R. (2026) · Developments in the Built Environment · doi:10.1016/j.dibe.2026.101047
Wang, L., Li, X., Liu, P., Milić, V., Guo, Z., & Wang, C. (2026) · Agriculture · doi:10.3390/agriculture16192040
Self-citationDing, X., Jiao, N., Chi, Y., et al. (2026) · Displays · doi:10.1016/j.displa.2026.103708
Safe Multi-Robot Coordination via VLM–LLM Reasoning and Reachability Analysis
Dwedar, M., Hafez, A. A., Jesser, A., & Alanwar, A. (2026) · IEEE Access · doi:10.1109/access.2026.3737314
“Vision-language models (VLMs) and large language models (LLMs) have demonstrated substantial capability in semantic scene interpretation, task decomposition, and high-level action selection for robotic systems [7]–[10].”
Greco, L., Rosa, F. D., Saggese, A., & Vento, M. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-39518-4_32
Liu, J., & Liu, H. (2026) · Applied Soft Computing · doi:10.1016/j.asoc.2026.116518
Feng, K., Tang, C., Ma, W., & Yankai, S. U. N. (2026) · Neurocomputing · doi:10.1016/j.neucom.2026.135296
- Authors
- Han, X., Chen, S., Fu, Z., Feng, Z., Fan, L., An, D., Wang, C., Guo, L., Meng, W., Zhang, X., Xu, R., & Xu, S.
- Affiliations
- Chinese Academy of Sciences (CN)
- Beijing University of Posts and Telecommunications (CN)
- Corresponding author
- Meng, Weiliang
- Xu, Rongtao
- ORCID
- Venue
- Information Fusion
- Publisher
- Elsevier BV
- Published
- 2026-02
- Volume
- 126
- Pages
- 103652
- ISSN
- 1566-2535 (ISSN-L)
- OpenAlex
- W4413788337
- Semantic Scholar
- b601ae26f7c7c0bcd81a64a781475bb16e6c5502
- Cited by
- 41
- References
- 286
- Citations by year
- 2026: 29 · 2025: 12
- FWCI
- 15.77
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- Information Fusion (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Natural Science Foundation of Beijing Municipality (L231013, JQ23014)
- National Natural Science Foundation of China (62262043, U22B2034, 62376271, 62172416, 62365014, 62271074, U21A20515, 62572059)
- Beijing Natural Science Foundation
- State Key Laboratory of Virtual Reality Technology and Systems (VRLAB2025B03)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Engineering › Control and Systems Engineering › Robot Manipulation and Learning
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Keywords
- multimodal fusion, vision-language models, robot vision, semantic scene understanding, SLAM, 3D object detection, robot navigation, robotic manipulation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-08.
Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-08.
Cited sources are checked by gpt-6-sol.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review