LL3DA: Visual Interactive Instruction Tuning for Omni-3D Understanding, Reasoning, and Planning
Open accessAbstract
Recent progress in Large Multimodal Models (LMM) has opened up great possibilities for various applications in the field of human-machine interactions. However, developing LMMs that can comprehend, reason, and plan in complex and diverse 3D environments remains a challenging topic, especially considering the demand for understanding permutation-invariant point cloud representations of the 3D scene. Existing works seek help from multi-view images by projecting 2D features to 3D space, which inevitably leads to huge computational overhead and performance degradation. In this paper, we present LL3DA, a Large Language 3D Assistant that takes point cloud as the direct input and responds to both text instructions and visual interactions. The additional visual interaction enables LMMs to better comprehend human interactions with the 3D environment and further remove the ambiguities within plain texts. Experiments show that LL3DA achieves remarkable results and surpasses various 3D vision-language models on both 3D Dense Captioning and 3D Question Answering.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Optimization techniques, gradient modulation, and adaptive architectures help balance training under modality imbalance.
“To mitigate this, optimization techniques, gradient modulation, and adaptive architectures [304,305] help balance training.”
From Multimodal fusion and vision–language models: A survey for robot vision (2026) · cites this paper as [304]
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 48,293 · see citing works
Decoupled Weight Decay Regularization
Loshchilov, I., & Hutter, F. (2017) · arXiv · doi:10.48550/arxiv.1711.05101 · Cited by 12,522 · see citing works
Kirillov, A., Mintun, E., Ravi, N., Mao, H., Rolland, C., Gustafson, L., Xiao, T., Whitehead, S., Berg, A. C., Lo, W.-Y., et al. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00371 · Cited by 11,953 · see citing works
(2004) · Cited by 9,718 · see citing works
CIDEr: Consensus-based image description evaluation
Vedantam, R., Zitnick, C. L., & Parikh, D. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7299087 · Cited by 4,845 · see citing works
ScanNet: Richly-Annotated 3D Reconstructions of Indoor Scenes
Dai, A., Chang, A. X., Savva, M., Halber, M., Funkhouser, T., & Niessner, M. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.261 · Cited by 4,368 · see citing works
InstructBLIP: Towards General-purpose Vision-Language Models with Instruction TuningPreprint
Dai, W., Li, J., Li, D., Tiong, A., Zhao, J.-Q., Wang, W., Li, B. A., Fung, P., & Hoi, S. (2023) · Neural Information Processing Systems · doi:10.48550/arxiv.2305.06500 · Cited by 4,191 · see citing works
(2005) · Cited by 4,152 · see citing works
Bleu: a method for automatic evaluation of machine translation
Papineni, K. (2024) · doi:10.57702/ljteqmrb · Cited by 3,682 · see citing works
Llama 2: Open Foundation and Fine-Tuned Chat Models
Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bashlykov, N., Batra, S., Bhargava, P., Bhosale, S., et al. (2023) · arXiv · doi:10.48550/arxiv.2307.09288 · Cited by 2,604 · see citing works
Liu, H., Li, C., Wu, Q., & Lee, Y. J. (2023) · Advances in Neural Information Processing Systems 36 · doi:10.52202/075280-1516 · Cited by 2,428 · see citing works
Proceedings of the 40th Annual Meeting on Association for Computational Linguistics
Isabelle, P. (2002) · Cited by 1,633 · see citing works
Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains
Tancik, M., Srinivasan, P. P., Mildenhall, B., Fridovich-Keil, S., Raghavan, N., Singhal, U., Ramamoorthi, R., Barron, J. T., & Ng, R. (2020) · arXiv · doi:10.48550/arxiv.2006.10739 · Cited by 1,459 · see citing works
Scaling Instruction-Finetuned Language Models
Chung, H. W., Hou, L., Longpre, S., Zoph, B., Tay, Y., Fedus, W., Li, Y., Wang, X., Dehghani, M., Brahma, S., et al. (2022) · arXiv · doi:10.48550/arxiv.2210.11416 · Cited by 1,441 · see citing works
Self-citationmPLUG-Owl: Modularization Empowers Large Language Models with MultimodalityPreprint
Ye, Q.-H., Xu, H., Xu, G., Ye, J., Yan, M., Zhou, Y., Wang, J., Hu, A., Shi, P.-C., Shi, Y., et al. (2023) · arXiv.org · doi:10.48550/arxiv.2304.14178 · Cited by 1,318 · see citing works
The Curious Case of Neural Text Degeneration
Holtzman, A., Buys, J., Du, L., Forbes, M., & Choi, Y. (2019) · arXiv · doi:10.48550/arxiv.1904.09751 · Cited by 1,304 · see citing works
Hierarchical Neural Story Generation
Fan, A., Lewis, M., & Dauphin, Y. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p18-1082 · Cited by 1,266 · see citing works
Hierarchical Neural Story Generation
Fan, A., Lewis, M., & Dauphin, Y. (2018) · arXiv · doi:10.48550/arxiv.1805.04833 · Cited by 1,242 · see citing works
Multimodal learning with transformers: A survey
Xu, P., Zhu, X., & Clifton, D. A. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3275156 · Cited by 1,072 · see citing works
Li, J., Li, D., Savarese, S., & Hoi, S. (2023) · arXiv · doi:10.48550/arxiv.2301.12597 · Cited by 908 · see citing works
A survey on multimodal large language models
Yin, S., Fu, C., Zhao, S., Li, K., Sun, X., Xu, T., & Chen, E. (2024) · National Science Review · doi:10.1093/nsr/nwae403 · Cited by 893 · see citing works
Murray, G., Renals, S. J., Carletta, J., & Moore, J. D. (2005) · Meeting of the Association for Computational Linguistics · Cited by 716 · see citing works
Liu, H., Li, C., Wu, Q., & Lee, Y. J. (2023) · arXiv · doi:10.48550/arxiv.2304.08485 · Cited by 682 · see citing works
ClipCap: CLIP Prefix for Image CaptioningPreprint
Mokady, R., Hertz, A., & Bermano, A. H. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.09734 · Cited by 605 · see citing works
PaLM-E: An Embodied Multimodal Language Model
Driess, D., Xia, F., Sajjadi, M. S. M., Lynch, C., Chowdhery, A., Ichter, B., Wahid, A., Tompson, J., Vuong, Q., Yu, T., et al. (2023) · arXiv · doi:10.48550/arxiv.2303.03378 · Cited by 591 · see citing works
An Embodied Generalist Agent in 3D WorldPreprint
Huang, J., Yong, S.-L., Ma, X., Linghu, X., Li, P.-H., Wang, Y., Li, Q., Zhu, S.-C., Jia, B., & Huang, S.-Y. (2023) · International Conference on Machine Learning · doi:10.48550/arxiv.2311.12871 · Cited by 475 · see citing works
MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models
Zhu, D., Chen, J., Shen, X., Li, X., & Elhoseiny, M. (2023) · arXiv · doi:10.48550/arxiv.2304.10592 · Cited by 469 · see citing works
PointLLM: Empowering Large Language Models to Understand Point CloudsPreprint
Xu, R.-S., Wang, X., Wang, T., et al. (2023) · European Conference on Computer Vision · doi:10.48550/arxiv.2308.16911 · Cited by 448 · see citing works
LLM-Planner: Few-Shot Grounded Planning for Embodied Agents with Large Language Models
Song, C. H., Sadler, B. M., Wu, J., Chao, W.-L., Washington, C., & Su, Y. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00280 · Cited by 435 · cited 1× · see citing works
“Except for this watermark, it is identical to the accepted version; the final published version of the proceedings is available on IEEE Xplore. bodied AI agents [21, 49].”
BackgroundOpt: Open pre-trained transformer language models
Zhang, S. (2025) · TIB Data Manager · doi:10.57702/dn0pmcbs · Cited by 405 · see citing works
GPT4RoI: Instruction Tuning Large Language Model on Region-of-InterestPreprint
Zhang, S., Sun, P., Chen, S., Xiao, M., Shao, W., Zhang, W., Liu, Y., Chen, K., & Luo, P. (2023) · ECCV Workshops · doi:10.48550/arxiv.2307.03601 · Cited by 390 · see citing works
Self-citationScanRefer: 3D Object Localization in RGB-D Scans Using Natural Language
Chen, D. Z., Chang, A. X., & Nießner, M. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58565-5_13 · Cited by 315 · see citing works
ReferIt3D: Neural Listeners for Fine-Grained 3D Object Identification in Real-World Scenes
Achlioptas, P., Abdelreheem, A., Xia, F., Elhoseiny, M., & Guibas, L. J. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58452-8_25 · Cited by 255 · see citing works
Driving with LLMs: Fusing Object-Level Vector Modality for Explainable Autonomous Driving
Chen, L., Sinavski, O., Hünermann, J., Karnsund, A., Willmott, A. J., Birch, D., Maund, D., & Shotton, J. (2024) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra57147.2024.10611018 · Cited by 203 · see citing works
ScanQA: 3D Question Answering for Spatial Scene Understanding
Azuma, D., Miyanishi, T., Kurita, S., & Kawanabe, M. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01854 · Cited by 197 · see citing works
Drive Like a Human: Rethinking Autonomous Driving with Large Language Models
Fu, D., Li, X., Wen, L., Dou, M., Cai, P., Shi, B., & Qiao, Y. (2024) · IEEE/CVF Winter Conference on Applications of Computer Vision Workshops (WACVW) · doi:10.1109/wacvw60836.2024.00102 · Cited by 195 · see citing works
AppAgent: Multimodal Agents as Smartphone Users
Zhang, C., Yang, Z., Liu, J., Li, Y., Han, Y., Chen, X., Huang, Z., Fu, B., & Yu, G. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713600 · Cited by 164 · see citing works
Self-citation3DVG-Transformer: Relation Modeling for Visual Grounding on Point Clouds
Zhao, L., Cai, D., Sheng, L., & Xu, D. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00292 · Cited by 153 · see citing works
MotionGPT: Human Motion as a Foreign Language
Jiang, B., Chen, X., Liu, W., Yu, J., Yu, G., & Chen, T. (2023) · arXiv · doi:10.48550/arxiv.2306.14795 · Cited by 149 · see citing works
Self-citationMotionGPT: Human Motion as a Foreign Language
Jiang, B., Chen, X., Wen, L. I. U., Jingyi Yu, A., Yu, G., & Chen, T. (2023) · neural information processing systems · doi:10.52202/075280-0880 · Cited by 149 · see citing works
Self-citation3D-LLM: Injecting the 3D World into Large Language Models
Hong, Y., Zhen, H., Chen, P., Zheng, S., Du, Y., Chen, Z., & Gan, C. (2023) · neural information processing systems · doi:10.52202/075280-0900 · Cited by 143 · see citing works
GPT4RoI: Instruction Tuning Large Language Model on Region-of-Interest
Zhang, S., Sun, P., Chen, S., Xiao, M., Shao, W., Zhang, W., Liu, Y., Chen, K., & Luo, P. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-91813-1_4 · Cited by 141 · see citing works
Self-citationScan2Cap: Context-aware Dense Captioning in RGB-D Scans
Chen, D. Z., Gholami, A., Nießner, M., & Chang, A. X. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00321 · Cited by 136 · see citing works
PLA: Language-Driven Open-Vocabulary 3D Scene Understanding
Ding, R., Yang, J., Xue, C., Zhang, W., Bai, S., & Qi, X. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.00677 · Cited by 131 · see citing works
PointLLM: Empowering Large Language Models to Understand Point Clouds
Xu, R., Wang, X., Wang, T., Chen, Y., Pang, J., & Lin, D. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72698-9_8 · Cited by 131 · see citing works
ChartLlama: A Multimodal LLM for Chart Understanding and GenerationPreprint
Han, Y., Zhang, C., Chen, X., Yang, X., Wang, Z., Yu, G., Fu, B., & Zhang, H. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2311.16483 · Cited by 126 · see citing works
Self-citation3D-LLM: Injecting the 3D World into Large Language Models
Hong, Y., Zhen, H., Chen, P., Zheng, S., Du, Y., Chen, Z., & Gan, C. (2023) · arXiv · doi:10.48550/arxiv.2307.12981 · Cited by 124 · see citing works
3DJCG: A Unified Framework for Joint Dense Captioning and Visual Grounding on 3D Point Clouds
Cai, D., Zhao, L., Zhang, J., Sheng, L., & Xu, D. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01597 · Cited by 93 · see citing works
TransRefer3D: Entity-and-Relation Aware Transformer for Fine-Grained 3D Visual Grounding
He, D., Zhao, Y., Luo, J., Hui, T., Huang, S., Zhang, A., & Liu, S. (2021) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3474085.3475397 · Cited by 81 · see citing works
OPT-IML: Scaling Language Model Instruction Meta Learning through the Lens of Generalization
Iyer, S., Lin, X. V., Pasunuru, R., Mihaylov, T., Simig, D., Yu, P., Shuster, K., Wang, T., Liu, Q., Koura, P. S., et al. (2022) · arXiv · doi:10.48550/arxiv.2212.12017 · Cited by 81 · see citing works
EDA: Explicit Text-Decoupling and Dense Alignment for 3D Visual Grounding
Wu, Y., Cheng, X., Zhang, R., Cheng, Z., & Zhang, J. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01843 · Cited by 73 · see citing works
Context-aware Alignment and Mutual Masking for 3D-Language Pre-training
Jin, Z., Hayat, M., Yang, Y., Guo, Y., & Lei, Y. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.01057 · Cited by 56 · see citing works
End-to-End 3D Dense Captioning with Vote2Cap-DETR
Chen, S., Zhu, H., Chen, X., Lei, Y., Yu, G., & Chen, T. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01070 · Cited by 53 · see citing works
Self-citationCLIP-Guided Vision-Language Pre-training for Question Answering in 3D Scenes
Parelli, M., Delitzas, A., Hars, N., Vlassis, G., Anagnostidis, S., Bachmann, G., & Hofmann, T. (2023) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw59228.2023.00593 · Cited by 48 · see citing works
Scalable 3D Captioning with Pretrained Models
Luo, T., Rockwell, C., Lee, H., & Johnson, J. (2023) · neural information processing systems · doi:10.52202/075280-3291 · Cited by 47 · see citing works
UniT3D: A Unified Transformer for 3D Dense Captioning and Visual Grounding
Chen, D. Z., Hu, R., Chen, X., Nießner, M., & Chang, A. X. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01660 · Cited by 43 · see citing works
Self-citation3D Concept Learning and Reasoning from Multi-View Images
Hong, Y., Lin, C., Du, Y., Chen, Z., Tenenbaum, J. B., & Gan, C. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00888 · Cited by 42 · see citing works
ShapeGPT: 3D Shape Generation With a Unified Multi-Modal Language Model
Yin, F., Chen, X., Zhang, C., Jiang, B., Zhao, Z., Liu, W., Yu, G., & Chen, T. (2025) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2025.3535389 · Cited by 40 · see citing works
Self-citationSpatiality-guided Transformer for 3D Dense Captioning on Point Clouds
Wang, H., Zhang, C., Yu, J., & Cai, W. (2022) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2022/194 · Cited by 39 · see citing works
Self-citationStableLLaVA: Enhanced Visual Instruction Tuning with Synthesized Image-Dialogue DataPreprint
Li, Y., Zhang, C., Yu, G., Wang, Z.-B., Fu, B., Lin, G., Shen, C.-H., Chen, L., & Wei, Y.-C. (2023) · arXiv.org · doi:10.48550/arxiv.2308.10253 · Cited by 37 · see citing works
Self-citationMORE: Multi-Order RElation Mining for Dense Captioning in 3D Scenes
Jiao, Y., Chen, S., Jie, Z., Chen, J., Ma, L., & Jiang, Y. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19833-5_31 · Cited by 34 · see citing works
Self-citationYe, S., Chen, D., Han, S., & Liao, J. (2022) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2022.3225327 · Cited by 32 · see citing works
Toward Explainable 3D Grounded Visual Question Answering: A New Benchmark and Strong Baseline
Zhao, L., Cai, D., Zhang, J., Sheng, L., Xu, D., Zheng, R., Zhao, Y., Wang, L., & Fan, X. (2022) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2022.3229081 · Cited by 31 · see citing works
Vote2Cap-DETR++: Decoupling Localization and Describing for End-to-End 3D Dense Captioning
Chen, S., Zhu, H., Li, M., Chen, X., Guo, P., Lei, Y., Yu, G., Li, T., & Chen, T. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3387838 · Cited by 28 · see citing works
Self-citationScalable 3D Captioning with Pretrained ModelsPreprint
Luo, T., Rockwell, C., Lee, H., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2306.07279 · Cited by 21 · see citing works
SQA3D: Situated Question Answering in 3D ScenesPreprint
Ma, X., Yong, S., Zheng, Z., Li, Q., Liang, Y., Zhu, S. C., & Huang, S. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2210.07474 · Cited by 16 · see citing works
Position-Enhanced Visual Instruction Tuning for Multimodal Large Language ModelsPreprint
Chen, C., Qin, R., Luo, F., Mi, X., Li, P., Sun, M., & Liu, Y. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2308.13437 · Cited by 13 · see citing works
M3DBench: Let's Instruct Large Models with Multi-modal 3D PromptsPreprint
Li, M., Chen, X., Zhang, C., Chen, S., Zhu, H., Yin, F., Yu, G., & Chen, T. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2312.10763 · Cited by 13 · see citing works
Self-citationComplete 3D Relationships Extraction Modality Alignment Network for 3D Dense Captioning
Mao, A., Yang, Z. X., Chen, W., Yi, R., & Liu, Y. (2023) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2023.3279204 · Cited by 12 · see citing works
Song, Z., Wang, C., Sheng, J., Zhang, C., Tang, S., Fan, J., & Chen, T. (2025) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3746027.3758252 · Cited by 12 · see citing works
Self-citationContextual Modeling for 3D Dense Captioning on Point CloudsPreprint
Zhong, Y., Long, X., Luo, J., & Ma, L. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2210.03925 · Cited by 11 · see citing works
AppAgent: Multimodal Agents as Smartphone Users
Zhang, C., Yang, Z., Liu, J., Li, Y., Han, Y., Chen, X., Huang, Z., Fu, B., & Yu, G. (2023) · arXiv · doi:10.48550/arxiv.2312.13771 · Cited by 11 · cited 1× · see citing works
“During this LLM carnival, researchers are also seeking generalized LLM solutions for various vision language tasks [37, 50, 62].”
BackgroundSelf-citationRegionBLIP: A Unified Multi-modal Pre-training Framework for Holistic and Regional ComprehensionPreprint
Zhou, Q., Yu, C., Zhang, S., Wu, S., Wang, Z., & Wang, F. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2308.02299 · Cited by 10 · see citing works
Cao, Y., Zeng, Y., Xu, H., & Xu, D. (2023) · Rare & Special e-Zone (The Hong Kong University of Science and Technology) · Cited by 9 · see citing works
Multi-CLIP: Contrastive Vision-Language Pre-training for Question Answering tasks in 3D ScenesPreprint
Delitzas, A., Parelli, M., Hars, N., Vlassis, G., Anagnostidis, S., Bachmann, G., & Hofmann, T. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2306.02329 · Cited by 7 · see citing works
Cao, Y., Zeng, Y., Xu, H., & Xu, D. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2310.02960 · Cited by 7 · see citing works
ShapeGPT: 3D Shape Generation with A Unified Multi-modal Language ModelPreprint
Yin, F., Chen, X., Zhang, C., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2311.17618 · Cited by 5 · see citing works
Self-citationSong, Z., Wang, C., Sheng, J., et al. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2403.01422 · Cited by 2 · see citing works
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805
OPT: Open Pre-trained Transformer Language Models
Zhang, S., Roller, S., Goyal, N., Artetxe, M., Chen, M., Chen, S., Dewan, C., Diab, M., Li, X., Lin, X. V., et al. (2022) · arXiv · doi:10.48550/arxiv.2205.01068
Self-citation
Flagged first, then by citations
9 of the 89 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
The Revolution of Multimodal Large Language Models: A Survey
Caffagni, D., Cocchi, F., Barsellotti, L., Moratelli, N., Sarto, S., Baraldi, L., Baraldi, L., Cornia, M., & Cucchiara, R. (2024) · Findings of the Association for Computational Linguistics ACL 2024 · doi:10.18653/v1/2024.findings-acl.807 · Cited by 98 · see citing works
“For input regions, GLaMM (Rasheed et al., 2023), GPT4RoI (Zhang et al., 2023g), ASM (Wang et al., 2023d) and ChatterBox (Tian et al., 2024b) leverage features of the image encoder to perform ROI align on the bounding box, whereas PVIT (Chen et al., 2023a) exploits Re-gionCLIP (Zhong et al., 2022).”
A Survey of Label-Efficient Deep Learning for 3D Point Clouds
Xiao, A., Zhang, X., Shao, L., & Lu, S. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3416302 · Cited by 76 · see citing works
Multimodal fusion and vision–language models: A survey for robot vision
Han, X., Chen, S., Fu, Z., Feng, Z., Fan, L., An, D., Wang, C., Guo, L., Meng, W., Zhang, X., et al. (2026) · Information Fusion · doi:10.1016/j.inffus.2025.103652 · Cited by 41 · see citing works
“Scene Understanding [303] 69.4 12.0 14.5 35.7 20.5 LL3DA [304] 76.8 13.5 15.9 37.3 - Qwen2-VL-7B [305] 53.9 3.0 11.4 29.3 -”
InfluentialSelf-citationLayoutVLM: Differentiable Optimization of 3D Layout via Vision-Language Models
Sun, F.-Y., Liu, W., Gu, S., Lim, D., Bhat, G., Tombari, F., Li, M., Haber, N., & Wu, J. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.02744 · Cited by 38 · see citing works
Self-citationLLaVA-3D: A Simple Yet Effective Pathway to Empowering LMMs with 3D Capabilities
Zhu, C., Wang, T., Zhang, W., Pang, J., & Liu, X. (2025) · 2025 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.00409 · Cited by 42 · see citing works
Wu, D., Zheng, P., Zhao, Q., Zhang, S., Qi, J., Hu, J., Zhu, G., & Wang, L. (2025) · Robotics and Computer-Integrated Manufacturing · doi:10.1016/j.rcim.2025.103064 · Cited by 35 · see citing works
MiniGPT-3D: Efficiently Aligning 3D Point Clouds with Large Language Models using 2D Priors
Tang, Y., Han, X., Li, X., Qiao, Y., Hao, Y., Hu, L., & Chen, M. (2024) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3664647.3681257 · Cited by 27 · see citing works
3UR-LLM: An End-to-End Multimodal Large Language Model for 3D Scene Understanding
Xiong, H., Zhuge, Y., Zhu, J., Zhang, L., & Lu, H. (2025) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2025.3557620 · Cited by 26 · see citing works
Unifying 3D Vision-Language Understanding via Promptable Queries
Zhu, Z., Zhang, Z., Ma, X., Niu, X., Chen, Y., Jia, B., Deng, Z., Huang, S., & Li, Q. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72784-9_11 · Cited by 24 · see citing works
LSceneLLM: Enhancing Large 3D Scene Understanding Using Adaptive Visual Preferences
Zhi, H., Chen, P., Li, J., Ma, S., Sun, X., Xiang, T., Lei, Y., Tan, M., & Gan, C. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.00356 · Cited by 21 · see citing works
Visual Instruction Tuning towards General-Purpose Multimodal Large Language Model: A Survey
Huang, J., Zhang, J., Jiang, K. Y., Qiu, H., Zhang, X., Shao, L., Lu, S., & Tao, D. (2025) · International Journal of Computer Vision · doi:10.1007/s11263-025-02572-7 · Cited by 21 · see citing works
TOD3Cap: Towards 3D Dense Captioning in Outdoor Scenes
Jin, B., Zheng, Y., Li, W., Li, W., Zheng, Y., Zhu, J., Liu, X., Zhu, J., Yan, Z., Sun, H., et al. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72649-1_21 · Cited by 20 · see citing works
Inst3D-LMM: Instance-Aware 3D Scene Understanding with Multi-modal Instruction Tuning
Yu, H., Li, W., Wang, S., Chen, J., & Zhu, J. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.01320 · Cited by 16 · see citing works
3D-LLaVA: Towards Generalist 3D LMMs with Omni Superpoint Transformer
Deng, J., He, T., Jiang, L., Wang, T., Dayoub, F., & Reid, I. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.00357 · Cited by 16 · see citing works
ScanReason: Empowering 3D Visual Grounding with Reasoning Capabilities
Zhu, C., Wang, T., Zhang, W., Chen, K., & Liu, X. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73242-3_9 · Cited by 16 · see citing works
MotionChain: Conversational Motion Controllers via Multimodal Prompts
Jiang, B., Chen, X., Zhang, C., Yin, F., Li, Z., Yu, G., & Fan, J. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73347-5_4 · Cited by 13 · see citing works
Self-citationRoss3d: Reconstructive Visual Instruction Tuning With 3D-Awareness
Wang, H., Zhao, Y., Wang, T., Fan, H., Zhang, X., & Zhang, Z. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.00866 · Cited by 11 · see citing works
View Selection for 3D Captioning via Diffusion Ranking
Luo, T., Johnson, J., & Lee, H. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72751-1_11 · Cited by 10 · see citing works
SpatialLLM: From multi-modality data to urban spatial intelligence
Chen, J., Wang, H., Li, J., Liu, Y., Dong, Z., & Yang, B. (2026) · International Journal of Applied Earth Observation and Geoinformation · doi:10.1016/j.jag.2026.105177 · Cited by 10 · see citing works
Shi, L., Kan, S., Jin, Y., Zhang, L., & Cen, Y. (2025) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2025.3557703 · Cited by 9 · see citing works
Reason3D: Searching and Reasoning 3D Segmentation via Large Language Model
Huang, K., Li, X., Qi, L., Yan, S., & Yang, M. (2025) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv66043.2025.00112 · Cited by 9 · see citing works
Embodied intelligence for 3D understanding: A survey on 3D Scene question answering
Li, Z., Yu, H., Ding, Y., Li, Y., He, Y., & Akhtar, N. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103624 · Cited by 8 · see citing works
Detect Anything 3D in the Wild
Zhang, H., Jiang, H., Yao, Q., Sun, Y., Zhang, R., Zhao, H., Li, H., Zhu, H., & Yang, Z. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.00480 · Cited by 6 · see citing works
Self-citationFei, H., Luo, M., Xu, J., Wu, S., Ji, W., Lee, M. L., & Hsu, W. (2024) · Proceedings of the 1st ACM Multimedia Workshop on Multi-modal Misinformation Governance in the Era of Foundation Models · doi:10.1145/3689090.3689388 · Cited by 6 · see citing works
Self-citationPARIS3D: Reasoning-Based 3D Part Segmentation Using Large Multimodal Model
Kareem, A., Lahoud, J., & Cholakkal, H. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73220-1_27 · Cited by 6 · see citing works
A Language-Driven Navigation Strategy Integrating Semantic Maps and Large Language Models
Zhong, Z., He, Y., Li, P., Yu, F. R., & Ma, F. (2024) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros58592.2024.10802631 · Cited by 5 · see citing works
Symmetry-Aware Advances in Multimodal Large Language Models: Architectures, Training, and Evaluation
Liu, X., & Liu, H. (2025) · Symmetry · doi:10.3390/sym17091400 · Cited by 5 · see citing works
From Multimodal LLM to Human-level AI: Modality, Instruction, Reasoning, Efficiency and beyond
Fei, H., Yao, Y., Zhang, Z., Liu, F., Zhang, A., & Chua, T. (2024) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/39sysjo27vbh · Cited by 5 · see citing works
Self-citationM3DMap: Object-Aware Multimodal 3D Mapping for Dynamic Environments
Yudin, D. A. (2025) · Optical Memory and Neural Networks · doi:10.3103/s1060992x25700092 · Cited by 4 · see citing works
Generation Models Know Space: Unleashing Implicit 3D Priors for Scene Understanding
Wu, X., Liang, D., Feng, T., Xia, K., Zhang, Y., Li, X., Tan, X., & Bai, X. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37235-2_15 · Cited by 4 · see citing works
PointCloud-Text Matching: Benchmark Dataset and Baseline
Feng, Y., Qin, Y., Peng, D., Zhu, H., Peng, X., & Hu, P. (2025) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2025.3590931 · Cited by 4 · see citing works
Self-citationCVP: Central-Peripheral Vision-Inspired Multimodal Model for Spatial Reasoning
Chen, Z., Zhang, X., Xu, H., Xie, J., & Tu, Z. (2026) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv61042.2026.00227 · Cited by 3 · see citing works
Huang, J., Jia, B., Wang, Y., Zhu, Z., Linghu, X., Li, Q., Zhu, S. C., & Huang, S. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.02288 · Cited by 3 · see citing works
3D-AVS: LiDAR-based 3D Auto-Vocabulary Segmentation
Wei, W., Ülger, O., Nejadasl, F. K., Gevers, T., & Oswald, M. R. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.00833 · Cited by 2 · see citing works
Mo, W., Chen, Q., Peng, Y., Huang, S., & Liu, Y. (2025) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3746027.3758244 · Cited by 2 · see citing works
Spatial 3D-LLM : Exploring Spatial Awareness in 3D Vision-Language Models
Wang, X., Li, Z., Xu, Y., Qi, J., Yang, Z., Ma, R., Liu, X., & Zhang, C. (2025) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme59968.2025.11208990 · Cited by 2 · see citing works
Self-citationAdaToken-3D: Dynamic Spatial Gating for Efficient 3D Large Multimodal-Models Reasoning
Zhang, K., Chen, X., & Zhang, X. (2025) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros60139.2025.11247780 · Cited by 2 · see citing works
Self-citationA review of recent advances in Gaussian splatting
Fan, S., Zeng, L., He, W., Zhang, L., Chen, W., & Pan, Z. (2026) · Applied Intelligence · doi:10.1007/s10489-026-07227-9 · Cited by 2 · see citing works
HIS-GPT: Towards 3D Human-In-Scene Multimodal Understanding
Zhao, J., Hou, R., Tian, Z., Chang, H., & Shan, S. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.00411 · Cited by 2 · see citing works
Towards Scalable Spatial Intelligence Via 2D-To-3D Data Lifting
Miao, X., Duan, H., Qian, Q., Wang, J., Long, Y., Shao, L., Zhao, D., Xu, R., & Zhang, G. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.00096 · Cited by 2 · see citing works
SpaceEra++: A Unified Framework Toward 3D Spatial Reasoning in Video
Guan, W., ZHANG, H., Liu, M., Xiang, Q., Wang, Y., & Nie, L. (2026) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2026.3692302 · Cited by 2 · see citing works
Argus: Leveraging Multiview Images for Improved 3-D Scene Understanding With Large Language Models
Xu, Y., Zhang, C., Jiang, H., Wang, X., Ma, R., Li, Y., Wu, Z., Li, Z., & Liu, X. (2025) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2025.3581411 · Cited by 2 · see citing works
Self-citationAdvancements in 3D vision understanding using multimodal large language models
Feng, M., Shen, J., Wu, Z., Peng, W., Zhong, H., Guo, Y., Shu, X., Zhang, H., Weisheng, D., & Wang, Y. (2025) · Journal of Image and Graphics · doi:10.11834/jig.240588 · Cited by 2 · see citing works
Learn 3D VQA Better with Active Selection and Reannotation
Zhou, S., Liu, Y., & Zheng, F. (2025) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3746027.3755515 · Cited by 2 · see citing works
3D-MoRe: Unified Modal-Contextual Reasoning for Embodied Question Answering
Xu, R., Gao, H., Yu, M., An, D. U., Chen, S., Wang, C., Guo, L., Liang, X., & Xu, S. (2025) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros60139.2025.11247159 · Cited by 1 · see citing works
Self-citationLLplace: Embodied 3D Indoor Layout Synthesis Framework with Large Language Model
Yang, Y., Lu, J., Zhao, Z., Luo, Z., Dong, W., Sanchez, V., & Zheng, F. (2025) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros60139.2025.11246995 · Cited by 1 · see citing works
Halacheva, A.-M., Zaech, J.-N., Wang, X., Paudel, D. P., & Gool, L. V. (2025) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2025.3623037 · Cited by 1 · see citing works
Patel, N., Krishnamurthy, P., & Khorrami, F. (2026) · IEEE Transactions on Robotics · doi:10.1109/tro.2026.3651674 · Cited by 1 · see citing works
Crafting Dynamic Virtual Activities with Advanced Multimodal Models
Li, C., Yan, Q., Kim, M., Li, Z., Xu, Y., & Yu, L.-F. (2025) · Proceedings - International Symposium on Mixed and Augmented Reality, ISMAR · doi:10.1109/ismar67309.2025.00025 · Cited by 1 · see citing works
Test-Time Correction: An Online 3D Detection System via Visual Prompting
Zhang, H., Yang, Z., Sun, Y., Chen, L., Xia, F., Güney, F., & Li, H. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2025.3642076 · Cited by 1 · see citing works
InstrucRobo: Object-centric multi-instruction decoupling model for explainable robotic manipulation
Yang, P., Jing, H., Zheng, N., & Ma, Y. (2026) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2026.114166 · Cited by 1 · see citing works
M3DBench: Towards Omni 3D Assistant with Interleaved Multi-modal Instructions
Li, M., Chen, X., Zhang, C., Chen, S., Zhu, H., Yin, F., Li, Z., Yu, G., & Chen, T. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73636-0_3 · Cited by 1 · see citing works
Self-citationUnrealgensyn: a framework for generating synthetic videos of Unfrequent human events
Mulero-Pérez, D., Benavent-Lledó, M., García‐Rodríguez, J., & Vincze, M. (2025) · Virtual Reality · doi:10.1007/s10055-025-01146-9 · Cited by 1 · see citing works
XFashion: Character Animation Generation via Facial-enhanced and Granularly Controlling
Zhao, Y., Li, B., & Fei, H. (2024) · International Workshop on Human-centric Multimedia Analysis (HCMA) · doi:10.1145/3688865.3689480
Self-citationThe ACM Multimedia 2024 Viual Spatial Description Grand Challenge
Zhao, Y., Fei, H., Li, B., Zhang, M., & Zhang, M. (2024) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3664647.3688989
Self-citationBowser, S., Matuszek, C., & Lukin, S. M. (2025) · doi:10.1145/3733566.3734430
Location-Reasoning for 3D Question Answering in Large Language Models
Yang, W., Liao, Z., Pan, X., Wu, K., & Yuan, J. (2024) · International Conference on Big Data, Artificial Intelligence and Internet of Things Engineering (ICBAIE) · doi:10.1109/icbaie63306.2024.11116802
HCNQA: Enhancing 3D VQA with Hierarchical Concentration Narrowing Supervision
Zhou, S., Zhu, J., Huang, Q., Wang, F., Zhang, Y., & Zheng, F. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-04546-1_3
Chu, X., Deng, J., You, G., Liu, W., Li, X., Ji, J., & Zhang, Y. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.00944
Injecting Cross-modal Fine-Grained Perception into LLMs for 3D Object-of-Interest Understanding
Sun, Q., Shi, L., Zhang, L., An, G., Jin, Y., Li, Y., & Cen, Y. (2025) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme59968.2025.11208909
Wang, J., Tian, Y., Li, J., Sun, H., Tian, H., & Zhang, P. (2025) · IEEE Transactions on Cognitive Communications and Networking · doi:10.1109/tccn.2025.3642880
Error-Driven Scene Editing for 3D Grounding in Large Language Models
Zhang, Y., Wang, Z., Lin, H., Li, J., Yang, J., Bitton, Y., Szpektor, I., & Bansal, M. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37029-7_24
Vision-Language Memory for Spatial Reasoning
Liu, Z., Du, Y., Fu, T., Su, S., Ho, C., & Wang, C. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37356-4_17
A Multi-Agent Workflow for 3D Visualization Code Generation and Evaluation Based on Spatial-MLLM
Li, R., Yan, W., Deng, Q., & Tong, J. (2025) · International Conference on Image Processing, Computer Vision and Machine Learning (ICICML) · doi:10.1109/icicml67980.2025.11333708
M3Fusion: a unified multi-view multi-modality fusion framework for embodied 3D perception
Mao, X., Zhu, C., Tai, W. A. N. G., & Pang, J. (2026) · Scientia Sinica Informationis · doi:10.1360/ssi-2025-0293
Language Generation-based 3D Scene Understanding for Spatial Media
Kim, Y., Yang, J., & Kang, J. (2026) · Journal of Broadcast Engineering · doi:10.5909/jbe.2026.31.1.41
Trends in Multi-modal Large Language Models for 3D Point Cloud Understanding
Son, S. B., Park, S., & Kim, J. (2025) · International Conference on Information and Communication Technology Convergence (ICTC) · doi:10.1109/ictc66702.2025.11388011
Open-Ended 3D Point Cloud Instance Segmentation
Nguyen, P., Luu, M. T. N., Tran, A., Pham, C., & Nguyen, K. (2025) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw69036.2025.00271
Zhang, J., Jiang, J., Li, H., Chen, Y., Jiang, K., & Chen, D. Z. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37356-4_31
Multi-View Images Suffice 3D Reasoning Through Chain-of-Thought Selection and Question-Guided Fusion
Xu, B., Wu, J., Zhang, W., Su, C., Liang, J., Sun, Z. X., & Lei, Z. (2026) · IEEE Transactions on Image Processing · doi:10.1109/tip.2026.3683282
Task-Aware Geometric Embeddings and Cross-Modal Interaction for 3D VQA
Gao, H., Xu, R., Wang, C., Guo, L., & Xu, S. (2026) · International Symposium on Computer Applications and Information Systems (ISCAIS) · doi:10.1109/iscait69154.2026.11477363
Reg3D: Reconstructive Geometry Instruction Tuning for 3D Scene Understanding
Zheng, H., Xiang, L., Lin, Q., Li, Z., Yang, Q., & Yin, H. (2026) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp55912.2026.11463814
Zero-Shot Multimodal Part Segmentation for Space Target via 3D-2D Semantic Priors Transfer
Feng, C., Gu, X., Wang, B., & Yang, X. (2026) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2026.3689873
Denoising diffusion bridge for cross-modal alignment in three-dimensional scene understanding
Li, J., Feng, Y., Luan, W., Xiao, P., Chen, Y., Duan, G., Tan, J., & Zhou, M. (2026) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2026.115295
Jiang, S., Yuan, Z., & Zhao, N. (2026) · International Conference on Multimedia Retrieval · doi:10.1145/3805622.3810869
Holo-Captioning: Toward the Text Equivalent of 3D Scenes
Lin, K.-Y., Bu, C., Li, Z., & Han, K. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37602-2_37
Feng, H., Wei, H., Wang, H., Li, S., Li, C., & Wu, Y. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37574-2_33
Talk2AnyPoint: Enabling Deep 3D-Text Dialogue with Chunked Point-Cloud Fusion
Shi, L., Zhang, Y., Bu, W., Zhang, B., & Liu, M. (2026) · Communications in computer and information science · doi:10.1007/978-981-92-3560-5_19
3D-Aware VLMs with Implicit and Explicit Geometries
Li, W., Jiang, X., Qian, Q., Zhao, D., Xu, R., Lu, S., & Zhang, G. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37092-1_10
Unsolvable Problem Detection and Trustworthy Reasoning in 3D-LLMs
Elgin, M., & Sheshappanavar, S. V. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-31673-8_45
Query-Driven Evidence Retrieval for Efficient 3-D Question Answering
Liu, H., & Wu, H. (2026) · IEEE Access · doi:10.1109/access.2026.3720265
3DZip: Spatial-Aware Feature Diversity-Guided Token Compression for 3D Question Answering
Baek, C., & Kong, K. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37321-2_25
Demystifying 3D Spatial Awareness via LLM Router
Tao, H., Zhang, L., Chen, L., Lin, T., Qin, H., Zhang, J., Kong, M., Zhu, Q., & Zhang, F. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-31397-3_29
Qwen-3D: A Generalist 3D Vision-Language Model for Spatial Understanding
Lin, L., Jain, A., Liu, Y., & Fragkiadaki, K. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37271-0_11
From Compression to Exploration: Active Querying Agent for 3D Physical Field Understanding
Gao, C., Zhou, H., Luo, Z., Jin, S., Yu, Q., Chen, T., & Li, J. (2026) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3770855.3819007
Self-citationHuang, T., Wan, W., & Fang, Z. (2026) · Knowledge-Based Systems · doi:10.1016/j.knosys.2026.116996
ORACLE-3D: Open-World Region-Aligned Cross-Modal Learning for Label-Efficient 3D Scene Understanding
Wang, Y., Liu, P., Wang, S., Zhang, Z., Lu, X., Cai, C., Li, H., Liu, H., Ning, Q., & Ma, J. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37026-6_2
Open-Vocabulary Instance Segmentation for Scene Understanding in Mobile Robots
Decena-Gimenez, M., Ojeda, P., Ruiz-Sarmiento, J.-R., & González-Jiménez, J. (2026) · Robotics · doi:10.3390/robotics15090175
Learning Generatable Mutual Distance for Scene-Aware Human Motion Generation
Yang, Z., Xing, C., Yin, Y., Liu, M., & Pan, L. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37258-1_9
C 2 ROPE: Causal Continuous Rotary Positional Encoding for 3D Large Multimodal-Models Reasoning
Ye, G., Zhao, Q., Yu, W., Zhang, X., Ji, J., Zhang, Y., & Yuen, K. (2026) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra57385.2026.11695834
Zhao, Y., Zheng, Z., Chen, C., Zhang, S., Zhang, H., Dai, W., He, F., & Xiong, H. (2026) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra57385.2026.11697495
Map2thought: Explicit 3D Spatial Reasoning via Metric Cognitive Maps
Gao, X., Zhang, Z., Chen, D. Z., Xu, S., Quan, L., Pérez-Pellitero, E., & Jang, Y. (2026) · doi:10.1109/cvprf71909.2026.00687
POMA-3D: The Point Map Way to 3D Scene Understanding
Mao, Y., Luo, W., Huang, R., Jing, J., & Mikolajczyk, K. (2026) · doi:10.1109/cvprf71909.2026.00699
Beyond Single Object: Learning 3D Relations with Large Language Models
Ide, K., Yamada, R., Qiu, Y., Ma, X., Fukuhara, Y., Kataoka, H., & Satoh, Y. (2026) · doi:10.1109/cvprf71909.2026.00927
Efficient3D: A Unified Framework for Adaptive and Debiased Token Reduction in 3D MLLMs
Lin, Y., Yu, S., Yang, Y., Cheng, G., & Xiao, J. (2026) · doi:10.1109/cvprf71909.2026.00858
Cho, S., Im, S., Chung, D., Kim, T., Shim, W., & 김형관Kim, H. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.7641925
Diagnosing Order Dependence in Vision-Language Task-Progress Judgment with Equal-State Supervision
Wang, C., Yang, X., Zhang, X., Tang, M., Xue, J., Gao, S., Zhang, J., geng, X., Zhang, J., Yuan, H., et al. (2026) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/2026.findings-emnlp.792
- Authors
- Chen, S., Chen, X., Zhang, C., Li, M., Yu, G., Fei, H., Zhu, H., Fan, J., & Chen, T.
- Affiliations
- Fudan University (CN)
- Tencent (China) (CN)
- National University of Singapore (SG)
- Agency for Science, Technology and Research (SG)
- Venue
- 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2024-06-16
- Pages
- 26418-26428
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W4402716423
- Cited by
- 92
- References
- 89
- Citations by year
- 2026: 39 · 2025: 40 · 2024: 13
- FWCI
- 19.05
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Engineering › Computational Mechanics › 3D Shape Modeling and Analysis
- Physical Sciences › Engineering › Control and Systems Engineering › Robot Manipulation and Learning
- Keywords
- instruction tuning, large language models, point clouds, visual interaction, 3D scene understanding, human-machine interaction
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
