Language-driven Grasp Detection
Open accessAbstract
Grasp detection is a persistent and intricate challenge with various industrial applications. Recently, many meth-ods and datasets have been proposed to tackle the grasp detection problem. However, most of them do not consider using natural language as a condition to detect the grasp poses. In this paper, we introduce Grasp-Anything++, a new language-driven grasp detection dataset featuring 1M samples, over 3M objects, and upwards of 10M grasping in-structions. We utilize foundation models to create a large-scale scene corpus with corresponding images and grasp prompts. We approach the language-driven grasp detection task as a conditional generation problem. Drawing on the success of diffusion models in generative tasks and given that language plays a vital role in this task, we propose a new language-driven grasp detection method based on dif-fusion models. Our key contribution is the contrastive training objective, which explicitly contributes to the denoising process to detect the grasp pose given the language instructions. We illustrate that our approach is theoretically sup-portive. The intensive experiments show that our method outperforms state-of-the-art approaches and allows real-world robotic grasping. Finally, we demonstrate our large-scale dataset enables zero-short grasp detection and is a challenging benchmark for future work.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Low-quality data challenges include noise pollution, missing modalities, modal quality imbalance, and dynamic changes in modal quality, and are widely found in autonomous driving, medical image analysis, and speech recognition.
“The challenges of low-quality data are mainly reflected in various aspects such as noise pollution, missing modalities, modal quality imbalance, and dynamic changes in modal quality, etc., which are widely found in many application scenarios such as autonomous driving [81,88,257], medical image analysis, speech recognition [296], etc., and pose great challenges to the validity and reliability of”
From Multimodal fusion and vision–language models: A survey for robot vision (2026) · cites this paper as [257]
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 48,293 · see citing works
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372 · see citing works
High-Resolution Image Synthesis with Latent Diffusion Models
Rombach, R., Blattmann, A., Lorenz, D., Esser, P., & Ommer, B. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01042 · Cited by 15,880 · see citing works
Learning Transferable Visual Models From Natural Language Supervision
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al. (2021) · arXiv · doi:10.48550/arxiv.2103.00020 · Cited by 15,293 · see citing works
Kaugeranna, R., Kaugeranna, E., & Aion, (. S. 4. 6. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.giscience.2023.43 · Cited by 13,695 · see citing works
Kirillov, A., Mintun, E., Ravi, N., Mao, H., Rolland, C., Gustafson, L., Xiao, T., Whitehead, S., Berg, A. C., Lo, W.-Y., et al. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00371 · Cited by 11,953 · see citing works
Denoising Diffusion Probabilistic Models
Ho, J., Jain, A., & Abbeel, P. (2020) · arXiv · doi:10.48550/arxiv.2006.11239 · Cited by 8,712 · see citing works
Diffusion Models Beat GANs on Image Synthesis
Dhariwal, P., & Nichol, A. (2021) · arXiv · doi:10.48550/arxiv.2105.05233 · Cited by 3,362 · see citing works
Towards A Rigorous Science of Interpretable Machine Learning
Doshi-Velez, F., & Kim, B. (2017) · arXiv · doi:10.48550/arxiv.1702.08608 · Cited by 3,121 · see citing works
BERTScore: Evaluating Text Generation with BERT
Zhang, T., Kishore, V., Wu, F., Weinberger, K. Q., & Artzi, Y. (2019) · arXiv · doi:10.48550/arxiv.1904.09675 · Cited by 2,865 · see citing works
Deep Unsupervised Learning using Nonequilibrium Thermodynamics
Sohl-Dickstein, J., Weiss, E. A., Maheswaranathan, N., & Ganguli, S. (2015) · arXiv · doi:10.48550/arxiv.1503.03585 · Cited by 2,637 · see citing works
Huang, L., Yu, W., Ma, W., Zhong, W., Feng, Z., Wang, H., Chen, Q., Peng, W., Feng, X., Qin, B., et al. (2023) · arXiv · doi:10.48550/arxiv.2311.05232 · Cited by 2,516 · cited 2× · see citing works
“Despite training on extensive datasets, Stable Diffusion [74] may produce subpar content, commonly termed as hallucination [36] when generating images from the text prompts.”
BackgroundHierarchical Text-Conditional Image Generation with CLIP Latents
Ramesh, A., Dhariwal, P., Nichol, A., Chu, C., & Chen, M. (2022) · arXiv · doi:10.48550/arxiv.2204.06125 · Cited by 2,511 · cited 1× · see citing works
“Instead of predicting x t as formulated by [33], we follow Ramesh et al. [72] and predict the signal itself, i.e., x t = f ( x t +1 , t + 1 , ˜ x 0 ) with the simple objective [33].”
MethodVision Transformers for Dense Prediction
Ranftl, R., Bochkovskiy, A., & Koltun, V. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01196 · Cited by 2,373 · cited 1× · see citing works
“First, future work could investigate the use of text or image-to-3D models [95] or image-to-depth [73] and reuse our dataset’s prompts and images to construct 3D language-driven grasp datasets.”
BackgroundHuang, L., Yu, W., Ma, W., Zhong, W., Feng, Z., Wang, H., Chen, Q., Peng, W., Feng, X., Qin, B., et al. (2025) · ACM Transactions on Information Systems · doi:10.1145/3703155 · Cited by 2,142 · see citing works
PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenes
Xiang, Y., Schmidt, T., Narayanan, V., & Fox, D. (2018) · Robotics: Science and Systems XIV · doi:10.15607/rss.2018.xiv.019 · Cited by 2,139 · see citing works
Grounding DINO: Marrying DINO with Grounded Pre-training for Open-Set Object Detection
Liu, S., Zeng, Z., Ren, T., Li, F., Zhang, H., Yang, J., Jiang, Q., Li, C., Yang, J., Su, H., et al. (2025) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-72970-6_3 · Cited by 2,074 · see citing works
Levine, S., Pastor, P., Krizhevsky, A., Ibarz, J., & Quillen, D. (2018) · The International Journal of Robotics Research · doi:10.1177/0278364917710318 · Cited by 1,971 · see citing works
Diffusion Models in Vision: A Survey
Croitoru, F.-A., Hondru, V., Ionescu, R. T., & Shah, M. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3261988 · Cited by 1,965 · see citing works
Conditional Prompt Learning for Vision-Language Models
Zhou, K., Yang, J., Loy, C. C., & Liu, Z. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01631 · Cited by 1,918 · see citing works
Diffusion Models: A Comprehensive Survey of Methods and Applications
Yang, L., Zhang, Z., Song, Y., Hong, S., Xu, R., Zhao, Y., Zhang, W., Cui, B., & Yang, M.-H. (2024) · ACM Computing Surveys · doi:10.1145/3626235 · Cited by 1,704 · see citing works
Noise-contrastive estimation: A new estimation principle for unnormalized statistical models
Gutmann, M. U., & Hyvärinen, A. (2010) · Edinburgh Research Explorer (University of Edinburgh) · Cited by 1,588 · see citing works
Mahler, J., Liang, J., Niyaz, S., Laskey, M., Doan, R., Liu, X., Aparicio, J., & Goldberg, K. Y. (2017) · Robotics: Science and Systems · doi:10.15607/rss.2017.xiii.058 · Cited by 1,299 · see citing works
LVIS: A Dataset for Large Vocabulary Instance Segmentation
Gupta, A., Dollar, P., & Girshick, R. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00550 · Cited by 1,274 · see citing works
Align before Fuse: Vision and Language Representation Learning with Momentum Distillation
Li, J., Selvaraju, R. R., Gotmare, A. D., Joty, S., Xiong, C., & Hoi, S. (2021) · arXiv · doi:10.48550/arxiv.2107.07651 · Cited by 1,078 · see citing works
Supersizing self-supervision: Learning to grasp from 50K tries and 700 robot hours
Pinto, L., & Gupta, A. (2016) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2016.7487517 · Cited by 1,060 · see citing works
LAION-5B: An open large-scale dataset for training next generation image-text models
Schuhmann, C., Beaumont, R., Vencu, R., Gordon, C., Wightman, R., Cherti, M., Coombes, T., Katta, A., Mullis, C., Wortsman, M., et al. (2022) · arXiv · doi:10.48550/arxiv.2210.08402 · Cited by 1,022 · see citing works
Summary of ChatGPT-Related research and perspective towards the future of large language models
Liu, Y., Han, T., Ma, S., Zhang, J., Yang, Y., Tian, J., He, H., Li, A., He, M., Liu, Z., et al. (2023) · Meta-Radiology · doi:10.1016/j.metrad.2023.100017 · Cited by 855 · see citing works
Objaverse: A Universe of Annotated 3D Objects
Deitke, M., Schwenk, D., Salvador, J., Weihs, L., Michel, O., VanderBilt, E., Schmidt, L., Ehsanit, K., Kembhavi, A., & Farhadi, A. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.01263 · Cited by 725 · see citing works
Align Your Latents: High-Resolution Video Synthesis with Latent Diffusion Models
Blattmann, A., Rombach, R., Ling, H., Dockhorn, T., Kim, S. W., Fidler, S., & Kreis, K. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.02161 · Cited by 724 · see citing works
GraspNet-1Billion: A Large-Scale Benchmark for General Object Grasping
Fang, H.-S., Wang, C., Gou, M., & Lu, C. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01146 · Cited by 683 · see citing works
Efficient grasping from RGBD images: Learning using a new rectangle representation
Yun, J., Moseson, S., & Saxena, A. (2011) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2011.5980145 · Cited by 677 · see citing works
Closing the Loop for Robotic Grasping: A Real-time, Generative Grasp Synthesis Approach
Morrison, D., Leitner, J., & Corke, P. (2018) · Robotics: Science and Systems · doi:10.15607/rss.2018.xiv.021 · Cited by 662 · see citing works
Do As I Can, Not As I Say: Grounding Language in Robotic Affordances
Ahn, M., Brohan, A., Brown, N., Chebotar, Y., Cortes, O., David, B., Finn, C., Fu, C., Gopalakrishnan, K., Hausman, K., et al. (2022) · arXiv · doi:10.48550/arxiv.2204.01691 · Cited by 624 · see citing works
RegionCLIP: Region-based Language-Image Pretraining
Zhong, Y., Yang, J., Zhang, P., Li, C., Codella, N., Li, L. H., Zhou, L., Dai, X., Yuan, L., Li, Y., et al. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01629 · Cited by 623 · see citing works
GLIGEN: Open-Set Grounded Text-to-Image Generation
Li, Y., Liu, H., Wu, Q., Mu, F., Yang, J., Gao, J., Li, C., & Lee, Y. J. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.02156 · Cited by 610 · see citing works
Ho, J., Salimans, T., Gritsenko, A. A., Chan, W., Norouzi, M., & Fleet, D. J. (2022) · neural information processing systems · doi:10.52202/068431-0628 · Cited by 603 · see citing works
PaLM-E: An Embodied Multimodal Language Model
Driess, D., Xia, F., Sajjadi, M. S. M., Lynch, C., Chowdhery, A., Ichter, B., Wahid, A., Tompson, J., Vuong, Q., Yu, T., et al. (2023) · arXiv · doi:10.48550/arxiv.2303.03378 · Cited by 591 · see citing works
6-DOF GraspNet: Variational Grasp Generation for Object Manipulation
Mousavian, A., Eppner, C., & Fox, D. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00299 · Cited by 585 · see citing works
ChatGPT for Robotics: Design Principles and Model Abilities
Vemprala, S. H., Bonatti, R., Bucker, A., & Kapoor, A. (2024) · IEEE Access · doi:10.1109/access.2024.3387941 · Cited by 570 · see citing works
Antipodal Robotic Grasping using Generative Residual Convolutional Neural Network
Kumra, S., Joshi, S., & Sahin, F. (2020) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros45743.2020.9340777 · Cited by 465 · see citing works
LLM-Planner: Few-Shot Grounded Planning for Embodied Agents with Large Language Models
Song, C. H., Sadler, B. M., Wu, J., Chao, W.-L., Washington, C., & Su, Y. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00280 · Cited by 435 · cited 1× · see citing works
“First, natural language is usually overlooked in existing grasp datasets [69] while training vision-and-language neural networks necessitates an excessive number of labeled examples [80].”
BackgroundJacquard: A Large Scale Dataset for Robotic Grasp Detection
Depierre, A., Dellandréa, E., & Chen, L. (2018) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2018.8593950 · Cited by 381 · see citing works
Ho, J., Salimans, T., Gritsenko, A., Chan, W., Norouzi, M., & Fleet, D. J. (2022) · arXiv · doi:10.48550/arxiv.2204.03458 · Cited by 301 · see citing works
Tevet, G., Raab, S., Gordon, B., Shafir, Y., Cohen-Or, D., & Bermano, A. H. (2022) · arXiv · doi:10.48550/arxiv.2209.14916 · Cited by 290 · see citing works
EDGE: Editable Dance Generation From Music
Tseng, J., Castellon, R., & Liu, C. K. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00051 · Cited by 275 · see citing works
Deep Learning Approaches to Grasp Synthesis: A Review
Newbury, R., Gu, M., Chumbley, L., Mousavian, A., Eppner, C., Leitner, J., Bohg, J., Morales, A., Asfour, T., Kragić, D., et al. (2023) · IEEE Transactions on Robotics · doi:10.1109/tro.2023.3280597 · Cited by 273 · see citing works
Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object DetectionPreprint
Liu, S., Zeng, Z., Ren, T., Li, F., Zhang, H., Yang, J., Jiang, Q., Li, C., Yang, J., Su, H., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2303.05499 · Cited by 247 · see citing works
ConceptFusion: Open-set multimodal 3D mapping
Jatavallabhula, K., Kuwajerwala, A., Gu, Q., Omama, M., Iyer, G., Saryazdi, S., Chen, T., Maalouf, A., Li, S., Keetha, N., et al. (2023) · Robotics: Science and Systems XIX · doi:10.15607/rss.2023.xix.066 · Cited by 228 · see citing works
CLIPort: What and Where Pathways for Robotic Manipulation
Shridhar, M., Manuelli, L., & Fox, D. (2021) · arXiv · doi:10.48550/arxiv.2109.12098 · Cited by 223 · see citing works
ROI-based Robotic Grasp Detection for Object Overlapping Scenes
Zhang, H., Lan, X., Bai, S., Zhou, X., Tian, Z., & Zheng, N. (2019) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros40897.2019.8967869 · Cited by 204 · see citing works
LM-Nav: Robotic Navigation with Large Pre-Trained Models of Language, Vision, and Action
Shah, D., Osinski, B., Ichter, B., & Levine, S. (2022) · arXiv · doi:10.48550/arxiv.2207.04429 · Cited by 179 · see citing works
Finding antipodal point grasps on irregularly shaped objects
Chen, I., & Burdick, J. W. (1993) · IEEE Transactions on Robotics and Automation · doi:10.1109/70.246063 · Cited by 170 · see citing works
Task allocation and planning for product disassembly with human–robot collaboration
Lee, M.-L., Behdad, S., Liang, X. P., & Zheng, M. (2022) · Robotics and Computer-Integrated Manufacturing · doi:10.1016/j.rcim.2021.102306 · Cited by 170 · see citing works
Asif, U., Tang, J., & Harrer, S. (2018) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2018/677 · Cited by 166 · see citing works
Diffusion Models for Implicit Image Segmentation EnsemblesPreprint
Wolleb, J., Sandkühler, R., Bieder, F., Valmaggia, P., & Cattin, P. C. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2112.03145 · Cited by 164 · see citing works
ACRONYM: A Large-Scale Grasp Dataset Based on Simulation
Eppner, C., Mousavian, A., & Fox, D. (2021) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48506.2021.9560844 · Cited by 162 · see citing works
Ainetter, S., & Fraundorfer, F. (2021) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48506.2021.9561398 · Cited by 159 · see citing works
Planning with Diffusion for Flexible Behavior Synthesis
Janner, M., Du, Y., Tenenbaum, J. B., & Levine, S. (2022) · arXiv · doi:10.48550/arxiv.2205.09991 · Cited by 140 · see citing works
EGAD! An Evolved Grasping Analysis Dataset for Diversity and Reproducibility in Robotic Manipulation
Morrison, D., Corke, P., & Leitner, J. (2020) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2020.2992195 · Cited by 130 · see citing works
Motion Planning Diffusion: Learning and Planning of Robot Motions with Diffusion Models
Carvalho, J., Le, A. T., Baierl, M., Koert, D., & Peters, J. (2023) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros55552.2023.10342382 · Cited by 129 · see citing works
Urain, J., Funk, N., Peters, J., & Chalvatzaki, G. (2023) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48891.2023.10161569 · Cited by 102 · see citing works
NeuralLift-360: Lifting an in-the-Wild 2D Photo to A 3D Object with 360° Views
Xu, D., Jiang, Y., Wang, P., Fan, Z., Wang, Y., & Wang, Z. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00435 · Cited by 93 · see citing works
Zero-Shot Contrastive Loss for Text-Guided Diffusion Image Style Transfer
Yang, S., Hwang, H., & Ye, J. C. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.02091 · Cited by 88 · see citing works
FedSeg: Class-Heterogeneous Federated Learning for Semantic Segmentation
Miao, J., Yang, Z., Fan, L., & Yang, Y. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00777 · Cited by 72 · see citing works
A Joint Modeling of Vision-Language-Action for Target-oriented Grasping in Clutter
Xu, K., Zhao, S., Zhou, Z., Li, Z., Pi, H., Zhu, Y., Wang, Y., & Xiong, R. (2023) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48891.2023.10161041 · Cited by 64 · see citing works
EmbodiedGPT: Vision-Language Pre-Training via Embodied Chain of ThoughtPreprint
Mu, Y., Zhang, Q., Hu, M., Wang, W., Ding, M., Jin, J., Wang, B., Dai, J., Qiao, Y., & Luo, P. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2305.15021 · Cited by 58 · see citing works
Fine-grained Angular Contrastive Learning with Coarse Labels
Bukchin, G., Schwartz, E., Saenko, K., Shahar, O., Feris, R., Giryes, R., & Karlinsky, L. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00862 · Cited by 55 · see citing works
Eppner, C., Mousavian, A., & Fox, D. (2022) · Springer proceedings in advanced robotics · doi:10.1007/978-3-030-95459-8_55 · Cited by 54 · see citing works
Going Denser with Open-Vocabulary Part Segmentation
Sun, P., Chen, S., Zhu, C., Xiao, F., Luo, P., Xie, S., & Yan, Z. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01417 · Cited by 53 · see citing works
INVIGORATE: Interactive Visual Grounding and Grasping in Clutter
Zhang, H., Lu, Y., Yu, C., Hsu, D., Lan, X., & Zheng, N. (2021) · Robotics: Science and Systems · doi:10.15607/rss.2021.xvii.020 · Cited by 52 · see citing works
Vu, M. N., Beck, F., Schwegel, M., Hartl-Nesic, C., Nguyen, A., & Kugi, A. (2023) · Mechatronics · doi:10.1016/j.mechatronics.2023.102970 · Cited by 48 · see citing works
Self-citationGrasp-Anything: Large-scale Grasp Dataset from Foundation Models
Vuong, A. D., Vu, M. N., Le, H., Huang, B., Bình, H. T. T., Vo, T., Kugi, A., & Nguyen, A. (2024) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra57147.2024.10611277 · Cited by 47 · see citing works
Self-citationInteractive Robotic Grasping with Attribute-Guided Disambiguation
Yang, Y., Lou, X., & Choi, C. (2022) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra46639.2022.9812360 · Cited by 38 · see citing works
Sim VQA: Exploring Simulated Environments for Visual Question Answering
Cascante-Bonilla, P., Wu, H., Wang, L., Feris, R., & Ordóñez, V. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00500 · Cited by 32 · see citing works
Open-Vocabulary Affordance Detection in 3D Point Clouds
Nguyen, T. Q., Vu, M. N., Vuong, A., Nguyen, D., Vo, T., Le, N., & Nguyen, A. (2023) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros55552.2023.10341553 · Cited by 32 · see citing works
Self-citationAMC-Loss: Angular Margin Contrastive Loss for Improved Explainability in Image Classification
Choi, H., Som, A., & Turaga, P. K. (2020) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw50498.2020.00427 · Cited by 31 · see citing works
EDMP: Ensemble-of-costs-guided Diffusion for Motion Planning
Saha, K., Mandadi, V., Reddy, J., Srikanth, A., Agarwal, A., Sen, B., Singh, A. K., & Krishna, M. (2024) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra57147.2024.10610519 · Cited by 30 · see citing works
Human-centric Scene Understanding for 3D Large-scale Scenarios
Xu, Y., Cong, P., Yao, Y., Chen, R., Hou, Y., Zhu, X., He, X., Jingyi Yu, A., & Ma, Y. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01861 · Cited by 25 · see citing works
VLMbench: A Compositional Benchmark for Vision-and-Language Manipulation
Zheng, K., Chen, X., Jenkins, O. C., & Wang, X. E. (2022) · arXiv · doi:10.48550/arxiv.2206.08522 · Cited by 24 · see citing works
VLMbench: A Compositional Benchmark for Vision-And-Language Manipulation
Zheng, K., Chen, X., Jenkins, O. C., & Wang, X. E. (2022) · neural information processing systems · doi:10.52202/068431-0048 · Cited by 24 · see citing works
Gilles, M., Chen, Y., Winter, T. R., Zeng, E. Z., & Wong, A. K. C. (2022) · IEEE International Conference on Automation Science and Engineering (CASE) · doi:10.1109/case49997.2022.9926427 · Cited by 23 · see citing works
Grasp Learning: Models, Methods, and Performance
Platt, R. (2022) · Annual Review of Control Robotics and Autonomous Systems · doi:10.1146/annurev-control-062122-025215 · Cited by 22 · see citing works
Compositional Transformers for Scene GenerationPreprint
Hudson, D. A., & Zitnick, C. L. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.08960 · Cited by 20 · see citing works
Target-referenced Reactive Grasping for Dynamic Objects
Liu, J., Zhang, R., Fang, H.-S., Gou, M., Fang, H., Wang, C., Xu, S., Yan, H., & Lu, C. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00852 · Cited by 20 · see citing works
Singularity Avoidance with Application to Online Trajectory Optimization for Serial Manipulators
Beck, F., Vu, M. N., Hartl-Nesic, C., & Kugi, A. (2023) · IFAC-PapersOnLine · doi:10.1016/j.ifacol.2023.10.1582 · Cited by 18 · see citing works
Self-citationLanguage-driven Scene Synthesis using Multi-conditional Diffusion ModelPreprint
Vuong, A., Vu, M., Nguyên, T., Huang, B.-R., Nguyen, D., Vo, T., & Nguyen, A. (2023) · Neural Information Processing Systems · doi:10.48550/arxiv.2310.15948 · Cited by 14 · see citing works
Self-citationEC2: Emergent Communication for Embodied Control
Mu, Y., Yao, S., Ding, M., Luo, P., & Gan, C. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00648 · Cited by 9 · see citing works
Object-Centric Inference for Language Conditioned Placement: A Foundation Model based Approach
Xu, Z., Xu, K., Xiong, R., & Wang, Y. (2023) · International Conference on Advanced Robotics and Mechatronics (ICARM) · doi:10.1109/icarm58088.2023.10218865 · Cited by 6 · see citing works
Language-driven Scene Synthesis using Multi-conditional Diffusion Model
Vuong, A. D., Vu, M. N., Nguyen, T., Huang, B., Nguyen, D., Vo, T., & Nguyen, A. (2023) · neural information processing systems · doi:10.52202/075280-1346 · Cited by 6 · see citing works
Self-citationNBMOD: Find It and Grasp It in Noisy BackgroundPreprint
Cao, B., Zhou, X., Guo, C., Zhang, B., Liu, Y., & Tan, Q. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2306.10265 · Cited by 4 · see citing works
Text-Conditioned Sampling Framework for Text-to-Image Generation with Masked Generative Models
Lee, J., Jang, S., Jo, J., Yoon, J., Kim, Y., Kim, J.-H., Ha, J.-W., & Hwang, S. J. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.02125 · Cited by 3 · see citing works
Grasp-Anything: Large-scale Grasp Dataset from Foundation ModelsPreprint
Vuong, A. D., Vu, M. N., Le, H., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2309.09818 · Cited by 2 · see citing works
Self-citationLanguage-Guided Diffusion Model for Visual GroundingPreprint
Chen, S., & Li, B. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2308.09599 · Cited by 1 · see citing works
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805
Flagged first, then by citations
1 of the 97 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Multimodal fusion and vision–language models: A survey for robot vision
Han, X., Chen, S., Fu, Z., Feng, Z., Fan, L., An, D., Wang, C., Guo, L., Meng, W., Zhang, X., et al. (2026) · Information Fusion · doi:10.1016/j.inffus.2025.103652 · Cited by 41 · see citing works
“Its design emphasizes transferring supervision signals from semantic segmentation to 3D detection, thereby improving the recognition of small and occluded objects.”
Diffusion models for robotic manipulation: a survey
Wolf, R., Shi, Y., Liu, S., & Rayyes, R. (2025) · Frontiers in Robotics and AI · doi:10.3389/frobt.2025.1606247 · Cited by 38 · see citing works
Integrating With Multimodal Information for Enhancing Robotic Grasping With Vision-Language Models
Zhao, Z., Zheng, D., Chen, Y., Luo, J., Wang, Y., Huang, P., & Yang, C. (2025) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2025.3550360 · Cited by 15 · see citing works
Lightweight Language-driven Grasp Detection using Conditional Consistency Model
Nguyen, N., Vu, M. N., Huang, B., Vuong, A., Le, N., Vo, T., & Nguyen, A. (2024) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros58592.2024.10802007 · Cited by 15 · see citing works
Self-citationLanguage-Driven 6-DoF Grasp Detection Using Negative Prompt Guidance
Nguyen, T., Vu, M. N., Huang, B., Vuong, A., Vuong, Q., Le, N., Vo, T. N., & Nguyen, A. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72655-2_21 · Cited by 13 · see citing works
Self-citationLanguage-driven Grasp Detection with Mask-guided Attention
Vo, T. V., Vu, M. N., Huang, B., Vuong, A., Le, N., Vo, T., & Nguyen, A. (2024) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros58592.2024.10802256 · Cited by 11 · see citing works
Self-citationGraspSAM: When Segment Anything Model Meets Grasp Detection
Noh, S., Kim, J., Nam, D., Back, S., Kang, R., & Lee, K. (2025) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra55743.2025.11128811 · Cited by 10 · see citing works
Gripping Success Metric for Robotic Fruit Harvesting
Seo, D., & Oh, I.-S. (2024) · Sensors · doi:10.3390/s25010181 · Cited by 8 · see citing works
LogiCzsl: Exploring Logic-induced Representation for Compositional Zero-shot Learning
Wu, P., Lu, X., Hu, H. P., Xian, Y., Shen, J., & Wang, W. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.02821 · Cited by 8 · see citing works
A Parameter-Efficient Tuning Framework for Language-Guided Object Grounding and Robot Grasping
Yu, H., Li, M., Rezazadeh, A., Yang, Y., & Choi, C. (2025) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra55743.2025.11128679 · Cited by 7 · see citing works
PhysVLM: Enabling Visual Language Models to Understand Robotic Physical Reachability
Zhou, W., Tao, M., Zhao, C., Guo, H., Dong, H., Tang, M., & Wang, J. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.00651 · Cited by 6 · see citing works
PAIR360: A Paired Dataset of High-Resolution 360$^{\circ }$ Panoramic Images and LiDAR Scans
Kim, G., Kim, D., Jang, J., & Hwang, H. (2024) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2024.3460418 · Cited by 4 · see citing works
Nguyen, H. H., Vuong, A., Nguyen, A., Reid, I. D., & Vu, M. N. (2025) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros60139.2025.11245817 · Cited by 5 · see citing works
Self-citationLanguage-driven closed-loop grasping with model-predictive trajectory optimization
Nguyen, H. H., Vu, M. N., Beck, F., Ebmer, G., Nguyen, A., Kemmetmüller, W., & Kugi, A. (2025) · Mechatronics · doi:10.1016/j.mechatronics.2025.103335 · Cited by 4 · see citing works
Self-citationAttribute-Based Object Grounding and Robot Grasp Detection with Spatial Reasoning
Yu, H., Zhou, Z., Sun, M., Ghasemalizadeh, O., Sun, Y., Kuo, C.-H., Sen, A., & Choi, C. (2025) · IEEE-RAS International Conference on Humanoid Robots (Humanoids) · doi:10.1109/humanoids65713.2025.11203093 · Cited by 3 · see citing works
Vision-based intelligent robot grasping using sparse neural network
Kushwaha, V., Shukla, P., & Nandi, G. C. (2025) · International Journal of Intelligent Robotics and Applications · doi:10.1007/s41315-025-00428-4 · Cited by 3 · see citing works
Command-driven semantic robotic grasping towards user-specified tasks
Lyu, Q., Ye, Q., Chen, X., & Zhang, Q. (2025) · Complex & Intelligent Systems · doi:10.1007/s40747-025-01981-y · Cited by 3 · see citing works
Robotic-CLIP: Fine-Tuning CLIP on Action Data for Robotic Applications
Nguyen, N. T., Vu, M. N., Ta, T. D., Huang, B., Vo, T., Le, N., & Nguyen, A. (2025) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra55743.2025.11127829 · Cited by 2 · see citing works
Self-citationJin, Y., Gao, Q., Zhu, X., Yue, Y., Lim, E. G., Wong, P. W. H., Chen, Y., & Chu, Y. (2026) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2026.3655915 · Cited by 2 · see citing works
Unified Sim-and-Real Scoring Methods for Robot Policy Selection
Im, S., & Lee, J. (2025) · The Journal of Korea Robotics Society · doi:10.7746/jkros.2025.20.3.371 · Cited by 2 · see citing works
Zafar, M. H., Moosavi, S. K. R., & Sanfilippo, F. (2025) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2025.3598695 · Cited by 1 · see citing works
Sparse-View 3-D Language Gaussian Splatting for Zero-Shot Robotic Grasping
Zhang, Y., Wang, Y., Chen, W., Xie, H., Xu, Z., Che, Z., Ren, P., Tang, J., & Sun, J. (2026) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2025.3645486 · Cited by 1 · see citing works
Oracle-grasp: zero-shot affordance-aligned robotic grasping using large multimodal models
Giuili, A., Atari, R., & Sintov, A. (2026) · Intelligent Service Robotics · doi:10.1007/s11370-026-00707-4 · Cited by 1 · see citing works
Shengjun, 徐. X., Zhiwei, 崔. C., Ya, 史. S., Xiaohan, 李. L., Erhu, 刘. L., & Abdelhamid, H. H. (2025) · Laser & Optoelectronics Progress · doi:10.3788/lop241866
Zhan, Z., Li, H., Wu, E., & Zhao, C. (2025) · Lecture notes in computer science · doi:10.1007/978-981-96-9921-6_5
Text-guided RGB-P grasp generation
Vu, V. D., Nguyen, V., Pham, N. H., Hoang, D.-C., & Tan, P. X. (2025) · PeerJ Computer Science · doi:10.7717/peerj-cs.3060
Online Trajectory Replanner for Dynamically Grasping Irregular Objects
Vu, M. N., Grander, F., Nguyen, A., & Unger, C. (2025) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra55743.2025.11128477
Self-citationImage-based 3D Grasp Pose Estimation for Efficient Targeted Grasping
Stephan, B., Fischedick, S. B., Müller, S., & Groß, H. (2025) · European Conference on Mobile Robots · doi:10.1109/ecmr65884.2025.11163001
GraspMAS: Zero-Shot Language-driven Grasp Detection with Multi-Agent System
Nguyen, Q., Le, T., Nguyen, H. A., Vo, T., Ta, T. D., Huang, B., Vu, M. N., & Nguyen, A. (2025) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros60139.2025.11246582
Self-citationIm, S., & Lee, J. (2026) · International Journal of Precision Engineering and Manufacturing · doi:10.1007/s12541-026-01499-4
Parameter-efficient fine-tuning for language-driven robotic grasping tasks
Xu, J., Zhang, T., & Xu, H. (2026) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2026.105588
DGSeg: Dynamic Gating of Semantic–Spatial Guided Predictions for Reasoning Segmentation
Zeng, R., Cao, S., Zhang, L., & Liu, Z. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37038-9_13
Event-Triggered Closed-Loop Semantic Control for Zero-Shot Multi-Step Robotic Manipulation
Bai, Y., Yi, P., Zhao, L., Fang, X., Li, H., & Wang, L. (2026) · International Conference on Electrical, Control and Information Technology (ECITech) · doi:10.1109/ecitech69277.2026.11601313
Zhang, S., Li, Z., Wang, Y., & Wang, S. (2026) · Sensors · doi:10.3390/s26165257
OPAL-Grasp: Object-Part Guided Alignment for Language-Driven Robotic Grasp Detection
Wang, Q., Wang, Y., Fu, Z., Li, Y., & Bai, C. (2026) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2026.3726329
InstructGrasp: A Real-Time Language-Guided Grasp Pose Detection for Robotic Manipulation
Yang, K. O., Kang, C., Go, Y., Moon, J., & Choi, J. W. (2026) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2026.3731854
Xu, J., Xu, H., Xue, T., & Zhang, T. (2026) · IEEE Transactions on Industrial Cyber-Physical Systems · doi:10.1109/ticps.2026.3732649
Riemannian Score-Based Diffusion for Language-Conditioned Grasp and Affordance Detection
Li, Y., Fu, Z., Lu, W., Zheng, J., Xu, J., Liao, J., & Zheng, J. (2026) · doi:10.1109/cvprf71909.2026.00153
A Lightweight One-Stage Framework for Language-Guided Robotic Grasp Detection
Zhang, G., Liu, Z., Wang, Z., & Lin, Z. (2026) · International Conference on Computer Application and Computer Graphics · doi:10.1145/3844212.3844232
- Authors
- Vuong, A. D., Vu, M. N., Huang, B., Nguyen, N., Le, H., Vo, T., & Nguyen, A.
- Affiliations
- FPT Software AI Center,Vietnam
- TU Wien (AT)
- Imperial College London (GB)
- Ton Duc Thang University (VN)
- University of Liverpool (GB)
- Venue
- 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2024-06-16
- Pages
- 17902-17912
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W4402702930
- Cited by
- 39
- References
- 97
- Citations by year
- 2026: 13 · 2025: 21 · 2024: 5
- FWCI
- 26.36
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Engineering › Control and Systems Engineering › Robot Manipulation and Learning
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Keywords
- grasp detection, grasp pose estimation, diffusion models, conditional generation, contrastive training, robotic grasping, large-scale datasets
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
