AffordanceNet: An End-to-End Deep Learning Approach for Object Affordance Detection
Open accessThe DOI matches its Crossref record; 31 of the 31 references it cites were found in a registry or an index.
Abstract
We propose AffordanceNet, a new deep learning approach to simultaneously detect multiple objects and their affordances from RGB images. Our AffordanceNet has two branches: an object detection branch to localize and classify the object, and an affordance detection branch to assign each pixel in the object to its most probable affordance label. The proposed framework employs three key components for effectively handling the multiclass problem in the affordance mask: a sequence of deconvolutional layers, a robust resizing strategy, and a multi-task loss function. The experimental results on the public datasets show that our AffordanceNet outperforms recent state-of-the-art methods by a fair margin, while its end-to-end architecture allows the inference at the speed of 150ms per image. This makes our AffordanceNet well suitable for real-time robotic applications. Furthermore, we demonstrate the effectiveness of AffordanceNet in different testing environments and in real robotic applications. The source code is available at https://github.com/nqanh/affordance-net.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2577031 · Cited by 73,472
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380
Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation
Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.81 · Cited by 32,684
The Ecological Approach to Visual Perception
Bornstein, M. H., & Gibson, J. J. (1980) · The Journal of Aesthetics and Art Criticism · doi:10.2307/429816 · Cited by 22,935
Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., & Yuille, A. L. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2699184 · Cited by 22,392
Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., & Yuille, A. L. (2016) · arXiv · doi:10.48550/arxiv.1606.00915 · Cited by 22,392
Caffe: Convolutional Architecture for Fast Feature Embedding
Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., & Darrell, T. (2014) · Proceedings of the 22nd ACM international conference on Multimedia · doi:10.1145/2647868.2654889 · Cited by 11,077
Caffe: Convolutional Architecture for Fast Feature Embedding
Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., & Darrell, T. J. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1408.5093 · Cited by 3,752
RefineNet: Multi-path Refinement Networks for High-Resolution Semantic Segmentation
Lin, G., Milan, A., Shen, C., & Reid, I. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.549 · Cited by 3,287
Self-citation
Research works printed but not linked: 2
- he, 2017, Mask R-CNN, ICCV
- girshick, 2015, Fast R-CNN, ICCV
Cited by
Items 301–327 of 327
Affordance Region Prediction Network Based on Semantic and Spatial Constraints
Xin, J., Jiang, Y., Zhang, Z., Chen, Z., Pan, Z., & Zhao, S. (2025) · IEEE International Conference on Sensors, Electronics and Computer Engineering (ICSECE) · doi:10.1109/icsece65727.2025.11257140
Semantics in Robotics: Environmental Data Can’t Yield Conventions of Human Behaviour
Freestone, J. (2025) · ACM Transactions on Human-Robot Interaction · doi:10.1145/3779300
ShapeAfford: Reconstructing 3D Shape With Manipulation Affordance via Geometry-Affordance Synergy
Zhao, G., Yang, S., Fan, J., Song, R., Jiang, Q., Han, L., & Zhang, W. (2025) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2025.3641144
From Words to Actions: Instance-Level Grasping via Visual Labeling and Language Models
Sun, H., Zhang, Y., Wang, P., & Hashimoto, K. (2025) · IEEE International Conference on Cyborg and Bionic Systems (CBS) · doi:10.1109/cbs65871.2025.11267599
Object Affordance Detection for Mobile Manipulation in Retail Environments
van Houtum, P. (. (2021) · Research Repository (Delft University of Technology) · Cited by 1
IntentFuse: Language-Guided 3D Scene Understanding via Prompt Filtering and Fusion
Ravendran, A., Perera, M., Xu, F., Petersson, L., Wang, D., & Li, X. (2025) · International Conference on Digital Image Computing: Techniques and Applications (DICTA) · doi:10.1109/dicta68720.2025.11302488
Integrating Affordances and Attention Models for Short-Term Object Interaction Anticipation
Mur-Labadia, L., Martínez-Cantín, R., Guerrero, J. J., Farinella, G. M., & Furnari, A. (2026) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2026.3652831 · Cited by 1
Affordance-driven symbol network construction via large language models
Arii, K., Liu, S., & Kurihara, S. (2026) · Artificial Life and Robotics · doi:10.1007/s10015-025-01106-1
On the role of Actions and Machine Learning in Artificial Agent Perception.
Caselles-Dupré, H. (2021) · theses.fr (ABES) · doi:10.70675/6cfe7f36z5adcz4a30za086z8f2bdc924098 · Cited by 1
Learning to Grasp in Robotics with Quality-Diversity Generated Data
Huber, J. B. (2024) · theses.fr (ABES) · doi:10.70675/b98f392dz4df9z4646z8b34z59f447564543
Weakly-Supervised Learning of Dense Functional Correspondences
Stojanov, S., Zhao, L., Zhang, Y., Yamins, D. L. K., & Wu, J. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.00656 · Cited by 2
Three-Dimensional Affordance Segmentation for Object Point Cloud Driven by Language Instructions
Du, J., Wu, H., Ma, Q., Tian, G., Zhao, Z., & Leng, S. (2026) · ENGINEERING Information Technology & Electronic Engineering · doi:10.1631/eng.itee.2026.0044
Can AI Systems Imagine? A Conceptual Engineering Perspective
Gan, N. (2026) · Synthese Library/Synthese library · doi:10.1007/978-3-032-10073-3_10 · Cited by 1
Synthetic Data Generation for Deep Learning-Based Subpart Detection in Robotics
Caldana, D., Castro, D. J., Coimbra, R., Filipe, V. M. D. J., Silva, M. F., & de Souza, J. P. C. (2026) · IEEE International Conference on Autonomous Robot Systems and Competitions · doi:10.1109/icarsc70216.2026.11523323
What Objects Enable, Not What They Are: Functional Latent Spaces for Affordance ReasoningPreprint
Siva, R., Bhatt, N. P., Yang, Y., Lee, S., Gadde, N., Ellis, C., Velasquez, A., Wang, Z., & Topcu, U. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2606.05533
A Robust Pose Estimation and Persistent Tracking Method for Safe Autonomous Robotic Manipulation
Li, M., Zhang, H., Xu, S., Chen, Y., Zhou, J., & Wang, L. (2026) · International Symposium on Intelligent Robotics and Systems (ISoIRS) · doi:10.1109/isoirs70157.2026.11545254
HAT-4D: Lifting Monocular Video for 4D Multi-object Interactions via Human-Agent Collaboration
Li, J., Wu, Y., Zhang, Z., Shi, X., Wang, H., Zhao, Y., Linxiang, S., Yu, C. S., Zhang, M., Ding, Y., et al. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37432-5_10
Kim, K.-H., Lee, S.-S., Kim, Y., Lee, M., Lee, E., & Jang, S. (2026) · IEEE Transactions on Very Large Scale Integration (VLSI) Systems · doi:10.1109/tvlsi.2026.3713578
A Review of Language-Guided Dexterous Grasping with Human-in-the-Loop Adaptation
Jeong, E., & Lim, S. (2026) · Mathematics · doi:10.3390/math14152709
Reproducible Multimodal Affordance PredictionPreprint
Apicella, T., Xompero, A., & Cavallaro, A. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2608.18317
Chiu, Y., Tsai, C., & Nisar, H. (2026) · Journal of King Saud University - Computer and Information Sciences · doi:10.1007/s44443-026-01148-w
Visual category-guided one-shot open affordance grounding
Wang, Y., Yu, H., & Li, X. (2026) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra57385.2026.11695897
ReaAct: Bridging Robotic Reasoning and Action Generation Toward Real-World Spatial Generalization
Yu, Y., Ding, Y., Tang, P., Yang, H., Qi, X., Wang, J., & Wang, X. (2026) · doi:10.1109/cvprf71909.2026.00143
UniffordInteractor: Reasoning the Unified Affordance for Human and Multi-Object Interaction
Su, C., Qi, X., 胡旭睿, Yang, B., Sun, M., Zhang, M., & Sun, Z. (2026) · ACM Transactions on Graphics · doi:10.1145/3842530
Yoshida, T., Kurita, S., Nishimura, T., & Mori, S. (2026) · Transactions of the Japanese Society for Artificial Intelligence · doi:10.1527/tjsai.41-6_e-q11
DeGSA: Semantic-Geometry Disentanglement for Robust 3D Affordance Grounding
Li, C., & Yang, X. (2026) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3767308.3835248
Metadata
- Authors
- Do, T.-T., Nguyen, A., & Reid, I.
- Affiliations
- Australian Centre for Robotic Vision (AU)
- Venue
- 2018 IEEE International Conference on Robotics and Automation (ICRA)
- Publisher
- IEEE
- Published
- 2018-05
- Pages
- 1-5
- ISSN
- 1050-4729 (ISSN-L)
- OpenAlex
- W2962984928
- Cited by
- 305
- References
- 31
- Citations by year
- 2026: 17 · 2025: 44 · 2024: 30 · 2023: 44 · 2022: 40 · 2021: 44 · 2020: 42 · 2019: 33 · 2018: 11
- FWCI
- 48.79
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- Proceedings - IEEE International Conference on Robotics and Automation/Proceedings (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- Adelaide Research & Scholarship (AR&S) (University of Adelaide) (submittedVersion)
- arXiv (Cornell University)
- Funders
- European Commission (611832, FP7 ICT, FP7 ICT 2013–10)
- Australian Centre for Robotic Vision (#CE140100016)
- FP7 Information and Communication Technologies (611832, FP7-ICT-2013-10)
- Topics
- Physical Sciences › Engineering › Control and Systems Engineering › Robot Manipulation and Learning
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- object detection, multi-task learning, multi-class segmentation, robotic manipulation, real-time inference, RGB images
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.