Hierarchical Semantic Labeling for Task-Relevant RGB-D Perception
Open accessThe DOI matches its Crossref record; 8 of the 8 references it cites were found in a registry or an index.
Abstract
Semantic labeling of RGB-D scenes is very important in enabling robots to perform mobile manipulation tasks, but different tasks may require entirely different sets of labels.For example, when navigating to an object, we may need only a single label denoting its class, but to manipulate it, we might need to identify individual parts.In this work, we present an algorithm that produces hierarchical labelings of a scene, following is-part-of and is-type-of relationships.Our model is based on a Conditional Random Field that relates pixel-wise and pair-wise observations to labels.We encode hierarchical labeling constraints into the model while keeping inference tractable.Our model thus predicts different specificities in labeling based on its confidence-if it is not sure whether an object is Pepsi or Sprite, it will predict soda rather than making an arbitrary choice.In extensive experiments, both offline on standard datasets as well as in online robotic experiments, we show that our model outperforms other stateof-the-art methods in labeling performance as well as in success rate for robotic tasks.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
Indoor segmentation and support inference from rgbd images
Silberman, N., Hoiem, D., Kohli, P., & Fergus, R. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-33715-4_54 · Cited by 5,760
iSAM2: Incremental smoothing and mapping using the Bayes tree
Kaess, M., Johannsson, H., Roberts, R., Ila, V., Leonard, J. J., & Dellaert, F. (2012) · The International Journal of Robotics Research · doi:10.1177/0278364911430419 · Cited by 1,362
SLAM++: Simultaneous Localisation and Mapping at the Level of Objects
Salas-Moreno, R. F., Newcombe, R. A., Strasdat, H., Kelly, P. H. J., & Davison, A. J. (2013) · 2013 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2013.178 · Cited by 945
Anticipating Human Activities Using Object Affordances for Reactive Robotic Response
Koppula, H. S., & Saxena, A. (2016) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2015.2430335 · Cited by 596
Self-citationUnsupervised feature learning for 3D scene labeling
Lai, K., Bo, L., & Fox, D. (2014) · 2014 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2014.6907298 · Cited by 317 · cited 1×
“Due to the availability of affordable RGB-D sensors, significant effort has been put into RGB-D scene understanding recently [45, 37, 24, 31, 4, 25, 14, 20, 19, 17, 27].”
BackgroundDeep Learning for Detecting Robotic Grasps
Lenz, I., Lee, H., & Saxena, A. (2013) · Robotics: Science and Systems IX · doi:10.15607/rss.2013.ix.012 · Cited by 126
Self-citationMulti-robot SLAM using condensed measurements
Lazaro, M. T., Paz, L. M., Pinies, P., Castellanos, J. A., & Grisetti, G. (2013) · 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems · doi:10.1109/iros.2013.6696483 · Cited by 89 · cited 1×
“Many works focused on improving SLAM techniques to better depict an environment for planning and navigation [34, 28], such as incremental smoothing and map-”
Background
Cited by
Past, present, and future of simultaneous localization and mapping: Toward the robust-perception age
Cadena, C., Carlone, L., Carrillo, H., Latif, Y., Scaramuzza, D., Neira, J., Reid, I. D., & Leonard, J. J. (2016) · Zurich Open Repository and Archive (University of Zurich) · doi:10.5167/uzh-138892 · Cited by 3,048
Tangent Convolutions for Dense Prediction in 3D
Tatarchenko, M., Park, J., Koltun, V., & Zhou, Q.-Y. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00409 · Cited by 593
Tell Me Dave: Context-Sensitive Grounding of Natural Language to Manipulation Instructions
Kumar Misra, D., Sung, J., Lee, K., & Saxena, A. (2014) · Robotics: Science and Systems X · doi:10.15607/rss.2014.x.005 · Cited by 59
Self-citationSung, J., Jin, S. H., & Saxena, A. (2018) · Springer Proceedings in Advanced Robotics · doi:10.1007/978-3-319-60916-4_40 · Cited by 52
“There has been great advancement in scene understanding [31, 41, 26, 61], in human activity detection [51, 19], and in features for RGB-D images and point-clouds [47, 29].”
BackgroundSelf-citationA fast, modular scene understanding system using context-aware object detection
Cadena, C., Dick, A., & Reid, I. D. (2015) · 2015 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2015.7139874 · Cited by 29
Metadata
- Authors
- Wu, C., Lenz, I., & Saxena, A.
- Venue
- Robotics: Science and Systems X
- Published
- 2014-07-12
- OpenAlex
- W2295647632
- Cited by
- 67
- References
- 8
- FWCI
- 7.03
- Type
- conference paper
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.