A Dataset for Improved RGBD-Based Object Detection and Pose Estimation for Warehouse Pick-and-Place
Open accessThe DOI matches its Crossref record; 32 of the 33 references it cites were found in a registry or an index.
Abstract
An important logistics application of robotics involves manipulators that pick-and-place objects placed in warehouse shelves. A critical aspect of this task corresponds to detecting the pose of a known object in the shelf using visual data. Solving this problem can be assisted by the use of an RGBD sensor, which also provides depth information beyond visual data. Nevertheless, it remains a challenging problem since multiple issues need to be addressed, such as low illumination inside shelves, clutter, texture-less and reflective objects as well as the limitations of depth sensors. This letter provides a new rich dataset for advancing the state-of-the-art in RGBD-based 3D object pose estimation, which is focused on the challenges that arise when solving warehouse pick-and-place tasks. The publicly available dataset includes thousands of images and corresponding ground truth data for the objects used during the first Amazon Picking Challenge at different poses and clutter conditions. Each image is accompanied with ground truth information to assist in the evaluation of algorithms for object detection. To show the utility of the dataset, a recent algorithm for RGBD-based pose estimation is evaluated in this letter. Given the measured performance of the algorithm on the dataset, this letter shows how it is possible to devise modifications and improvements to increase the accuracy of pose estimation algorithms. This process can be easily applied to a variety of different methodologies for object pose detection and improve performance in the domain of warehouse pick-and-place.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380
The Pascal Visual Object Classes Challenge: A Retrospective
Everingham, M., Eslami, S. M. A., Van Gool, L., Williams, C. K. I., Winn, J., & Zisserman, A. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-014-0733-5 · Cited by 7,139 · cited 1×
“Some standard RGB benchmarks for the task include CIFAR-10/100 [6], ImageNet [7], and PASCAL VOC [8].”
MethodIndoor segmentation and support inference from rgbd images
Silberman, N., Hoiem, D., Kohli, P., & Fergus, R. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-33715-4_54 · Cited by 5,760
A large-scale hierarchical multi-view RGB-D object dataset
Lai, K., Bo, L., Ren, X., & Fox, D. (2011) · 2011 IEEE International Conference on Robotics and Automation · doi:10.1109/icra.2011.5980382 · Cited by 1,316
Hinterstoisser, S., Lepetit, V., Ilic, S., Holzer, S., Bradski, G., Konolige, K., & Navab, N. (2013) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-37331-2_42 · Cited by 1,161
Model globally, match locally: Efficient and robust 3D object recognition
Drost, B., Ulrich, M., Navab, N., & Ilic, S. (2010) · 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2010.5540108 · Cited by 981
Gradient Response Maps for Real-Time Detection of Textureless Objects
Hinterstoisser, S., Cagniart, C., Ilic, S., Sturm, P., Navab, N., Fua, P., & Lepetit, V. (2012) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2011.206 · Cited by 610
CAD-model recognition and 6DOF pose estimation using 3D cues
Aldoma, A., Vincze, M., Blodow, N., Gossow, D., Gedikli, S., Rusu, R. B., & Bradski, G. (2011) · 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops) · doi:10.1109/iccvw.2011.6130296 · Cited by 294
Object recognition and full pose registration from a single image for robotic manipulation
Collet, A., Berenson, D., Srinivasa, S. S., & Ferguson, D. (2009) · 2009 IEEE International Conference on Robotics and Automation · doi:10.1109/robot.2009.5152739 · Cited by 275
A Category-Level 3D Object Dataset: Putting the Kinect to Work
Janoch, A., Karayev, S., Jia, Y., Barron, J. T., Fritz, M., Saenko, K., & Darrell, T. (2013) · Advances in Computer Vision and Pattern Recognition · doi:10.1007/978-1-4471-4640-7_8 · Cited by 254
Benchmarking in Manipulation Research: The YCB Object and Model Set and Benchmarking Protocols
Çallı, B., Walsman, A., Singh, A., Srinivasa, S. S., Abbeel, P., & Dollar, A. M. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1502.03143 · Cited by 203
Voting-based pose estimation for robotic assembly using a 3D sensor
Choi, C., Taguchi, Y., Tuzel, O., Liu, M.-Y., & Ramalingam, S. (2012) · 2012 IEEE International Conference on Robotics and Automation · doi:10.1109/icra.2012.6225371 · Cited by 180
Depth-Encoded Hough Voting for Joint Object Detection and Shape Recovery
Sun, M., Bradski, G. R., Xu, B., & Savarese, S. (2010) · Lecture notes in computer science · doi:10.1007/978-3-642-15555-0_48 · Cited by 156
(2009) · doi:10.1075/is.10.3.06wis · Cited by 124
Dealing with Difficult Instances of Object Rearrangement
Krontiris, A., & Bekris, K. (2015) · Robotics: Science and Systems XI · doi:10.15607/rss.2015.xi.045 · Cited by 112
Self-citationRecognition and Pose Estimation of Rigid Transparent Objects with a Kinect Sensor
Lysenkov, I., Eruhimov, V., & Bradski, G. R. (2012) · Robotics: Science and Systems · doi:10.15607/rss.2012.viii.035 · Cited by 78
Localizing Handle-Like Grasp Affordances in 3D Point Clouds
ten Pas, A., & Platt, R. (2015) · Springer tracts in advanced robotics · doi:10.1007/978-3-319-23778-7_41 · Cited by 62
Using Geometry to Detect Grasps in 3D Point CloudsPreprint
ten Pas, A., & Platt, R. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1501.03100 · Cited by 60
Lessons from the Amazon Picking Challenge.Preprint
Correll, N., Bekris, K. E., Berenson, D., Brock, O., Causo, A., Hauser, K. K., Okada, K., Rodríguez, A., Romano, J. M., & Wurman, P. R. (2016) · arXiv (Cornell University) · Cited by 58
Self-citation3D object recognition in range images using visibility context
Kim, E., & Medioni, G. G. (2011) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2011.6094527 · Cited by 54
RoboCup@Home: Demonstrating Everyday Manipulation Skills in RoboCup@Home
Stückler, J., Holz, D., & Behnke, S. (2012) · IEEE Robotics & Automation Magazine · doi:10.1109/mra.2012.2191993 · Cited by 53
Segmentation and Unsupervised Part-based Discovery of Repetitive Objects
Triebel, R., Shin, J., & Siegwart, R. (2010) · Repository for Publications and Research Data (ETH Zurich) · doi:10.3929/ethz-a-010027929 · Cited by 52
A new distance for scale-invariant 3D shape recognition and registration
Pham, M.-T., Woodford, O. J., Perbet, F., Maki, A., Stenger, B., & Cipolla, R. (2011) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2011.6126236 · Cited by 47
Cloud Automation: Precomputing Roadmaps for Flexible Manipulation
Bekris, K. E., Shome, R., Krontiris, A., & Dobson, A. (2015) · IEEE Robotics & Automation Magazine · doi:10.1109/mra.2015.2401291 · Cited by 36
Self-citationVaškevičius, N., Pathak, K., Ichim, A.-E., & Birk, A. (2012) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2012.6225335 · Cited by 23
Unsupervised Dense Object Discovery, Detection, Tracking and Reconstruction
Ma, L., & Sibley, G. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-10605-2_6 · Cited by 22
Simultaneous localization, mapping, and manipulation for unsupervised object discovery
Ma, L., Ghafarianzadeh, M., Coleman, D., Correll, N., & Sibley, G. (2015) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2015.7139365 · Cited by 20
Range sensor and silhouette fusion for high-quality 3D Scanning
Narayan, K. S., Sha, J., Singh, A., & Abbeel, P. (2015) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2015.7139701 · Cited by 17
Scene Interpretation for Self-Aware Cognitive Robots
Ozturk, M. D., Ersen, M., Kapotoglu, M., Koc, C., Sarıel, S., & Yalçın, H. (2014) · Cited by 15
3D object recognition in range images using visibility context
Kim, E. Y., & Medioni, G. G. (2011) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2011.6048203 · Cited by 14
(2009) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings
(2000) · Medical Entomology and Zoology
Research works printed but not linked: 4
- choi, 0, Proc IEEE/RSJ Int Conf Intell Robots Syst (IROS), 3342
- lai, 0, Proc AAAI Conf Artif Intell, 1474
- muja, 0, Proc IEEE Int Conf Robot Autom (ICRA), 2939
- steder, 0, Proc (IEEE/RSJ IROS) Workshop Defin Solv Realistic Percept Prob Pers Robot
Cited by
PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenes
Xiang, Y., Schmidt, T., Narayanan, V., & Fox, D. (2018) · Robotics: Science and Systems XIV · doi:10.15607/rss.2018.xiv.019 · Cited by 2,139
Du, G., Wang, K., Lian, S., & Zhao, K. (2021) · Artificial Intelligence Review · doi:10.1007/s10462-020-09888-5 · Cited by 475
BOP: Benchmark for 6D Object Pose Estimation
Hodaň, T., Michel, F., Brachmann, E., Kehl, W., Buch, A. G., Kraft, D., Drost, B., Vidal, J., Ihrke, S., Zabulis, X., et al. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01249-6_2 · Cited by 470
Multi-view self-supervised deep learning for 6D pose estimation in the Amazon Picking Challenge
Zeng, A., Yu, K.-T., Song, S., Suo, D., Walker, E., Rodriguez, A., & Xiao, J. (2017) · 2017 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2017.7989165 · Cited by 457
Analysis and Observations From the First Amazon Picking Challenge
Correll, N., Bekris, K. E., Berenson, D., Brock, O., Causo, A., Hauser, K., Okada, K., Rodriguez, A., Romano, J. M., & Wurman, P. R. (2018) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2016.2600527 · Cited by 425
“The remaining nine answers each report distinct combinations of color and 3D information including [17], [18].”
BackgroundSelf-citation
Metadata
- Authors
- Rennie, C., Shome, R., Bekris, K. E., & De Souza, A. F.
- Affiliations
- Rutgers, The State University of New Jersey (US)
- Venue
- IEEE Robotics and Automation Letters
- Published
- 2016-07
- Volume
- 1
- Issue
- 2
- Pages
- 1179-1185
- ISSN
- 2377-3766 (ISSN-L)
- 2377-3774
- OpenAlex
- W2221752211
- Cited by
- 182
- References
- 33
- Citations by year
- 2026: 6 · 2025: 15 · 2024: 20 · 2023: 17 · 2022: 17 · 2021: 24 · 2020: 14 · 2019: 24 · 2018: 17 · 2017: 12 · 2016: 16
- FWCI
- 19.32
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE Robotics and Automation Letters (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- Rutgers University Community Repository (Rutgers University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Engineering › Control and Systems Engineering › Robot Manipulation and Learning
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Computer Graphics and Computer-Aided Design › 3D Shape Modeling and Analysis
- Keywords
- RGB-D, 3D object pose estimation, object detection, depth sensing, cluttered scenes, low-light conditions, textureless objects, robotic manipulation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.