A Dataset for Improved RGBD-Based Object Detection and Pose Estimation for Warehouse Pick-and-Place
Open accessThe DOI matches its Crossref record; 32 of the 33 references it cites were found in a registry or an index.
Abstract
An important logistics application of robotics involves manipulators that pick-and-place objects placed in warehouse shelves. A critical aspect of this task corresponds to detecting the pose of a known object in the shelf using visual data. Solving this problem can be assisted by the use of an RGBD sensor, which also provides depth information beyond visual data. Nevertheless, it remains a challenging problem since multiple issues need to be addressed, such as low illumination inside shelves, clutter, texture-less and reflective objects as well as the limitations of depth sensors. This letter provides a new rich dataset for advancing the state-of-the-art in RGBD-based 3D object pose estimation, which is focused on the challenges that arise when solving warehouse pick-and-place tasks. The publicly available dataset includes thousands of images and corresponding ground truth data for the objects used during the first Amazon Picking Challenge at different poses and clutter conditions. Each image is accompanied with ground truth information to assist in the evaluation of algorithms for object detection. To show the utility of the dataset, a recent algorithm for RGBD-based pose estimation is evaluated in this letter. Given the measured performance of the algorithm on the dataset, this letter shows how it is possible to devise modifications and improvements to increase the accuracy of pose estimation algorithms. This process can be easily applied to a variety of different methodologies for object pose detection and improve performance in the domain of warehouse pick-and-place.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380 · cited 1×
“Some standard RGB benchmarks for the task include CIFAR-10/100 [6], ImageNet [7], and PASCAL VOC [8].”
MethodThe Pascal Visual Object Classes Challenge: A Retrospective
Everingham, M., Eslami, S. M. A., Van Gool, L., Williams, C. K. I., Winn, J., & Zisserman, A. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-014-0733-5 · Cited by 7,139 · cited 1×
“Some standard RGB benchmarks for the task include CIFAR-10/100 [6], ImageNet [7], and PASCAL VOC [8].”
MethodIndoor segmentation and support inference from rgbd images
Silberman, N., Hoiem, D., Kohli, P., & Fergus, R. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-33715-4_54 · Cited by 5,760 · cited 2×
“NYU Depth Dataset v2 [10]: The NYU dataset also focuses on indoor scenes, but ground truth labels are presented as full image segmentation.”
BackgroundResultA large-scale hierarchical multi-view RGB-D object dataset
Lai, K., Bo, L., Ren, X., & Fox, D. (2011) · 2011 IEEE International Conference on Robotics and Automation · doi:10.1109/icra.2011.5980382 · Cited by 1,316 · cited 7×
“Table-Top Object Dataset [12]: A collaboration between Willow Garage and the Univ. of Michigan, this dataset consists of ∼ 1,000 2.5D images with ground truth labels for 480 frames.”
ResultInfluentialHinterstoisser, S., Lepetit, V., Ilic, S., Holzer, S., Bradski, G., Konolige, K., & Navab, N. (2013) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-37331-2_42 · Cited by 1,161 · cited 8×
“Table-Top Object Dataset [12]: A collaboration between Willow Garage and the Univ. of Michigan, this dataset consists of ∼ 1,000 2.5D images with ground truth labels for 480 frames.”
BackgroundMethodInfluentialModel globally, match locally: Efficient and robust 3D object recognition
Drost, B., Ulrich, M., Navab, N., & Ilic, S. (2010) · 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2010.5540108 · Cited by 981 · cited 1×
“of different algorithms for this problem, given the rich literature on the subject [17], [18], [19], [20], [21], [22], [23], [24], [25], [26], [27], [28], [29].”
BackgroundGradient Response Maps for Real-Time Detection of Textureless Objects
Hinterstoisser, S., Cagniart, C., Ilic, S., Sturm, P., Navab, N., Fua, P., & Lepetit, V. (2012) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2011.206 · Cited by 610
CAD-model recognition and 6DOF pose estimation using 3D cues
Aldoma, A., Vincze, M., Blodow, N., Gossow, D., Gedikli, S., Rusu, R. B., & Bradski, G. (2011) · 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops) · doi:10.1109/iccvw.2011.6130296 · Cited by 294 · cited 1×
“of different algorithms for this problem, given the rich literature on the subject [17], [18], [19], [20], [21], [22], [23], [24], [25], [26], [27], [28], [29].”
BackgroundObject recognition and full pose registration from a single image for robotic manipulation
Collet, A., Berenson, D., Srinivasa, S. S., & Ferguson, D. (2009) · 2009 IEEE International Conference on Robotics and Automation · doi:10.1109/robot.2009.5152739 · Cited by 275 · cited 1×
“of different algorithms for this problem, given the rich literature on the subject [17], [18], [19], [20], [21], [22], [23], [24], [25], [26], [27], [28], [29].”
BackgroundA Category-Level 3D Object Dataset: Putting the Kinect to Work
Janoch, A., Karayev, S., Jia, Y., Barron, J. T., Fritz, M., Saenko, K., & Darrell, T. (2013) · Advances in Computer Vision and Pattern Recognition · doi:10.1007/978-1-4471-4640-7_8 · Cited by 254 · cited 3×
“Segmented Scenes Datasets B3DO [9]: A project by UC Berkeley, the dataset contains >3,000 2.”
MethodResult
Research works printed but not linked: 4
- choi, 0, Proc IEEE/RSJ Int Conf Intell Robots Syst (IROS), 3342
- lai, 0, Proc AAAI Conf Artif Intell, 1474
- muja, 0, Proc IEEE Int Conf Robot Autom (ICRA), 2939
- steder, 0, Proc (IEEE/RSJ IROS) Workshop Defin Solv Realistic Percept Prob Pers Robot
Cited by
PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenes
Xiang, Y., Schmidt, T., Narayanan, V., & Fox, D. (2018) · Robotics: Science and Systems XIV · doi:10.15607/rss.2018.xiv.019 · Cited by 2,139
Du, G., Wang, K., Lian, S., & Zhao, K. (2021) · Artificial Intelligence Review · doi:10.1007/s10462-020-09888-5 · Cited by 475
BOP: Benchmark for 6D Object Pose Estimation
Hodaň, T., Michel, F., Brachmann, E., Kehl, W., Buch, A. G., Kraft, D., Drost, B., Vidal, J., Ihrke, S., Zabulis, X., et al. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01249-6_2 · Cited by 470
Multi-view self-supervised deep learning for 6D pose estimation in the Amazon Picking Challenge
Zeng, A., Yu, K.-T., Song, S., Suo, D., Walker, E., Rodriguez, A., & Xiao, J. (2017) · 2017 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2017.7989165 · Cited by 457
“Previous efforts to construct benchmark datasets include Berkeley’s dataset [15] with a number of objects from and beyond APC 2015 and Rutgers’s dataset [16] with semi-automatically labeled data.”
BackgroundAnalysis and Observations From the First Amazon Picking Challenge
Correll, N., Bekris, K. E., Berenson, D., Brock, O., Causo, A., Hauser, K., Okada, K., Rodriguez, A., Romano, J. M., & Wurman, P. R. (2018) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2016.2600527 · Cited by 425
“The remaining nine answers each report distinct combinations of color and 3D information including [17], [18].”
BackgroundSelf-citationShape completion enabled robotic grasping
Varley, J., DeChant, C., Richardson, A., Ruales, J., & Allen, P. (2017) · 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2017.8206060 · Cited by 290
“A alternative to general shape completion in the robotics community is object recognition and 3D pose detection [4][5][6].”
BackgroundBOP Challenge 2020 on 6D Object Localization
Hodaň, T., Sundermeyer, M., Drost, B., Labbé, Y., Brachmann, E., Michel, F., Rother, C., & Matas, J. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-66096-3_39 · Cited by 268
Falling Things: A Synthetic Dataset for 3D Object Detection and Pose Estimation
Tremblay, J., To, T., & Birchfield, S. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2018.00275 · Cited by 189
Georgiou, T., Liu, Y., Chen, W., & Lew, M. (2020) · International Journal of Multimedia Information Retrieval · doi:10.1007/s13735-019-00183-w · Cited by 187
HomebrewedDB: RGB-D Dataset for 6D Pose Estimation of 3D Objects
Kaskman, R., Zakharov, S., Shugurov, I., & Ilic, S. (2019) · 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW) · doi:10.1109/iccvw.2019.00338 · Cited by 163
“Additional datasets, which are suited for robotic tasks are Rutgers dataset [25], Amazon Picking Challange [7], BIG-Bird [26] etc. Datasets introducing light change challenge are TUD-Light and Toyota Light, both used and referenced in the benchmark paper of Hodan et al .”
Critical Design and Control Issues of Indoor Autonomous Mobile Robots: A Review
Niloy, M. A., Shama, A., Chakrabortty, R. K., Ryan, M. J., Badal, F. R., Tasneem, Z., Ahamed, M. H., Moyeen, S. I., Das, S. K., Ali, M. F., et al. (2021) · IEEE Access · doi:10.1109/access.2021.3062557 · Cited by 192
“…Recognition of gesture, voice and some other activity and semantic place classifications Detection of environmental changes Classification of terrain [57], All the examples are achieved with the help of some spe-cific and highly precise sensors such as various vision sensors and proximity sensors [9].”
RGBD Datasets: Past, Present and Future
Firman, M. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2016.88 · Cited by 147
“One solution has been to fix the target objects to a calibration board to allow for ground-truth tracking using the RGB channels [45], while [87] and [85] have the poses manually aligned.”
MethodPhotorealistic Image Synthesis for Object Instance Detection
Hodaň, T., Vineet, V., Gal, R., Shalev, E., Hanzelka, J., Connell, T., Urbina, P., Sinha, S. N., & Guenter, B. K. (2019) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2019.8803821 · Cited by 186
“75IoU on real test images from Rutgers APC [2], and 11% on real test images from LineModOccluded [3, 4].”
MethodInfluentialObjectron: A Large Scale Dataset of Object-Centric Videos in the Wild with Pose Annotations
Ahmadyan, A., Zhang, L., Ablavatski, A., Wei, J., & Grundmann, M. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00773 · Cited by 261
Lessons from the Amazon Picking Challenge: Four Aspects of Building Robotic Systems
Eppner, C., Höfer, S., Jonschkowski, R., Martín-Martín, R., Sieverling, A., Wall, V., & Brock, O. (2016) · Robotics: Science and Systems XII · doi:10.15607/rss.2016.xii.036 · Cited by 124
Pauly, L., Rharbaoui, W., Shneider, C., Rathinam, A., Gaudillière, V., & Aouada, D. (2023) · Acta Astronautica · doi:10.1016/j.actaastro.2023.08.001 · Cited by 121
“Though DL algorithms are evolving towards few-shot [152] and zero-shot [18] learning, solving 6 DoF pose prediction problems with high accuracy still depends on large datasets with images spanning a wide range of scenarios [134, 179].”
BackgroundLabel Fusion: A Pipeline for Generating Ground Truth Labels for Real RGBD Data of Cluttered Scenes
Marion, P., Florence, P., Manuelli, L., & Tedrake, R. (2018) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2018.8460950 · Cited by 126
“As compared to existing methods such as [8], [27], [1] our method requires no sophisticated calibration, works for arbitrary rigid objects in general environments, and requires”
MethodA review on object pose recovery: From 3D bounding box detectors to full 6D pose estimators
Şahin, C., Garcia-Hernando, G., Sock, J., & Kim, T. (2020) · Image and Vision Computing · doi:10.1016/j.imavis.2020.103898 · Cited by 135
6D pose estimation using an improved method based on point pair features
Vidal, J., Lin, C., & Martí, R. (2018) · International Conference on Control, Automation and Robotics (ICCAR) · doi:10.1109/iccar.2018.8384709 · Cited by 122
Tyree, S., Tremblay, J., To, T., Cheng, J., Mosier, T., Smith, J., & Birchfield, S. T. (2022) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros47612.2022.9981838 · Cited by 172
“It is composed of the following datasets: LineMOD [34], LineMOD-Occluded [30], T-LESS [20], ITODD [35], HomebrewedDB [36], YCB-Video [22], Rutgers APC [16], IC-BIN [37], IC-MI [38], TUD Light [14], Toyota Light [14], and now HOPE.”
BackgroundMethodMitash, C., Bekris, K. E., & Boularias, A. (2017) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2017.8202206 · Cited by 120
“Several RGB-D datasets have been released in the setting of the Amazon Picking Challenge [3], [4], [5].”
MethodSelf-citationBOP Challenge 2022 on Detection, Segmentation and Pose Estimation of Specific Rigid Objects
Sundermeyer, M., Hodaň, T., Labbé, Y., Wang, G., Brachmann, E., Drost, B., Rother, C., & Matas, J. (2023) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw59228.2023.00279 · Cited by 138
“HB* [24] YCB-V* [54] RU-APC [40] IC-BIN* [4]”
BlenderProc: Reducing the Reality Gap with Photorealistic Rendering
Denninger, M., Sundermeyer, M., Winkelbauer, D., Olefir, D., Hodaň, T., Zidan, Y., Elbadrawy, M., Knauer, M., Katam, H., & Lodhi, A. (2020) · elib (German Aerospace Center) · Cited by 144
Tagliavini, L., Colucci, G. P., Botta, A., Cavallone, P., Baglieri, L., & Quaglia, G. (2022) · Journal of Intelligent & Robotic Systems · doi:10.1007/s10846-022-01745-7 · Cited by 105
“Some practical examples of robotic perception subareas are obstacle detection [49, 50] and (time for reconfiguration with sharp corner trajectory) Little sensitivity to floor condition recognition [51, 52], semantic place [53, 54], activity [55] and terrain classification [56], 3D environment…”
BackgroundLarge-scale 6D Object Pose Estimation Dataset for Industrial Bin-Picking
Kleeberger, K., Landgraf, C., & Huber, M. F. (2019) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros40897.2019.8967594 · Cited by 112
“marker no partly no yes/yes no partly Rutgers APC [15] RGB-D 24 10,000/- (semi) manual no no no yes/yes no no T-Less [7] RGB-D 30 48,000/77,000 semi-auto.”
BackgroundA Comprehensive Review on 3D Object Detection and 6D Pose Estimation With Deep Learning
Hoque, S., Arafat, M. Y., Xu, S., Maiti, A., & Wei, Y.-C. (2021) · IEEE Access · doi:10.1109/access.2021.3114399 · Cited by 107
MVTec D2S: Densely Segmented Supermarket Dataset
Follmann, P., Böttger, T., Härtinger, P., König, R., & Ulrich, M. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01249-6_35 · Cited by 96
The ACRV picking benchmark: A robotic shelf picking benchmark to foster reproducible research
Leitner, J., Tow, A. W., Sunderhauf, N., Dean, J. E., Durham, J. W., Cooper, M., Eich, M., Lehnert, C., Mangels, R., McCool, C., et al. (2017) · 2017 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2017.7989545 · Cited by 78
“for object segmentation [15] and 6D pose estimation [16] can be contributed to the ubiquity of low-cost depth sensors.”
BackgroundSuctionNet-1Billion: A Large-Scale Benchmark for Suction Grasping
Cao, H., Fang, H.-S., Liu, W., & Lu, C. (2021) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2021.3115406 · Cited by 103
“An online evaluation system is further provided to enable algorithm evaluation without the need to access real robots Meanwhile, our dataset contains 6D poses and thus can be used by researchers in 6D pose estimation community [13]– [15].”
MethodBOP Challenge 2023 on Detection, Segmentation and Pose Estimation of Seen and Unseen Rigid Objects
Hodaň, T., Sundermeyer, M., Labbé, Y., Nguyen, V. N., Wang, G., Brachmann, E., Drost, B., Lepetit, V., Rother, C., & Matas, J. (2024) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw63382.2024.00570 · Cited by 131
Eversberg, L., & Lambrecht, J. (2021) · Sensors · doi:10.3390/s21237901 · Cited by 95
“With high quality render settings (average render time of 720 s per image) they got an improvement of 6% over low quality settings (15 s) on the LineMod-Occluded [27,28] dataset but no improvement on the Rutgers APC [29] dataset.”
Probabilistic multi-class segmentation for the Amazon Picking Challenge
Jonschkowski, R., Eppner, C., Hofer, S., Martin-Martin, R., & Brock, O. (2016) · 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2016.7758087 · Cited by 72
“In order to create an multi-class segmentation method for the APC setting, we analyze the problem to identify how to address its challenges by leveraging useful problem structure.”
MethodA Comprehensive Study of 3-D Vision-Based Robot Manipulation
Cong, Y., Chen, R., Ma, B., Liu, H., Hou, D., & Yang, C. (2021) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2021.3108165 · Cited by 99
Krontiris, A., & Bekris, K. E. (2016) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2016.7487581 · Cited by 79
“Future efforts should also aim towards robust rearrangement trajectories under the presence of uncertainty, which can arise from pose estimation processes [35].”
BackgroundSelf-citationDeep Learning Frontiers in 3D Object Detection: A Comprehensive Review for Autonomous Driving
Pravallika, A., Hashmi, M. F., & Gupta, A. D. (2024) · IEEE Access · doi:10.1109/access.2024.3456893 · Cited by 92
DPODv2: Dense Correspondence-Based 6 DoF Pose Estimation
Shugurov, I., Zakharov, S., & Ilić, S. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3118833 · Cited by 75
“Over the past decade, numerous datasets for object detection and 6 DoF pose estimation have been proposed [5], [7], [18], [52], [76], [77], [78], [79], [80].”
BackgroundLessons from the Amazon Picking Challenge.Preprint
Correll, N., Bekris, K. E., Berenson, D., Brock, O., Causo, A., Hauser, K. K., Okada, K., Rodríguez, A., Romano, J. M., & Wurman, P. R. (2016) · arXiv (Cornell University) · Cited by 58
Self-citationRGB-D Images for Objects Recognition using 3D Point Clouds and RANSAC Plane Fitting
Jalal, A., Sarwar, M. Z., & Kim, K. (2021) · International Bhurban Conference on Applied Sciences and Technology (IBCAST) · doi:10.1109/ibcast51254.2021.9393166 · Cited by 57
“Our proposed approach to compute geometric artifacts from depth clues [42].”
MethodTowards Robust Product Packing with a Minimalistic End-Effector
Shome, R., Tang, W., Song, C., Mitash, C., Kourtev, H., Yu, J., Boularias, A., & Bekris, K. E. (2019) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2019.8793966 · Cited by 66
Self-citationIntelligent Robotic Perception Systems
Premebida, C., Ambrus, R., & Marton, Z.-C. (2019) · Applications of Mobile Robots · doi:10.5772/intechopen.79742 · Cited by 52
Du, G., Wang, K., & Lian, S. (2019) · arXiv (Cornell University) · Cited by 55
Robotic Grasping With Multi-View Image Acquisition and Model-Based Pose Estimation
Lin, H., Liang, S.-C., & Chen, Y. (2020) · IEEE Sensors Journal · doi:10.1109/jsen.2020.3030791 · Cited by 55
“In the case of 3D object segmentation and pose estimation for pick-and-place applications, the dataset [10] contains 14 objects from 2015 Amazon Picking Challenge for grabbing.”
Wang, P., Jung, H., Li, Y., Shen, S., Srikanth, R. P., Garattoni, L., Meier, S., Navab, N., & Busam, B. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.02054 · Cited by 70
The Freiburg Groceries DatasetPreprint
Jund, P., Abdo, N., Eitel, A., & Burgard, W. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.05799 · Cited by 97
“…challenge of recognizing objects in this domain due to the large variation in shape, color, and appearance of everyday products even within the the same class. understanding and place recognition [28, 34], as well as object recognition, manipulation and pose estimation for robots [5, 11, 17, 27].”
BackgroundDetection and Classification of Printed Circuit Boards Using YOLO Algorithm
Glučina, M., Anđelić, N., Lorencin, I., & Car, Z. (2023) · Electronics · doi:10.3390/electronics12030667 · Cited by 50
Challenges for Monocular 6-D Object Pose Estimation in Robotics
Thalhammer, S., Bauer, D., Hönig, P., Weibel, J.-B., García‐Rodríguez, J., & Vincze, M. (2024) · IEEE Transactions on Robotics · doi:10.1109/tro.2024.3433870 · Cited by 78
“Only few datasets assume more complex scene backgrounds [98], [211].”
BackgroundAre we done with object recognition? The iCub robot’s perspective
Pasquale, G., Ciliberto, C., Odone, F., Rosasco, L., & Natale, L. (2018) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2018.11.001 · Cited by 46
“In the literature, several datasets have been used to benchmark visual object recognition in robotics: COIL (Nene et al 1996), ALOI (Geusebroek et al 2005), Washington RGB-D (Lai et al 2011), KIT (Kasper et al 2012), SHORT-100 (Rivera-Rubio et al 2014), BigBIRD (Singh et al 2014), Rutgers Amazon Picking Challenge RGB-…”
BackgroundIndoor Synthetic Data Generation: A Systematic Review
Schieber, H., Demir, K. C., Kleinbeck, C., Yang, S. H., & Roth, D. (2024) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2023.103907 · Cited by 52
Jung, H., Wu, S., Ruhkamp, P., Zhai, G., Schieber, H., Rizzoli, G., Wang, P., Zhao, H., Garattoni, L., Roth, D., et al. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02123 · Cited by 99
“However, those datasets still rely on checkerboard-based camera localization, or human-powered annotation [47], or a rotating table [15] to provide tolerable annotations.”
BackgroundUnknown Object Segmentation from Stereo Images
Durner, M., Boerdijk, W., Sundermeyer, M., Friedl, W., Márton, Z.-C., & Triebel, R. (2021) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros51168.2021.9636281 · Cited by 46
“There exist several datasets of objects on table-top surfaces, and many provide segmentation [32], [52], bounding box [53], [54] or point cloud [55] annotations.”
BackgroundA Survey on Deep Learning Based Methods and Datasets for Monocular 3D Object Detection
Kim, S., & Hwang, Y. (2021) · Electronics · doi:10.3390/electronics10040517 · Cited by 43
BOP: Benchmark for 6D Object Pose EstimationPreprint
Hodaň, T., Michel, F., Brachmann, E., Kehl, W., Buch, A. G., Kraft, D., Drost, B., Vidal, J., Ihrke, S., Zabulis, X., et al. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1808.08319 · Cited by 105
“LM/LM-O [14,1] IC-MI [34] IC-BIN [7] T-LESS [16] RU-APC [28] TUD-L - new TYO-L - new”
InfluentialARMBench: An Object-centric Benchmark Dataset for Robotic Manipulation
Mitash, C., Wang, F., Lu, S., Terhuja, V., Garaas, T. W., Polido, F., & Nambi, M. (2023) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48891.2023.10160846 · Cited by 54
“The Amazon robotics challenge [18], [24] initiated the development of other relevant datasets [55], [41], [68].”
BackgroundFour aspects of building robotic systems: lessons from the Amazon Picking Challenge 2015
Eppner, C., Höfer, S., Jonschkowski, R., Martín-Martín, R., Sieverling, A., Wall, V., & Brock, O. (2018) · Autonomous Robots · doi:10.1007/s10514-018-9761-2 · Cited by 40
“By making suitable assumptions, our approach outperforms generic off-the-shelf methods in the specific settings of the Amazon Picking Challenge (Rennie et al. 2016; Jonschkowski et al. 2016) (Sect.”
MethodReal-Time Object Pose Estimation with Pose Interpreter NetworksPreprint
Wu, J., Zhou, B., Russell, R. L., Kee, V., Wagner, S., Hebert, M. R., Torralba, A. B., & Johnson, D. M. S. (2018) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 35
DoraPicker: An autonomous picking system for general objects
Zhang, H., Long, P., Zhou, D., Qian, Z., Wang, Z., Wan, W., Manocha, D., Park, C., Hu, T., Cao, C., et al. (2016) · IEEE International Conference on Automation Science and Engineering (CASE) · doi:10.1109/coase.2016.7743473 · Cited by 37
“As described in [12], LINEMOD is an object detection and pose estimation method, which takes a 3D mesh object model generated in advance as the input.”
BackgroundMethodImproving 6D Pose Estimation of Objects in Clutter Via Physics-Aware Monte Carlo Tree Search
Mitash, C., Boularias, A., & Bekris, K. E. (2018) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2018.8461163 · Cited by 31
Self-citationHurtado, E. G. (2019) · IntechOpen eBooks · doi:10.5772/intechopen.74181 · Cited by 38
Real-Time Object Pose Estimation with Pose Interpreter Networks
Wu, J., Zhou, B., Russell, R. L., Kee, V., Wagner, S., Hebert, M., Torralba, A. B., & Johnson, D. M. S. (2018) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2018.8593662 · Cited by 39
“annotate and were often hand annotated in the past [4], [12].”
BackgroundSemiautomatic Labeling for Deep Learning in Robotics
Gregorio, D. D., Tonioni, A., Palli, G., & Stefano, L. D. (2019) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2019.2938316 · Cited by 38
“Although a number of high-quality, large, multi-category training data sets are publicly available [3], [16], [26], they usually concern general classes (e.”
Background[POSTER] A Benchmark Dataset for 6DoF Object Pose Tracking
Wu, P.-C., Lee, Y.-Y., Tseng, H.-Y., Ho, H.-I., Yang, M., & Chien, S. (2017) · Proceedings - International Symposium on Mixed and Augmented Reality, ISMAR · doi:10.1109/ismar-adjunct.2017.62 · Cited by 36
“Rennie [16] Kinect v1 Robotic Arm Manual No 24 - 10,368”
BackgroundMethodGrasping through dynamic weaving with entangled closed loops
Kang, G., Kim, Y., Lee, S. E., Kim, S. K., Lee, D., & Song, K. (2023) · Nature Communications · doi:10.1038/s41467-023-40358-y · Cited by 34
“Two notable features exist in the woven structure: (1) the lines are collected to the center by moving through linear motion, and (2) the lines have position differences along the z-axis relative to the adjacent lines.”
BackgroundPIZZA: A Powerful Image-only Zero-Shot Zero-CAD Approach to 6 DoF Tracking
Nguyen, V. N., Du, Y., Xiao, Y., Ramamonjisoa, M., & Lepetit, V. (2022) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv57658.2022.00063 · Cited by 34
Multi-view Fusion for Multi-level Robotic Scene Understanding
Lin, Y., Tremblay, J., Tyree, S., Vela, P. A., & Birchfield, S. T. (2021) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros51168.2021.9635994 · Cited by 44
“…tremendous progress made in the computer vision community on solving problems such as 3D reconstruction [1], [2], [3], [4] and object pose estimation [5], [6], [7], [8], [9], [10], [11], [12], [13], [14], existing deployed robotic manipulators have limited, if any, perception of their surroundings.”
BackgroundHousehold Cloth Object Set: Fostering Benchmarking in Deformable Object Manipulation
Garcia-Camacho, I., Borràs, J., Çallı, B., Norton, A., & Alenyà, G. (2022) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2022.3158428 · Cited by 37
“9 GraspNet [10] 2020 88 RGB-D images Grasp evaluation No No Grasp pose annotation 3D mesh models 10 Warehouse P&P [11] 2016 25 RGB-D dataset 3D pose estimation No No 11 Rothlin [12] 2007 21 3D models Grasping No No 12 DefGraspSim [13] 2021 34 Mesh models Grasp evaluation No No 13 Yamazaki [14] 2013 21 RGB images Class…”
MethodSIDOD: A Synthetic Image Dataset for 3D Object Pose Recognition With Distractors
Jalal, M., Spjut, J., Boudaoud, B., & Betke, M. (2019) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2019.00063 · Cited by 41
“While a number of datasets to support object recognition research already exist, which provide a variety of labels [3, 4, 5, 7], there is still a need for larger, high-quality datasets.”
BackgroundClose the Optical Sensing Domain Gap by Physics-Grounded Active Stereo Sensor Simulation
Zhang, X., Chen, R., Li, A., Xiang, F., Qin, Y., Gu, J., Ling, Z., Liu, M., Zeng, P., Han, S., et al. (2023) · IEEE Transactions on Robotics · doi:10.1109/tro.2023.3235591 · Cited by 53
“pose of camera frames, and computing the object pose for each frame [73], [74], but it is undesirable to leave markers in captured images.”
BackgroundSoltan, S., Oleinikov, A., Demirci, M. F., & Shintemirov, A. (2020) · Robotics · doi:10.3390/robotics9030063 · Cited by 28
“The current state of technology facilitated the wide practical application of compact RGB-D sensors for real-time capturing of 2D visual and depth image data, allowing on fly reconstruction of the 3D working scene without the need for additional scanning robot motions [8].”
BackgroundDeep Active Contours for Real-time 6-DoF Object Tracking
Wang, L., Yan, S., Zhen, J., Liu, Y., Zhang, M., Zhang, G., & Zhou, X. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01290 · Cited by 38
“We train our model on six datasets from the BOP [10] challenge, namely IC-BIN [5], T-LESS [8], TUD-L [9], LM [7], YCB-V [54], and RU-APC [29].”
MethodSemi-infinite programming for trajectory optimization with non-convex obstacles
Hauser, K. K. (2021) · The International Journal of Robotics Research · doi:10.1177/0278364920983353 · Cited by 23
“In the APC + YCB rows, the object sets are drawn at random from the YCB (Calli et al., 2017) and APC (Rennie et al., 2016) object datasets.”
MethodDeep Template-based Object Instance Detection
Mercier, J.-P., Garon, M., Giguère, P., & Lalonde, J. (2021) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv48630.2021.00155 · Cited by 36
“Despite this, efforts have been made to capture such real datasets [3, 13, 16, 33, 7, 34] and to combine them together [17].”
BackgroundSafe and Efficient Robot Manipulation: Task-Oriented Environment Modeling and Object Pose Estimation
Zhang, Y., Tian, G., & Shao, X. (2021) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2021.3071222 · Cited by 22
“To address the challenging problems such as low illumination, Rennie et al. [22] adopted the depth information to estimate the pose of objects.”
BackgroundCMA: Cross-modal attention for 6D object pose estimation
Zou, L., Huang, Z., Wang, F., Yang, Z., & Wang, G. (2021) · Computers & Graphics · doi:10.1016/j.cag.2021.04.018 · Cited by 22
A Vision-based inventory method for stacked goods in stereoscopic warehouse
Yin, H., Chen, C., Hao, C., & Huang, B. (2022) · Neural Computing and Applications · doi:10.1007/s00521-022-07551-4 · Cited by 25
“[28] evaluated algorithm for RGBD-based 3D object pose estimation and improved its performance in warehousing scenario.”
MethodEgoObjects: A Large-Scale Egocentric Dataset for Fine-Grained Object Understanding
Zhu, C., Xiao, F., Alvarado, A., Babaei, Y., Hu, J., El-Mohri, H., Culatana, S. C., Sumbaly, R., & Yan, Z. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01840 · Cited by 66
“In [51], a small dataset of 10K RGBD images of 24 object instances were created for object detection and pose estimation.”
Learning Physically Realizable Skills for Online Packing of General 3D Shapes
Zhao, H., Pan, Z., Yu, Y., & Xu, K. (2023) · ACM Transactions on Graphics · doi:10.1145/3603544 · Cited by 38
“2015], the APC dataset [Rennie et al. 2016], and the YCB dataset [Çalli et al.”
Physics-based scene-level reasoning for object pose estimation in clutter
Mitash, C., Boularias, A., & Bekris, K. E. (2019) · The International Journal of Robotics Research · doi:10.1177/0278364919846551 · Cited by 22
“The recent Amazon Picking Challenge ( APC ) (Correll et al. (2016)) has reinforced this realization and has led into the development of datasets specifically for the detection of objects inside cluttered, shelving units (Singh et al. (2014); Rennie et al. (2016); Zeng et al. (2017)).”
BackgroundSelf-citationSynPick: A Dataset for Dynamic Bin Picking Scene Understanding
Periyasamy, A. S., Schwarz, M., & Behnke, S. (2021) · IEEE International Conference on Automation Science and Engineering (CASE) · doi:10.1109/case49439.2021.9551599 · Cited by 22
“Existing datasets for 6D object pose estimation include YCB-Video [2], Linemod-Occluded [3], HomebrewedDB [4], TUD-Light (TUD-L) [1], Toyota Light (TYOL) [1], and Rutgers APC [8].”
BackgroundDevelopment of library management robotic system
Angal, Y. S., & Gade, A. (2017) · International Conference on Data Management, Analytics and Innovation (ICDMAI) · doi:10.1109/icdmai.2017.8073520 · Cited by 19
I Like to Move It: 6D Pose Estimation as an Action Decision ProcessPreprint
Busam, B., Jung, H. J., & Navab, N. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2009.12678 · Cited by 33
“Other datasets focus on textureless industrial [26,16] and household [63,15,71] objects.”
BackgroundComprehensive Review of Robotized Freight Packing
Pantoja-Benavides, G. F., Giraldo, D., Montes, A., García, A., Rodríguez, C., Moreno, C. A. M., & Álvarez‐Martínez, D. (2024) · Logistics · doi:10.3390/logistics8030069 · Cited by 17
Learning Visual Affordances for Robotic Manipulation
Zeng, A. (2019) · Cited by 23
High Accuracy and Efficiency Grasp Pose Detection Scheme with Dense Predictions
Cheng, H., Ho, D., & Meng, M. Q. ‐. (2020) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra40945.2020.9197333 · Cited by 20
A Large-Scale 3D Object Recognition Dataset
Sølund, T., Buch, A. G., Krüger, N., & Aanæs, H. (2016) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2016.16 · Cited by 19
“the Line-Mod dataset [17] or the recent published RGB-D dataset for warehouse pick-and-place tasks [27].”
MethodCarton dataset synthesis method for loading-and-unloading carton detection based on deep learning
Gou, L., Wu, S., Yang, J., Yu, H., Lin, C., Li, X., & Deng, C. (2022) · The International Journal of Advanced Manufacturing Technology · doi:10.1007/s00170-022-08721-3 · Cited by 15
Robotic Grasping Using Semantic Segmentation and Primitive Geometric Model Based 3D Pose Estimation
Sun, G.-J., & Lin, H. (2020) · IEEE/SICE International Symposium on System Integration · doi:10.1109/sii46433.2020.9026297 · Cited by 14
“In the field of 3D object segmentation and pose estimation for pick-and-place applications, the dataset [20] includes 14 kinds of objects for grabbing from the 2015 Amazon Picking Challenge.”
BackgroundSuemitsu, I., Bhamgara, H. K., Utsugi, K., Hashizume, J., & Ito, K. (2022) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2022.3185778 · Cited by 14
“Among these, automated storage and retrieval systems (AS/RS) [3], automated guided vehicle (AGV) picking systems [4], [5] and piece picking robots [6], [7] are commonly employed.”
BackgroundAttitude Estimation Based on Arrays of Passive RFID Tags
Álvarez-Narciandi, G., Laviada, J., Pino, M. R., & Las‐Heras, F. (2018) · IEEE Transactions on Antennas and Propagation · doi:10.1109/tap.2018.2811779 · Cited by 14
“This information can be obtained with inertial sensors (although they are more expensive than RFID tags and require batteries), using RFID tags and antennas with high directivity as proposed in [12], or by means of a computer vision system (which requires line-of-sight conditions) [13].”
BackgroundVisual Object Recognition in Indoor Environments Using Topologically Persistent Features
Samani, E. U., Yang, X., & Banerjee, A. G. (2021) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2021.3099460 · Cited by 21
“On the other hand, the dataset in [33] provides images with several objects but only in a single environment.”
BackgroundAlam, M. M., Ugli, R. A. N., Tanha, K. J., & Jun, T. (2026) · ICT Express · doi:10.1016/j.icte.2026.01.009 · Cited by 18
Block, L., Raiser, A., Schön, L., Braun, F., & Riedel, O. (2022) · Procedia CIRP · doi:10.1016/j.procir.2022.05.004 · Cited by 12
Persistent Homology Meets Object Unity: Object Recognition in Clutter
Samani, E. U., & Banerjee, A. G. (2023) · IEEE Transactions on Robotics · doi:10.1109/tro.2023.3343994 · Cited by 17
“RGB-D RGB-D LM-O HOPE Rutgers BigBird ARID OCID UW-IS UW-IS Object [26] Scenes [49] [50], [51] Image [52] APC [53] [54] [55] [48] [11] Occluded”
BackgroundStrategic Digitalization in Oil and Gas: A Case Study on Mixed Reality and Digital Twins
Aiken, W., Carden, L. L., Bhabhrawala, A., Branco, P., Jourdan, G.-V., & Berg, A. (2024) · IEEE Access · doi:10.1109/access.2024.3417391 · Cited by 13
“Their rendering architecture comes with several unique contributions to the SDG field, such as: 1) Support for at least 15 3D object detection datasets including T-LESS [53] and Rutgers APC [54].”
BackgroundDeep Convolutional Neural Network Design Approach for 3D Object Detection for Robotic Grasping
Sharma, P., & Valles, D. (2020) · IEEE Annual Computing and Communication Workshop and Conference (CCWC) · doi:10.1109/ccwc47524.2020.9031186 · Cited by 11
Towards Intelligent Autonomous Sorting of Unclassified Nuclear Wastes
Basso, A., Hlaváč, V., Hůlka, J., Jilich, M., Kršek, P., Malassiotis, S., Molfino, R., Smutný, V., Wagner, L., & Zoppi, M. (2017) · Procedia Manufacturing · doi:10.1016/j.promfg.2017.07.122 · Cited by 11
“Pick-n-place operations under clutter were also considered in the recent Amazon picking challenge [8].”
BackgroundA 2D laser rangefinder scans dataset of standard EUR pallets
Mohamed, I. S., Capitanelli, A., Mastrogiovanni, F., Rovetta, S., & Zaccaria, R. (2019) · Data in Brief · doi:10.1016/j.dib.2019.103837 · Cited by 13
“• To our knowledge, this is the first dataset for pallet localization and tracking using only 2D Laser Rangefinder data, as opposed to previous datasets aimed at generic AGV and/or more complex sensors [2–5].”
MethodBack, S., Lee, J., Kim, K., Rho, H., Lee, G., Kang, R., Lee, S., Noh, S., Lee, Y., Lee, T., et al. (2025) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2025.3601045 · Cited by 23
“Recent significant progress has been driven by comprehensive datasets [4], [7], [9], [10], [11], [12], [13], [14], [15], enabling deep learning to achieve better generalization.”
Experience-based optimization of robotic perception
Durner, M., Kriegel, S., Riedel, S., Brucker, M., Márton, Z.-C., Bálint-Benczédi, F., & Triebel, R. (2017) · International Conference on Advanced Robotics · doi:10.1109/icar.2017.8023493 · Cited by 12
“described in [1], where the challenges of the object recognition task necessary for the Amazon Picking Challenge resulted in the recording of a new dataset and the manual introduction of a few additional processing and filtering steps around an existing object detection method.”
BackgroundStanding on Giant’s Shoulders: Newcomer’s Experience from the Amazon Robotics Challenge 2017
Ricardez, G. A. G., Hafi, L. E., & von Drigalski, F. (2020) · Springer eBooks · doi:10.1007/978-3-030-35679-8_8 · Cited by 10
Towards Co-Evaluation of Cameras, HDR, and Algorithms for Industrial-Grade 6DoF Pose Estimation
Kalra, A., Stoppi, G., Marin, D., Taamazyan, V., Shandilya, A., Agarwal, R., Boykov, A., Chong, T. H., & Stark, M. M. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02141 · Cited by 22
“This may seem surprising, considering that 6DoF pose estimation has been an active area of research in terms of both algorithms [9, 11–13, 15, 16, 31, 38, 42, 44, 47, 48, 52, 54, 65, 70, 72] and benchmarks [21, 22, 26, 32– 34, 37, 44, 45, 55, 61, 63, 67, 71] for over a decade.”
MethodInfluential
Metadata
- Authors
- Rennie, C., Shome, R., Bekris, K. E., & De Souza, A. F.
- Affiliations
- Rutgers, The State University of New Jersey (US)
- Venue
- IEEE Robotics and Automation Letters
- Published
- 2016-07
- Volume
- 1
- Issue
- 2
- Pages
- 1179-1185
- ISSN
- 2377-3766 (ISSN-L)
- 2377-3774
- OpenAlex
- W2221752211
- Cited by
- 182
- References
- 33
- Citations by year
- 2026: 6 · 2025: 15 · 2024: 20 · 2023: 17 · 2022: 17 · 2021: 24 · 2020: 14 · 2019: 24 · 2018: 17 · 2017: 12 · 2016: 16
- FWCI
- 19.32
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE Robotics and Automation Letters (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- Rutgers University Community Repository (Rutgers University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Engineering › Control and Systems Engineering › Robot Manipulation and Learning
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Computer Graphics and Computer-Aided Design › 3D Shape Modeling and Analysis
- Keywords
- RGB-D, 3D object pose estimation, object detection, depth sensing, cluttered scenes, low-light conditions, textureless objects, robotic manipulation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.