RGBD Datasets: Past, Present and Future
Open accessThe DOI matches its Crossref record; 14 of the 14 references it cites were found in a registry or an index.
Abstract
Since the launch of the Microsoft Kinect, scores of RGBD datasets have been released. These have propelled advances in areas from reconstruction to gesture recognition. In this paper we explore the field, reviewing datasets across eight categories: semantics, object pose estimation, camera tracking, scene reconstruction, object tracking, human actions, faces and identification. By extracting relevant information in each category we help researchers to find appropriate data for their needs, and we consider which datasets have succeeded in driving computer vision forward and why. Finally, we examine the future of RGBD datasets. We identify key areas which are currently underexplored, and suggest that future directions may include synthetic data and dense reconstructions of static and dynamic scenes.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Indoor segmentation and support inference from rgbd images
Silberman, N., Hoiem, D., Kohli, P., & Fergus, R. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-33715-4_54 · Cited by 5,760
Human3.6M: Large Scale Datasets and Predictive Methods for 3D Human Sensing in Natural Environments
Ionescu, C., Papava, D., Olaru, V., & Sminchisescu, C. (2014) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.248 · Cited by 3,653 · cited 1×
“Four datasets stand out for capturing humans with a full MoCap setup [23, 34, 82, 51], while the Manipulation Action Dataset [1] is unique in providing semantic segmentation of objects as they are manipulated.”
MethodNTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysis
Shahroudy, A., Liu, J., Ng, T.-T., & Wang, G. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.115 · Cited by 3,148
SUN RGB-D: A RGB-D scene understanding benchmark suite
Song, S., Lichtenberg, S. P., & Xiao, J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298655 · Cited by 1,943 · cited 4×
“We also see new labels applied to existing data [41] and previous releases being recompiled into new offerings [95].”
BackgroundMethodInfluentialA Naturalistic Open Source Movie for Optical Flow Evaluation
Butler, D. J., Wulff, J., Stanley, G. B., & Black, M. J. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-33783-3_44 · Cited by 1,842
Hinterstoisser, S., Lepetit, V., Ilic, S., Holzer, S., Bradski, G., Konolige, K., & Navab, N. (2013) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-37331-2_42 · Cited by 1,161
The YCB object and model set: Towards common benchmarks for manipulation research
Calli, B., Singh, A., Walsman, A., Srinivasa, S., Abbeel, P., & Dollar, A. M. (2015) · 2015 International Conference on Advanced Robotics (ICAR) · doi:10.1109/icar.2015.7251504 · Cited by 838 · cited 1×
“YCB Object and Model Setc [15] Asus Xtion Pro, DSLR 88 XX ’15”
BackgroundBosphorus Database for 3D Face Analysis
Savran, A., Alyüz, N., Dibeklioğlu, H., Çeliktutan, O., Gökberk, B., Sankur, B., & Akarun, L. (2008) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-89991-4_6 · Cited by 812 · cited 1×
“scanner 15 expressions performed by one face - ’07 CASIA 3D Face Database [19] 123 Minolta Vivid 910 4624 images of various expressions, poses and lighting Expression performed ’08 Bosphorus Database [4] 105 Inspeck Mega Capturor II 3D Faces performing expressions at different rotations Expression and pose ’08 ETH Fac…”
MethodHuman fall detection on embedded platform using depth maps and wireless accelerometer
Kwolek, B., & Kepski, M. (2014) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2014.09.005 · Cited by 650 · cited 2×
“Three datasets of humans falling over [60, 39, 38] reflect an interest in use of RGBD sensors for monitoring vulnerable humans in their daily lives.”
BackgroundCombining embedded accelerometers with computer vision for recognizing food preparation activities
Stein, S., & McKenna, S. J. (2013) · Proceedings of the 2013 ACM international joint conference on Pervasive and ubiquitous computing · doi:10.1145/2493432.2493482 · Cited by 448 · cited 2×
“Others are more niche: 50 Salads [96] features over 4 hours of people preparing mixed salads.”
BackgroundUnsupervised feature learning for 3D scene labeling
Lai, K., Bo, L., & Fox, D. (2014) · 2014 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2014.6907298 · Cited by 317 · cited 1×
“RGBD Scenes dataset v2 [61] 14 scenes X # Items from the RGBD Objects dataset labeled on reconstructed point cloud ’14”
MethodRGBD-HuDaAct: A color-depth video database for human daily activity recognition
Ni, B., Wang, G., & Moulin, P. (2011) · 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops) · doi:10.1109/iccvw.2011.6130379 · Cited by 304
ChaLearn Looking at People Challenge 2014: Dataset and Results
Escalera, S., Baró, X., Gonzàlez, J., Bautista, M. A., Madadi, M., Reyes, M., Ponce-López, V., Escalante, H. J., Shotton, J., & Guyon, I. (2015) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-16178-5_32 · Cited by 211 · cited 2×
“Montalbano Gesture [26] 27 20 13858 X Italian hand gestures ’14”
BackgroundA dataset for improved RGBD-based object detection and pose estimation for warehouse pick-and-place
Rennie, C., Shome, R., Bekris, K. E., & De Souza, A. F. (2016) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2016.2532924 · Cited by 182
Cited by
ScanNet: Richly-Annotated 3D Reconstructions of Indoor Scenes
Dai, A., Chang, A. X., Savva, M., Halber, M., Funkhouser, T., & Niessner, M. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.261 · Cited by 4,368
Matterport3D: Learning from RGB-D Data in Indoor Environments
Chang, A., Dai, A., Funkhouser, T., Halber, M., Niebner, M., Savva, M., Song, S., Zeng, A., & Zhang, Y. (2017) · 2017 International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2017.00081 · Cited by 1,858
Deep multimodal learning: a survey on recent advances and trends
Ramachandram, D., & Taylor, G. W. (2017) · IEEE Signal Processing Magazine · doi:10.1109/msp.2017.2738401 · Cited by 1,139
T-LESS: An RGB-D Dataset for 6D Pose Estimation of Texture-Less Objects
Hodan, T., Haluza, P., Obdrzalek, S., Matas, J., Lourakis, M., & Zabulis, X. (2017) · 2017 IEEE Winter Conference on Applications of Computer Vision (WACV) · doi:10.1109/wacv.2017.103 · Cited by 514
Vision-based human activity recognition: a survey
Beddiar, D. R., Nini, B., Sabokrou, M., & Hadid, A. (2020) · Multimedia Tools and Applications · doi:10.1007/s11042-020-09004-3 · Cited by 505
Metadata
- Authors
- Firman, M.
- Venue
- 2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)
- Publisher
- IEEE
- Published
- 2016-06
- Pages
- 661-673
- OpenAlex
- W2337010728
- Cited by
- 147
- References
- 14
- FWCI
- 9.63
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.