The DOI matches its Crossref record; 65 of the 71 references it cites were found in a registry or an index.
Human3.6M: Large Scale Datasets and Predictive Methods for 3D Human Sensing in Natural Environments
Abstract
We introduce a new dataset, Human3.6M, of 3.6 Million accurate 3D Human poses, acquired by recording the performance of 5 female and 6 male subjects, under 4 different viewpoints, for training realistic human sensing systems and for evaluating the next generation of human pose estimation models and algorithms. Besides increasing the size of the datasets in the current state-of-the-art by several orders of magnitude, we also aim to complement such datasets with a diverse set of motions and poses encountered as part of typical human activities (taking photos, talking on the phone, posing, greeting, eating, etc.), with additional synchronized image, human motion capture, and time of flight (depth) data, and with accurate 3D body scans of all the subject actors involved. We also provide controlled mixed reality evaluation scenarios where 3D human models are animated using motion capture and inserted using correct 3D geometry, in complex real environments, viewed with moving cameras, and under occlusion. Finally, we provide a set of large-scale statistical models and detailed evaluation baselines for the dataset illustrating its diversity and the scope for improvement by future work in the research community. Our experiments show that our best large-scale model can leverage our full training set to obtain a 20% improvement in performance compared to a training set of the scale of the largest existing public dataset for this problem. Yet the potential for improvement by leveraging higher capacity, more complex models with our large dataset, is substantially vaster and should stimulate future research. The dataset together with code for the associated large-scale learning models, features, visualization tools, as well as the evaluation server, is available online at http://vision.imar.ro/human3.6m.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Multiple View Geometry in Computer Vision
Hartley, R., & Zisserman, A. (2004) · doi:10.1017/cbo9780511811685 · Cited by 21,070
Nonlinear Component Analysis as a Kernel Eigenvalue Problem
Schölkopf, B., Smola, A. J., & Müller, K. (1998) · Neural Computation · doi:10.1162/089976698300017467 · Cited by 8,196
Real-time human pose recognition in parts from single depth images
Shotton, J., Fitzgibbon, A. W., Cook, M., Sharp, T., Finocchio, M., Moore, R., Kipman, A., & Blake, A. (2011) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2011.5995316 · Cited by 3,545
Multiple View Geometry in Computer Vision
(2001) · Kybernetes · doi:10.1108/k.2001.30.9_10.1333.2 · Cited by 3,376
Random Features for Large-Scale Kernel Machines
Rahimi, A., & Recht, B. (2007) · Cited by 2,783
Kernel methods in machine learning
Hofmann, T., Schölkopf, B., & Smola, A. J. (2008) · The Annals of Statistics · doi:10.1214/009053607000000677 · Cited by 2,292
SCAPE: shape completion and animation of people
Anguelov, D., Srinivasan, P., Koller, D., Thrun, S., Rodgers, J., & Davis, J. (2005) · ACM Transactions on Graphics · doi:10.1145/1073204.1073207 · Cited by 1,489
KERNEL METHODS IN MACHINE LEARNING
Hofmann, T., Schölkopf, B., & Smola, A. J. (2008) · Cited by 1,408
Sigal, L., Bălan, A. O., & Black, M. J. (2009) · International Journal of Computer Vision · doi:10.1007/s11263-009-0273-6 · Cited by 1,205
Poselets: Body part detectors trained using 3D human pose annotations
Bourdev, L., & Malik, J. (2009) · 2009 IEEE 12th International Conference on Computer Vision · doi:10.1109/iccv.2009.5459303 · Cited by 961
Articulated body motion capture by annealed particle filtering
Deutscher, J., Blake, A., & Reid, I. D. (2002) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2000.854758 · Cited by 868
Cover trees for nearest neighbor
Beygelzimer, A., Kakade, S. M., & Langford, J. C. (2006) · International Conference on Machine Learning · doi:10.1145/1143844.1143857 · Cited by 830
Fast pose estimation with parameter-sensitive hashing
Shakhnarovich, Viola, & Darrell (2003) · IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2003.1238424 · Cited by 770
Recovering 3D human pose from monocular images
Agarwal, A., & Triggs, B. (2006) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2006.21 · Cited by 730
Stochastic Tracking of 3D Human Figures Using 2D Image Motion
Sidenbladh, H., Black, M. J., & Fleet, D. J. (2000) · Lecture notes in computer science · doi:10.1007/3-540-45053-x_45 · Cited by 701
Monocular 3D pose estimation and tracking by detection
Andriluka, M., Roth, S., & Schiele, B. (2010) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2010.5540156 · Cited by 507
Surface Capture for Performance-Based Animation
Starck, J., & Hilton, A. (2007) · IEEE Computer Graphics and Applications · doi:10.1109/mcg.2007.68 · Cited by 460
Nearest-Neighbor Methods in Learning and Vision
(2006) · The MIT Press eBooks · doi:10.7551/mitpress/4908.001.0001 · Cited by 442
Anguelov, D., Srinivasan, P., Koller, D., Thrun, S., Rodgers, J., & Davis, J. (2005) · ACM SIGGRAPH Conference (SIGGRAPH) · doi:10.1145/1186822.1073207 · Cited by 410
Nearest-neighbor methods in learning and vision : theory and practice
Shakhnarovich, G., Indyk, P., & Darrell, T. J. (2005) · Cited by 400
Sigal, L., Bhatia, S., Roth, S., Black, M. J., & Isard, M. (2004) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2004.1315063 · Cited by 393
3D human pose from silhouettes by relevance vector regression
Agarwal, A., & Triggs, B. (2004) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2004.1315258 · Cited by 386
Real time motion capture using a single time-of-flight camera
Ganapathi, V., Plagemann, C., Koller, D., & Thrun, S. (2010) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2010.5540141 · Cited by 371
Guan, P., Reiss, L., Hirshberg, D. A., Weiss, A., & Black, M. J. (2012) · ACM Transactions on Graphics · doi:10.1145/2185520.2185531 · Cited by 299
Covariance scaled sampling for monocular 3D body tracking
Sminchisescu, C., & Triggs, B. (2005) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2001.990509 · Cited by 290
Self-citationKinematic jump processes for monocular 3D human tracking
Sminchisescu, C., & Triggs, B. (2003) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2003.1211339 · Cited by 270
Self-citationPractical motion capture in everyday surroundings
Vlasic, D., Adelsberger, R., Vannucci, G., Barnwell, J., Groß, M. H., Matusik, W., & Popović, J. (2007) · ACM SIGGRAPH Conference (SIGGRAPH) · doi:10.1145/1275808.1276421 · Cited by 269
Latent structured models for human pose estimation
Ionescu, C., Li, F., & Sminchisescu, C. (2011) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2011.6126500 · Cited by 262
Self-citationPriors for people tracking from small training sets
Urtasun, R., Fleet, D. J., Hertzmann, A., & Fua, P. (2005) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2005.193 · Cited by 255
Twin Gaussian Processes for Structured Prediction
Bo, L., & Sminchisescu, C. (2009) · International Journal of Computer Vision · doi:10.1007/s11263-008-0204-y · Cited by 243
Self-citationOptimization and Filtering for Human Motion Capture
Gall, J., Rosenhahn, B., Brox, T., & Seidel, H. (2008) · International Journal of Computer Vision · doi:10.1007/s11263-008-0173-1 · Cited by 234
Articulated people detection and pose estimation: Reshaping the future
Pishchulin, L., Jain, A., Andriluka, M., Thormählen, T., & Schiele, B. (2012) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2012.6248052 · Cited by 228
Cascaded Models for Articulated Pose Estimation
Sapp, B., Toshev, A., & Taskar, B. (2010) · Lecture notes in computer science · doi:10.1007/978-3-642-15552-9_30 · Cited by 223
Combined discriminative and generative articulated pose and non-rigid shape estimation
Sigal, L., Balan, A., & Black, M. J. (2007) · Cited by 192
Pose search: Retrieving people using their pose
Ferrari, V., Marín-Jiménez, M. J., & Zisserman, A. (2009) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2009.5206495 · Cited by 154
Weston, J., Chapelle, O., Elisseeff, A., Schölkopf, B., & Vapnik, V. N. (2002) · Cited by 148
Learning hierarchical poselets for human parsing
Wang, Y., Tran, D., & Liao, Z. (2011) · doi:10.1109/cvpr.2011.5995519 · Cited by 146
Computational Studies of Human Motion: Part 1, Tracking and Motion Synthesis
Forsyth, D. A., Arıkan, O., Ikemoto, L., O’Brien, J. F., & Ramanan, D. (2005) · now publishers, Inc. eBooks · doi:10.1561/0600000005 · Cited by 145
Markerless Motion Capture of Multiple Characters Using Multiview Image Segmentation
Liu, Y., Gall, J., Stoll, C., Dai, Q., Seidel, H., & Theobalt, C. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.47 · Cited by 136
Motion capture from body-mounted cameras
Shiratori, T., Park, H. S., Sigal, L., Sheikh, Y. A., & Hodgins, J. K. (2011) · ACM SIGGRAPH Conference (SIGGRAPH) · doi:10.1145/1964921.1964926 · Cited by 118
Articulated soft objects for video-based body modeling
Plankers, R., & Fua, P. (2002) · IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2001.937545 · Cited by 118
We Are Family: Joint Pose Estimation of Multiple Persons
Eichner, M. L., & Ferrari, V. (2010) · Lecture notes in computer science · doi:10.1007/978-3-642-15549-9_17 · Cited by 109
Learning Joint Top-Down and Bottom-up Processes for 3D Visual Inference
Sminchisescu, C., Kanaujia, A., & Metaxas, D. (2006) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2006.169 · Cited by 97
Self-citationLearning Body Pose via Specialized Maps
Rosales, R., & Sclaroff, S. (2001) · Cited by 96
Training Deformable Models for Localization
Ramanan, D., & Sminchisescu, C. (2006) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2006.315 · Cited by 89
Self-citationA general regression technique for learning transductions
Cortes, C., Mohri, M., & Weston, J. (2005) · International Conference on Machine Learning · doi:10.1145/1102351.1102371 · Cited by 88
Outdoor human motion capture using inverse kinematics and von mises-fisher sampling
Pons‐Moll, G., Baak, A., Gall, J., Leal-Taixé, L., Müller, M., Seidel, H., & Rosenhahn, B. (2011) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2011.6126375 · Cited by 86
Semi-supervised Hierarchical Models for 3D Human Pose Reconstruction
Kanaujia, A., Sminchisescu, C., & Metaxas, D. (2007) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2007.383341 · Cited by 85
Self-citationVisual Analysis of Humans: Looking at People
Moeslund, T. B., Hilton, A., Krger, V., & Sigal, L. (2013) · Surrey Research Insight Open Access (The University of Surrey) · doi:10.1007/978-0-85729-997-0 · Cited by 83
BM³E : Discriminative Density Propagation for Visual Tracking
Sminchisescu, C., Kanaujia, A., & Metaxas, D. (2007) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2007.1111 · Cited by 74
Self-citationThe Kneed Walker for human pose tracking
Brubaker, M. A., & Fleet, D. J. (2008) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2008.4587546 · Cited by 72
Monocular Tracking of 3D Human Motion with a Coordinated Mixture of Factor Analyzers
Li, R., Yang, M., Sclaroff, S., & Tian, T.-P. (2006) · Lecture notes in computer science · doi:10.1007/11744047_11 · Cited by 61
A Local Basis Representation for Estimating Human Pose from Cluttered Images
Agarwal, A., & Triggs, B. (2006) · Lecture notes in computer science · doi:10.1007/11612032_6 · Cited by 60
Structured output-associative regression
Bo, L., & Sminchisescu, C. (2009) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2009.5206699 · Cited by 56
Self-citationBakir, G. H., Weston, J., Schölkopf, B., Thrun, S., Saul, L. K., & Schölkopf, B. (2004) · Max Planck Institute for Plasma Physics · Cited by 54
3D Human Motion Analysis in Monocular Video Techniques and Challenges
Sminchisescu, C. (2006) · IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS) · doi:10.1109/avss.2006.3 · Cited by 43
Self-citationNo Bias Left behind: Covariate Shift Adaptation for Discriminative 3D Pose Estimation
Yamada, M., Sigal, L., & Raptis, M. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-33765-9_48 · Cited by 39
Variational mixture smoothing for non-linear dynamical systems
Sminchisescu, C., & Jepson, A. S. (2004) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2004.1315220 · Cited by 35
Self-citationBuilding Roadmaps of Minima and Transitions in Visual Models
Sminchisescu, C., & Triggs, B. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000042935.43630.46 · Cited by 31
Self-citationChebyshev approximations to the histogram χ2 kernel
Li, F., Lebanon, G., & Sminchisescu, C. (2012) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2012.6247956 · Cited by 26
Self-citationRosenhahn, B., Metaxas, D., & Klette, R. (2007) · Computational imaging and vision · doi:10.1007/978-1-4020-6693-1 · Cited by 24
3D Human Motion Analysis in Monocular Video: Techniques and Challenges
Sminchisescu, C. (2007) · Computational imaging and vision · doi:10.1007/978-1-4020-6693-1_8 · Cited by 23
Self-citationPictorial Human Spaces: How Well Do Humans Perceive a 3D Articulated Pose?
Marinoiu, E., Papava, D., & Sminchisescu, C. (2013) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2013.163 · Cited by 22
Self-citationHuman Motion: Understanding, Modelling, Capture, and Animation (Computational Imaging and Vision)
Rosenhahn, B., Klette, R., & Metaxas, D. (2007) · Cited by 12
Chebyshev Approximations to the Histogram $\chi^2$ KernelPreprint
Li, F., Lebanon, G., & Sminchisescu, C. (2012) · arXiv (Cornell University) · Cited by 3
Self-citation
Research works printed but not linked: 3
- 0, INRIA 4D
- 0, CMU HMC
- sminchisescu, 2011, Feature-based Human Pose Estimation in Guide to Visual Analysis of Humans Looking at People
Cited by
Sorted by citations
ScanNet: Richly-Annotated 3D Reconstructions of Indoor Scenes
Dai, A., Chang, A. X., Savva, M., Halber, M., Funkhouser, T., & Niessner, M. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.261 · Cited by 4,368
Transformers in Vision: A Survey
Khan, S., Naseer, M., Hayat, M., Zamir, S. W., Khan, F. S., & Shah, M. (2022) · ACM Computing Surveys · doi:10.1145/3505244 · Cited by 3,595
2D Human Pose Estimation: New Benchmark and State of the Art Analysis
Andriluka, M., Pishchulin, L., Gehler, P., & Schiele, B. (2014) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2014.471 · Cited by 2,699
“6M” [10] that includes images and 3D poses of people but are captured in the controlled indoor environments, whereas our dataset includes real-world images but provides 2D poses only.”
BackgroundSUN RGB-D: A RGB-D scene understanding benchmark suite
Song, S., Lichtenberg, S. P., & Xiao, J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298655 · Cited by 1,943
“There are many existing RGB-D datasets available [54] [47] [1] [25] [44] [60] [49] [45] [29] [66] [57] [52] [46] [16] [21] [73] [35] [67] [3] [41] [10] [63] [42] [64] [65] [48] [12] [33] [8] [50] [26] [6].”
End-to-End Recovery of Human Shape and Pose
Kanazawa, A., Black, M. J., Jacobs, D. W., & Malik, J. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00744 · Cited by 1,962
Metadata
- Authors
- Ionescu, C., Papava, D., Olaru, V., & Sminchisescu, C.
- Affiliations
- IMAR - Institutul de Matematică „Simion Stoilow” al Academiei Române (RO)
- Lund University (SE)
- Venue
- IEEE Transactions on Pattern Analysis and Machine Intelligence
- Published
- 2014-07
- Volume
- 36
- Issue
- 7
- Pages
- 1325-1339
- ISSN
- 0162-8828 (ISSN-L)
- 2160-9292
- OpenAlex
- W2101032778
- Cited by
- 3,653
- References
- 71
- Citations by year
- 2026: 349 · 2025: 529 · 2024: 581 · 2023: 543 · 2022: 390 · 2021: 429 · 2020: 305 · 2019: 235 · 2018: 133 · 2017: 78 · 2016: 48 · 2015: 20 · 2014: 10
- FWCI
- 46.29
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- IEEE Transactions on Pattern Analysis and Machine Intelligence (publishedVersion)
- PubMed (publishedVersion)
- Lund University Publications (Lund University) (submittedVersion)
- Funders
- Universitatea Politehnica din București
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Human Pose and Action Recognition
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Face recognition and analysis
- Physical Sciences › Computer Science › Human-Computer Interaction › Hand Gesture Recognition Systems
- Keywords
- 3D human pose estimation, natural environment, 3D body scans, occlusion, human activities
- MeSH terms
- Algorithms, Biometry, Humans, Image Enhancement, Image Interpretation, Computer-Assisted, Pattern Recognition, Automated, Photography, Posture, Sensitivity and Specificity, Subtraction Technique, Reproducibility of Results, Databases, Factual, Information Storage and Retrieval, Ecosystem, Imaging, Three-Dimensional, Whole Body Imaging
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
