The DOI matches its Crossref record; 39 of the 39 references it cites were found in a registry or an index.
Monocular 3D Pose and Shape Estimation of Multiple People in Natural Scenes: The Importance of Multiple Scene Constraints
Abstract
Human sensing has greatly benefited from recent advances in deep learning, parametric human modeling, and large scale 2d and 3d datasets. However, existing 3d models make strong assumptions about the scene, considering either a single person per image, full views of the person, a simple background or many cameras. In this paper, we leverage state-of-the-art deep multi-task neural networks and parametric human and scene modeling, towards a fully automatic monocular visual sensing system for multiple interacting people, which (i) infers the 2d and 3d pose and shape of multiple people from a single image, relying on detailed semantic representations at both model and image level, to guide a combined optimization with feedforward and feedback components, (ii) automatically integrates scene constraints including ground plane support and simultaneous volume occupancy by multiple people, and (iii) extends the single image model to video by optimally solving the temporal person assignment problem and imposing coherent temporal pose and motion reconstructions while preserving image alignment fidelity. We perform experiments on both single and multi-person datasets, and systematically evaluate each component of the model, showing improved performance and extensive multiple human sensing capability. We also apply our method to images with multiple people, severe occlusions and diverse backgrounds captured in challenging natural scenes, and obtain results of good perceptual quality.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Realtime Multi-person 2D Pose Estimation Using Part Affinity Fields
Cao, Z., Simon, T., Wei, S.-E., & Sheikh, Y. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.143 · Cited by 7,539
Loper, M., Mahmood, N., Romero, J., Pons-Moll, G., & J, B. M. (2015) · ACM Transactions on Graphics · doi:10.1145/2816795.2818013 · Cited by 3,734
Human3.6M: Large Scale Datasets and Predictive Methods for 3D Human Sensing in Natural Environments
Ionescu, C., Papava, D., Olaru, V., & Sminchisescu, C. (2014) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.248 · Cited by 3,653
Self-citationDeepPose: Human Pose Estimation via Deep Neural Networks
Toshev, A., & Szegedy, C. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.214 · Cited by 3,185
Keep It SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Image
Bogo, F., Kanazawa, A., Lassner, C., Gehler, P., Romero, J., & Black, M. J. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46454-1_34 · Cited by 1,498
A Simple Yet Effective Baseline for 3d Human Pose Estimation
Martinez, J., Hossain, R., Romero, J., & Little, J. J. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.288 · Cited by 1,462
DeeperCut: A Deeper, Stronger, and Faster Multi-person Pose Estimation Model
Insafutdinov, E., Pishchulin, L., Andres, B., Andriluka, M., & Schiele, B. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46466-4_3 · Cited by 1,130
VNect: real-time 3D human pose estimation with a single RGB camera
Mehta, D., Sridhar, S., Sotnychenko, O., Rhodin, H., Shafiei, M., Seidel, H.-P., Xu, W., Casas, D., & Theobalt, C. (2017) · ACM Transactions on Graphics · doi:10.1145/3072959.3073596 · Cited by 1,113
On Human Motion Prediction Using Recurrent Neural Networks
Martinez, J., Black, M. J., & Romero, J. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.497 · Cited by 984
Joint Training of a Convolutional Network and a Graphical Model for Human Pose Estimation
Tompson, J., Jain, A., LeCun, Y., & Bregler, C. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1406.2984 · Cited by 915
Coarse-to-Fine Volumetric Prediction for Single-Image 3D Human Pose
Pavlakos, G., Zhou, X., Derpanis, K. G., & Daniilidis, K. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.139 · Cited by 884
Recurrent Network Models for Human Dynamics
Fragkiadaki, K., Levine, S., Felsen, P., & Malik, J. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.494 · Cited by 860
Towards Accurate Multi-person Pose Estimation in the Wild
Papandreou, G., Zhu, T., Kanazawa, N., Toshev, A., Tompson, J., Bregler, C., & Murphy, K. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.395 · Cited by 860
SMPL: A Skinned Multi-Person Linear Model
Loper, M., Mahmood, N., Romero, J., Pons-Moll, G., & Black, M. J. (2023) · Seminal Graphics Papers: Pushing the Boundaries, Volume 2 · doi:10.1145/3596711.3596800 · Cited by 768
Panoptic Studio: A Massively Multiview System for Social Motion Capture
Joo, H., Liu, H., Tan, L., Gui, L., Nabbe, B., Matthews, I., Kanade, T., Nobuhara, S., & Sheikh, Y. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.381 · Cited by 546
Gaussian Process Dynamical Models
Wang, J., Hertzmann, A., & Blei, D. M. (2005) · Cited by 438
Sparseness Meets Deepness: 3D Human Pose Estimation from Monocular Video
Zhou, X., Zhu, M., Leonardos, S., Derpanis, K. G., & Daniilidis, K. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.537 · Cited by 413
Reconstructing 3D Human Pose from 2D Image Landmarks
Ramakrishna, V., Kanade, T., & Sheikh, Y. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-33765-9_41 · Cited by 296
Kinematic jump processes for monocular 3D human tracking
Sminchisescu, C., & Triggs, B. (2003) · 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings. · doi:10.1109/cvpr.2003.1211339 · Cited by 270
Self-citation3D Pictorial Structures for Multiple Human Pose Estimation
Belagiannis, V., Amin, S., Andriluka, M., Schiele, B., Navab, N., & Ilic, S. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.216 · Cited by 258
Estimating Articulated Human Motion With Covariance Scaled Sampling
Sminchisescu, C., & Triggs, B. (2003) · The International Journal of Robotics Research · doi:10.1177/027836403128965132 · Cited by 244
Self-citationLearning to Fuse 2D and 3D Image Cues for Monocular Body Pose Estimation
Tekin, B., Marquez-Neila, P., Salzmann, M., & Fua, P. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.425 · Cited by 242
MoCap-guided Data Augmentation for 3D Pose Estimation in the Wild
Rogez, G., & Schmid, C. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1607.02046 · Cited by 210
A Dual-Source Approach for 3D Pose Estimation from a Single Image
Yasin, H., Iqbal, U., Kruger, B., Weber, A., & Gall, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.535 · Cited by 189
Combined discriminative and generative articulated pose and non-rigid shape estimation
Sigal, L., Bălan, A. O., & Black, M. J. (2007) · Cited by 186
Efficient ConvNet-based marker-less motion capture in general scenes with a low number of cameras
Elhayek, A., de Aguiar, E., Jain, A., Tompson, J., Pishchulin, L., Andriluka, M., Bregler, C., Schiele, B., & Theobalt, C. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7299005 · Cited by 149
Deep Multitask Architecture for Integrated 2D and 3D Human Sensing
Popa, A.-I., Zanfir, M., & Sminchisescu, C. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.501 · Cited by 141
Self-citationA Joint Model for 2D and 3D Pose Estimation from a Single Image
Simo‐Serra, E., Quattoni, A., Torras, C., et al. (2013) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2013.466 · Cited by 140
Markerless Motion Capture of Multiple Characters Using Multiview Image Segmentation
Liu, Y., Gall, J., Stoll, C., Dai, Q., Seidel, H., & Theobalt, C. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.47 · Cited by 136
3D Human Sensing, Action and Emotion Recognition in Robot Assisted Therapy of Children with Autism
Marinoiu, E., Zanfir, M., Olaru, V., & Sminchisescu, C. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00230 · Cited by 110
Self-citationZanfir, M., Popa, A.-I., Zanfir, A., & Sminchisescu, C. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00565 · Cited by 102
Self-citationIterated Second-Order Label Sensitive Pooling for 3D Human Pose Estimation
Ionescu, C., Carreira, J., & Sminchisescu, C. (2014) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2014.215 · Cited by 99
Self-citationGeneral Automatic Human Shape and Motion Capture Using Volumetric Contour Cues
Rhodin, H., Robertini, N., Casas, D., Richardt, C., Seidel, H., & Theobalt, C. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46454-1_31 · Cited by 98
Learning Joint Top-Down and Bottom-up Processes for 3D Visual Inference
Sminchisescu, C., Kanaujia, A., & Metaxas, D. (2006) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2006.169 · Cited by 96
Self-citationTraining Deformable Models for Localization
Ramanan, D., & Sminchisescu, C. (2006) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2006.315 · Cited by 91
Self-citationMARCOnI—ConvNet-Based MARker-Less Motion Capture in Outdoor and Indoor Scenes
Elhayek, A., de Aguiar, E., Jain, A., Thompson, J., Pishchulin, L., Andriluka, M., Bregler, C., Schiele, B., & Theobalt, C. (2016) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2557779 · Cited by 71
Multi-view Dynamic Shape Refinement Using Local Temporal Integration
Leroy, V., Franco, J.-S., & Boyer, E. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.336 · Cited by 65
Variational mixture smoothing for non-linear dynamical systems
Sminchisescu, C., & Jepson, A. S. (2004) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2004.1315220 · Cited by 32
Self-citationSurface Motion Capture Transfer with Gaussian Process Regression
Boukhayma, A., Franco, J.-S., & Boyer, E. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.379 · Cited by 14
Cited by
Monocular human pose estimation: A survey of deep learning-based methods
Chen, Y., Tian, Y., & He, M. (2020) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2019.102897 · Cited by 441
PyMAF: 3D Human Pose and Shape Regression with Pyramidal Mesh Alignment Feedback Loop
Zhang, H., Tian, Y., Zhou, X., Ouyang, W., Liu, Y., Wang, L., & Sun, Z. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01125 · Cited by 341
STAR: Sparse Trained Articulated Human Body Regressor
Osman, A. A. A., Bolkart, T., & Black, M. J. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58539-6_36 · Cited by 252
A Systematic Review of Recent Deep Learning Approaches for 3D Human Pose Estimation
El Kaid, A., & Baïna, K. (2023) · Journal of Imaging · doi:10.3390/jimaging9120275 · Cited by 44
Metadata
- Authors
- Zanfir, A., Marinoiu, E., & Sminchisescu, C.
- Venue
- 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition
- Publisher
- IEEE
- Published
- 2018-06
- Pages
- 2148-2157
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W2798411580
- Cited by
- 292
- References
- 39
- FWCI
- 11.08
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.