The DOI matches its Crossref record; 41 of the 41 references it cites were found in a registry or an index.
Learning from Synthetic Humans
Open accessAbstract
Estimating human pose, shape, and motion from images and videos are fundamental challenges with many applications. Recent advances in 2D human pose estimation use large amounts of manually-labeled training data for learning convolutional neural networks (CNNs). Such data is time consuming to acquire and difficult to extend. Moreover, manual labeling of 3D pose, depth and motion is impractical. In this work we present SURREAL (Synthetic hUmans foR REAL tasks): a new large-scale dataset with synthetically-generated but realistic images of people rendered from 3D sequences of human motion capture data. We generate more than 6 million frames together with ground truth pose, depth maps, and segmentation masks. We show that CNNs trained on our synthetic dataset allow for accurate human depth estimation and human part segmentation in real RGB images. Our results and the new dataset open up new possibilities for advancing person analysis using cheap and large-scale synthetic data.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Stacked Hourglass Networks for Human Pose Estimation
Newell, A., Yang, K., & Deng, J. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46484-8_29 · Cited by 5,634
Flownet: Learning optical flow with convolutional networks
Dosovitskiy, A., Fischer, P., Ilg, E., Hausser, P., Hazirbas, C., Golkov, V., Smagt, P. V. D., Cremers, D., & Brox, T. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.316 · Cited by 4,313
Loper, M., Mahmood, N., Romero, J., Pons-Moll, G., & J, B. M. (2015) · ACM Transactions on Graphics · doi:10.1145/2816795.2818013 · Cited by 3,734
Self-citationHuman3.6M: Large Scale Datasets and Predictive Methods for 3D Human Sensing in Natural Environments
Ionescu, C., Papava, D., Olaru, V., & Sminchisescu, C. (2014) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.248 · Cited by 3,653 · cited 3×
“n poses, clothing, hair styles, body shapes, occlusions, viewpoints, motion blur and other factors. Much of these variations, however, can be synthesized using existing 3D motion capture (MoCap) data [3,16] and modern tools for realistic rendering. Provided sufficient realism, such an approach would be WILLOW project-t…”
BackgroundMethodInfluentialReal-time human pose recognition in parts from single depth images
Shotton, J., Fitzgibbon, A., Cook, M., Sharp, T., Finocchio, M., Moore, R., Kipman, A., & Blake, A. (2011) · CVPR 2011 · doi:10.1109/cvpr.2011.5995316 · Cited by 3,491
Depth Map Prediction from a Single Image using a Multi-Scale Deep Network
Eigen, D., Puhrsch, C., & Fergus, R. (2014) · arXiv · doi:10.48550/arxiv.1406.2283 · Cited by 3,442
Wei, S.-E., Ramakrishna, V., Kanade, T., & Sheikh, Y. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.511 · Cited by 2,966
Eigen, D., & Fergus, R. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.304 · Cited by 2,862
2D Human Pose Estimation: New Benchmark and State of the Art Analysis
Andriluka, M., Pishchulin, L., Gehler, P., & Schiele, B. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.471 · Cited by 2,688
Real-time human pose recognition in parts from single depth images
Shotton, J., Sharp, T., Kipman, A., Fitzgibbon, A., Finocchio, M., Blake, A., Cook, M., & Moore, R. (2013) · Communications of the ACM · doi:10.1145/2398356.2398381 · Cited by 2,044
Attention to Scale: Scale-Aware Semantic Image Segmentation
Chen, L.-C., Yang, Y., Wang, J., Xu, W., & Yuille, A. L. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.396 · Cited by 1,377
LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop
Yu, F., Seff, A., Zhang, Y., Song, S., Funkhouser, T., & Xiao, J. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1506.03365 · Cited by 1,245
VirtualWorlds as Proxy for Multi-object Tracking Analysis
Gaidon, A., Wang, Q., Cabon, Y., & Vig, E. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.470 · Cited by 1,079
Deep convolutional neural fields for depth estimation from a single image
Liu, F., Shen, C., & Lin, G. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7299152 · Cited by 892
SMPL: A Skinned Multi-Person Linear Model
Loper, M., Mahmood, N., Romero, J., Pons-Moll, G., & Black, M. J. (2023) · Seminal Graphics Papers: Pushing the Boundaries, Volume 2 · doi:10.1145/3596711.3596800 · Cited by 768
Self-citationRender for CNN: Viewpoint Estimation in Images Using CNNs Trained with Rendered 3D Model Views
Su, H., Qi, C. R., Li, Y., & Guibas, L. J. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.308 · Cited by 680
MODEC: Multimodal Decomposable Models for Human Pose Estimation
Sapp, B., & Taskar, B. (2013) · 2013 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2013.471 · Cited by 423
Sparseness Meets Deepness: 3D Human Pose Estimation from Monocular Video
Zhou, X., Zhu, M., Leonardos, S., Derpanis, K. G., & Daniilidis, K. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.537 · Cited by 413
MoSh: motion and shape capture from sparse markers
Loper, M., Mahmood, N., & Black, M. J. (2014) · ACM Transactions on Graphics · doi:10.1145/2661229.2661273 · Cited by 344
Self-citationLearning Deep Object Detectors from 3D Models
Peng, X., Sun, B., Ali, K., & Saenko, K. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.151 · Cited by 338
Dyna: a model of dynamic human shape in motion
Pons-Moll, G., Romero, J., Mahmood, N., & Black, M. J. (2015) · ACM Transactions on Graphics · doi:10.1145/2766993 · Cited by 323
Self-citationLatent structured models for human pose estimation
Ionescu, C., Li, F., & Sminchisescu, C. (2011) · 2011 International Conference on Computer Vision · doi:10.1109/iccv.2011.6126500 · Cited by 258
Virtual Worlds as Proxy for Multi-Object Tracking Analysis
Gaidon, A., Wang, Q., Cabon, Y., & Vig, E. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1605.06457 · Cited by 252
Synthesizing Training Images for Boosting Human 3D Pose Estimation
Chen, W., Wang, H., Li, Y., Su, H., Wang, Z., Tu, C., Lischinski, D., Cohen-Or, D., & Chen, B. (2016) · 2016 Fourth International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2016.58 · Cited by 238
Articulated people detection and pose estimation: Reshaping the future
Pishchulin, L., Jain, A., Andriluka, M., Thormahlen, T., & Schiele, B. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6248052 · Cited by 232
MoCap-guided Data Augmentation for 3D Pose Estimation in the Wild
Rogez, G., & Schmid, C. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1607.02046 · Cited by 210
Self-citationA Dual-Source Approach for 3D Pose Estimation from a Single Image
Yasin, H., Iqbal, U., Kruger, B., Weber, A., & Gall, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.535 · Cited by 189
3D Action Recognition from Novel Viewpoints
Rahmani, H., & Mian, A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.167 · Cited by 187
Learning appearance in virtual scenarios for pedestrian detection
Marin, J., Vazquez, D., Geronimo, D., & Lopez, A. M. (2010) · 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2010.5540218 · Cited by 168
Rhodin, H., Richardt, C., Casas, D., Insafutdinov, E., Shafiei, M., Seidel, H., Schiele, B., & Theobalt, C. (2016) · ACM Transactions on Graphics · doi:10.1145/2980179.2980235 · Cited by 156
Learning a non-linear knowledge transfer model for cross-view action recognition
Rahmani, H., & Mian, A. S. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7298860 · Cited by 145
Marker-Less 3D Human Motion Capture with Monocular Image Sequence and Height-Maps
Du, Y., Wong, Y., Liu, Y., Han, F., Gui, Y., Wang, Z., Kankanhalli, M., & Geng, W. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46493-0_2 · Cited by 108
Iterated Second-Order Label Sensitive Pooling for 3D Human Pose Estimation
Ionescu, C., Carreira, J., & Sminchisescu, C. (2014) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2014.215 · Cited by 99
Deep learning for human part discovery in images
Oliveira, G. L., Valada, A., Bollen, C., Burgard, W., & Brox, T. (2016) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2016.7487304 · Cited by 97
Learning Joint Top-Down and Bottom-up Processes for 3D Visual Inference
Sminchisescu, C., Kanaujia, A., & Metaxas, D. (2006) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2006.169 · Cited by 96
Relevant Feature Selection for Human Pose Estimation and Localization in Cluttered Images
Okada, R., & Soatto, S. (2008) · Lecture notes in computer science · doi:10.1007/978-3-540-88688-4_32 · Cited by 79
Learning people detection models from few training samples
Pishchulin, L., Jain, A., Wojek, C., Andriluka, M., Thormählen, T., & Schiele, B. (2011) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2011.5995574 · Cited by 74
Learning to be a depth camera for close-range human capture and interaction
Fanello, S., Keskin, C., Izadi, S., Kohli, P., Kim, D., Sweeney, D., Criminisi, A., Shotton, J., Kang, S. B., & Paek, T. (2014) · ACM Transactions on Graphics · doi:10.1145/2601097.2601223 · Cited by 70
Synthesizing Training Images for Boosting Human 3D Pose EstimationPreprint
Chen, W., Wang, H., Li, Y., Su, H., Wang, Z., Tu, C., Lischinski, D., Cohen‐Or, D., & Chen, B. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1604.02703 · Cited by 54
Learning Camera Viewpoint Using CNN to Improve 3D Body Pose Estimation
Fathollahi, M., Kasturi, R., & Sarkar, S. (2016) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2016.75 · Cited by 50
FlowCap: 2D Human Pose from Optical Flow
Romero, J., Loper, M., & Black, M. J. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24947-6_34 · Cited by 18
Self-citation
Cited by
End-to-End Recovery of Human Shape and Pose
Kanazawa, A., Black, M. J., Jacobs, D. W., & Malik, J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00744 · Cited by 1,870
Self-citationAMASS: Archive of Motion Capture As Surface Shapes
Mahmood, N., Ghorbani, N., Troje, N. F., Pons-Moll, G., & Black, M. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00554 · Cited by 1,323
Self-citationRomero (2017) · doi:10.1145/3130800.3130883 · Cited by 1,056
Learning to Estimate 3D Human Pose and Shape from a Single Color Image
Pavlakos, G., Zhu, L., Zhou, X., & Daniilidis, K. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00055 · Cited by 607
Neural Body Fitting: Unifying Deep Learning and Model Based Human Pose and Shape Estimation
Omran, M., Lassner, C., Pons-Moll, G., Gehler, P., & Schiele, B. (2018) · 2018 International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2018.00062 · Cited by 498
Metadata
- Authors
- Varol, G., Romero, J., Martin, X., Mahmood, N., Black, M. J., Laptev, I., & Schmid, C.
- Venue
- 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2017-07
- Pages
- 4627-4635
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W2576289912
- Cited by
- 908
- References
- 41
- FWCI
- 27.10
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
