The DOI matches its Crossref record; 72 of the 72 references it cites were found in a registry or an index.
3D Human Pose Estimation in Video With Temporal Convolutions and Semi-Supervised Training
Open accessAbstract
In this work, we demonstrate that 3D poses in video can be effectively estimated with a fully convolutional model based on dilated temporal convolutions over 2D keypoints. We also introduce back-projection, a simple and effective semi-supervised training method that leverages unlabeled video data. We start with predicted 2D keypoints for unlabeled video, then estimate 3D poses and finally back-project to the input 2D keypoints. In the supervised setting, our fully-convolutional model outperforms the previous best result from the literature by 6 mm mean per-joint position error on Human3.6M, corresponding to an error reduction of 11%, and the model also shows significant improvements on HumanEva-I. Moreover, experiments with back-projection show that it comfortably outperforms previous state-of-the-art results in semi-supervised settings where labeled data is scarce. Code and models are available at https://github.com/facebookresearch/VideoPose3D.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2015) · arXiv · doi:10.48550/arxiv.1506.01497 · Cited by 70,812
Microsoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460
(2014) · Cited by 33,419
Feature Pyramid Networks for Object Detection
Lin, T.-Y., Dollar, P., Girshick, R., He, K., Hariharan, B., & Belongie, S. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.106 · Cited by 30,939
He, K., Gkioxari, G., Dollar, P., & Girshick, R. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.322 · Cited by 29,137
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S., & Szegedy, C. (2015) · arXiv · doi:10.48550/arxiv.1502.03167 · Cited by 27,818
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S. (2024) · arXiv (Cornell University) · doi:10.57702/o9raffed · Cited by 14,214
Multi-scale context aggregation by dilated convolutions
Yu, F., & Koltun, V. (2015) · arXiv · doi:10.48550/arxiv.1511.07122 · Cited by 7,738
Caruana, R. (1997) · Machine Learning · doi:10.1023/a:1007379606734 · Cited by 6,636
Stacked Hourglass Networks for Human Pose Estimation
Newell, A., Yang, K., & Deng, J. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46484-8_29 · Cited by 5,634
WaveNet: a generative model for raw audio
Oord, A. V. D., Dieleman, S., Zen, H., Simonyan, K., Vinyals, O., Graves, A., Kalchbrenner, N., Senior, A., & Kavukcuoglu, K. (2016) · arXiv · doi:10.48550/arxiv.1609.03499 · Cited by 4,911
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2015) · arXiv · doi:10.48550/arxiv.1512.03385 · Cited by 4,196
Human3.6M: Large Scale Datasets and Predictive Methods for 3D Human Sensing in Natural Environments
Ionescu, C., Papava, D., Olaru, V., & Sminchisescu, C. (2014) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.248 · Cited by 3,653 · cited 1×
“Before the success of deep learning, most approaches to 3D pose estimation were based on feature engineering and assumptions about skeletons and joint mobility [48, 42, 20, 18].”
BackgroundConvolutional Sequence to Sequence Learning
Gehring, J., Auli, M., Grangier, D., Yarats, D., & Dauphin, Y. N. (2017) · arXiv · doi:10.48550/arxiv.1705.03122 · Cited by 2,850 · cited 1×
“On the other hand, convolutional networks have been very successful in modeling temporal information in tasks that were traditionally tackled with RNNs, such as neural machine translation [11], language modeling [7], speech generation [57], and speech recognition [6].”
BackgroundSelf-citation2D Human Pose Estimation: New Benchmark and State of the Art Analysis
Andriluka, M., Pishchulin, L., Gehler, P., & Schiele, B. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.471 · Cited by 2,688
End-to-End Recovery of Human Shape and Pose
Kanazawa, A., Black, M. J., Jacobs, D. W., & Malik, J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00744 · Cited by 1,870
Cascaded Pyramid Network for Multi-person Pose Estimation
Chen, Y., Wang, Z., Peng, Y., Zhang, Z., Yu, G., & Sun, J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00742 · Cited by 1,799
On the Convergence of Adam and Beyond
Reddi, S. J., Kale, S., & Kumar, S. (2019) · arXiv · doi:10.48550/arxiv.1904.09237 · Cited by 1,771
Keep It SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Image
Bogo, F., Kanazawa, A., Lassner, C., Gehler, P., Romero, J., & Black, M. J. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46454-1_34 · Cited by 1,498
A Simple Yet Effective Baseline for 3d Human Pose Estimation
Martinez, J., Hossain, R., Romero, J., & Little, J. J. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.288 · Cited by 1,462
Sigal, L., Balan, A. O., & Black, M. J. (2010) · International Journal of Computer Vision · doi:10.1007/s11263-009-0273-6 · Cited by 1,164
Monocular 3D Human Pose Estimation in the Wild Using Improved CNN Supervision
Mehta, D., Rhodin, H., Casas, D., Fua, P., Sotnychenko, O., Xu, W., & Theobalt, C. (2017) · 2017 International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2017.00064 · Cited by 1,142
VNect: real-time 3D human pose estimation with a single RGB camera
Mehta, D., Sridhar, S., Sotnychenko, O., Rhodin, H., Shafiei, M., Seidel, H.-P., Xu, W., Casas, D., & Theobalt, C. (2017) · ACM Transactions on Graphics · doi:10.1145/3072959.3073596 · Cited by 1,113
Caruana, R. (1998) · Cited by 1,110
Understanding Back-Translation at Scale
Edunov, S., Ott, M., Auli, M., & Grangier, D. (2018) · Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d18-1045 · Cited by 993
Self-citationNeural Machine Translation of Rare Words with Subword Units
Sennrich, R., Haddow, B., & Birch, A. (2016) · Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p16-1162 · Cited by 974
Neural Machine Translation of Rare Words with Subword Units
Sennrich, R., Haddow, B., & Birch, A. (2015) · arXiv · doi:10.48550/arxiv.1508.07909 · Cited by 950
Coarse-to-Fine Volumetric Prediction for Single-Image 3D Human Pose
Pavlakos, G., Zhou, X., Derpanis, K. G., & Daniilidis, K. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.139 · Cited by 884
(2017) · International Conference on Machine Learning · Cited by 758
A Real-Time Algorithm for Signal Analysis with the Help of the Wavelet Transform
Holschneider, M., Kronland-Martinet, R., Morlet, J., & Tchamitchian, P. (1989) · Inverse Problems and Theoretical Imaging · doi:10.1007/978-3-642-97177-8_28 · Cited by 633
Towards 3D Human Pose Estimation in the Wild: A Weakly-Supervised Approach
Zhou, X., Huang, Q., Sun, X., Xue, X., & Wei, Y. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.51 · Cited by 599
Unsupervised Machine Translation Using Monolingual Corpora Only
Lample, G., Conneau, A., Denoyer, L., & Ranzato, M. (2017) · arXiv · doi:10.48550/arxiv.1711.00043 · Cited by 584
3D Human Pose Estimation = 2D Pose Estimation + Matching
Chen, C.-H., & Ramanan, D. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.610 · Cited by 566
2D/3D Pose Estimation and Action Recognition Using Multitask Deep Learning
Luvizon, D. C., Picard, D., & Tabia, H. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00539 · Cited by 555
Compositional Human Pose Regression
Sun, X., Shang, J., Liang, S., & Wei, Y. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.284 · Cited by 542
Lifting from the Deep: Convolutional 3D Pose Estimation from a Single Image
Tome, D., Russell, C., & Agapito, L. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.603 · Cited by 504
3D Human Pose Estimation from Monocular Images with Deep Convolutional Neural Network
Li, S., & Chan, A. B. (2015) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-16808-1_23 · Cited by 443
Neural Machine Translation in Linear Time
Kalchbrenner, N., Espeholt, L., Simonyan, K., Oord, A. V. D., Graves, A., & Kavukcuoglu, K. (2016) · arXiv · doi:10.48550/arxiv.1610.10099 · Cited by 414 · cited 1×
“Architectures with dilated convolutions have been successful for audio generation [57], semantic segmentation [59] and machine translation [22].”
MethodSparseness Meets Deepness: 3D Human Pose Estimation from Monocular Video
Zhou, X., Zhu, M., Leonardos, S., Derpanis, K. G., & Daniilidis, K. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.537 · Cited by 413
Ordinal Depth Supervision for 3D Human Pose Estimation
Pavlakos, G., Zhou, X., & Daniilidis, K. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00763 · Cited by 411
3D Human Pose Estimation in the Wild by Adversarial Learning
Yang, W., Ouyang, W., Wang, X., Ren, J., Li, H., & Wang, X. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00551 · Cited by 406
Feature Pyramid Networks for Object Detection
Lin, T.-Y., Dollár, P., Girshick, R., He, K., Hariharan, B., & Belongie, S. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1612.03144 · Cited by 380
Learning Pose Grammar to Encode Human Body Configuration for 3D Pose Estimation
Fang, H.-S., Xu, Y., Wang, W., Liu, X., & Zhu, S.-C. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.12270 · Cited by 332
Understanding Back-Translation at ScalePreprint
Edunov, S., Ott, M., Auli, M., & Grangier, D. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1808.09381 · Cited by 330
Self-citationReconstructing 3D Human Pose from 2D Image Landmarks
Ramakrishna, V., Kanade, T., & Sheikh, Y. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-33765-9_41 · Cited by 296
Exploiting Temporal Information for 3D Human Pose Estimation
Hossain, M. R. I., & Little, J. J. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01249-6_5 · Cited by 287
Latent structured models for human pose estimation
Ionescu, C., Li, F., & Sminchisescu, C. (2011) · 2011 International Conference on Computer Vision · doi:10.1109/iccv.2011.6126500 · Cited by 258
Learning to Fuse 2D and 3D Image Cues for Monocular Body Pose Estimation
Tekin, B., Marquez-Neila, P., Salzmann, M., & Fua, P. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.425 · Cited by 242
Propagating LSTM: 3D Pose Estimation Based on Joint Interdependency
Lee, K., Lee, I., & Lee, S. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01234-2_8 · Cited by 235
Wav2Letter: an End-to-End ConvNet-based Speech Recognition System
Collobert, R., Puhrsch, C., & Synnaeve, G. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1609.03193 · Cited by 230
Unsupervised Geometry-Aware Representation for 3D Human Pose Estimation
Rhodin, H., Salzmann, M., & Fua, P. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01249-6_46 · Cited by 218
Direct Prediction of 3D Body Poses from Motion Compensated Sequences
Tekin, B., Rozantsev, A., Lepetit, V., & Fua, P. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.113 · Cited by 215
Norm matters: efficient and accurate normalization schemes in deep networksPreprint
Hoffer, E., Banner, R., Golan, I., & Soudry, D. (2018) · Neural Information Processing Systems · doi:10.48550/arxiv.1803.01814 · Cited by 200
Maximum-Margin Structured Learning with Deep Networks for 3D Human Pose Estimation
Li, S., Zhang, W., & Chan, A. B. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.326 · Cited by 181
Multi-band Polarization Imaging and Applications
Zhao, Y., Yi, C., Kong, S. G., Pan, Q., & Cheng, Y. (2016) · Advances in Computer Vision and Pattern Recognition · doi:10.1007/978-3-662-49373-1 · Cited by 180
3D Human Pose Estimation Using Convolutional Neural Networks with 2D Pose Information
Park, S., Hwang, J., & Kwak, N. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-49409-8_15 · Cited by 160
Tung, H.-Y. F., Harley, A. W., Seto, W., & Fragkiadaki, K. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.467 · Cited by 143
A simple yet effective baseline for 3d human pose estimationPreprint
Martínez, J., Hossain, R., Romero, J., & Little, J. J. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.03098 · Cited by 121
Can 3D Pose Be Learned from 2D Projections Alone?
Drover, D., Rohith, M. V., Chen, C.-H., Agrawal, A., Tyagi, A., & Huynh, C. P. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-11018-5_7 · Cited by 111
Recurrent 3D Pose Sequence Machines
Lin, M., Lin, L., Liang, X., Wang, K., & Cheng, H. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.588 · Cited by 110
Iterated Second-Order Label Sensitive Pooling for 3D Human Pose Estimation
Ionescu, C., Carreira, J., & Sminchisescu, C. (2014) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2014.215 · Cited by 99
Rethinking Pose in 3D: Multi-stage Refinement and Recovery for Markerless Motion Capture
Tomè, D., Toso, M., Agapito, L., & Russell, C. (2018) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2018.00061 · Cited by 89
Maximum-Margin Structured Learning with Deep Networks for 3D Human Pose Estimation
Li, S., Zhang, W., & Chan, A. B. (2016) · International Journal of Computer Vision · doi:10.1007/s11263-016-0962-x · Cited by 79
Learning Latent Representations of 3D Human Pose with Deep Neural Networks
Katircioglu, I., Tekin, B., Salzmann, M., Lepetit, V., & Fua, P. (2018) · International Journal of Computer Vision · doi:10.1007/s11263-018-1066-6 · Cited by 75
3D Human Pose Reconstruction Using Millions of Exemplars
Jiang, H. (2010) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr.2010.414 · Cited by 70
3D Human Pose Estimation via Deep Learning from 2D Annotations
Brau, E., & Jiang, H. (2016) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2016.84 · Cited by 54
3D Human Motion Analysis in Monocular Video Techniques and Challenges
Sminchisescu, C. (2006) · IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS) · doi:10.1109/avss.2006.3 · Cited by 43
Structured Prediction of 3D Human Pose with Deep Neural NetworksPreprint
Tekin, B., Katircioglu, I., Salzmann, M., Lepetit, V., & Fua, P. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1605.05180 · Cited by 39
3D Human Motion Analysis in Monocular Video: Techniques and Challenges
Sminchisescu, C. (2007) · Computational imaging and vision · doi:10.1007/978-1-4020-6693-1_8 · Cited by 23
Monocular 3D Human Pose Estimation In The Wild Using Improved CNN SupervisionPreprint
Mehta, D., Rhodin, H., Casas, D., Fua, P., Sotnychenko, O., Xu, W., & Theobalt, C. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.09813 · Cited by 22
Learning to Fuse 2D and 3D Image Cues for Monocular Body Pose EstimationPreprint
Tekin, B., Márquez-Neila, P., Salzmann, M., & Fua, P. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.05708 · Cited by 12
Cited by
wav2vec: Unsupervised Pre-Training for Speech Recognition
Schneider, S., Baevski, A., Collobert, R., & Auli, M. (2019) · Interspeech 2019 · doi:10.21437/interspeech.2019-1873 · Cited by 1,290
Self-citationLearning 3D Human Dynamics From Video
Kanazawa, A., Zhang, J. Y., Felsen, P., & Malik, J. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00576 · Cited by 531
MHFormer: Multi-Hypothesis Transformer for 3D Human Pose Estimation
Li, W., Liu, H., Tang, H., Wang, P., & Van Gool, L. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01280 · Cited by 458
MixSTE: Seq2seq Mixed Spatio-Temporal Encoder for 3D Human Pose Estimation in Video
Zhang, J., Tu, Z., Yang, J., Chen, Y., & Yuan, J. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01288 · Cited by 426
Learnable Triangulation of Human Pose
Iskakov, K., Burkov, E., Lempitsky, V., & Malkov, Y. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00781 · Cited by 414
Metadata
- Authors
- Pavllo, D., Feichtenhofer, C., Grangier, D., & Auli, M.
- Venue
- 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2019-06
- Pages
- 7745-7754
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W2903549000
- Cited by
- 1,297
- References
- 72
- FWCI
- 45.12
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.