The DOI matches its Crossref record; 91 of the 91 references it cites were found in a registry or an index.
A Comprehensive Analysis of Deep Regression
Open accessAbstract
Deep learning revolutionized data science, and recently its popularity has grown exponentially, as did the amount of papers employing deep networks. Vision tasks, such as human pose estimation, did not escape from this trend. There is a large number of deep models, where small changes in the network architecture, or in the data pre-processing, together with the stochastic nature of the optimization procedures, produce notably different results, making extremely difficult to sift methods that significantly outperform others. This situation motivates the current study, in which we perform a systematic evaluation and statistical analysis of vanilla deep regression, i.e., convolutional neural networks with a linear regression top layer. This is the first comprehensive analysis of deep regression techniques. We perform experiments on four vision problems, and report confidence intervals for the median performance as well as the statistical significance of the results, if any. Surprisingly, the variability due to different data pre-processing procedures generally eclipses the variability due to modifications in the network architecture. Our results reinforce the hypothesis according to which, in general, a general-purpose network (e.g., VGG-16 or ResNet-50) adequately tuned can yield results close to the state-of-the-art without having to resort to more complex and ad-hoc regression models.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2577031 · Cited by 73,472
Going deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562
(2014) · Cited by 33,419
Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation
Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.81 · Cited by 32,684
Rethinking the Inception Architecture for Computer Vision
Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., & Wojna, Z. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.308 · Cited by 32,138
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S., & Szegedy, C. (2015) · arXiv · doi:10.48550/arxiv.1502.03167 · Cited by 27,818
A Simple Sequentially Rejective Multiple Test Procedure
(1979) · Scandinavian Journal of Statistics · doi:10.2307/4615733 · Cited by 21,526
Reducing the dimensionality of data with neural networks
Hinton, G. E., & Salakhutdinov, R. R. (2006) · Science · doi:10.1126/science.1127647 · Cited by 20,928
Individual Comparisons by Ranking Methods
Wilcoxon, F. (1945) · Biometrics Bulletin · doi:10.2307/3001968 · Cited by 15,354
Practical Nonparametric Statistics
Groggel, D. J. (2000) · Technometrics · doi:10.1080/00401706.2000.10486067 · Cited by 11,041
Statistical Methods for Research Workers
(1945) · American Journal of Clinical Pathology · doi:10.1093/ajcp/15.11.552b · Cited by 10,880
(2010) · Cited by 8,261
Realtime Multi-person 2D Pose Estimation Using Part Affinity Fields
Cao, Z., Simon, T., Wei, S.-E., & Sheikh, Y. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.143 · Cited by 7,539
Greff, K., Srivastava, R. K., Koutnik, J., Steunebrink, B. R., & Schmidhuber, J. (2017) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2016.2582924 · Cited by 6,955
How transferable are features in deep neural networks?
Yosinski, J., Clune, J., Bengio, Y., & Lipson, H. (2014) · arXiv · doi:10.48550/arxiv.1411.1792 · Cited by 6,061
Stacked Hourglass Networks for Human Pose Estimation
Newell, A., Yang, K., & Deng, J. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46484-8_29 · Cited by 5,634
Derrac, J., García, S., Molina, D., & Herrera, F. (2011) · Swarm and Evolutionary Computation · doi:10.1016/j.swevo.2011.02.002 · Cited by 5,512
ADADELTA: An Adaptive Learning Rate Method
Zeiler, M. D. (2012) · arXiv · doi:10.48550/arxiv.1212.5701 · Cited by 5,322
Multiple comparisons among means
Dunn, O. J. (1961) · Journal of the American Statistical Association · doi:10.1080/01621459.1961.10482090 · Cited by 5,006
A sharper Bonferroni procedure for multiple tests of significance
HOCHBERG, Y. (1988) · Biometrika · doi:10.1093/biomet/75.4.800 · Cited by 4,829
(2013) · Cited by 3,740
Return of the Devil in the Details: Delving Deep into Convolutional Nets
Chatfield, K., Simonyan, K., Vedaldi, A., & Zisserman, A. (2014) · Proceedings of the British Machine Vision Conference 2014 · doi:10.5244/c.28.6 · Cited by 3,335
Convolutional Neural Networks for Medical Image Analysis: Full Training or Fine Tuning?
Tajbakhsh, N., Shin, J. Y., Gurudu, S. R., Hurst, R. T., Kendall, C. B., Gotway, M. B., & Liang, J. (2016) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2016.2535302 · Cited by 3,277
DeepPose: Human Pose Estimation via Deep Neural Networks
Toshev, A., & Szegedy, C. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.214 · Cited by 3,185
Evolving artificial neural networks
Yao, X. (1999) · Proceedings of the IEEE · doi:10.1109/5.784219 · Cited by 2,981
2D Human Pose Estimation: New Benchmark and State of the Art Analysis
Andriluka, M., Pishchulin, L., Gehler, P., & Schiele, B. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.471 · Cited by 2,688
OverFeat: Integrated Recognition, Localization and Detection using Convolutional Networks
Sermanet, P., Eigen, D., Zhang, X., Mathieu, M., Fergus, R., & LeCun, Y. (2013) · arXiv · doi:10.48550/arxiv.1312.6229 · Cited by 2,680
Why Does Unsupervised Pre-training Help Deep Learning?
Erhan, D., Bengio, Y., Courville, A. C., Manzagol, P.-A., Vincent, P., & Samy Bengio, A. (2010) · Journal of Machine Learning Research · Cited by 2,631
Practical Recommendations for Gradient-Based Training of Deep Architectures
Bengio, Y. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-35289-8_26 · Cited by 2,259
Statistical Methods for Research Workers.
I., L., & Fisher, R. A. (1926) · Journal of the Royal Statistical Society · doi:10.2307/2341488 · Cited by 2,128
Face detection, pose estimation, and landmark localization in the wild
Zhu, X., & Ramanan, D. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6248014 · Cited by 2,120
Scientific method: Statistical errors
Nuzzo, R. (2014) · Nature · doi:10.1038/506150a · Cited by 1,536
Sterne, J. A. C. (2001) · BMJ · doi:10.1136/bmj.322.7280.226 · Cited by 1,466
Bulat, A., & Tzimiropoulos, G. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.116 · Cited by 1,464
Practical Nonparametric Statistics
Coakley, C. W., & Conover, W. J. (2000) · Journal of the American Statistical Association · doi:10.2307/2669565 · Cited by 1,396
Deep Convolutional Network Cascade for Facial Point Detection
Sun, Y., Wang, X., & Tang, X. (2013) · 2013 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2013.446 · Cited by 1,382
A stagewise rejective multiple test procedure based on a modified Bonferroni test
HOMMEL, G. (1988) · Biometrika · doi:10.1093/biomet/75.2.383 · Cited by 1,367
Statistical Methods for Research Workers
(1926) · Journal of Comparative Pathology and Therapeutics · doi:10.1016/s0368-1742(26)80018-9 · Cited by 1,283
Morgan, M. C. (2025) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.17575540 · Cited by 1,283
Bulat, A., & Tzimiropoulos, G. (2017) · Nottingham ePrints (University of Nottingham) · Cited by 1,277
Joint Training of a Convolutional Network and a Graphical Model for Human Pose Estimation
Tompson, J., Jain, A., LeCun, Y., & Bregler, C. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1406.2984 · Cited by 915
Deep Expectation of Real and Apparent Age from a Single Image Without Facial Landmarks
Rothe, R., Timofte, R., & Van Gool, L. (2018) · International Journal of Computer Vision · doi:10.1007/s11263-016-0940-3 · Cited by 896
Articulated Human Detection with Flexible Mixtures of Parts
Yang, Y., & Ramanan, D. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2012.261 · Cited by 852
Clustered Pose and Nonlinear Appearance Models for Human Pose Estimation
Johnson, S., & Everingham, M. (2010) · Procedings of the British Machine Vision Conference 2010 · doi:10.5244/c.24.12 · Cited by 819
Pictorial structures revisited: People detection and articulated pose estimation
Andriluka, M., Roth, S., & Schiele, B. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206754 · Cited by 799
Robust Face Landmark Estimation under Occlusion
Burgos-Artizzu, X. P., Perona, P., & Dollar, P. (2013) · 2013 IEEE International Conference on Computer Vision · doi:10.1109/iccv.2013.191 · Cited by 775
Xie, L., & Yuille, A. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.154 · Cited by 740
Multi-context Attention for Human Pose Estimation
Chu, X., Yang, W., Ouyang, W., Ma, C., Yuille, A. L., & Wang, X. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.601 · Cited by 737
Image-Based Human Age Estimation by Manifold Learning and Locally Adjusted Robust Regression
Guo, G., Fu, Y., Dyer, C. R., & Huang, T. S. (2008) · IEEE Transactions on Image Processing · doi:10.1109/tip.2008.924280 · Cited by 645
Evaluation of convolutional neural networks for visual recognition
Nebauer, C. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/72.701181 · Cited by 597
Random Forests for Real Time 3D Face Analysis
Fanelli, G., Dantone, M., Gall, J., Fossati, A., & Van Gool, L. (2013) · International Journal of Computer Vision · doi:10.1007/s11263-012-0549-0 · Cited by 566
Learning Feature Pyramids for Human Pose Estimation
Yang, W., Li, S., Ouyang, W., Li, H., & Wang, X. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.144 · Cited by 536
Real time head pose estimation with random regression forests
Fanelli, G., Gall, J., & Van Gool, L. (2011) · CVPR 2011 · doi:10.1109/cvpr.2011.5995458 · Cited by 436
A sequentially rejective test procedure based on a modified Bonferroni inequality
ROM, D. M. (1990) · Biometrika · doi:10.1093/biomet/77.3.663 · Cited by 421
Learning to parse images of articulated bodies
Ramanan, D. (2007) · Advances in Neural Information Processing Systems 19 · doi:10.7551/mitpress/7503.003.0146 · Cited by 414
3D human pose from silhouettes by relevance vector regression
Agarwal, A., & Triggs, B. (2004) · Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004. · doi:10.1109/cvpr.2004.1315258 · Cited by 386
Xie, L., & Yuille, A. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · Cited by 377
An Improved Sequentially Rejective Bonferroni Test Procedure
Holland, B. S., & Copenhaver, M. D. (1987) · Biometrics · doi:10.2307/2531823 · Cited by 372
Real-time facial feature detection using conditional regression forests
Dantone, M., Gall, J., Fanelli, G., & Van Gool, L. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6247976 · Cited by 368
Adversarial PoseNet: A Structure-Aware Convolutional Network for Human Pose Estimation
Chen, Y., Shen, C., Wei, X.-S., Liu, L., & Yang, J. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.137 · Cited by 357
Recurrent Human Pose Estimation
Belagiannis, V., & Zisserman, A. (2017) · 2017 12th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2017) · doi:10.1109/fg.2017.64 · Cited by 313
Systematic evaluation of convolution neural network advances on the Imagenet
Mishkin, D., Sergievskiy, N., & Matas, J. (2017) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2017.05.007 · Cited by 308
LCR-Net: Localization-Classification-Regression for Human Pose
Rogez, G., Weinzaepfel, P., & Schmid, C. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.134 · Cited by 296
Multi-source Deep Learning for Human Pose Estimation
Ouyang, W., Chu, X., & Wang, X. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.299 · Cited by 262
Articulated people detection and pose estimation: Reshaping the future
Pishchulin, L., Jain, A., Andriluka, M., Thormahlen, T., & Schiele, B. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6248052 · Cited by 232
Deep Head Pose: Gaze-Direction Estimation in Multimodal Video
Mukherjee, S. S., & Robertson, N. M. (2015) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2015.2482819 · Cited by 182
Robust Optimization for Deep Regression
Belagiannis, V., Rupprecht, C., Carneiro, G., & Navab, N. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.324 · Cited by 179
Convolutional Neural FabricsPreprint
Saxena, S., & Verbeek, J. J. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1606.02492 · Cited by 178
Learning Auto-Structured Regressor from Uncertain Nonnegative Labels
Yan, S., Wang, H., Tang, X., & Huang, T. S. (2007) · 2007 IEEE 11th International Conference on Computer Vision · doi:10.1109/iccv.2007.4409050 · Cited by 173
Fashion Landmark Detection in the Wild
Liu, Z., Yan, S., Luo, P., Wang, X., & Tang, X. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46475-6_15 · Cited by 169
Conditional regression forests for human pose estimation
Sun, M., Kohli, P., & Shotton, J. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6248079 · Cited by 167
P-Values: Misunderstood and Misused
Vidgen, B., & Yasseri, T. (2016) · Frontiers in Physics · doi:10.3389/fphy.2016.00006 · Cited by 143
Geodesic Regression for Image Time-Series
Niethammer, M., Huang, Y., & Vialard, F. (2011) · Lecture notes in computer science · doi:10.1007/978-3-642-23629-7_80 · Cited by 139
Human Pose Estimation with Parsing Induced Learner
Nie, X., Feng, J., Zuo, Y., & Yan, S. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00224 · Cited by 107
3D head pose estimation with convolutional neural network trained on synthetic images
Liu, X., Liang, W. G., Wang, Y., Li, S., & Pei, M. (2016) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2016.7532566 · Cited by 104
A practical guide to CNNs and Fisher Vectors for image instance retrieval
Chandrasekhar, V., Lin, J., Morère, O., Goh, H., & Veillard, A. (2016) · Signal Processing · doi:10.1016/j.sigpro.2016.05.021 · Cited by 92
Robust Head-Pose Estimation Based on Partially-Latent Mixture of Linear Regressions
Drouard, V., Horaud, R., Deleforge, A., et al. (2017) · IEEE Transactions on Image Processing · doi:10.1109/tip.2017.2654165 · Cited by 82
Self-citationOn Regression Losses for Deep Depth Estimation
Carvalho, M., Saux, B. L., Trouvé-Peloux, P., Almansa, A., & Champagnat, F. (2018) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2018.8451312 · Cited by 78
Pictorial structures revisited: People detection and articulated pose estimation
Andriluka, M., Roth, S., & Schiele, B. (2009) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvprw.2009.5206754 · Cited by 76
Deep Convolutional Neural Network Design PatternsPreprint
Smith, L. N., & Topin, N. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.00847 · Cited by 65
2D/3D image registration using regression learning
Chou, C.-R., Frederick, C. B., Mageras, G. S., Chang, S., & Pizer, S. M. (2013) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2013.02.009 · Cited by 64
Deep Mixture of Linear Inverse Regressions Applied to Head-Pose Estimation
Lathuilière, S., Juge, R., Mesejo, P., Muñoz‐Salinas, R., & Horaud, R. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.756 · Cited by 62
Self-citationHead Pose Estimation with Combined 2D SIFT and 3D HOG Features
Wang, B., Liang, W. G., Wang, Y., & Liang, Y. (2013) · International Conference on Image and Graphics · doi:10.1109/icig.2013.133 · Cited by 45
Artificial Neuron–Glia Networks Learning Approach Based on Cooperative Coevolution
Mesejo, P., Ibáñez, O. C., Fernández-Blanco, E., Cedrón, F., Pazos, A., & Porto-Pazos, A. B. (2015) · International Journal of Neural Systems · doi:10.1142/s0129065715500124 · Cited by 37
Self-citationDeepGUM: Learning Deep Robust Regression with a Gaussian-Uniform Mixture Model
Lathuilière, S., Mesejo, P., Alameda-Pineda, X., et al. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01228-1_13 · Cited by 28
Self-citationViraliency: Pooling Local Virality
Alameda-Pineda, X., Pilzer, A., Xu, D., et al. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.59 · Cited by 20
Self-citationIthapu, V. K., Ravi, S. N., & Singh, V. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1702.08670 · Cited by 14
Cited by
A review on deep learning in UAV remote sensing
Osco, L. P., Marcato Junior, J., Marques Ramos, A. P., de Castro Jorge, L. A., Fatholahi, S. N., de Andrade Silva, J., Matsubara, E. T., Pistori, H., Gonçalves, W. N., & Li, J. (2021) · International Journal of Applied Earth Observation and Geoinformation · doi:10.1016/j.jag.2021.102456 · Cited by 458
Remaining useful life prediction using multi-scale deep convolutional neural network
Li, H., Zhao, W., Zhang, Y., & Zio, E. (2020) · Applied Soft Computing · doi:10.1016/j.asoc.2020.106113 · Cited by 390
Zhang, J., Petitjean, C., & Ainouz, S. (2022) · Journal of Imaging · doi:10.3390/jimaging8020023 · Cited by 21
Metadata
- Authors
- Lathuiliere, S., Mesejo, P., Alameda-Pineda, X., & Horaud, R.
- Venue
- IEEE Transactions on Pattern Analysis and Machine Intelligence
- Published
- 2020-09-01
- Volume
- 42
- Issue
- 9
- Pages
- 2065-2081
- ISSN
- 0162-8828 (ISSN-L)
- 2160-9292
- 1939-3539
- OpenAlex
- W2793745122
- Cited by
- 326
- References
- 91
- FWCI
- 22.10
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.