The DOI matches its DataCite record; 54 of the 54 references it cites were found in a registry or an index.
Deep Residual Learning for Image Recognition
Open accessAbstract
Deeper neural networks are more difficult to train. We present a residual learning framework to ease the training of networks that are substantially deeper than those used previously. We explicitly reformulate the layers as learning residual functions with reference to the layer inputs, instead of learning unreferenced functions. We provide comprehensive empirical evidence showing that these residual networks are easier to optimize, and can gain accuracy from considerably increased depth. On the ImageNet dataset we evaluate residual nets with a depth of up to 152 layers---8x deeper than VGG nets but still having lower complexity. An ensemble of these residual nets achieves 3.57% error on the ImageNet test set. This result won the 1st place on the ILSVRC 2015 classification task. We also present analysis on CIFAR-10 with 100 and 1000 layers. The depth of representations is of central importance for many visual recognition tasks. Solely due to our extremely deep representations, we obtain a 28% relative improvement on the COCO object detection dataset. Deep residual nets are foundations of our submissions to ILSVRC & COCO 2015 competitions, where we also won the 1st places on the tasks of ImageNet detection, ImageNet localization, COCO detection, and COCO segmentation.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“ResNet-50 is a deep neural network with 50 layers, which can mitigate the vanishing and exploding gradient problems that are commonly encountered in substantially deeper neural networks through the utilization of a deep residual learning framework and residual mapping technique.”
From SAFER: Situation Aware Facial Emotion Recognition (2023) · cites this paper as [36]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“In ILSVRC and COCO 2015 competitions, Faster R-CNN and RPN are the basis of several 1st-place entries [18] in the tracks of ImageNet detection, ImageNet localization, COCO detection, and COCO segmentation.”
From Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks (2017) · cites this paper as [18]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“the use of deep convolutional neural networks (CNNs) has grown due in part to their success in many challenging image classification problems [15-19], surpassing methods relying on hand-crafted features”
From EEGNet: a compact convolutional neural network for EEG-based brain-computer interfaces (2018) · cites this paper as [18]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“The same system [18] also won the 1st place in the ILSVRC 2015 object detection competition, surpassing the second place by absolute 8.5%.”
From Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks (2017) · cites this paper as [18]
The cited source says what the claim attributes to it.
In the citing paper
“Only by replacing VGG-16 with a 101layer residual net (ResNet-101) [18], the Faster R-CNN system increases the mAP from 41.5%/21.2% (VGG16) to 48.4%/27.2% (ResNet-101) on the COCO val set.”
From Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks (2017) · cites this paper as [18]
The cited source says what the claim attributes to it.
In the citing paper
“RPN is also a building block of the 1st-place winning entries in ILSVRC 2015 localization and COCO 2015 segmentation competitions, for which the details are available in [18] and [15] respectively.”
From Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks (2017) · cites this paper as [18]
The cited source says what the claim attributes to it.
In the citing paper
“This observation is still valid even when one increases the depth substantially to over 100 layers [18].”
From Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks (2017) · cites this paper as [18]
The cited source says what the claim attributes to it.
In the citing paper
“RPNs completely learn to propose regions from data, and thus can easily benefit from deeper and more expressive features (such as the 101-layer residual nets adopted in [18]).”
From Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks (2017) · cites this paper as [18]
The cited source says what the claim attributes to it.
In the citing paper
“With other improvements orthogonal to Faster RCNN, He et al . [18] obtained a single-model result of 55.7%/34.9% and an ensemble result of 59.0%/37.4% on the COCO test-dev set, which won the 1st place in the COCO 2015 object detection competition.”
From Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks (2017) · cites this paper as [18]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148 · cited 7×
“On the ImageNet dataset we evaluate residual nets with a depth of up to 152 layers—8× deeper than VGG nets [41] but still having lower complexity.”
BackgroundMethodInfluentialImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · cited 4×
“Our implementation for ImageNet follows the practice in [21, 40].”
BackgroundMethodInfluentialHochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · cited 1×
“Concurrent with our work, “highway networks” [42, 43] present shortcut connections with gating functions [15].”
BackgroundFaster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2577031 · Cited by 73,472 · cited 9×
“The above results are only based on the proposal network (RPN) in Faster R-CNN [32].”
BackgroundMethodInfluentialSelf-citationFaster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2015) · arXiv · doi:10.48550/arxiv.1506.01497 · Cited by 70,812
Self-citationGoing deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562
Microsoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460
Fully convolutional networks for semantic segmentation
Long, J., Shelhamer, E., & Darrell, T. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298965 · Cited by 38,834
Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation
Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.81 · Cited by 32,684
Girshick, R. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.169 · Cited by 28,105
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S., & Szegedy, C. (2015) · arXiv · doi:10.48550/arxiv.1502.03167 · Cited by 27,818 · cited 6×
“We do not use dropout [14], following the practice in [16].”
BackgroundMethodInfluentialLearning Multiple Layers of Features from Tiny Images
Krizhevsky, A. (2024) · doi:10.57702/zp44cu3g · Cited by 23,889
Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification
He, K., Zhang, X., Ren, S., & Sun, J. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.123 · Cited by 19,741 · cited 7×
“1 PReLU-net [13] 21.”
BackgroundMethodInfluentialSelf-citationThe pascal visual object classes (VOC) challenge
Everingham, M., Van Gool, L., Williams, C. K. I., Winn, J., & Zisserman, A. (2010) · International Journal of Computer Vision · doi:10.1007/s11263-009-0275-4 · Cited by 19,599
Neural networks for pattern recognition
Bishop, C. (1994) · Choice Reviews Online · doi:10.5860/choice.31-5500 · Cited by 18,537
Visualizing and understanding convolutional networks
Zeiler, M. D., & Fergus, R. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10590-1_53 · Cited by 15,839
(2010) · Cited by 14,134
(2010) · International Conference on Machine Learning · Cited by 13,995
Backpropagation Applied to Handwritten Zip Code Recognition
LeCun, Y., Boser, B., Denker, J. S., Henderson, D., Howard, R. E., Hubbard, W., & Jackel, L. D. (1989) · Neural Computation · doi:10.1162/neco.1989.1.4.541 · Cited by 11,891 · cited 2×
“15 competitions1, where we also won the 1st places on the tasks of ImageNet detection, ImageNet localization, COCO detection, and COCO segmentation. 1. Introduction Deep convolutional neural networks [22,21] have led to a series of breakthroughs for image classification [21, 50,40]. Deep networks naturally integrate lo…”
BackgroundMethodLearning long-term dependencies with gradient descent is difficult
Bengio, Y., Simard, P., & Frasconi, P. (1994) · IEEE Transactions on Neural Networks · doi:10.1109/72.279181 · Cited by 8,542 · cited 1×
“Driven by the significance of depth, a question arises: Is learning better networks as easy as stacking more layers? An obstacle to answering this question was the notorious problem of vanishing/exploding gradients [1, 9], which hamper convergence from the beginning.”
BackgroundPattern Recognition and Neural Networks
Ripley, B. D. (1996) · Cambridge University Press eBooks · doi:10.1017/cbo9780511812651 · Cited by 6,500
Improving neural networks by preventing co-adaptation of feature detectors
Hinton, G. E., Srivastava, N., Krizhevsky, A., Sutskever, I., & Salakhutdinov, R. R. (2012) · arXiv · doi:10.48550/arxiv.1207.0580 · Cited by 6,382 · cited 2×
“Strong regularization such as maxout [9] or dropout [13] is applied to obtain the best results ([9, 25, 24, 34]) on this dataset.”
MethodCaffe: Convolutional Architecture for Fast Feature EmbeddingPreprint
Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., & Darrell, T. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1408.5093 · Cited by 5,299 · cited 2×
“, Caffe [19]) without modifying the solvers.”
Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10578-9_23 · Cited by 3,034 · cited 1×
“Many other nontrivial visual recognition tasks [8, 12, 7, 32, 27] have also”
BackgroundSelf-citationModern Applied Statistics with S-Plus.
Venables, W. N., & Ripley, B. D. (1996) · Biometrics · doi:10.2307/2532871 · Cited by 3,012 · cited 3×
“An early practice of training multi-layer perceptrons (MLPs) is to add a linear layer connected from the network input to the output [34, 49].”
BackgroundProduct Quantization for Nearest Neighbor Search
Jégou, H., Douze, M., & Schmid, C. (2011) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2010.57 · Cited by 2,956 · cited 1×
“For vector quantization, encoding residual vectors [17] is shown to be more effective than encoding original vectors.”
BackgroundFitNets: Hints for Thin Deep Nets
Romero, A., Ballas, N., Kahou, S. E., Chassang, A., Gatta, C., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1412.6550 · Cited by 2,859 · cited 3×
“networks such as FitNet [35] and Highway [42] (Table 6), yet is among the state-of-the-art results (6.”
BackgroundMethodInfluentialVedaldi, A., & Fulkerson, B. (2010) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/1873951.1874249 · Cited by 2,691 · cited 1×
“Both of them are powerful shallow representations for image retrieval and classification [4, 47].”
BackgroundBriggs, W. L. (1987) · Cited by 1,871
Deeply-Supervised NetsPreprint
Lee, C., Xie, S., Gallagher, P., Zhang, Z., & Tu, Z. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1409.5185 · Cited by 1,693
Fisher Kernels on Visual Vocabularies for Image Categorization
Perronnin, F., & Dance, C. (2007) · 2007 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2007.383266 · Cited by 1,641 · cited 1×
“In image recognition, VLAD [18] is a representation that encodes by the residual vectors with respect to a dictionary, and Fisher Vector [30] can be formulated as a probabilistic version [18] of VLAD.”
MethodConvolutional neural networks at constrained time cost
He, K., & Sun, J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7299173 · Cited by 1,558 · cited 1×
“Unexpectedly, such degradation is not caused by overfitting, and adding more layers to a suitably deep model leads to higher training error, as reported in [11, 42] and thoroughly verified by our experiments.”
ResultSelf-citationTraining Very Deep NetworksPreprint
Srivastava, R. K., Greff, K., & Schmidhuber, J. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1507.06228 · Cited by 1,504 · cited 2×
“Concurrent with our work, “highway networks” [42, 43] present shortcut connections with gating functions [15].”
BackgroundMethodAggregating Local Image Descriptors into Compact Codes
Jegou, H., Perronnin, F., Douze, M., Sanchez, J., Perez, P., & Schmid, C. (2012) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2011.235 · Cited by 1,503 · cited 1×
“In image recognition, VLAD [18] is a representation that encodes by the residual vectors with respect to a dictionary, and Fisher Vector [30] can be formulated as a probabilistic version [18] of VLAD.”
MethodLeCun, Y., Bottou, L., Orr, G., & Müller, K. (1998) · neural information processing systems · Cited by 1,333
Exact solutions to the nonlinear dynamics of learning in deep linear neural networksPreprint
Saxe, A., McClelland, J. L., & Ganguli, S. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1312.6120 · Cited by 1,010 · cited 1×
“This problem, however, has been largely addressed by normalized initialization [23, 9, 37, 13] and intermediate normalization layers [16], which enable networks with tens of layers to start converging for stochastic gradient descent (SGD) with backpropagation [22].”
BackgroundModern Applied Statistics with S-PLUS
Venables, W. N., & Ripley, B. D. (1999) · Statisctics and computing/Statistics and computing · doi:10.1007/978-1-4757-3121-7 · Cited by 990
Goodfellow, I. J., Warde-Farley, D., Mirza, M., Courville, A., & Bengio, Y. (2013) · arXiv · doi:10.48550/arxiv.1302.4389 · Cited by 935 · cited 2×
“en shortcut connections are used, they are connected to the pairs of 3 3 layers (totally 3nshortcuts). On this dataset we use identity shortcuts in all cases (i.e., option A), method error (%) Maxout [10] 9.38 NIN [25] 8.81 DSN [24] 8.22 # layers # params FitNet [35] 19 2.5M 8.39 Highway [42,43] 19 2.3M 7.54 (7.72 0.1…”
MethodInfluentialLeCun, Y., Bottou, L., Orr, G., & Müller, K. R. (1998) · Lecture notes in computer science · doi:10.1007/3-540-49430-8_2 · Cited by 920
Srivastava, R. K., Greff, K., & Schmidhuber, J. (2015) · arXiv · doi:10.48550/arxiv.1505.00387 · Cited by 912 · cited 6×
“16) Highway [41, 42] 32 1.”
BackgroundResultInfluentialOn the Number of Linear Regions of Deep Neural Networks
Montúfar, G., Pascanu, R., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1402.1869 · Cited by 840 · cited 1×
“See [28].”
The devil is in the details: an evaluation of recent feature encoding methods
Chatfield, K., Lempitsky, V., Vedaldi, A., & Zisserman, A. (2011) · Procedings of the British Machine Vision Conference 2011 · doi:10.5244/c.25.76 · Cited by 834 · cited 1×
“with respect to a dictionary, and Fisher Vector [30] can be formulated as a probabilistic version [18] of VLAD. Both of them are powerful shallow representations for image retrieval and classification [4,48]. For vector quantization, encoding residual vectors [17] is shown to be more effective than encoding original ve…”
BackgroundObject Detection via a Multi-region and Semantic Segmentation-Aware CNN Model
Gidaris, S., & Komodakis, N. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.135 · Cited by 801
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2014) · arXiv · doi:10.48550/arxiv.1409.0575 · Cited by 567
Object Detection Networks on Convolutional Feature Maps
Ren, S., He, K., Girshick, R., Zhang, X., & Sun, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2601099 · Cited by 454
Self-citationObject Detection Networks on Convolutional Feature Maps
Ren, S., He, K., Girshick, R., Zhang, X., & Sun, J. (2015) · arXiv · doi:10.48550/arxiv.1504.06066 · Cited by 454
Self-citationVisualizing and Understanding Convolutional Neural NetworksPreprint
Zeiler, M. D., & Fergus, R. (2013) · arXiv (Cornell University) · Cited by 442
Lin, M., Chen, Q., & Yan, S. (2013) · arXiv · doi:10.48550/arxiv.1312.4400 · Cited by 252 · cited 1×
“Strong regularization such as maxout [10] or dropout [14] is applied to obtain the best results ([10, 25, 24, 35]) on this dataset.”
MethodDeep Learning Made Easier by Linear Transformations in Perceptrons
Raiko, T., Harri Valpola, B., & LeCun, Y. (2012) · Cited by 171
Fast surface interpolation using hierarchical basis functions
Szeliski, R. (1990) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.56188 · Cited by 166
Locally adapted hierarchical basis preconditioning
Szeliski, R. (2006) · ACM Transactions on Graphics · doi:10.1145/1141911.1142005 · Cited by 128
Centering Neural Network Gradient Factors
Schraudolph, N. N. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-35289-8_14 · Cited by 39
Vatanen, T., Raiko, T., Harri Valpola, B., & LeCun, Y. (2013) · Lecture notes in computer science · doi:10.1007/978-3-642-42054-2_55 · Cited by 31
Accelerated gradient descent by factor-centering decomposition
Schraudolph, N. N. (1998) · Repository for Publications and Research Data (ETH Zurich) · doi:10.3929/ethz-a-004263473 · Cited by 27 · cited 1×
“The papers of [39, 38, 31, 47] propose methods for centering layer responses, gradients, and propagated errors, implemented by shortcut connections.”
Method
Research works printed but not linked: 2
- [7][7] R. Girshick. Fast R-CNN. In ICCV , 2015. -
- [40][40] P. Sermanet, D. Eigen, X. Zhang, M. Mathieu, R. Fergus, and Y. LeCun. Overfeat: Integrated recognition, localization and detection using convolutional networks. In ICLR , 2014. -
Cited by
Sorted by citations
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2015) · arXiv · doi:10.48550/arxiv.1506.01497 · Cited by 70,812
Self-citationFaster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2577031 · Cited by 73,472
Self-citationFeature Pyramid Networks for Object Detection
Lin, T.-Y., Dollar, P., Girshick, R., He, K., Hariharan, B., & Belongie, S. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.106 · Cited by 30,939
Self-citationMobileNetV2: Inverted Residuals and Linear Bottlenecks
Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., & Chen, L.-C. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00474 · Cited by 28,755
Metadata
- Authors
- He, K., Zhang, X., Ren, S., & Sun, J.
- Affiliations
- Microsoft (United States) (US)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2015
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1512.03385
- OpenAlex
- W2949650786
- Cited by
- 4,196
- References
- 50
- Citations by year
- 2026: 230 · 2025: 339 · 2024: 371 · 2023: 338 · 2022: 177 · 2021: 659 · 2020: 605 · 2019: 529 · 2018: 479 · 2017: 334 · 2016: 131 · 2015: 4
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Image and Video Retrieval Techniques
- Keywords
- ResNet, image recognition, convolutional neural networks, reading lists
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
