The DOI matches its DataCite record; 23 of the 23 references it cites were found in a registry or an index.
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Open accessAbstract
Training Deep Neural Networks is complicated by the fact that the distribution of each layer's inputs changes during training, as the parameters of the previous layers change. This slows down the training by requiring lower learning rates and careful parameter initialization, and makes it notoriously hard to train models with saturating nonlinearities. We refer to this phenomenon as internal covariate shift, and address the problem by normalizing layer inputs. Our method draws its strength from making normalization a part of the model architecture and performing the normalization for each training mini-batch. Batch Normalization allows us to use much higher learning rates and be less careful about initialization. It also acts as a regularizer, in some cases eliminating the need for Dropout. Applied to a state-of-the-art image classification model, Batch Normalization achieves the same accuracy with 14 times fewer training steps, and beats the original model by a significant margin. Using an ensemble of batch-normalized networks, we improve upon the best published result on ImageNet classification: reaching 4.9% top-5 validation error (and 4.8% test error), exceeding the accuracy of human raters.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source doesn’t support the claim as written.
In the citing paper
“This method ensures the performance of the output activations [81]. This performance follows a unit Gaussian distribution.”
From Review of deep learning: concepts, CNN architectures, challenges, applications, future directions (2021) · cites this paper as [81]
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Since the addition of three branches introduces the covariate shift issue (Ioffe and Szegedy 2015)”
From Make RepVGG Greater Again: A Quantization-Aware Approach (2024) · cites this paper as Ioffe and Szegedy 2015
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“SGD can be applied naively to the federated optimization problem, where a single batch gradient calculation (say on a randomly selected client) is done per round of communication. This approach is computationally efficient, but requires very large numbers of rounds of training to produce good models (e.g., even using an advanced approach like batch normalization, Ioffe and Szegedy [21] trained MNI”
From Communication-Efficient Learning of Deep Networks from Decentralized Data (2016) · cites this paper as [21]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“It mitigates the internal covariate shift that occurs as the distribution of each hidden layer's inputs changes during the training (as the parameters of the preceding layers change) and slows down the learning process.”
From Empirical asset pricing via machine learning: evidence from the European stock market (2021) · cites this paper as Ioffe and Szegedy (2015)
The cited source says what the claim attributes to it.
In the citing paper
“Second, we adopt the batch normalization algorithm introduced by Ioffe and Szegedy (2015).”
From Empirical asset pricing via machine learning: evidence from the European stock market (2021) · cites this paper as Ioffe and Szegedy (2015)
The cited source says what the claim attributes to it.
In the citing paper
“It occurs throughout training, because the weight updates to deeper layers result in a continuously changing distribution of signal at higher layers, which hinders the convergence of their weights.”
From Efficient multi-scale 3D CNN with fully connected CRF for accurate brain lesion segmentation (2017) · cites this paper as Ioffe and Szegedy, 2015
The cited source says what the claim attributes to it.
In the citing paper
“Batch normalization standardizes intermediate outputs of the network to zero mean and unit variance for a batch of training examples [Ioffe and Szegedy, 2015].”
From Deep learning with convolutional neural networks for EEG decoding and visualization (2017) · cites this paper as Ioffe and Szegedy, 2015
The cited source says what the claim attributes to it.
In the citing paper
“Batch normalization is a type of supplemental layer which adaptively normalizes the input values of the following layer, mitigating the risk of overfitting, as well as improving gradient flow through the network, allowing higher learning rates, and reducing the dependence on initialization [28].”
From Convolutional neural networks: an overview and application in radiology (2018) · cites this paper as [28]
The cited source says what the claim attributes to it.
In the citing paper
“Unlike BatchNorm [Ioffe and Szegedy, 2015], commonly used in CNNs, LayerNorm cannot be folded into the previous layer due to its dynamic computational property, so we have to quantize it separately.”
From FQ-ViT: Post-Training Quantization for Fully Quantized Vision Transformer (2022) · cites this paper as [Ioffe and Szegedy, 2015]
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“However, in integer-only quantization, all the operations are performed using low-precision integer arithmetic [113,132,154,193,267], as illustrated in Figure 6 (Right).This permits the entire inference to be carried out with efficient integer arithmetic, without any floating point dequantization of any parameters or activations.”
From A Survey of Quantization Methods for Efficient Neural Network Inference (2022) · cites this paper as [113]
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“CMSIS-NN [136] is a library from ARM that helps quantizing and deploying NN models onto the ARM Cortex-M cores.Specifically, the library leverages fixed-point quantization [113,154,267] with power-oftwo scaling factors so that quantization and dequantization processes can be carried out efficiently with bit shifting operations.”
From A Survey of Quantization Methods for Efficient Neural Network Inference (2022) · cites this paper as [113]
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“as neural networks have the capability of learning directly from raw data [49].”
From Investigating Enhancements to Contrastive Predictive Coding for Human Activity Recognition (2023) · cites this paper as [49]
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Recent advances in light-weight neural network architectures [15,13,14,9,11,10,12] are mostly based on the metric of FLOPs and do not consider these properties above.”
From ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design (2018) · cites this paper as [13]
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Following the common practice [15,13,14], all networks in comparison have four levels of computational complexity, i.e. about 40, 140, 300 and 500+ MFLOPs.”
From ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design (2018) · cites this paper as [13]
Experimental — accuracy not yet measuredIn the citing paper
“Our reproduced GoogleNet has center crop top-1 error of 26.4%, close to that reported in [32] (25.2%).”
From Object Detection Networks on Convolutional Feature Maps (2017) · cites this paper as [32]
In the citing paper
“To avoid overfitting, both dropout [116] and batch normalization [33] can help to achieve better regularization.”
From Deep learning for human affect recognition: Insights and new developments (2021) · cites this paper as [33]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805
(2014) · Cited by 33,419
Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification
He, K., Zhang, X., Ren, S., & Sun, J. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.123 · Cited by 19,741
(2010) · Cited by 14,134
(2010) · International Conference on Machine Learning · Cited by 13,995
Independent component analysis: algorithms and applications
Hyvärinen, A., & Erkki Oja, A. (2000) · Neural Networks · doi:10.1016/s0893-6080(00)00026-5 · Cited by 8,926
(2010) · Cited by 8,261
On the difficulty of training recurrent neural networks
Pascanu, R., Mikolov, T., & Bengio, Y. (2012) · arXiv · doi:10.48550/arxiv.1211.5063 · Cited by 4,310
(2013) · Cited by 3,740
Large Scale Distributed Deep Networks
Dean, J. B., Corrado, G. S., Monga, R., Chen, K., Devin, M., Mao, M., Ranzato, M., Senior, A., Tucker, P. A., Yang, K., et al. (2012) · Cited by 3,202
Flagged first, then by citations
Cited by
Sorted by citations
Items 301–400 of 1,339
Speaker Recognition from Raw Waveform with SincNet
Ravanelli, M., & Bengio, Y. (2018) · 2018 IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt.2018.8639585 · Cited by 896
Multi-Similarity Loss With General Pair Weighting for Deep Metric Learning
Wang, X., Han, X., Huang, W., Dong, D., & Scott, M. R. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00516 · Cited by 923
Data augmentation: A comprehensive survey of modern approaches
Mumuni, A., & Mumuni, F. (2022) · Array · doi:10.1016/j.array.2022.100258 · Cited by 891
Conformer: Local Features Coupling Global Representations for Visual Recognition
Peng, Z., Huang, W. F., Gu, S., Xie, L., Wang, Y., Jiao, J., & Ye, Q. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00042 · Cited by 922
Attentional Local Contrast Networks for Infrared Small Target Detection
Dai, Y., Wu, Y., Zhou, F., & Barnard, K. (2021) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2020.3044958 · Cited by 924
Shape Robust Text Detection With Progressive Scale Expansion Network
Wang, W., Xie, E., Li, X., Hou, W., Lü, T., Yu, G., & Shao, S. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00956 · Cited by 881
Show and Tell: Lessons Learned from the 2015 MSCOCO Image Captioning Challenge
Vinyals, O., Toshev, A., Bengio, S., & Erhan, D. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2587640 · Cited by 875
Cole, J. H., Poudel, R. P. K., Tsagkrasoulis, D., Caan, M. W. A., Steves, C., Spector, T. D., & Montana, G. (2017) · NeuroImage · doi:10.1016/j.neuroimage.2017.07.059 · Cited by 872
“The architecture contains repeated 5 blocks of a (3 3 3) convolutional layer (with stride of 1), a rectified linear unit (ReLU), a (3 3 3) convolutional layer (with stride of 1), a 3D batch-normalization layer (Ioffe and Szegedy, 2015), a ReLU and finally a (2 2 2) max-pooling layer (with stride of 2).”
BackgroundFFDNet: Toward a fast and flexible solution for CNN-Based image denoising
Zhang, K., Zuo, W., & Zhang, L. (2018) · TUScholarShare (Temple University) · Cited by 916
Evaluation of output embeddings for fine-grained image classification
Akata, Z., Reed, S., Walter, D., Lee, H., & Schiele, B. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7298911 · Cited by 1,022
Pose-Driven Deep Convolutional Model for Person Re-identification
Su, C., Li, J., Zhang, S. S., Xing, J., Gao, W., & Tian, Q. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.427 · Cited by 850
You Only Look One-level Feature
Chen, Q., Wang, Y., Yang, T., Zhang, X., Cheng, J., & Sun, J. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01284 · Cited by 871
Structural Design of Convolutional Neural Networks for Steganalysis
Xu, G., Wu, H., & Shi, Y.-Q. (2016) · IEEE Signal Processing Letters · doi:10.1109/lsp.2016.2548421 · Cited by 882
Brain tumor classification for MR images using transfer learning and fine-tuning
Swati, Z. N. K., Zhao, Q., Kabir, M., Ali, F., Ali, Z., Ahmad, S., & Lu, J. (2019) · Computerized Medical Imaging and Graphics · doi:10.1016/j.compmedimag.2019.05.001 · Cited by 895
Adaptive Graph Convolutional Neural Networks
Li, R., Wang, S., Zhu, F., & Huang, J. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11691 · Cited by 842
Liang, J., Hu, D., & Feng, J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.08546 · Cited by 869
Benchmark analysis of representative deep neural network architectures
Bianco, S., Cadene, R., Celona, L., & Napoletano, P. (2018) · IEEE Access · doi:10.1109/access.2018.2877890 · Cited by 838
Image Generation from Scene Graphs
Johnson, J., Gupta, A., & Fei-Fei, L. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00133 · Cited by 836
AttGAN: Facial attribute editing by only changing what you want
He, Z., Zuo, W., Kan, M., Shan, S., & Chen, X. (2019) · IEEE Transactions on Image Processing · doi:10.1109/tip.2019.2916751 · Cited by 833
Deep Learning Hierarchical Representations for Image Steganalysis
Ye, J., Ni, J., & Yi, Y. (2017) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2017.2710946 · Cited by 865
Arandjelović, R., & Zisserman, A. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.73 · Cited by 836
Speeding Up Distributed Machine Learning Using Codes
Lee, K., Lam, M., Pedarsani, R., Papailiopoulos, D., & Ramchandran, K. (2018) · IEEE Transactions on Information Theory · doi:10.1109/tit.2017.2736066 · Cited by 829
CHAOS Challenge - combined (CT-MR) healthy abdominal organ segmentation
Kavur, A. E., Gezer, N. S., BARIŞ, M. M., Aslan, S., Conze, P.-H., Groza, V., Pham, D. D., Chatterjee, S., Ernst, P., Özkan, S., et al. (2020) · Medical Image Analysis · doi:10.1016/j.media.2020.101950 · Cited by 893
Wen, J., Thibeau-Sutre, E., Diaz-Melo, M., Samper-González, J., Routier, A., Bottani, S., Dormont, D., Durrleman, S., Burgos, N., & Colliot, O. (2020) · Medical Image Analysis · doi:10.1016/j.media.2020.101694 · Cited by 809
Weinan, E., Han, J., & Jentzen, A. (2017) · Communications in Mathematics and Statistics · doi:10.1007/s40304-017-0117-6 · Cited by 876
Learning to Compare: Relation Network for Few-Shot Learning
Sung, F., Yang, Y., Zhang, L., Xiang, T., Torr, P. H. S., & Hospedales, T. M. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00131 · Cited by 819
Iizuka, S., Simo‐Serra, E., & Ishikawa, H. (2016) · ACM Transactions on Graphics · doi:10.1145/2897824.2925974 · Cited by 830
Improved Recurrent Neural Networks for Session-based Recommendations
Tan, Y., Xu, X., & Liu, Y. (2016) · Workshop on Deep Learning for Recommender Systems (DLRS) · doi:10.1145/2988450.2988452 · Cited by 845
Training Spiking Neural Networks Using Lessons From Deep Learning
Eshraghian, J. K., Ward, M., Neftci, E., Wang, X., Lenz, G., Dwivedi, G., Bennamoun, M., Jeong, D. S., & Lu, W. D. (2023) · Proceedings of the IEEE · doi:10.1109/jproc.2023.3308088 · Cited by 1,106
ResMLP: Feedforward Networks for Image Classification With Data-Efficient Training
Touvron, H., Bojanowski, P., Caron, M., Cord, M., El-Nouby, A., Grave, E., Izacard, G., Joulin, A., Synnaeve, G., Verbeek, J., et al. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3206148 · Cited by 807
Deep bilateral learning for real-time image enhancement
Gharbi, M., Chen, J., Barron, J. T., Hasinoff, S. W., & Durand, F. (2017) · ACM Transactions on Graphics · doi:10.1145/3072959.3073592 · Cited by 819
Strategies for Pre-training Graph Neural NetworksPreprint
Hu, W., Liu, B., Gomes, J., Žitnik, M., Liang, P., Pande, V. S., & Leskovec, J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.12265 · Cited by 1,410
Object Detection via a Multi-region and Semantic Segmentation-Aware CNN Model
Gidaris, S., & Komodakis, N. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.135 · Cited by 801
Learning Deep Representations of Fine-Grained Visual Descriptions
Reed, S., Akata, Z., Lee, H., & Schiele, B. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.13 · Cited by 818
BSN: Boundary Sensitive Network for Temporal Action Proposal Generation
Lin, T., Zhao, X., Su, H., Wang, C., & Yang, M. Z. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01225-0_1 · Cited by 816
Deep Learning in Medical Ultrasound Analysis: A Review
Liu, S., Wang, Y., Yang, X., Lei, B., Liu, L., Li, S. X., Ni, D., & Wang, T. (2019) · Engineering · doi:10.1016/j.eng.2018.11.020 · Cited by 798
On instabilities of deep learning in image reconstruction and the potential costs of AI
Antun, V., Renna, F., Poon, C., Adcock, B., & Hansen, A. C. (2020) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1907377117 · Cited by 845
Do ImageNet Classifiers Generalize to ImageNet?
Recht, B., Roelofs, R., Schmidt, L., & Shankar, V. (2019) · arXiv · doi:10.48550/arxiv.1902.10811 · Cited by 790
Disentangling by FactorisingPreprint
Kim, H., & Mnih, A. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.05983 · Cited by 886
Convolutional networks for fast, energy-efficient neuromorphic computing
Esser, S. K., Merolla, P. A., Arthur, J. V., Cassidy, A. S., Appuswamy, R., Andreopoulos, A., Berg, D. J., McKinstry, J. L., Melano, T., Barch, D. R., et al. (2016) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1604850113 · Cited by 814
Learning Deep Structure-Preserving Image-Text Embeddings
Wang, L., Li, Y., & Lazebnik, S. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.541 · Cited by 795
Evolving Deep Convolutional Neural Networks for Image Classification
Sun, Y., Xue, B., Zhang, M., & Yen, G. G. (2019) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2019.2916183 · Cited by 794
HashNet: Deep Learning to Hash by Continuation
Cao, Z., Long, M., Wang, J., & Yu, P. S. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.598 · Cited by 777
Convolutional Neural Networks for Inverse Problems in Imaging: A Review
McCann, M. T., Jin, K. H., & Unser, M. A. (2017) · IEEE Signal Processing Magazine · doi:10.1109/msp.2017.2739299 · Cited by 794
Multi-Classification of Brain Tumor Images Using Deep Neural Network
Sultan, H. H., Salem, N. M., & Al-Atabany, W. (2019) · IEEE Access · doi:10.1109/access.2019.2919122 · Cited by 775
Scale Aggregation Network for Accurate and Efficient Crowd Counting
Cao, X., Wang, Z., Zhao, Y., & Su, F. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01228-1_45 · Cited by 768
Network Traffic Classifier With Convolutional and Recurrent Neural Networks for Internet of Things
López‐Martín, M., Carro, B., Sánchez-Esguevillas, A., & Lloret, J. (2017) · IEEE Access · doi:10.1109/access.2017.2747560 · Cited by 772
Learning a Deep Embedding Model for Zero-Shot Learning
Zhang, L., Xiang, T., & Gong, S. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.321 · Cited by 771
Wu, B., Zhou, X., Zhao, S., Yue, X., & Keutzer, K. (2019) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2019.8793495 · Cited by 785
Video Frame Synthesis Using Deep Voxel Flow
Liu, Z., Yeh, R. A., Tang, X., Liu, Y., & Agarwala, A. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.478 · Cited by 766
Adversarially Learned One-Class Classifier for Novelty Detection
Sabokrou, M., Khalooei, M., Fathy, M., & Adeli, E. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00356 · Cited by 765
Rethinking Classification and Localization for Object Detection
Wu, Y., Chen, Y., Yuan, L., Liu, Z., Wang, L., Li, H., & Fu, Y. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.01020 · Cited by 780
Towards K-means-friendly Spaces: Simultaneous Deep Learning and ClusteringPreprint
Yang, B., Fu, X., Sidiropoulos, N. D., & Hong, M. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1610.04794 · Cited by 782
Realistic Evaluation of Deep Semi-Supervised Learning Algorithms
Oliver, A., Odena, A., Raffel, C., Cubuk, E. D., & Goodfellow, I. J. (2018) · arXiv · doi:10.48550/arxiv.1804.09170 · Cited by 772
ABCDM: An Attention-based Bidirectional CNN-RNN Deep Model for sentiment analysis
Basiri, M. E., Nemati, S., Abdar, M., Cambria, E., & Acharya, U. R. (2020) · Future Generation Computer Systems · doi:10.1016/j.future.2020.08.005 · Cited by 770
Ball, J. E., Anderson, D. T., & Chan, C. S. (2017) · Journal of Applied Remote Sensing · doi:10.1117/1.jrs.11.042609 · Cited by 746
Urban, G., Tripathi, P. V., Alkayali, T., Mittal, M., Jalali, F., Karnes, W. E., & Baldi, P. (2018) · Gastroenterology · doi:10.1053/j.gastro.2018.06.037 · Cited by 763
Deep Learning for Electromyographic Hand Gesture Signal Classification Using Transfer Learning
Côté‐Allard, U., Fall, C. L., Drouin, A., Campeau‐Lecours, A., Gosselin, C., Glette, K., Laviolette, F., & Gosselin, B. J. (2019) · IEEE Transactions on Neural Systems and Rehabilitation Engineering · doi:10.1109/tnsre.2019.2896269 · Cited by 789
Video Frame Interpolation via Adaptive Separable Convolution
Niklaus, S., Mai, L., & Liu, F. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.37 · Cited by 746
Expectation-Maximization Attention Networks for Semantic Segmentation
Li, X., Zhong, Z., Wu, J., Yang, Y., Lin, Z., & Liu, H. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00926 · Cited by 747
Xie, L., & Yuille, A. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.154 · Cited by 919
Density estimation using Real NVP
Dinh, L., Sohl-Dickstein, J., & Bengio, S. (2016) · arXiv · Cited by 2,067
Invariance Matters: Exemplar Memory for Domain Adaptive Person Re-Identification
Zhong, Z., Zheng, L., Luo, Z., Li, S., & Yang, Y. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00069 · Cited by 729
Certified Robustness to Adversarial Examples with Differential Privacy
Lecuyer, M., Atlidakis, V., Geambasu, R., Hsu, D., & Jana, S. (2019) · 2019 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp.2019.00044 · Cited by 750
Learning Spatio-Temporal Features with 3D Residual Networks for Action Recognition
Hara, K., Kataoka, H., & Satoh, Y. (2017) · 2017 IEEE International Conference on Computer Vision Workshops (ICCVW) · doi:10.1109/iccvw.2017.373 · Cited by 726
An empirical survey of data augmentation for time series classification with neural networks
Iwana, B. K., & Uchida, S. (2021) · PLOS ONE · doi:10.1371/journal.pone.0254841 · Cited by 725
Singh, K. K., & Lee, Y. J. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.381 · Cited by 724
A CNN-RNN framework for crop yield prediction
Khaki, S., Wang, L., & Archontoulis, S. V. (2020) · Frontiers in Plant Science · doi:10.3389/fpls.2019.01750 · Cited by 722
Automatic diagnosis of the 12-lead ECG using a deep neural network
Ribeiro, A. H., Ribeiro, M. H., Paixao, G. M., Oliveira, D. M., Gomes, P. R., Canazart, J. A., Ferreira, M. P. S., Andersson, C. R., Macfarlane, P. W., Meira, W., et al. (2020) · RePEc: Research Papers in Economics · doi:10.5281/zenodo.3625006 · Cited by 722
Shi, Q., Liu, M., Li, S., Liu, X., Wang, F., & Zhang, L. (2021) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2021.3085870 · Cited by 711
DER: Dynamically Expandable Representation for Class Incremental Learning
Yan, S., Xie, J., & He, X. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00303 · Cited by 803
Learning Deep Context-Aware Features over Body and Latent Parts for Person Re-identification
Li, D., Chen, X., Zhang, Z., & Huang, K. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.782 · Cited by 710
BACH: Grand challenge on breast cancer histology images
Aresta, G., Araújo, T., Kwok, S., Chennamsetty, S. S., Safwan, M., Alex, V., Marami, B., Prastawa, M. W., Chan, M. S. M., Donovan, M. J., et al. (2019) · Medical Image Analysis · doi:10.1016/j.media.2019.05.010 · Cited by 755
An Attention-Based Deep Learning Approach for Sleep Stage Classification With Single-Channel EEG
Eldele, E., Chen, Z., Liu, C., Wu, M., Kwoh, C. K., Li, X., & Guan, C. (2021) · IEEE Transactions on Neural Systems and Rehabilitation Engineering · doi:10.1109/tnsre.2021.3076234 · Cited by 741
A Deep CNN-LSTM Model for Particulate Matter (PM2.5) Forecasting in Smart Cities
Huang, C., & Kuo, P. (2018) · Sensors · doi:10.3390/s18072220 · Cited by 727
Zhu, Y., & Zabaras, N. (2018) · Journal of Computational Physics · doi:10.1016/j.jcp.2018.04.018 · Cited by 704
A Comprehensive Overhaul of Feature Distillation
Heo, B., Kim, J., Yun, S., Park, H., Kwak, N., & Choi, J. Y. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00201 · Cited by 706
Semi-Supervised Domain Adaptation via Minimax Entropy
Saito, K., Kim, D., Sclaroff, S., Darrell, T., & Saenko, K. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00814 · Cited by 705
Progressive Differentiable Architecture Search: Bridging the Depth Gap Between Search and Evaluation
Chen, X., Xie, L., Wu, J. H., & Tian, Q. C. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00138 · Cited by 731
Direct Training for Spiking Neural Networks: Faster, Larger, Better
Wu, Y., Deng, L., Li, G., Zhu, J., Xie, Y., & Shi, L. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33011311 · Cited by 721
Adversarial Attacks and Defenses in Images, Graphs and Text: A Review
Xu, H., Ma, Y., Liu, H., Deb, D., Liu, H., Tang, J., & Jain, A. K. (2020) · Machine Intelligence Research · doi:10.1007/s11633-019-1211-x · Cited by 756
PACT: Parameterized Clipping Activation for Quantized Neural Networks
Choi, J., Wang, Z., Venkataramani, S., Chuang, P. I.-J., Srinivasan, V., & Gopalakrishnan, K. (2018) · arXiv · doi:10.48550/arxiv.1805.06085 · Cited by 703
Automatic Multi-Organ Segmentation on Abdominal CT With Dense V-Networks
Gibson, E., Giganti, F., Hu, Y., Bonmati, E., Bandula, S., Gurusamy, K. S., Davidson, B. R., Pereira, S. P., Clarkson, M. J., & Barratt, D. C. (2018) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2018.2806309 · Cited by 701
Deep Learning for Classification of Hyperspectral Data: A Comparative Review
Audebert, N., Saux, B. L., & Lefèvre, S. (2019) · IEEE Geoscience and Remote Sensing Magazine · doi:10.1109/mgrs.2019.2912563 · Cited by 700
Multisource Remote Sensing Data Classification Based on Convolutional Neural Network
Xu, X., Li, W., Ran, Q., Du, Q., Gao, L., & Zhang, B. (2017) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2017.2756851 · Cited by 699
TuckER: Tensor Factorization for Knowledge Graph Completion
Balazevic, I., Allen, C., & Hospedales, T. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1522 · Cited by 696
HINet: Half Instance Normalization Network for Image Restoration
Chen, L., Lu, X., Zhang, J., Chu, X., & Chen, C. (2021) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw53098.2021.00027 · Cited by 698
MFNet: Towards real-time semantic segmentation for autonomous vehicles with multi-spectral scenes
Ha, Q., Watanabe, K., Karasawa, T., Ushiku, Y., & Harada, T. (2017) · 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2017.8206396 · Cited by 693
Deep Pyramidal Residual Networks
Han, D., Kim, J., & Kim, J. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.668 · Cited by 715
Spatiotemporal Multiplier Networks for Video Action Recognition
Feichtenhofer, C., Pinz, A. J., & Wildes, R. P. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.787 · Cited by 692
Xu, C., Lopez, R., Mehlman, E., Regier, J., Jordan, M. I., & Yosef, N. (2021) · Molecular Systems Biology · doi:10.15252/msb.20209620 · Cited by 691
“We use deterministic warmup [67] and batch normalization [68] in order to learn an expressive model.”
MethodAttention-guided CNN for image denoising
Tian, C., Xu, Y., Li, Z., Zuo, W., Fei, L., & Liu, H. (2020) · Neural Networks · doi:10.1016/j.neunet.2019.12.024 · Cited by 688
Few-Shot Adversarial Learning of Realistic Neural Talking Head Models
Zakharov, E., Shysheya, A., Burkov, E., & Lempitsky, V. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00955 · Cited by 687
Automatic skin lesion segmentation using deep fully convolutional networks with jaccard distance.
Yuan, Y., Chao, M., & Lo, Y.-C. (2017) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2017.2695227 · Cited by 687
A review on deep convolutional neural networks
Aloysius, N., & Geetha, M. K. (2017) · International Conference on Communications and Signal Processing (ICCSP) · doi:10.1109/iccsp.2017.8286426 · Cited by 723
A Strong Baseline and Batch Normalization Neck for Deep Person Re-Identification
Luo, H., Jiang, W., Gu, Y., Liu, F., Liao, X., Lai, S., & Gu, J. (2019) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2019.2958756 · Cited by 702
Wireless Networks Design in the Era of Deep Learning: Model-Based, AI-Based, or Both?
Zappone, A., Renzo, M. D., & Debbah, M. A. (2019) · IEEE Transactions on Communications · doi:10.1109/tcomm.2019.2924010 · Cited by 703
Semantics-Guided Neural Networks for Efficient Skeleton-Based Human Action Recognition
Zhang, P., Lan, C., Zeng, W., Xing, J., Xue, J., & Zheng, N. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00119 · Cited by 692
Deep Feature Flow for Video Recognition
Zhu, X., Xiong, Y., Dai, J., Yuan, L., & Wei, Y. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.441 · Cited by 695
A survey and critique of multiagent deep reinforcement learning
Hernández-Leal, P., Kartal, B., & Taylor, M. E. (2019) · Autonomous Agents and Multi-Agent Systems · doi:10.1007/s10458-019-09421-1 · Cited by 799
Metadata
- Authors
- Ioffe, S., & Szegedy, C.
- Affiliations
- Google (United States) (US)
- ORCID
- Szegedy, Christian
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2015
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1502.03167
- OpenAlex
- W1836465849
- Cited by
- 27,818
- References
- 23
- Citations by year
- 2026: 152 · 2025: 1317 · 2024: 2504 · 2023: 3465 · 2022: 3990 · 2021: 4698 · 2020: 4028 · 2019: 3832 · 2018: 2356 · 2017: 1153 · 2016: 273 · 2015: 43
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Stochastic Gradient Optimization Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- batch normalization, internal covariate shift, deep neural network training, training speed, learning rate, parameter initialization, regularization, dropout, image classification, ensemble learning, ImageNet
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
