The DOI matches its DataCite record; 23 of the 23 references it cites were found in a registry or an index.
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Open accessAbstract
Training Deep Neural Networks is complicated by the fact that the distribution of each layer's inputs changes during training, as the parameters of the previous layers change. This slows down the training by requiring lower learning rates and careful parameter initialization, and makes it notoriously hard to train models with saturating nonlinearities. We refer to this phenomenon as internal covariate shift, and address the problem by normalizing layer inputs. Our method draws its strength from making normalization a part of the model architecture and performing the normalization for each training mini-batch. Batch Normalization allows us to use much higher learning rates and be less careful about initialization. It also acts as a regularizer, in some cases eliminating the need for Dropout. Applied to a state-of-the-art image classification model, Batch Normalization achieves the same accuracy with 14 times fewer training steps, and beats the original model by a significant margin. Using an ensemble of batch-normalized networks, we improve upon the best published result on ImageNet classification: reaching 4.9% top-5 validation error (and 4.8% test error), exceeding the accuracy of human raters.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source doesn’t support the claim as written.
In the citing paper
“This method ensures the performance of the output activations [81]. This performance follows a unit Gaussian distribution.”
From Review of deep learning: concepts, CNN architectures, challenges, applications, future directions (2021) · cites this paper as [81]
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Since the addition of three branches introduces the covariate shift issue (Ioffe and Szegedy 2015)”
From Make RepVGG Greater Again: A Quantization-Aware Approach (2024) · cites this paper as Ioffe and Szegedy 2015
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“SGD can be applied naively to the federated optimization problem, where a single batch gradient calculation (say on a randomly selected client) is done per round of communication. This approach is computationally efficient, but requires very large numbers of rounds of training to produce good models (e.g., even using an advanced approach like batch normalization, Ioffe and Szegedy [21] trained MNI”
From Communication-Efficient Learning of Deep Networks from Decentralized Data (2016) · cites this paper as [21]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“It mitigates the internal covariate shift that occurs as the distribution of each hidden layer's inputs changes during the training (as the parameters of the preceding layers change) and slows down the learning process.”
From Empirical asset pricing via machine learning: evidence from the European stock market (2021) · cites this paper as Ioffe and Szegedy (2015)
The cited source says what the claim attributes to it.
In the citing paper
“Second, we adopt the batch normalization algorithm introduced by Ioffe and Szegedy (2015).”
From Empirical asset pricing via machine learning: evidence from the European stock market (2021) · cites this paper as Ioffe and Szegedy (2015)
The cited source says what the claim attributes to it.
In the citing paper
“It occurs throughout training, because the weight updates to deeper layers result in a continuously changing distribution of signal at higher layers, which hinders the convergence of their weights.”
From Efficient multi-scale 3D CNN with fully connected CRF for accurate brain lesion segmentation (2017) · cites this paper as Ioffe and Szegedy, 2015
The cited source says what the claim attributes to it.
In the citing paper
“Batch normalization standardizes intermediate outputs of the network to zero mean and unit variance for a batch of training examples [Ioffe and Szegedy, 2015].”
From Deep learning with convolutional neural networks for EEG decoding and visualization (2017) · cites this paper as Ioffe and Szegedy, 2015
The cited source says what the claim attributes to it.
In the citing paper
“Batch normalization is a type of supplemental layer which adaptively normalizes the input values of the following layer, mitigating the risk of overfitting, as well as improving gradient flow through the network, allowing higher learning rates, and reducing the dependence on initialization [28].”
From Convolutional neural networks: an overview and application in radiology (2018) · cites this paper as [28]
The cited source says what the claim attributes to it.
In the citing paper
“Unlike BatchNorm [Ioffe and Szegedy, 2015], commonly used in CNNs, LayerNorm cannot be folded into the previous layer due to its dynamic computational property, so we have to quantize it separately.”
From FQ-ViT: Post-Training Quantization for Fully Quantized Vision Transformer (2022) · cites this paper as [Ioffe and Szegedy, 2015]
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“However, in integer-only quantization, all the operations are performed using low-precision integer arithmetic [113,132,154,193,267], as illustrated in Figure 6 (Right).This permits the entire inference to be carried out with efficient integer arithmetic, without any floating point dequantization of any parameters or activations.”
From A Survey of Quantization Methods for Efficient Neural Network Inference (2022) · cites this paper as [113]
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“CMSIS-NN [136] is a library from ARM that helps quantizing and deploying NN models onto the ARM Cortex-M cores.Specifically, the library leverages fixed-point quantization [113,154,267] with power-oftwo scaling factors so that quantization and dequantization processes can be carried out efficiently with bit shifting operations.”
From A Survey of Quantization Methods for Efficient Neural Network Inference (2022) · cites this paper as [113]
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“as neural networks have the capability of learning directly from raw data [49].”
From Investigating Enhancements to Contrastive Predictive Coding for Human Activity Recognition (2023) · cites this paper as [49]
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Recent advances in light-weight neural network architectures [15,13,14,9,11,10,12] are mostly based on the metric of FLOPs and do not consider these properties above.”
From ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design (2018) · cites this paper as [13]
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Following the common practice [15,13,14], all networks in comparison have four levels of computational complexity, i.e. about 40, 140, 300 and 500+ MFLOPs.”
From ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design (2018) · cites this paper as [13]
Experimental — accuracy not yet measuredIn the citing paper
“Our reproduced GoogleNet has center crop top-1 error of 26.4%, close to that reported in [32] (25.2%).”
From Object Detection Networks on Convolutional Feature Maps (2017) · cites this paper as [32]
In the citing paper
“To avoid overfitting, both dropout [116] and batch normalization [33] can help to achieve better regularization.”
From Deep learning for human affect recognition: Insights and new developments (2021) · cites this paper as [33]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805
(2014) · Cited by 33,419
Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification
He, K., Zhang, X., Ren, S., & Sun, J. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.123 · Cited by 19,741
(2010) · Cited by 14,134
(2010) · International Conference on Machine Learning · Cited by 13,995
Independent component analysis: algorithms and applications
Hyvärinen, A., & Erkki Oja, A. (2000) · Neural Networks · doi:10.1016/s0893-6080(00)00026-5 · Cited by 8,926
(2010) · Cited by 8,261
On the difficulty of training recurrent neural networks
Pascanu, R., Mikolov, T., & Bengio, Y. (2012) · arXiv · doi:10.48550/arxiv.1211.5063 · Cited by 4,310
(2013) · Cited by 3,740
Large Scale Distributed Deep Networks
Dean, J. B., Corrado, G. S., Monga, R., Chen, K., Devin, M., Mao, M., Ranzato, M., Senior, A., Tucker, P. A., Yang, K., et al. (2012) · Cited by 3,202
Flagged first, then by citations
Cited by
Sorted by citations
Items 701–800 of 1,339
Context-Aware Synthesis for Video Frame Interpolation
Niklaus, S., & Liu, F. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00183 · Cited by 415
Hierarchical Recurrent Neural Encoder for Video Representation with Application to Captioning
Pan, P., Xu, Z., Yang, Y., Wu, F. Q., & Zhuang, Y. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.117 · Cited by 420
Thin MobileNet: An Enhanced MobileNet Architecture
Sinha, D., & El‐Sharkawy, M. (2019) · IEEE Annual Ubiquitous Computing, Electronics & Mobile Communication Conference (UEMCON) · doi:10.1109/uemcon47517.2019.8993089 · Cited by 436
Unsupervised Person Re-Identification via Multi-Label Classification
Wang, D., & Zhang, S. S. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.01099 · Cited by 418
FaceNet2ExpNet: Regularizing a Deep Face Recognition Net for Expression Recognition
Ding, H., Zhou, S. K., & Chellappa, R. (2017) · 2017 12th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2017) · doi:10.1109/fg.2017.23 · Cited by 416
SonoNet: Real-Time Detection and Localisation of Fetal Standard Scan Planes in Freehand Ultrasound
Baumgartner, C. F., Kamnitsas, K., Matthew, J., Fletcher, T. P., Smith, S., Koch, L. M., Kainz, B., & Rueckert, D. (2017) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2017.2712367 · Cited by 416
Deep residual learning-based fault diagnosis method for rotating machinery
Zhang, W., Li, X., & Ding, Q. (2019) · ISA Transactions · doi:10.1016/j.isatra.2018.12.025 · Cited by 416
Know Your Surroundings: Exploiting Scene Information for Object Tracking
Bhat, G., Danelljan, M., Gool, L. V., & Timofte, R. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58592-1_13 · Cited by 419
Avoiding Overfitting: A Survey on Regularization Methods for Convolutional Neural Networks
Santos, C. F. G. D., & Papa, J. P. (2022) · ACM Computing Surveys · doi:10.1145/3510413 · Cited by 415
“For example, Cutout [7] and MaxDropout [44] original codes show they do not execute anything during the inference, but the BatchNormalization [25] executes its algorithm in deducing the test set.”
BackgroundLight Gated Recurrent Units for Speech Recognition
Ravanelli, M., Brakel, P., Omologo, M., & Bengio, Y. (2018) · IEEE Transactions on Emerging Topics in Computational Intelligence · doi:10.1109/tetci.2017.2762739 · Cited by 434
Deep Hyperspectral Image Sharpening
Dian, R., Shutao Li, A., Guo, A., & Fang, L. (2018) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2018.2798162 · Cited by 414
Neural network analysis of sleep stages enables efficient diagnosis of narcolepsy
Stephansen, J. B., Olesen, A. N., Olsen, M., Ambati, A., Leary, E. B., Moore, H. E., Carrillo, O., Lin, L., Han, F., Yan, H., et al. (2018) · Nature Communications · doi:10.1038/s41467-018-07229-3 · Cited by 455
Neural Aggregation Network for Video Face Recognition
Yang, J., Ren, P., Zhang, D., Chen, D., Wen, F., Li, H., & Hua, G. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.554 · Cited by 423
Skeleton-based action recognition via spatial and temporal transformer networks
Plizzari, C., Cannici, M., & Matteucci, M. (2021) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2021.103219 · Cited by 409
Inception-v3 for flower classification
Xia, X., Cui, X., & Nan, B. (2017) · International Conference on Image, Vision and Computing (ICIVC) · doi:10.1109/icivc.2017.7984661 · Cited by 409
Temporal Generative Adversarial Nets with Singular Value Clipping
Saito, M., Matsumoto, E., & Saito, S. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.308 · Cited by 422
Appearance-and-Relation Networks for Video Classification
Wang, L., Li, W., Li, W., & Gool, L. V. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00155 · Cited by 411
Multi-Agent Reinforcement Learning: A Review of Challenges and Applications
Canese, L., Cardarilli, G. C., Nunzio, L. D., Fazzolari, R., Giardino, D., Re, M., & Spanò, S. (2021) · Applied Sciences · doi:10.3390/app11114948 · Cited by 454
H2Former: An Efficient Hierarchical Hybrid Transformer for Medical Image Segmentation
He, A., Wang, K., Li, T., Du, C., Xia, S., & Fu, H. (2023) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2023.3264513 · Cited by 436
Dynamic Fusion With Intra- and Inter-Modality Attention Flow for Visual Question Answering
Gao, P., Jiang, Z., You, H., Lu, P., Hoi, S. C. H., Wang, X., & Li, H. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00680 · Cited by 420
Camouflaged Object Detection with Feature Decomposition and Edge Reconstruction
He, C., Li, K., Zhang, Y., Tang, L., Zhang, Y., Guo, Z., & Li, X. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.02111 · Cited by 406
With a Little Help from My Friends: Nearest-Neighbor Contrastive Learning of Visual Representations
Dwibedi, D., Aytar, Y., Tompson, J., Sermanet, P., & Zisserman, A. (2021) · arXiv · doi:10.48550/arxiv.2104.14548 · Cited by 405
With a Little Help from My Friends: Nearest-Neighbor Contrastive Learning of Visual Representations
Dwibedi, D., Aytar, Y., Tompson, J., Sermanet, P., & Zisserman, A. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00945 · Cited by 404
Intelligent rotating machinery fault diagnosis based on deep learning using data augmentation
Li, X., We, Z., Ding, Q., & Sun, J. (2018) · Journal of Intelligent Manufacturing · doi:10.1007/s10845-018-1456-1 · Cited by 420
Deep Networks for Direction-of-Arrival Estimation in Low SNR
Papageorgiou, G. K., Sellathurai, M., & Eldar, Y. C. (2021) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2021.3089927 · Cited by 427
StyleSpace Analysis: Disentangled Controls for StyleGAN Image Generation
Wu, Z., Lischinski, D., & Shechtman, E. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01267 · Cited by 435
PlaNet - Photo Geolocation with Convolutional Neural Networks
Weyand, T., Kostrikov, I., & Philbin, J. F. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46484-8_3 · Cited by 439
Charting the Right Manifold: Manifold Mixup for Few-shot Learning
Mangla, P., Singh, M., Sinha, A., Kumari, N., Balasubramanian, V. N., & Krishnamurthy, B. (2020) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv45572.2020.9093338 · Cited by 402
Cao, C., Liu, X., Yan, S., Yu, Y., Wang, J., Wang, Z., Huang, Y., Wang, L., Huang, C., Xu, W., et al. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.338 · Cited by 416
A cloud detection algorithm for satellite imagery based on deep learning
Jeppesen, J. H., Jacobsen, R. H., Inceoglu, F., & Toftegaard, T. S. (2019) · Remote Sensing of Environment · doi:10.1016/j.rse.2019.03.039 · Cited by 402
Neural Machine Translation: A Review
Stahlberg, F. (2020) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.12007 · Cited by 495
Residual Networks of Residual Networks: Multilevel Residual Networks
Zhang, K., Sun, M., Han, T. X., Yuan, X., Guo, L., & Liu, T. (2017) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2017.2654543 · Cited by 413
Exploit the Unknown Gradually: One-Shot Video-Based Person Re-identification by Stepwise Learning
Wu, Y., Lin, Y., Dong, X., Yan, Y., Ouyang, W., & Yang, Y. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00543 · Cited by 401
A novel attention-based hybrid CNN-RNN architecture for sEMG-based gesture recognition
Hu, Y., Wong, Y., Wei, W., Du, Y., Kankanhalli, M., & Geng, W. (2018) · PLoS ONE · doi:10.1371/journal.pone.0206049 · Cited by 403
Early Detection of Parkinson’s Disease Using Deep Learning and Machine Learning
Wang, W., Lee, J., Harrou, F., & Sun, Y. (2020) · IEEE Access · doi:10.1109/access.2020.3016062 · Cited by 423
AASIST: Audio Anti-Spoofing Using Integrated Spectro-Temporal Graph Attention Networks
Jung, J.-W., Heo, H.-S., Tak, H., Shim, H.-J., Chung, J. S., Lee, B.-J., Yu, H.-J., & Evans, N. (2022) · ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp43922.2022.9747766 · Cited by 399
Probabilistic forecasting with temporal convolutional neural network
Chen, Y., Kang, Y., Chen, Y., & Wang, Z. (2020) · Neurocomputing · doi:10.1016/j.neucom.2020.03.011 · Cited by 416
Li, Y., Song, Y., Jia, L., Gao, S., Li, Q., & Qiu, M. (2020) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2020.3008010 · Cited by 404
Xiao, J., Fu, X., Liu, A., Wu, F. H., & Zha, Z.-J. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3183612 · Cited by 420
Sagawa, S., Koh, P. W., Hashimoto, T., & Liang, P. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.08731 · Cited by 1,178
Du, Y., & Li, F. F. (2019) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2019.2930299 · Cited by 396
A deep learning method for online capacity estimation of lithium-ion batteries
Shen, S., Sadoughi, M., Chen, X., Hong, M., & Hu, C. K. (2019) · Journal of Energy Storage · doi:10.1016/j.est.2019.100817 · Cited by 407
Hyperspectral Image Classification Using Group-Aware Hierarchical Transformer
Mei, S., Song, C., Ma, M., & Xu, F. (2022) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2022.3207933 · Cited by 416
Valverde, S., Cabezas, M., Roura, E., González-Villà, S., Pareto, D., Vilanova, J. C., Ramió‐Torrentà, L., Rovira, À., Oliver, A., & Lladó, X. (2017) · NeuroImage · doi:10.1016/j.neuroimage.2017.04.034 · Cited by 407
Particle swarm optimization of deep neural networks architectures for image classification
Fernandes, F. E., & Yen, G. G. (2019) · Swarm and Evolutionary Computation · doi:10.1016/j.swevo.2019.05.010 · Cited by 403
Hafemann, L. G., Sabourin, R., & Oliveira, L. S. (2017) · Pattern Recognition · doi:10.1016/j.patcog.2017.05.012 · Cited by 406
Object-Part Attention Model for Fine-Grained Image Classification
Peng, Y., He, X., & Zhao, J. (2017) · IEEE Transactions on Image Processing · doi:10.1109/tip.2017.2774041 · Cited by 399
Guo, J., Wen, C.-K., Jin, S., & Li, G. Y. (2020) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2020.2968430 · Cited by 401
Kraus, M., Feuerriegel, S., & Oztekin, A. (2020) · European Journal of Operational Research · doi:10.1016/j.ejor.2019.09.018 · Cited by 392
Self-Correction for Human Parsing
Li, P., Xu, Y., Wei, Y., & Yang, Y. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.3048039 · Cited by 452
Chaitanya, K., Erdil, E., Karani, N., & Konukoglu, E. (2020) · Neural Information Processing Systems · doi:10.3929/ethz-b-000443425 · Cited by 720
Learning to Read Chest X-Rays: Recurrent Neural Cascade Model for Automated Image Annotation
Shin, H.-C., Roberts, K. E., Lü, L., Demner‐Fushman, D., Yao, J., & Summers, R. M. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.274 · Cited by 401
Scale-Transferrable Object Detection
Zhou, P., Ni, B., Geng, C., Hu, J., & Xu, Y. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00062 · Cited by 391
Aad, G., Abbott, B., Abbott, D. C., Abud, A. A., Abeling, K., Abhayasinghe, D. K., Abidi, S. H., AbouZeid, O. S., Abraham, N. L., Abramowicz, H., et al. (2019) · The European Physical Journal C · doi:10.1140/epjc/s10052-019-7450-8 · Cited by 390
A Survey of Deep Convolutional Neural Networks Applied for Prediction of Plant Leaf Diseases
Dhaka, V. S., Meena, S. V., Rani, G., Sinwar, D., Kavita, K., Ijaz, M. F., & Woźniak, M. (2021) · Sensors · doi:10.3390/s21144749 · Cited by 439
Salvi, M., Acharya, U. R., Molinari, F., & Meiburger, K. M. (2020) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2020.104129 · Cited by 420
Deep Semi-Supervised Anomaly Detection
Ruff, L., Vandermeulen, R. A., Görnitz, N., Binder, A., Müller, E., Müller, K.-R., & Kloft, M. (2019) · arXiv · doi:10.48550/arxiv.1906.02694 · Cited by 388
Deep learning for load forecasting with smart meter data: Online Adaptive Recurrent Neural Network
Fekri, M. N., Patel, H., Grolinger, K., & Sharma, V. K. (2020) · Applied Energy · doi:10.1016/j.apenergy.2020.116177 · Cited by 387
Spectral–Spatial Morphological Attention Transformer for Hyperspectral Image Classification
Roy, S. K., Deria, A., Shah, C., Haut, J. M., Du, Q., & Plaza, A. (2023) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2023.3242346 · Cited by 387
High Quality Monocular Depth Estimation via Transfer LearningPreprint
Alhashim, I., & Wonka, P. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1812.11941 · Cited by 524
Lu, Y., Kumar, A., Zhai, S., Cheng, Y., Javidi, T., & Feris, R. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.126 · Cited by 386
Progressive Pose Attention Transfer for Person Image Generation
Zhu, Z., Huang, T., Shi, B., Yu, M., Wang, B., & Bai, X. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00245 · Cited by 388
He, J., Zhao, L., Yang, H., Zhang, M., & Li, W. (2019) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2019.2934760 · Cited by 395
Deep learning methods in network intrusion detection: A survey and an objective comparison
Gamage, S., & Samarabandu, J. K. (2020) · Journal of Network and Computer Applications · doi:10.1016/j.jnca.2020.102767 · Cited by 422
Semantic Segmentation of Earth Observation Data Using Multimodal and Multi-scale Deep Networks
Audebert, N., Saux, B. L., & Lefèvre, S. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-54181-5_12 · Cited by 386
An Intrusion Detection Model Based on Feature Reduction and Convolutional Neural Networks
Xiao, Y., Xing, C., Zhang, T., & Zhao, Z. (2019) · IEEE Access · doi:10.1109/access.2019.2904620 · Cited by 384
Deep Deterministic Policy Gradient (DDPG)-Based Energy Harvesting Wireless Communications
Qiu, C., Hu, Y., Chen, Y., & Zeng, B. (2019) · IEEE Internet of Things Journal · doi:10.1109/jiot.2019.2921159 · Cited by 393
TAM: Temporal Adaptive Module for Video Recognition
Liu, Z., Wang, L., Wu, W., Qian, C., & Lü, T. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01345 · Cited by 405
Empirical Asset Pricing via Machine Learning
Gu, S., Kelly, B., & Xiu, D. (2018) · National Bureau of Economic Research · doi:10.3386/w25398 · Cited by 423
Mo, S., Zhu, Y., Zabaras, N., Shi, X., & Wu, J. (2018) · Water Resources Research · doi:10.1029/2018wr023528 · Cited by 394
Variational Convolutional Neural Network Pruning
Zhao, C., Ni, B., Zhang, J., Zhao, Q., Zhang, W., & Tian, Q. C. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00289 · Cited by 379
Video Representation Learning by Dense Predictive Coding
Han, T., Xie, W., & Zisserman, A. (2019) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw.2019.00186 · Cited by 393
Rethinking Transformer-based Set Prediction for Object Detection
Sun, Z., Cao, S., Yang, Y., & Kitani, K. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00359 · Cited by 402
Pathology image analysis using segmentation deep learning algorithms
Wang, S., Yang, D. M., Rong, R., Zhan, X., & Xiao, G. (2019) · The American Journal of Pathology · doi:10.1016/j.ajpath.2019.05.007 · Cited by 382
DeepUNet: A Deep Fully Convolutional Network for Pixel-Level Sea-Land Segmentation
Li, R., Liu, W., Yang, L., Sun, S., Hu, W., Zhang, F., & Li, W. (2018) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2018.2833382 · Cited by 403
A Fast Dense Spectral–Spatial Convolution Network Framework for Hyperspectral Images Classification
Wang, W., Dou, S., Jiang, Z., & Sun, L. (2018) · Remote Sensing · doi:10.3390/rs10071068 · Cited by 381
Chang, J. Y., Moon, G., & Lee, K. M. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00533 · Cited by 400
Exploring Simple Siamese Representation Learning
Chen, X., & He, K. (2020) · arXiv · doi:10.48550/arxiv.2011.10566 · Cited by 381
Zhao, M., Kang, M., Tang, B., & Pecht, M. (2018) · IEEE Transactions on Industrial Electronics · doi:10.1109/tie.2017.2762639 · Cited by 380
Ma, Y., Zhou, G., Wang, S., Zhao, H., & Jung, W. (2018) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3191755 · Cited by 396
DC-SPP-YOLO: Dense connection and spatial pyramid pooling based YOLO for object detection
Huang, Z., Wang, J., Fu, X., Yu, T., Guo, Y., & Wang, R. (2020) · Information Sciences · doi:10.1016/j.ins.2020.02.067 · Cited by 392
MPViT: Multi-Path Vision Transformer for Dense Prediction
Lee, Y., Kim, J., Willette, J., & Hwang, S. J. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00714 · Cited by 401
Artificial Neural Networks for Neuroscientists: A Primer
Yang, G. R., & Wang, X. (2020) · Neuron · doi:10.1016/j.neuron.2020.09.005 · Cited by 411
Raju, V. N. G., Lakshmi, K. P., Jain, V. M., Kalidindi, A., & Padma, V. S. (2020) · International Conference on Smart Systems and Inventive Technology (ICSSIT) · doi:10.1109/icssit48917.2020.9214160 · Cited by 379
DeepSpectra: An end-to-end deep learning approach for quantitative spectral analysis
Zhang, X., Lin, T., Xu, J., Luo, X., & Ying, Y. B. (2019) · Analytica Chimica Acta · doi:10.1016/j.aca.2019.01.002 · Cited by 393
Moon, G., & Lee, K. M. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58571-6_44 · Cited by 398
Accurate deep neural network inference using computational phase-change memory
Joshi, V., Gallo, M. L., Haefeli, S., Boybat, I., Nandakumar, S. R., Piveteau, C., Dazzi, M., Rajendran, B., Sebastian, A., & Eleftheriou, E. S. (2020) · RePEc: Research Papers in Economics · doi:10.3929/ethz-b-000416797 · Cited by 378
Xie, L., & Yuille, A. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · Cited by 377
Khened, M., Kollerathu, V. A., & Krishnamurthi, G. (2018) · Medical Image Analysis · doi:10.1016/j.media.2018.10.004 · Cited by 387
SRM: A Style-Based Recalibration Module for Convolutional Neural Networks
Lee, H., Kim, H., & Nam, H. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00194 · Cited by 379
DyTox: Transformers for Continual Learning with DYnamic TOken eXpansion
Douillard, A., Ramé, A., Couairon, G., & Cord, M. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00907 · Cited by 411
Tinier-YOLO: A Real-Time Object Detection Method for Constrained Environments
Fang, W., Wang, L., & Ren, P. (2019) · IEEE Access · doi:10.1109/access.2019.2961959 · Cited by 385
Al‐masni, M. A., Kim, D., & Kim, T. (2020) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2020.105351 · Cited by 385
Real-time detection of steel strip surface defects based on improved yolo detection network
Li, J., Su, Z., Geng, J., & Yin, Y. (2018) · IFAC-PapersOnLine · doi:10.1016/j.ifacol.2018.09.412 · Cited by 371
A review on modern defect detection models using DCNNs – Deep convolutional neural networks
Tulbure, A.-A., Tulbure, A.-A., & Dulf, E.-H. (2022) · Journal of Advanced Research · doi:10.1016/j.jare.2021.03.015 · Cited by 372
Very deep convolutional neural networks for raw waveforms
Dai, W., Dai, C., Qu, S., Li, J., & Das, S. (2017) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2017.7952190 · Cited by 370
Imbalanced Deep Learning by Minority Class Incremental Rectification
Dong, Q., Gong, S., & Zhu, X. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2832629 · Cited by 378
Predicting protein–protein interactions through sequence-based deep learning
Hashemifar, S. S., Neyshabur, B., Khan, A. A., & Xu, J. (2018) · Bioinformatics · doi:10.1093/bioinformatics/bty573 · Cited by 378
A deep convolutional neural network model for rapid prediction of fluvial flood inundation
Kabir, S., Patidar, S., Xia, X., Liang, Q., Neal, J. C., & Pender, G. (2020) · Journal of Hydrology · doi:10.1016/j.jhydrol.2020.125481 · Cited by 387
Efficient Deep Network Architectures for Fast Chest X-Ray Tuberculosis Screening and Visualization
Pasa, F., Golkov, V., Pfeiffer, F., Cremers, D., & Pfeiffer, D. (2019) · Scientific Reports · doi:10.1038/s41598-019-42557-4 · Cited by 372
Metadata
- Authors
- Ioffe, S., & Szegedy, C.
- Affiliations
- Google (United States) (US)
- ORCID
- Szegedy, Christian
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2015
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1502.03167
- OpenAlex
- W1836465849
- Cited by
- 27,818
- References
- 23
- Citations by year
- 2026: 152 · 2025: 1317 · 2024: 2504 · 2023: 3465 · 2022: 3990 · 2021: 4698 · 2020: 4028 · 2019: 3832 · 2018: 2356 · 2017: 1153 · 2016: 273 · 2015: 43
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Stochastic Gradient Optimization Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- batch normalization, internal covariate shift, deep neural network training, training speed, learning rate, parameter initialization, regularization, dropout, image classification, ensemble learning, ImageNet
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
