The DOI matches its DataCite record; 23 of the 23 references it cites were found in a registry or an index.
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Open accessAbstract
Training Deep Neural Networks is complicated by the fact that the distribution of each layer's inputs changes during training, as the parameters of the previous layers change. This slows down the training by requiring lower learning rates and careful parameter initialization, and makes it notoriously hard to train models with saturating nonlinearities. We refer to this phenomenon as internal covariate shift, and address the problem by normalizing layer inputs. Our method draws its strength from making normalization a part of the model architecture and performing the normalization for each training mini-batch. Batch Normalization allows us to use much higher learning rates and be less careful about initialization. It also acts as a regularizer, in some cases eliminating the need for Dropout. Applied to a state-of-the-art image classification model, Batch Normalization achieves the same accuracy with 14 times fewer training steps, and beats the original model by a significant margin. Using an ensemble of batch-normalized networks, we improve upon the best published result on ImageNet classification: reaching 4.9% top-5 validation error (and 4.8% test error), exceeding the accuracy of human raters.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source doesn’t support the claim as written.
In the citing paper
“This method ensures the performance of the output activations [81]. This performance follows a unit Gaussian distribution.”
From Review of deep learning: concepts, CNN architectures, challenges, applications, future directions (2021) · cites this paper as [81]
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Since the addition of three branches introduces the covariate shift issue (Ioffe and Szegedy 2015)”
From Make RepVGG Greater Again: A Quantization-Aware Approach (2024) · cites this paper as Ioffe and Szegedy 2015
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“SGD can be applied naively to the federated optimization problem, where a single batch gradient calculation (say on a randomly selected client) is done per round of communication. This approach is computationally efficient, but requires very large numbers of rounds of training to produce good models (e.g., even using an advanced approach like batch normalization, Ioffe and Szegedy [21] trained MNI”
From Communication-Efficient Learning of Deep Networks from Decentralized Data (2016) · cites this paper as [21]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“It mitigates the internal covariate shift that occurs as the distribution of each hidden layer's inputs changes during the training (as the parameters of the preceding layers change) and slows down the learning process.”
From Empirical asset pricing via machine learning: evidence from the European stock market (2021) · cites this paper as Ioffe and Szegedy (2015)
The cited source says what the claim attributes to it.
In the citing paper
“Second, we adopt the batch normalization algorithm introduced by Ioffe and Szegedy (2015).”
From Empirical asset pricing via machine learning: evidence from the European stock market (2021) · cites this paper as Ioffe and Szegedy (2015)
The cited source says what the claim attributes to it.
In the citing paper
“It occurs throughout training, because the weight updates to deeper layers result in a continuously changing distribution of signal at higher layers, which hinders the convergence of their weights.”
From Efficient multi-scale 3D CNN with fully connected CRF for accurate brain lesion segmentation (2017) · cites this paper as Ioffe and Szegedy, 2015
The cited source says what the claim attributes to it.
In the citing paper
“Batch normalization standardizes intermediate outputs of the network to zero mean and unit variance for a batch of training examples [Ioffe and Szegedy, 2015].”
From Deep learning with convolutional neural networks for EEG decoding and visualization (2017) · cites this paper as Ioffe and Szegedy, 2015
The cited source says what the claim attributes to it.
In the citing paper
“Batch normalization is a type of supplemental layer which adaptively normalizes the input values of the following layer, mitigating the risk of overfitting, as well as improving gradient flow through the network, allowing higher learning rates, and reducing the dependence on initialization [28].”
From Convolutional neural networks: an overview and application in radiology (2018) · cites this paper as [28]
The cited source says what the claim attributes to it.
In the citing paper
“Unlike BatchNorm [Ioffe and Szegedy, 2015], commonly used in CNNs, LayerNorm cannot be folded into the previous layer due to its dynamic computational property, so we have to quantize it separately.”
From FQ-ViT: Post-Training Quantization for Fully Quantized Vision Transformer (2022) · cites this paper as [Ioffe and Szegedy, 2015]
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“However, in integer-only quantization, all the operations are performed using low-precision integer arithmetic [113,132,154,193,267], as illustrated in Figure 6 (Right).This permits the entire inference to be carried out with efficient integer arithmetic, without any floating point dequantization of any parameters or activations.”
From A Survey of Quantization Methods for Efficient Neural Network Inference (2022) · cites this paper as [113]
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“CMSIS-NN [136] is a library from ARM that helps quantizing and deploying NN models onto the ARM Cortex-M cores.Specifically, the library leverages fixed-point quantization [113,154,267] with power-oftwo scaling factors so that quantization and dequantization processes can be carried out efficiently with bit shifting operations.”
From A Survey of Quantization Methods for Efficient Neural Network Inference (2022) · cites this paper as [113]
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“as neural networks have the capability of learning directly from raw data [49].”
From Investigating Enhancements to Contrastive Predictive Coding for Human Activity Recognition (2023) · cites this paper as [49]
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Recent advances in light-weight neural network architectures [15,13,14,9,11,10,12] are mostly based on the metric of FLOPs and do not consider these properties above.”
From ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design (2018) · cites this paper as [13]
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Following the common practice [15,13,14], all networks in comparison have four levels of computational complexity, i.e. about 40, 140, 300 and 500+ MFLOPs.”
From ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design (2018) · cites this paper as [13]
Experimental — accuracy not yet measuredIn the citing paper
“Our reproduced GoogleNet has center crop top-1 error of 26.4%, close to that reported in [32] (25.2%).”
From Object Detection Networks on Convolutional Feature Maps (2017) · cites this paper as [32]
In the citing paper
“To avoid overfitting, both dropout [116] and batch normalization [33] can help to achieve better regularization.”
From Deep learning for human affect recognition: Insights and new developments (2021) · cites this paper as [33]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805
(2014) · Cited by 33,419
Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification
He, K., Zhang, X., Ren, S., & Sun, J. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.123 · Cited by 19,741
(2010) · Cited by 14,134
(2010) · International Conference on Machine Learning · Cited by 13,995
Independent component analysis: algorithms and applications
Hyvärinen, A., & Erkki Oja, A. (2000) · Neural Networks · doi:10.1016/s0893-6080(00)00026-5 · Cited by 8,926
(2010) · Cited by 8,261
On the difficulty of training recurrent neural networks
Pascanu, R., Mikolov, T., & Bengio, Y. (2012) · arXiv · doi:10.48550/arxiv.1211.5063 · Cited by 4,310
(2013) · Cited by 3,740
Large Scale Distributed Deep Networks
Dean, J. B., Corrado, G. S., Monga, R., Chen, K., Devin, M., Mao, M., Ranzato, M., Senior, A., Tucker, P. A., Yang, K., et al. (2012) · Cited by 3,202
Flagged first, then by citations
Cited by
Sorted by citations
Items 1,201–1,300 of 1,339
FPGA Implementation for CNN-Based Optical Remote Sensing Object Detection
Zhang, N., Wei, X., Chen, H., & Liu, W. (2021) · Electronics · doi:10.3390/electronics10030282 · Cited by 85
Traffic Scene Segmentation Based on RGB-D Image and Deep Learning
Li, L., Qian, B., Lian, J., Zheng, W., & Zhou, Y. (2018) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2017.2724138 · Cited by 77
Automatic Annotation of Hip Anatomy in Fluoroscopy for Robust and Efficient 2D/3D Registration
Grupp, R. B., Unberath, M., Gao, C., Hegeman, R. A., Murphy, R. J., Alexander, C. P., Otake, Y., McArthur, B. A., Armand, M., & Taylor, R. H. (2020) · International Journal of Computer Assisted Radiology and Surgery · doi:10.1007/s11548-020-02162-7 · Cited by 83
QuipuNet: Convolutional Neural Network for Single-Molecule Nanopore Sensing
Misiunas, K., Ermann, N., & Keyser, U. F. (2018) · Nano Letters · doi:10.1021/acs.nanolett.8b01709 · Cited by 82
ChannelNets: Compact and Efficient Convolutional Neural Networks via Channel-Wise Convolutions
Gao, H., Wang, Z., Cai, L., & Ji, S. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.2975796 · Cited by 81
Qi, X., de Almeida, G. A. M., & Maldonado, S. (2024) · Journal of Hydrology · doi:10.1016/j.jhydrol.2024.131263 · Cited by 78
Hierarchical Recurrent Attention Networks for Structured Online Maps
Homayounfar, N., Ma, W.-C., Lakshmikanth, S. K., & Urtasun, R. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00360 · Cited by 71
Lost data neural semantic recovery framework for structural health monitoring based on deep learning
Jiang, K., Han, Q., & Du, X. (2022) · Computer-Aided Civil and Infrastructure Engineering · doi:10.1111/mice.12850 · Cited by 77
Deep neural networks predicting oil movement in a development unit
Temirchev, P., Simonov, M., Kostoev, R., Burnaev, E., Oseledets, I., Akhmetov, A., Margarit, A., Sitnikov, A., & Koroteev, D. (2020) · Journal of Petroleum Science and Engineering · doi:10.1016/j.petrol.2019.106513 · Cited by 75
Li, S., Luo, Y., Sun, K., Yadav, N., & Choi, K. K. (2020) · IEEE Access · doi:10.1109/access.2020.3000009 · Cited by 75
A Disentangling Invertible Interpretation Network for Explaining Latent Representations
Esser, P., Rombach, R., & Ommer, B. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00924 · Cited by 72
An Energy-Efficient Digital ReRAM-Crossbar-Based CNN With Bitwise Parallelism
Ni, L., Liu, Z., Yu, H., & Joshi, R. V. (2017) · IEEE Journal on Exploratory Solid-State Computational Devices and Circuits · doi:10.1109/jxcdc.2017.2697910 · Cited by 74
Liao, Y., Qing, X., Wang, Y., & Zhang, F. (2023) · Composite Structures · doi:10.1016/j.compstruct.2023.116871 · Cited by 73
A Multi-modal Convolutional Neural Network Framework for the Prediction of Alzheimer’s Disease
Spasov, S. E., Passamonti, L., Duggento, A., Lio, P., & Toschi, N. (2018) · 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) · doi:10.1109/embc.2018.8512468 · Cited by 73
NTIRE 2023 Challenge on Efficient Super-Resolution: Methods and Results
Li, Y., Zhang, Y., Timofte, R., Van Gool, L., Yu, L., Li, Y., Li, X., Jiang, T., Wu, Q., Han, M., et al. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw59228.2023.00189 · Cited by 72
Graph Attention Networks for Anti-Spoofing
Tak, H., Jung, J.-W., Patino, J., Todisco, M., & Evans, N. (2021) · Interspeech 2021 · doi:10.21437/interspeech.2021-993 · Cited by 79
CondenseNet V2: Sparse Feature Reactivation for Deep Networks
Yang, L., Jiang, H., Cai, R., Wang, Y., Song, S., Huang, G., & Tian, Q. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00357 · Cited by 71
Prototype Propagation Networks (PPN) for Weakly-supervised Few-shot Learning on Category Graph
Liu, L., Zhou, T., Long, G., Jiang, J., Yao, L., & Zhang, C. (2019) · Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2019/418 · Cited by 65
Autoregressive Unsupervised Image Segmentation
Ouali, Y., Hudelot, C., & Tami, M. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58571-6_9 · Cited by 75
Mathur, A., Zhang, T., Bhattacharya, S., Velickovic, P., Joffe, L., Lane, N. D., Kawsar, F., & Lio, P. (2018) · 2018 17th ACM/IEEE International Conference on Information Processing in Sensor Networks (IPSN) · doi:10.1109/ipsn.2018.00048 · Cited by 70
Multi-Modality Multi-Task Recurrent Neural Network for Online Action Detection
Liu, J., Li, Y., Song, S., Xing, J., Lan, C., & Zeng, W. (2019) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2018.2799968 · Cited by 62
Integrating uncertainty in deep neural networks for MRI based stroke analysis
Herzog, L., Murina, E., Dürr, O., Wegener, S., & Sick, B. (2020) · Medical Image Analysis · doi:10.1016/j.media.2020.101790 · Cited by 66
Yoo, Y., Tang, L. Y. W., Li, D. K. B., Metz, L., Kolind, S., Traboulsee, A. L., & Tam, R. C. (2019) · Computer Methods in Biomechanics and Biomedical Engineering: Imaging & Visualization · doi:10.1080/21681163.2017.1356750 · Cited by 67
Learning Deep Multimodal Feature Representation with Asymmetric Multi-layer Fusion
Wang, Y., Sun, F., Lu, M., & Yao, A. (2020) · Proceedings of the 28th ACM International Conference on Multimedia · doi:10.1145/3394171.3413621 · Cited by 68
Combining deep learning with 3D stereophotogrammetry for craniosynostosis diagnosis
de Jong, G., Bijlsma, E., Meulstee, J., Wennen, M., van Lindert, E., Maal, T., Aquarius, R., & Delye, H. (2020) · Scientific Reports · doi:10.1038/s41598-020-72143-y · Cited by 62
Wang, J., Li, S., Han, B., An, Z., Xin, Y., Qian, W., & Wu, Q. (2019) · Measurement Science and Technology · doi:10.1088/1361-6501/aaf319 · Cited by 61
Empirical asset pricing via machine learning: evidence from the European stock market
Drobetz, W., & Otto, T. (2021) · Journal of Asset Management · doi:10.1057/s41260-021-00237-x · Cited by 61
Robust Multimodal Learning with Missing Modalities via Parameter-Efficient Adaptation
Reza, M. K., Prater-Bennette, A., & Asif, M. S. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3476487 · Cited by 78
Theory In, Theory Out: The Uses of Social Theory in Machine Learning for Social Science
Radford, J., & Joseph, K. (2020) · Frontiers in Big Data · doi:10.3389/fdata.2020.00018 · Cited by 58
Liao, Z., Girgis, H., Abdi, A., Vaseli, H., Hetherington, J., Rohling, R., Gin, K., Tsang, T., & Abolmaesumi, P. (2020) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2019.2959209 · Cited by 56
Masoudnia, S., Mersa, O., Araabi, B. N., Vahabie, A.-H., Sadeghi, M. A., & Ahmadabadi, M. N. (2019) · Expert Systems with Applications · doi:10.1016/j.eswa.2019.03.040 · Cited by 57
Li, J., Wu, G., Liao, T., Fan, M., Mao, X., & Pedrycz, W. (2023) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2022.3233358 · Cited by 61
TOW-IDS: Intrusion detection system based on three overlapped wavelets for automotive ethernet
Han, M. L., Kwak, B. I., & Kim, H. K. (2023) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2022.3221893 · Cited by 58
Inference-Time Intervention: Eliciting Truthful Answers from a Language Model
Li, K., Patel, O., Viégas, F., Pfister, H., & Wattenberg, M. (2023) · Advances in Neural Information Processing Systems 36 · doi:10.52202/075280-1797 · Cited by 117
“The latter connects to the whitening and coloring transformation commonly used in deep learning (Ioffe and Szegedy, 2015; Huang and Belongie, 2017).”
BackgroundDNN-based Emotion Recognition Based on Bottleneck Acoustic Features and Lexical Features
Kim, E., & Shin, J. W. (2019) · ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2019.8683077 · Cited by 54
Interpreting the retinal neural code for natural scenes: from computations to neurons
Maheswaranathan, N., McIntosh, L. T., Tanaka, H., Grant, S., Kastner, D. B., Melander, J. B., Nayebi, A., Brezovec, L. E., Wang, J. H., Ganguli, S., et al. (2023) · Neuron · doi:10.1016/j.neuron.2023.06.007 · Cited by 52
Lessons Learned from Designing an AI-Enabled Diagnosis Tool for Pathologists
Gu, H., Huang, J., Hung, L., & Chen, X. '. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3449084 · Cited by 56
Character-Level Language Modeling with Deeper Self-Attention
Al-Rfou, R., Choe, D., Constant, N., Guo, M., & Jones, L. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33013159 · Cited by 56
Evolving the olfactory system with machine learning
Wang, P. Y., Sun, Y., Axel, R., Abbott, L. F., & Yang, G. R. (2021) · Neuron · doi:10.1016/j.neuron.2021.09.010 · Cited by 54
Petascale neural circuit reconstruction: automated methods
Macrina, T., Lee, K., Lu, R., Turner, N. L., Wu, J., Popovych, S., Silversmith, W., Kemnitz, N., Bae, J. A., Castro, M. A., et al. (2021) · doi:10.1101/2021.08.04.455162 · Cited by 55
Mapping model units to visual neurons reveals population code for social behaviour
Cowley, B. R., Calhoun, A. J., Rangarajan, N., Ireland, E., Turner, M. H., Pillow, J. W., & Murthy, M. (2024) · Nature · doi:10.1038/s41586-024-07451-8 · Cited by 55
A systematic deep learning model selection for p300-based brain-computer interfaces
Abibullaev, B., & Zollanvari, A. (2022) · IEEE Transactions on Systems Man and Cybernetics Systems · doi:10.1109/tsmc.2021.3051136 · Cited by 53
Artificial intelligence and its potential in oncology
Londhe, V. Y., & Bhasin, B. (2019) · Drug Discovery Today · doi:10.1016/j.drudis.2018.10.005 · Cited by 53
Chen, W., Gao, Y., Gao, L., & Li, X. (2018) · Procedia CIRP · doi:10.1016/j.procir.2018.03.264 · Cited by 56
Hoai Thu, N. T., & Han, D. S. (2021) · IEEE Access · doi:10.1109/access.2021.3122298 · Cited by 54
Make RepVGG Greater Again: A Quantization-Aware Approach
Chu, X., Li, L., & Zhang, B. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i10.29045 · Cited by 53
Deep Self-Supervised Representation Learning for Free-Hand Sketch
Xu, P., Song, Z., Yin, Q., Song, Y.-Z., & Wang, L. (2021) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2020.3003048 · Cited by 53
Multi-Modal Sensor Fusion-Based Semantic Segmentation for Snow Driving Scenarios
Vachmanus, S., Ravankar, A. A., Emaru, T., & Kobayashi, Y. (2021) · IEEE Sensors Journal · doi:10.1109/jsen.2021.3077029 · Cited by 53
Expression Recognition Method Based on a Lightweight Convolutional Neural Network
Zhao, G., Yang, H., & Yu, M. (2020) · IEEE Access · doi:10.1109/access.2020.2964752 · Cited by 53
Learning to Set Waypoints for Audio-Visual Navigation
Chen, C., Majumder, S., Al-Halah, Z., Gao, R., Ramakrishnan, S. K., & Grauman, K. (2020) · arXiv · doi:10.48550/arxiv.2008.09622 · Cited by 53
Object Detection Based on Lightweight YOLOX for Autonomous Driving
He, Q., xu, A., Ye, Z., Zhou, W., & Cai, T. (2023) · doi:10.21203/rs.3.rs-3053457/v1 · Cited by 53
A bioelectric neural interface towards intuitive prosthetic control for amputees
Nguyen, A. T., Xu, J., Jiang, M., Luu, D. K., Wu, T., Tam, W.-K., Zhao, W., Drealan, M. W., Overstreet, C. K., Zhao, Q., et al. (2020) · Journal of Neural Engineering · doi:10.1088/1741-2552/abc3d3 · Cited by 52
Mohlman, J. S., Leventhal, S. D., Hansen, T., Kohan, J., Pascucci, V., & Salama, M. E. (2020) · American Journal of Clinical Pathology · doi:10.1093/ajcp/aqaa001 · Cited by 50
A Dynamic Deep Neural Network Design for Efficient Workload Allocation in Edge Computing
Lo, C., Su, Y.-Y., Lee, C.-Y., & Chang, S.-C. (2017) · 2017 IEEE International Conference on Computer Design (ICCD) · doi:10.1109/iccd.2017.49 · Cited by 48
Visual Scene Understanding for Autonomous Driving Using Semantic Segmentation
Hofmarcher, M., Unterthiner, T., Arjona-Medina, J., Klambauer, G., Hochreiter, S., & Nessler, B. (2019) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-28954-6_15 · Cited by 51
Convolutional neural network and long short-term memory models for ice-jam predictions
Madaeni, F., Chokmani, K., Lhissou, R., Homayouni, S., Gauthier, Y., & Tolszczuk-Leclerc, S. (2022) · The Cryosphere · doi:10.5194/tc-16-1447-2022 · Cited by 51
Learning From Synthetic Photorealistic Raindrop for Single Image Raindrop Removal
Hao, Z., You, S., Li, Y., Li, K., & Lu, F. (2019) · 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW) · doi:10.1109/iccvw.2019.00534 · Cited by 54
High Speed and High Dynamic Range Video with an Event Camera
Rebecq, H., Ranftl, R., Koltun, V., & Scaramuzza, D. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2963386 · Cited by 50
Wu, B., Zhao, S., Sun, G., Zhang, X., Su, Z., Zeng, C., & Liu, Z. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00220 · Cited by 44
Enhancing Reliability of Analog Neural Network Processors
Moon, S., Shin, K., & Jeon, D. (2019) · IEEE Transactions on Very Large Scale Integration (VLSI) Systems · doi:10.1109/tvlsi.2019.2893256 · Cited by 45
Automated 3D ultrasound image analysis for first trimester assessment of fetal health
Ryou, H., Yaqub, M., Cavallaro, A., Papageorghiou, A. T., & Alison Noble, J. (2019) · Physics in Medicine & Biology · doi:10.1088/1361-6560/ab3ad1 · Cited by 44
Query-Key Normalization for Transformers
Henry, A., Dachapally, P. R., Pawar, S. S., & Chen, Y. (2020) · Findings of the Association for Computational Linguistics: EMNLP 2020 · doi:10.18653/v1/2020.findings-emnlp.379 · Cited by 45
Multimodal and Multi-view Models for Emotion Recognition
Aguilar, G., Rozgic, V., Wang, W., & Wang, C. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1095 · Cited by 45
The Robustness of Modern Deep Learning Architectures against Single Event Upset Errors
Arechiga, A. P., & Michaels, A. J. (2018) · 2018 IEEE High Performance extreme Computing Conference (HPEC) · doi:10.1109/hpec.2018.8547532 · Cited by 47
Kwon, I., Wang, S.-G., Shin, S.-C., Cheon, Y.-I., Lee, B.-J., Lee, J.-C., Lim, D.-W., Jo, C., Cho, Y., & Shin, B.-J. (2025) · Journal of Voice · doi:10.1016/j.jvoice.2022.07.007 · Cited by 43
Modeling multimodal cues in a deep learning-based framework for emotion recognition in the wild
Pini, S., Ahmed, O. B., Cornia, M., Baraldi, L., Cucchiara, R., & Huet, B. (2017) · Proceedings of the 19th ACM International Conference on Multimodal Interaction · doi:10.1145/3136755.3143006 · Cited by 44
Automated ultrasound assessment of amniotic fluid index using deep learning
Cho, H. C., Sun, S., Min Hyun, C., Kwon, J.-Y., Kim, B., Park, Y., & Seo, J. K. (2021) · Medical Image Analysis · doi:10.1016/j.media.2020.101951 · Cited by 43
Subcortical segmentation of the fetal brain in 3D ultrasound using deep learning
Hesse, L. S., Aliasi, M., Moser, F., Haak, M. C., Xie, W., Jenkinson, M., & Namburete, A. I. L. (2022) · NeuroImage · doi:10.1016/j.neuroimage.2022.119117 · Cited by 44
Deep learning in spatial transcriptomics: Learning from the next next-generation sequencing
Heydari, A. A., & Sindi, S. S. (2023) · Biophysics Reviews · doi:10.1063/5.0091135 · Cited by 44
Crosstalk Removal in Forward Scan Sonar Image Using Deep Learning for Object Detection
Sung, M., Cho, H., Kim, T., Joe, H., & Yu, S.-C. (2019) · IEEE Sensors Journal · doi:10.1109/jsen.2019.2925830 · Cited by 43
RLDS: An explainable residual learning diagnosis system for fetal congenital heart disease
Qiao, S., Pang, S., Luo, G., Pan, S., Yu, Z., Chen, T., & Lv, Z. (2022) · Future Generation Computer Systems · doi:10.1016/j.future.2021.10.001 · Cited by 42
ScaleNet: Guiding Object Proposal Generation in Supermarkets and Beyond
Qiao, S., Shen, W., Qiu, W., Liu, C., & Yuille, A. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.199 · Cited by 41
Roggeveen, L., el Hassouni, A., Ahrendt, J., Guo, T., Fleuren, L., Thoral, P., Girbes, A. R., Hoogendoorn, M., & Elbers, P. W. (2021) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2020.102003 · Cited by 41
Recognition of persisting emotional valence from EEG using convolutional neural networks
Yanagimoto, M., & Sugimoto, C. (2016) · 2016 IEEE 9th International Workshop on Computational Intelligence and Applications (IWCIA) · doi:10.1109/iwcia.2016.7805744 · Cited by 43
Passive mobile sensing and psychological traits for large scale mood prediction
Spathis, D., Servia-Rodriguez, S., Farrahi, K., Mascolo, C., & Rentfrow, J. (2019) · Proceedings of the 13th EAI International Conference on Pervasive Computing Technologies for Healthcare · doi:10.1145/3329189.3329213 · Cited by 40
Learning Weight Uncertainty with Stochastic Gradient MCMC for Shape Classification
Li, C., Stevens, A., Chen, C., Pu, Y., Gan, Z., & Carin, L. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.611 · Cited by 39
Self-Supervised Wasserstein Pseudo-Labeling for Semi-Supervised Image Classification
Taherkhani, F., Dabouei, A., Soleymani, S., Dawson, J., & Nasrabadi, N. M. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01209 · Cited by 45
Forecasting Energy Consumption of a Public Building Using Transformer and Support Vector Regression
Huang, J., & Kaewunruen, S. (2023) · Energies · doi:10.3390/en16020966 · Cited by 38
EEG-based emotion recognition with autoencoder feature fusion and MSC-TimesNet model
Yin, J., Qiao, Z., Han, L., & Zhang, X. (2026) · Computer Methods in Biomechanics and Biomedical Engineering · doi:10.1080/10255842.2025.2477801 · Cited by 41
An Empirical Evaluation on Robustness and Uncertainty of Regularization Methods
Chun, S., Oh, S. J., Yun, S., Han, D., Choe, J., & Yoo, Y. (2020) · arXiv · doi:10.48550/arxiv.2003.03879 · Cited by 37
Borra, D., Fantozzi, S., & Magosso, E. (2021) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2021.655840 · Cited by 36
Predicting modular functions and neural coding of behavior from a synaptic wiring diagram
Vishwanathan, A., Sood, A., Wu, J., Ramirez, A. D., Yang, R., Kemnitz, N., Ih, D., Turner, N., Lee, K., Tartavull, I., et al. (2024) · Nature Neuroscience · doi:10.1038/s41593-024-01784-3 · Cited by 36
An ADC-Less RRAM-Based Computing-in-Memory Macro With Binary CNN for Efficient Edge AI
Li, Y., Chen, J., Wang, L., Zhang, W., Guo, Z., Wang, J., Han, Y., Li, Z., Wang, F., Dou, C., et al. (2023) · IEEE Transactions on Circuits and Systems II: Express Briefs · doi:10.1109/tcsii.2022.3233396 · Cited by 38
IENet: a robust convolutional neural network for EEG based brain-computer interfaces
Du, Y., & Liu, J. (2022) · Journal of Neural Engineering · doi:10.1088/1741-2552/ac7257 · Cited by 34
“We adopt batch normalization [67] as the regularization method in IE module.”
MethodCai, L., Huang, S., Zhang, Y., Lu, J., & Zhang, Y. (2025) · Medical Image Analysis · doi:10.1016/j.media.2025.103631 · Cited by 33
Recognition of facial expression of fetuses by artificial intelligence (AI)
Miyagi, Y., Hata, T., Bouno, S., Koyanagi, A., & Miyake, T. (2021) · Journal of Perinatal Medicine · doi:10.1515/jpm-2020-0537 · Cited by 34
A Generalization Performance Study Using Deep Learning Networks in Embedded Systems
Gorospe, J., Mulero, R., Arbelaitz, O., Muguerza, J., & Antón, M. Á. (2021) · Sensors · doi:10.3390/s21041031 · Cited by 32
Baydargil, H. B., Park, J.-S., & Kang, D.-Y. (2021) · Applied Sciences · doi:10.3390/app11052187 · Cited by 30
Agent with Warm Start and Adaptive Dynamic Termination for Plane Localization in 3D Ultrasound
Yang, X., Dou, H., Huang, R., Xue, W., Huang, Y., Qian, J., Zhang, Y., Luo, H., Guo, H., Wang, T., et al. (2021) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2021.3069663 · Cited by 32
Predictive Coding: Towards a Future of Deep Learning beyond Backpropagation?
Millidge, B., Salvatori, T., Song, Y., Bogacz, R., & Lukasiewicz, T. (2022) · arXiv · doi:10.48550/arxiv.2202.09467 · Cited by 31
Memory-Efficient Synaptic Connectivity for Spike-Timing- Dependent Plasticity
Pedroni, B. U., Joshi, S., Deiss, S. R., Sheik, S., Detorakis, G., Paul, S., Augustine, C., Neftci, E. O., & Cauwenberghs, G. (2019) · Frontiers in Neuroscience · doi:10.3389/fnins.2019.00357 · Cited by 29
A Deep Multi-Modal Explanation Model for Zero-Shot Learning
Liu, Y., & Tuytelaars, T. (2020) · IEEE Transactions on Image Processing · doi:10.1109/tip.2020.2975980 · Cited by 31
Activity Driven Weakly Supervised Object Detection
Yang, Z., Mahajan, D., Ghadiyaram, D., Nevatia, R., & Ramanathan, V. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00303 · Cited by 27
Zhang, F., Yang, F., Li, C., & Yuan, G. (2019) · IEEE Access · doi:10.1109/access.2019.2919103 · Cited by 29
“A batch normalization layer [50] is added on all of the convolutional layers to help regularize the network.”
BackgroundLearning cortical representations through perturbed and adversarial dreaming
Deperrois, N., Petrovici, M. A., Senn, W., & Jordan, J. (2022) · eLife · doi:10.7554/elife.76384 · Cited by 28
Progressive modality cooperation for multi-modality domain adaptation
Zhang, W., Xu, D., Zhang, J., & Ouyang, W. (2021) · IEEE Transactions on Image Processing · doi:10.1109/tip.2021.3052083 · Cited by 25
Kumar, D., Sankar, V., Clausi, D., Taylor, G. W., & Wong, A. (2019) · IEEE Access · doi:10.1109/access.2019.2945524 · Cited by 26
Qian, W., Li, S., Wang, J., Xin, Y., & Ma, H. (2018) · 2018 Prognostics and System Health Management Conference (PHM-Chongqing) · doi:10.1109/phm-chongqing.2018.00180 · Cited by 28
Object Localization Based on Proposal Fusion
Tang, S., Li, Y., Deng, L., & Zhang, Y. (2017) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2017.2729786 · Cited by 25
Investigating Enhancements to Contrastive Predictive Coding for Human Activity Recognition
Haresamudram, H., Essa, I., & Plötz, T. (2023) · 2023 IEEE International Conference on Pervasive Computing and Communications (PerCom) · doi:10.1109/percom56429.2023.10099197 · Cited by 27
Metadata
- Authors
- Ioffe, S., & Szegedy, C.
- Affiliations
- Google (United States) (US)
- ORCID
- Szegedy, Christian
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2015
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1502.03167
- OpenAlex
- W1836465849
- Cited by
- 27,818
- References
- 23
- Citations by year
- 2026: 152 · 2025: 1317 · 2024: 2504 · 2023: 3465 · 2022: 3990 · 2021: 4698 · 2020: 4028 · 2019: 3832 · 2018: 2356 · 2017: 1153 · 2016: 273 · 2015: 43
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Stochastic Gradient Optimization Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- batch normalization, internal covariate shift, deep neural network training, training speed, learning rate, parameter initialization, regularization, dropout, image classification, ensemble learning, ImageNet
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
