The DOI matches its DataCite record; 23 of the 23 references it cites were found in a registry or an index.
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Open accessAbstract
Training Deep Neural Networks is complicated by the fact that the distribution of each layer's inputs changes during training, as the parameters of the previous layers change. This slows down the training by requiring lower learning rates and careful parameter initialization, and makes it notoriously hard to train models with saturating nonlinearities. We refer to this phenomenon as internal covariate shift, and address the problem by normalizing layer inputs. Our method draws its strength from making normalization a part of the model architecture and performing the normalization for each training mini-batch. Batch Normalization allows us to use much higher learning rates and be less careful about initialization. It also acts as a regularizer, in some cases eliminating the need for Dropout. Applied to a state-of-the-art image classification model, Batch Normalization achieves the same accuracy with 14 times fewer training steps, and beats the original model by a significant margin. Using an ensemble of batch-normalized networks, we improve upon the best published result on ImageNet classification: reaching 4.9% top-5 validation error (and 4.8% test error), exceeding the accuracy of human raters.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source doesn’t support the claim as written.
In the citing paper
“This method ensures the performance of the output activations [81]. This performance follows a unit Gaussian distribution.”
From Review of deep learning: concepts, CNN architectures, challenges, applications, future directions (2021) · cites this paper as [81]
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Since the addition of three branches introduces the covariate shift issue (Ioffe and Szegedy 2015)”
From Make RepVGG Greater Again: A Quantization-Aware Approach (2024) · cites this paper as Ioffe and Szegedy 2015
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“SGD can be applied naively to the federated optimization problem, where a single batch gradient calculation (say on a randomly selected client) is done per round of communication. This approach is computationally efficient, but requires very large numbers of rounds of training to produce good models (e.g., even using an advanced approach like batch normalization, Ioffe and Szegedy [21] trained MNI”
From Communication-Efficient Learning of Deep Networks from Decentralized Data (2016) · cites this paper as [21]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“It mitigates the internal covariate shift that occurs as the distribution of each hidden layer's inputs changes during the training (as the parameters of the preceding layers change) and slows down the learning process.”
From Empirical asset pricing via machine learning: evidence from the European stock market (2021) · cites this paper as Ioffe and Szegedy (2015)
The cited source says what the claim attributes to it.
In the citing paper
“Second, we adopt the batch normalization algorithm introduced by Ioffe and Szegedy (2015).”
From Empirical asset pricing via machine learning: evidence from the European stock market (2021) · cites this paper as Ioffe and Szegedy (2015)
The cited source says what the claim attributes to it.
In the citing paper
“It occurs throughout training, because the weight updates to deeper layers result in a continuously changing distribution of signal at higher layers, which hinders the convergence of their weights.”
From Efficient multi-scale 3D CNN with fully connected CRF for accurate brain lesion segmentation (2017) · cites this paper as Ioffe and Szegedy, 2015
The cited source says what the claim attributes to it.
In the citing paper
“Batch normalization standardizes intermediate outputs of the network to zero mean and unit variance for a batch of training examples [Ioffe and Szegedy, 2015].”
From Deep learning with convolutional neural networks for EEG decoding and visualization (2017) · cites this paper as Ioffe and Szegedy, 2015
The cited source says what the claim attributes to it.
In the citing paper
“Batch normalization is a type of supplemental layer which adaptively normalizes the input values of the following layer, mitigating the risk of overfitting, as well as improving gradient flow through the network, allowing higher learning rates, and reducing the dependence on initialization [28].”
From Convolutional neural networks: an overview and application in radiology (2018) · cites this paper as [28]
The cited source says what the claim attributes to it.
In the citing paper
“Unlike BatchNorm [Ioffe and Szegedy, 2015], commonly used in CNNs, LayerNorm cannot be folded into the previous layer due to its dynamic computational property, so we have to quantize it separately.”
From FQ-ViT: Post-Training Quantization for Fully Quantized Vision Transformer (2022) · cites this paper as [Ioffe and Szegedy, 2015]
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“However, in integer-only quantization, all the operations are performed using low-precision integer arithmetic [113,132,154,193,267], as illustrated in Figure 6 (Right).This permits the entire inference to be carried out with efficient integer arithmetic, without any floating point dequantization of any parameters or activations.”
From A Survey of Quantization Methods for Efficient Neural Network Inference (2022) · cites this paper as [113]
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“CMSIS-NN [136] is a library from ARM that helps quantizing and deploying NN models onto the ARM Cortex-M cores.Specifically, the library leverages fixed-point quantization [113,154,267] with power-oftwo scaling factors so that quantization and dequantization processes can be carried out efficiently with bit shifting operations.”
From A Survey of Quantization Methods for Efficient Neural Network Inference (2022) · cites this paper as [113]
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“as neural networks have the capability of learning directly from raw data [49].”
From Investigating Enhancements to Contrastive Predictive Coding for Human Activity Recognition (2023) · cites this paper as [49]
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Recent advances in light-weight neural network architectures [15,13,14,9,11,10,12] are mostly based on the metric of FLOPs and do not consider these properties above.”
From ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design (2018) · cites this paper as [13]
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Following the common practice [15,13,14], all networks in comparison have four levels of computational complexity, i.e. about 40, 140, 300 and 500+ MFLOPs.”
From ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design (2018) · cites this paper as [13]
Experimental — accuracy not yet measuredIn the citing paper
“Our reproduced GoogleNet has center crop top-1 error of 26.4%, close to that reported in [32] (25.2%).”
From Object Detection Networks on Convolutional Feature Maps (2017) · cites this paper as [32]
In the citing paper
“To avoid overfitting, both dropout [116] and batch normalization [33] can help to achieve better regularization.”
From Deep learning for human affect recognition: Insights and new developments (2021) · cites this paper as [33]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805
(2014) · Cited by 33,419
Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification
He, K., Zhang, X., Ren, S., & Sun, J. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.123 · Cited by 19,741
(2010) · Cited by 14,134
(2010) · International Conference on Machine Learning · Cited by 13,995
Independent component analysis: algorithms and applications
Hyvärinen, A., & Erkki Oja, A. (2000) · Neural Networks · doi:10.1016/s0893-6080(00)00026-5 · Cited by 8,926
(2010) · Cited by 8,261
On the difficulty of training recurrent neural networks
Pascanu, R., Mikolov, T., & Bengio, Y. (2012) · arXiv · doi:10.48550/arxiv.1211.5063 · Cited by 4,310
(2013) · Cited by 3,740
Large Scale Distributed Deep Networks
Dean, J. B., Corrado, G. S., Monga, R., Chen, K., Devin, M., Mao, M., Ranzato, M., Senior, A., Tucker, P. A., Yang, K., et al. (2012) · Cited by 3,202
Flagged first, then by citations
Cited by
Sorted by citations
Items 901–1,000 of 1,339
Deep Convolutional Neural Networks Enable Discrimination of Heterogeneous Digital Pathology Images
Khosravi, P., Kazemi, E., Imielinski, M., Elemento, O., & Hajirasouliha, I. (2018) · EBioMedicine · doi:10.1016/j.ebiom.2017.12.026 · Cited by 334
Identity-Aware Convolutional Neural Network for Facial Expression Recognition
Meng, Z., Liu, P., Cai, J., Han, S., & Tong, Y. (2017) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg.2017.140 · Cited by 335
Latern: Dynamic Continuous Hand Gesture Recognition Using FMCW Radar Sensor
Zhang, Z., Tian, Z., & Zhou, M. (2018) · IEEE Sensors Journal · doi:10.1109/jsen.2018.2808688 · Cited by 334
Understanding the Disharmony Between Dropout and Batch Normalization by Variance Shift
Li, X., Chen, S., Hu, X., & Yang, J. J. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00279 · Cited by 356
MoDNN: Local distributed mobile computing system for Deep Neural Network
Mao, J., Chen, X., Nixon, K. W., Krieger, C., & Chen, Y. (2017) · Design, Automation & Test in Europe Conference & Exhibition (DATE), 2017 · doi:10.23919/date.2017.7927211 · Cited by 331
Interpreting Deep Visual Representations via Network Dissection
Zhou, B., Bau, D., Oliva, A., & Torralba, A. B. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2858759 · Cited by 347
Ge, W., Lin, X., & Yu, Y. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00315 · Cited by 337
Photographic Text-to-Image Synthesis with a Hierarchically-Nested Adversarial Network
Zhang, Z., Xie, Y., & Yang, L. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00649 · Cited by 332
Local Relation Learning for Face Forgery Detection
Chen, S., Yao, T., Chen, Y., Ding, S., Li, J., & Ji, R. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i2.16193 · Cited by 334
CSI: Novelty Detection via Contrastive Learning on Distributionally Shifted Instances
Tack, J., Mo, S., Jeong, J., & Shin, J. (2020) · arXiv · doi:10.48550/arxiv.2007.08176 · Cited by 351
Bilinear Convolutional Neural Networks for Fine-Grained Visual Recognition
Lin, T., RoyChowdhury, A., & Maji, S. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2723400 · Cited by 408
Simple Black-Box Adversarial Attacks on Deep Neural Networks
Narodytska, N., & Kasiviswanathan, S. P. (2017) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2017.172 · Cited by 345
Iqbal, I., Younus, M., Walayat, K., Kakar, M. U., & Ma, J. (2020) · Computerized Medical Imaging and Graphics · doi:10.1016/j.compmedimag.2020.101843 · Cited by 343
Context Autoencoder for Self-supervised Representation Learning
Chen, X., Ding, M., Wang, X., Xin, Y., Mo, S., Wang, Y., Han, S., Luo, P., Zeng, G., & Wang, J. (2023) · International Journal of Computer Vision · doi:10.1007/s11263-023-01852-4 · Cited by 446
DF-GAN: A Simple and Effective Baseline for Text-to-Image Synthesis
Tao, M., Tang, H., Wu, F., Jing, X., Bao, B., & Xu, C. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01602 · Cited by 513
Focal Sparse Convolutional Networks for 3D Object Detection
Chen, Y., Li, Y., Zhang, X., Sun, J. Z., & Jia, J. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00535 · Cited by 346
Surface Water Mapping by Deep Learning
Isikdogan, F., Bovik, A. C., & Passalacqua, P. (2017) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2017.2735443 · Cited by 292
A Comprehensive Analysis of Deep Regression
Lathuilière, S., Mesejo, P., Alameda-Pineda, X., & Horaud, R. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2910523 · Cited by 358
Yedroudj-Net: An Efficient CNN for Spatial Steganalysis
Yedroudj, M., Comby, F., & Chaumont, M. (2018) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2018.8461438 · Cited by 356
C3SRAM: An In-Memory-Computing SRAM Macro Based on Robust Capacitive Coupling Computing Mechanism
Jiang, Z., Yin, S., Seo, J.-S., & Seok, M. (2020) · IEEE Journal of Solid-State Circuits · doi:10.1109/jssc.2020.2992886 · Cited by 328
Flow Prediction in Spatio-Temporal Networks Based on Multitask Deep Learning
Zhang, J., Zheng, Y., Sun, J., & Qi, D. (2019) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2019.2891537 · Cited by 334
Regularizing Hyperspectral and Multispectral Image Fusion by CNN Denoiser
Dian, R., Shutao Li, A., & Kang, X. (2020) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.2980398 · Cited by 327
View-GCN: View-Based Graph Convolutional Network for 3D Shape Analysis
Wei, X., Yu, R., & Sun, J. Z. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00192 · Cited by 340
Accurate and versatile 3D segmentation of plant tissues at cellular resolution
Wolny, A., Cerrone, L., Vijayan, A., Tofanelli, R., Barro, A. V., Louveaux, M., Wenzl, C., Strauss, S., Wilson‐Sánchez, D., Lymbouridou, R., et al. (2020) · eLife · doi:10.7554/elife.57613 · Cited by 347
VITON-HD: High-Resolution Virtual Try-On via Misalignment-Aware Normalization
Choi, S.-H., Park, S., Lee, M., & Choo, J. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01391 · Cited by 360
Data Augmentation in Classification and Segmentation: A Survey and New Strategies
Alomar, K., Aysel, H. I., & Cai, X. (2023) · Journal of Imaging · doi:10.3390/jimaging9020046 · Cited by 426
A 64-Tile 2.4-Mb In-Memory-Computing CNN Accelerator Employing Charge-Domain Compute
Valavi, H., Ramadge, P. J., Nestler, E., & Verma, N. (2019) · IEEE Journal of Solid-State Circuits · doi:10.1109/jssc.2019.2899730 · Cited by 328
In-place Activated BatchNorm for Memory-Optimized Training of DNNs
Bulò, S. R., Porzi, L., & Kontschieder, P. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00591 · Cited by 329
An application of cascaded 3D fully convolutional networks for medical image segmentation
Roth, H. R., Oda, H., Zhou, X., Shimizu, N., Yang, Y., Hayashi, Y., Oda, M., Fujiwara, M., Misawa, K., & Mori, K. (2018) · Computerized Medical Imaging and Graphics · doi:10.1016/j.compmedimag.2018.03.001 · Cited by 330
Ghamisi, P., Maggiori, E., Shutao Li, A., de Souza, R. M., Tarablaka, Y., Moser, G., Giorgi, A. D., Fang, L., Chen, Y., Chi, M., et al. (2018) · IEEE Geoscience and Remote Sensing Magazine · doi:10.1109/mgrs.2018.2854840 · Cited by 336
Diagnosisformer: An efficient rolling bearing fault diagnosis method based on improved Transformer
Hou, Y., Wang, J., Chen, Z., Ma, J., & Li, T. (2023) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2023.106507 · Cited by 337
Hypercorrelation Squeeze for Few-Shot Segmenation
Min, J., Kang, D., & Cho, M. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00686 · Cited by 332
Interleaved Group Convolutions
Zhang, T., Qi, G.-J., Xiao, B., & Wang, J. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.469 · Cited by 323
Recurrent Human Pose Estimation
Belagiannis, V., & Zisserman, A. (2017) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg.2017.64 · Cited by 336
Li, P., Xie, J., Wang, Q., & Gao, Z. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00105 · Cited by 327
Context Prior for Scene Segmentation
Yu, C., Wang, J., Gao, C., Yu, G., Shen, C., & Sang, N. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.01243 · Cited by 323
Diving deeper into underwater image enhancement: A survey
Anwar, S., & Li, C. (2020) · Signal Processing Image Communication · doi:10.1016/j.image.2020.115978 · Cited by 342
Walk in the Cloud: Learning Curves for Point Clouds Shape Analysis
Xiang, T., Zhang, C., Song, Y., Yu, J., & Cai, W. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00095 · Cited by 340
High-Performance Large-Scale Image Recognition Without NormalizationPreprint
Brock, A., De, S., Smith, S. L., & Simonyan, K. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.06171 · Cited by 394
TransGAN: Two Pure Transformers Can Make One Strong GAN, and That Can Scale UpPreprint
Jiang, Y., Chang, S., & Wang, Z. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.07074 · Cited by 419
A White Paper on Neural Network Quantization
Nagel, M., Fournarakis, M., Amjad, R. A., Bondarenko, Y., van Baalen, M., & Blankevoort, T. (2021) · arXiv · doi:10.48550/arxiv.2106.08295 · Cited by 323
Point-Voxel CNN for Efficient 3D Deep LearningPreprint
Liu, Z., Tang, H., Lin, Y., & Han, S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1907.03739 · Cited by 379
Deep Roots: Improving CNN Efficiency with Hierarchical Filter Groups
Ioannou, Y., Robertson, D., Cipolla, R., & Criminisi, A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.633 · Cited by 321
Street-view change detection with deconvolutional networks
Alcantarilla, P. F., Stent, S., Ros, G., Arroyo, R., & Gherardi, R. (2018) · Autonomous Robots · doi:10.1007/s10514-018-9734-5 · Cited by 323
Dynamic Multi-Scale Filters for Semantic Segmentation
He, J., Deng, Z., & Qiao, Y. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00366 · Cited by 320
DENSE-INception U-net for medical image segmentation
Zhang, Z., Wu, C., Coleman, S., & Kerr, D. (2020) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2020.105395 · Cited by 326
Transfer learning for time series classification
Ismail Fawaz, H., Forestier, G., Weber, J., Idoumghar, L., & Muller, P.-A. (2018) · 2018 IEEE International Conference on Big Data (Big Data) · doi:10.1109/bigdata.2018.8621990 · Cited by 319
Li, Z., Shen, H., Cheng, Q., Liu, Y., You, S., & He, Z. (2019) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2019.02.017 · Cited by 340
RESA: Recurrent Feature-Shift Aggregator for Lane Detection
Zheng, T., Fang, H., Zhang, Y., Tang, W., Yang, Z., Li, X., & Cai, D. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i4.16469 · Cited by 369
Deep Image: Scaling up Image RecognitionPreprint
Wu, R., Yan, S., Shan, Y., Dang, Q., & Sun, G. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1501.02876 · Cited by 345
Mei, S., Ji, J., Geng, Y., Zhang, Z., Xu, L., & Du, Q. (2019) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2019.2908756 · Cited by 318
AutoGAN: Neural Architecture Search for Generative Adversarial Networks
Gong, X., Chang, S., Jiang, Y., & Wang, Z. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00332 · Cited by 323
A Data-Driven Multi-Agent Autonomous Voltage Control Framework Using Deep Reinforcement Learning
Wang, S., Duan, J., Shi, D., Xu, C., Li, H., Diao, R., & Wang, Z. (2020) · IEEE Transactions on Power Systems · doi:10.1109/tpwrs.2020.2990179 · Cited by 318
An Overview of Deep-Learning-Based Audio-Visual Speech Enhancement and Separation
Michelsanti, D., Tan, Z., Zhang, S.-X., Xu, Y., Yu, M., Yu, D., & Jensen, J. (2021) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2021.3066303 · Cited by 344
Fake News identification on Twitter with Hybrid CNN and RNN Models
Ajao, O., Bhowmik, D., & Zargari, S. A. (2018) · e-space (Manchester Metropolitan University) · Cited by 318
Learning and Using the Arrow of Time
Wei, D., Lim, J. J., Zisserman, A., & Freeman, W. T. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00840 · Cited by 329
Fan, S., Li, J., Zhang, Y., Tian, X., Wang, Q., He, X., Zhang, C., & Huang, W. (2020) · Journal of Food Engineering · doi:10.1016/j.jfoodeng.2020.110102 · Cited by 317
Complex-Valued Networks for Automatic Modulation Classification
Tu, Y., Lin, Y., Hou, C., & Mao, S. (2020) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2020.3005707 · Cited by 319
Improving Calibration for Long-Tailed Recognition
Zhong, Z., Cui, J., Liu, S., & Jia, J. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01622 · Cited by 339
Fu, S., Wang, T.-W., Tsao, Y., Lu, X., & Kawai, H. (2018) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2018.2821903 · Cited by 314
MMTM: Multimodal Transfer Module for CNN Fusion
Joze, H. R. V., Shaban, A., Iuzzolino, M. L., & Koishida, K. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.01330 · Cited by 337
Deep learning in robotics: A review of recent research
Pierson, H. A., & Gashler, M. S. (2017) · Advanced Robotics · doi:10.1080/01691864.2017.1365009 · Cited by 331
Applying Deep Learning for Epilepsy Seizure Detection and Brain Mapping Visualization
Hossain, M. S., Amin, S. U., Alsulaiman, M., & Muhammad, G. (2019) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3241056 · Cited by 316
Zhou, H., He, S., Cai, Y., Wang, M., & Su, S. (2019) · Sustainable Cities and Society · doi:10.1016/j.scs.2019.101605 · Cited by 313
Split Computing and Early Exiting for Deep Learning Applications: Survey and Research Challenges
Matsubara, Y., Levorato, M., & Restuccia, F. (2022) · ACM Computing Surveys · doi:10.1145/3527155 · Cited by 357
Mei, S., Ji, J., Hou, J., Li, X., & Du, Q. (2017) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2017.2693346 · Cited by 320
Universal Denoising Networks : A Novel CNN Architecture for Image Denoising
Lefkimmiatis, S. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00338 · Cited by 310
EPIC-Fusion: Audio-Visual Temporal Binding for Egocentric Action Recognition
Kazakos, E., Nagrani, A., Zisserman, A., & Damen, D. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00559 · Cited by 10
Concrete bridge surface damage detection using a single-stage detector
Zhang, C., Chang, C. C., & Jamshidi, M. (2019) · Computer-Aided Civil and Infrastructure Engineering · doi:10.1111/mice.12500 · Cited by 347
EDRNet: Encoder–Decoder Residual Network for Salient Object Detection of Strip Steel Surface Defects
Song, G., Song, K., & Yan, Y. (2020) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2020.3002277 · Cited by 314
Learning by Aligning: Visible-Infrared Person Re-identification using Cross-Modal Correspondences
Park, H., Lee, S., Lee, J., & Ham, B. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01183 · Cited by 322
The Devil Is in the Details: Window-based Attention for Image Compression
Zou, R., Song, C., & Zhang, Z. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01697 · Cited by 337
Deep Generalized Unfolding Networks for Image Restoration
Mou, C., Wang, Q., & Zhang, J. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01688 · Cited by 315
MegDet: A Large Mini-Batch Object Detector
Peng, C., Xiao, T., Li, Z., Jiang, Y., Zhang, X., Jia, K., Yu, G., & Sun, J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00647 · Cited by 311
Raghunath, S., Pfeifer, J. M., Ulloa-Cerna, A. E., Nemani, A., Carbonati, T., Jing, L., vanMaanen, D. P., Hartzel, D. N., Ruhl, J. A., Lagerman, B. F., et al. (2021) · Circulation · doi:10.1161/circulationaha.120.047829 · Cited by 312
Beck, C., Weinan, E., & Jentzen, A. (2019) · Journal of Nonlinear Science · doi:10.1007/s00332-018-9525-3 · Cited by 345
Jampani, V., Sun, D., Liu, M.-Y., Yang, M., & Kautz, J. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01234-2_22 · Cited by 312
A point-cloud deep learning framework for prediction of fluid flow fields on irregular geometries
Kashefi, A., Rempe, D., & Guibas, L. (2021) · Physics of Fluids · doi:10.1063/5.0033376 · Cited by 313
Image super-resolution: A comprehensive review, recent trends, challenges and applications
Lepcha, D. C., Goyal, B., Dogra, A., & Goyal, V. (2022) · Information Fusion · doi:10.1016/j.inffus.2022.10.007 · Cited by 342
Review of deep convolution neural network in image classification
Al-Saffar, A. A. M., Hai, T., & Talab, M. A. (2017) · International Conference on Radar, Antenna, Microwave, Electronics, and Telecommunications (ICRAMET) · doi:10.1109/icramet.2017.8253139 · Cited by 316
High-Fidelity Generative Image CompressionPreprint
Mentzer, F., Toderici, G. D., Tschannen, M., & Agustsson, E. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.09965 · Cited by 347
Systematic evaluation of convolution neural network advances on the Imagenet
Mishkin, D., Sergievskiy, N., & Matas, J. (2017) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2017.05.007 · Cited by 346
Learning Steerable Filters for Rotation Equivariant CNNs
Weiler, M., Hamprecht, F. A., & Storath, M. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00095 · Cited by 309
Brain Tumor Classification via Convolutional Neural Network and Extreme Learning Machines
Pashaei, A., Sajedi, H., & Jazayeri, N. (2018) · International Conference on Computer and Knowledge Engineering · doi:10.1109/iccke.2018.8566571 · Cited by 320
Bai, X., Wang, X., Liu, X., Liu, Q., Song, J., Sebe, N., & Kim, B. (2021) · Pattern Recognition · doi:10.1016/j.patcog.2021.108102 · Cited by 313
Imaging and fusing time series for wearable sensor-based human activity recognition
Qin, Z., Zhang, Y., Meng, S., Qin, Z., & Choo, K.-K. R. (2020) · Information Fusion · doi:10.1016/j.inffus.2019.06.014 · Cited by 309
Learning to optimize: Training deep neural networks for wireless resource management
Sun, H., Chen, X., Shi, Q., Hong, M., Fu, X., & Sidiropoulos, N. D. (2017) · 2017 IEEE 18th International Workshop on Signal Processing Advances in Wireless Communications (SPAWC) · doi:10.1109/spawc.2017.8227766 · Cited by 308
Li, G., Yang, Y., Li, S., Qu, X., Lyu, N., & Li, S. E. (2022) · Transportation Research Part C: Emerging Technologies · doi:10.1016/j.trc.2021.103452 · Cited by 308
Liang, X., Zhang, J., Li, Z., Li, Y., & Tian, Q. (2019) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2019.2905881 · Cited by 311
A Multi-Branch 3D Convolutional Neural Network for EEG-Based Motor Imagery Classification
Zhao, X., Zhang, H., Zhu, G., You, F., Kuang, S., & Sun, L. (2019) · IEEE Transactions on Neural Systems and Rehabilitation Engineering · doi:10.1109/tnsre.2019.2938295 · Cited by 308
DensePoint: Learning Densely Contextual Representation for Efficient Point Cloud Processing
Liu, Y., Fan, B., Meng, G., Lu, J., Xiang, S., & Pan, C. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00534 · Cited by 306
Objects are Different: Flexible Monocular 3D Object Detection
Zhang, Y., Lu, J., & Zhou, J. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00330 · Cited by 310
Zhang, A. A., Wang, K. C. P., Fei, Y., Liu, Y., Tao, S., Chen, C., Li, Q., & Li, B. (2018) · Journal of Computing in Civil Engineering · doi:10.1061/(asce)cp.1943-5487.0000775 · Cited by 317
Knowledge Transfer for Out-of-Knowledge-Base Entities : A Graph Neural Network Approach
Hamaguchi, T., Oiwa, H., Shimbo, M., & Matsumoto, Y. (2017) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2017/250 · Cited by 322
Yu, K., Lin, T. R., Ma, H., Li, X., & Li, X. (2020) · Mechanical Systems and Signal Processing · doi:10.1016/j.ymssp.2020.107043 · Cited by 305
A-CNN: Annularly Convolutional Neural Networks on Point Clouds
Komarichev, A., Zhong, Z., & Hua, J. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00760 · Cited by 306
More is Less: A More Complicated Network with Less Inference Complexity
Dong, X., Huang, J., Yang, Y., & Yan, S. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.205 · Cited by 305
Fully automatic acute ischemic lesion segmentation in DWI using convolutional neural networks
Chen, L., Bentley, P., & Rueckert, D. (2017) · NeuroImage Clinical · doi:10.1016/j.nicl.2017.06.016 · Cited by 307
Vibration-based structural state identification by a 1-dimensional convolutional neural network
Zhang, Y., Miyamori, Y., Mikami, S., & Saito, T. (2019) · Computer-Aided Civil and Infrastructure Engineering · doi:10.1111/mice.12447 · Cited by 314
Graph Stacked Hourglass Networks for 3D Human Pose Estimation
Xu, T., & Takano, W. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01584 · Cited by 312
Metadata
- Authors
- Ioffe, S., & Szegedy, C.
- Affiliations
- Google (United States) (US)
- ORCID
- Szegedy, Christian
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2015
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1502.03167
- OpenAlex
- W1836465849
- Cited by
- 27,818
- References
- 23
- Citations by year
- 2026: 152 · 2025: 1317 · 2024: 2504 · 2023: 3465 · 2022: 3990 · 2021: 4698 · 2020: 4028 · 2019: 3832 · 2018: 2356 · 2017: 1153 · 2016: 273 · 2015: 43
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Stochastic Gradient Optimization Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- batch normalization, internal covariate shift, deep neural network training, training speed, learning rate, parameter initialization, regularization, dropout, image classification, ensemble learning, ImageNet
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
