The DOI matches its Crossref record; 250 of the 250 references it cites were found in a registry or an index.
Normalization Techniques in Training DNNs: Methodology, Analysis and Application
Open accessAbstract
Normalization techniques are essential for accelerating the training and improving the generalization of deep neural networks (DNNs), and have successfully been used in various applications. This paper reviews and comments on the past, present and future of normalization methods in the context of DNN training. We provide a unified picture of the main motivation behind different approaches from the perspective of optimization, and present a taxonomy for understanding the similarities and differences between them. Specifically, we decompose the pipeline of the most representative normalizing activation methods into three components: the normalization area partitioning, normalization operation and normalization representation recovery. In doing so, we provide insight for designing new normalization technique. Finally, we discuss the current progress in understanding normalization methods, and provide a comprehensive review of the applications of normalization for particular tasks, in which it can effectively solve the key issues.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
Exploiting Generative AI to Scale up Intelligent Tutoring Systems
Jakubův, J., Chvalovský, K., Goertzel, Z., Kaliszyk, C., Olšák, M., Piotrowski, B., Schulz, S., Suda, M., & Urban, J. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,658
Maximum Likelihood from Incomplete Data Via the EM Algorithm
Dempster, A. P., Laird, N. M., & Rubin, D. B. (1977) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1977.tb01600.x · Cited by 50,444
Densely Connected Convolutional Networks
Huang, G., Liu, Z., Van Der Maaten, L., & Weinberger, K. Q. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.243 · Cited by 48,566
Going deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562
Microsoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380
Squeeze-and-Excitation Networks
Hu, J., Shen, L., & Sun, G. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00745 · Cited by 35,667
(2014) · Cited by 33,419
Rethinking the Inception Architecture for Computer Vision
Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., & Wojna, Z. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.308 · Cited by 32,138
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S., & Szegedy, C. (2015) · arXiv · doi:10.48550/arxiv.1502.03167 · Cited by 27,818
Unpaired image-to-image translation using cycle-consistent adversarial networks
Zhu, J.-Y., Park, T., Isola, P., & Efros, A. A. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.244 · Cited by 23,602
Learning Multiple Layers of Features from Tiny Images
Krizhevsky, A. (2024) · doi:10.57702/zp44cu3g · Cited by 23,395
Reducing the dimensionality of data with neural networks
Hinton, G. E., & Salakhutdinov, R. R. (2006) · Science · doi:10.1126/science.1127647 · Cited by 20,928
Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification
He, K., Zhang, X., Ren, S., & Sun, J. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.123 · Cited by 19,741
Labaca-Castro, R. (2023) · Machine Learning under Malware Attack · doi:10.1007/978-3-658-40442-0_9 · Cited by 19,272
Zhao, H., Shi, J., Qi, X., Wang, X., & Jia, J. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.660 · Cited by 16,438
(2010) · Cited by 14,134
(2010) · International Conference on Machine Learning · Cited by 13,995
Momentum Contrast for Unsupervised Visual Representation Learning
He, K., Fan, H., Wu, Y., Xie, S., & Girshick, R. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00975 · Cited by 13,347
Aggregated Residual Transformations for Deep Neural Networks
Xie, S., Girshick, R., Dollar, P., Tu, Z., & He, K. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.634 · Cited by 12,465
Unsupervised representation learning with deep convolutional generative adversarial networks
Radford, A., Metz, L., & Chintala, S. (2015) · arXiv · doi:10.48550/arxiv.1511.06434 · Cited by 12,235
Squeeze-and-Excitation Networks
Hu, J., Shen, L., Albanie, S., Sun, G., & Wu, E. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2913372 · Cited by 12,157
TensorFlow: A system for large-scale machine learning
Abadi, M., Barham, P., Chen, J., Chen, Z., Davis, A., Dean, J., Devin, M., Ghemawat, S., Irving, G., Isard, M., et al. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1605.08695 · Cited by 11,822
A Style-Based Generator Architecture for Generative Adversarial Networks
Karras, T., Laine, S., & Aila, T. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00453 · Cited by 11,714
(2017) · Cited by 11,625
Continuous control with deep reinforcement learning
Lillicrap, T. P., Hunt, J. J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., & Wierstra, D. (2015) · arXiv · doi:10.48550/arxiv.1509.02971 · Cited by 10,782
Identity mappings in deep residual networks
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46493-0_38 · Cited by 10,741
Identity Mappings in Deep Residual Networks
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · arXiv · doi:10.48550/arxiv.1603.05027 · Cited by 10,665
Path Aggregation Network for Instance Segmentation
Liu, S., Qi, L., Qin, H., Shi, J., & Jia, J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00913 · Cited by 10,011
PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation
Charles, R. Q., Su, H., Kaichun, M., & Guibas, L. J. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.16 · Cited by 9,237
Conditional Generative Adversarial Nets
Mirza, M., & Osindero, S. (2014) · arXiv · doi:10.48550/arxiv.1411.1784 · Cited by 8,863
(2010) · Cited by 8,261
(2007) · doi:10.5555/1283383.1283494 · Cited by 7,364
Zagoruyko, S., & Komodakis, N. (2016) · Procedings of the British Machine Vision Conference 2016 · doi:10.5244/c.30.87 · Cited by 7,244
Asynchronous Methods for Deep Reinforcement Learning
Mnih, V., Badia, A. P., Mirza, M., Graves, A., Lillicrap, T. P., Harley, T., Silver, D., & Kavukcuoglu, K. (2016) · arXiv · doi:10.48550/arxiv.1602.01783 · Cited by 7,132
Improved Techniques for Training GANs
Salimans, T., Goodfellow, I., Zaremba, W., Cheung, V., Radford, A., & Chen, X. (2016) · arXiv · doi:10.48550/arxiv.1606.03498 · Cited by 6,965
Image Style Transfer Using Convolutional Neural Networks
Gatys, L. A., Ecker, A. S., & Bethge, M. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.265 · Cited by 6,025
Arbitrary Style Transfer in Real-Time with Adaptive Instance Normalization
Huang, X., & Belongie, S. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.167 · Cited by 5,076
On the difficulty of training recurrent neural networks
Pascanu, R., Mikolov, T., & Bengio, Y. (2012) · arXiv · doi:10.48550/arxiv.1211.5063 · Cited by 4,310
Large Scale GAN Training for High Fidelity Natural Image Synthesis
Brock, A., Donahue, J., & Simonyan, K. (2018) · arXiv · doi:10.48550/arxiv.1809.11096 · Cited by 3,717
Spectral Normalization for Generative Adversarial Networks
Miyato, T., Kataoka, T., Koyama, M., & Yoshida, Y. (2018) · arXiv · doi:10.48550/arxiv.1802.05957 · Cited by 3,438
Self-Attention Generative Adversarial Networks
Zhang, H., Goodfellow, I., Metaxas, D., & Odena, A. (2018) · arXiv · doi:10.48550/arxiv.1805.08318 · Cited by 3,353
Optimization Methods for Large-Scale Machine Learning
Bottou, L., Curtis, F. E., & Nocedal, J. (2018) · SIAM Review · doi:10.1137/16m1080173 · Cited by 3,128
StackGAN: Text to Photo-Realistic Image Synthesis with Stacked Generative Adversarial Networks
Zhang, H., Xu, T., Li, H., Zhang, S., Wang, X., Huang, X., & Metaxas, D. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.629 · Cited by 3,021
Semantic Image Synthesis With Spatially-Adaptive Normalization
Park, T., Liu, M.-Y., Wang, T.-C., & Zhu, J.-Y. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00244 · Cited by 2,974
Understanding deep learning requires rethinking generalization
Zhang, C., Bengio, S., Hardt, M., Recht, B., & Vinyals, O. (2016) · arXiv · doi:10.48550/arxiv.1611.03530 · Cited by 2,838 · cited 1×
“Despite the recent progress of deep learning in terms of representation [278], optimization [279] and generalization [280], the networks investigated theoretically are usually different from those used in practice [199].”
MethodInstance Normalization: The Missing Ingredient for Fast Stylization
Ulyanov, D., Vedaldi, A., & Lempitsky, V. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1607.08022 · Cited by 2,775
Learning Efficient Convolutional Networks through Network Slimming
Liu, Z., Li, J., Shen, Z., Huang, G., Yan, S., & Zhang, C. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.298 · Cited by 2,761
Generative Adversarial Text to Image Synthesis
Reed, S., Akata, Z., Yan, X., Logeswaran, L., Schiele, B., & Lee, H. (2016) · arXiv · doi:10.48550/arxiv.1605.05396 · Cited by 2,494
Multimodal Unsupervised Image-to-Image Translation
Huang, X., Liu, M.-Y., Belongie, S., & Kautz, J. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01219-9_11 · Cited by 2,451
ShapeNet: An Information-Rich 3D Model Repository
Chang, A. X., Funkhouser, T., Guibas, L., Hanrahan, P., Huang, Q., Li, Z., Savarese, S., Savva, M., Song, S., Su, H., et al. (2015) · arXiv · doi:10.48550/arxiv.1512.03012 · Cited by 2,330
Arjovsky, M., Chintala, S., & Bottou, L. (2017) · arXiv · doi:10.48550/arxiv.1701.07875 · Cited by 2,225
What is the best multi-stage architecture for object recognition?
Jarrett, K., Kavukcuoglu, K., Ranzato, M. A., & LeCun, Y. (2009) · 2009 IEEE 12th International Conference on Computer Vision · doi:10.1109/iccv.2009.5459469 · Cited by 2,153
Functions of Matrices: Theory and Computation
Higham, N. J. (2008) · doi:10.1137/1.9780898717778 · Cited by 2,113
On First-Order Meta-Learning Algorithms
Nichol, A., Achiam, J., & Schulman, J. (2018) · arXiv · doi:10.48550/arxiv.1803.02999 · Cited by 1,438
Averaging Weights Leads to Wider Optima and Better Generalization
Izmailov, P., Podoprikhin, D., Garipov, T., Vetrov, D., & Wilson, A. G. (2018) · arXiv · doi:10.48550/arxiv.1803.05407 · Cited by 1,409
(1992) · Cited by 1,405
How Does Batch Normalization Help Optimization?
Santurkar, S., Tsipras, D., Ilyas, A., & Madry, A. (2018) · arXiv · doi:10.48550/arxiv.1805.11604 · Cited by 1,092
Exact solutions to the nonlinear dynamics of learning in deep linear neural networks
Saxe, A., McClelland, J. L., & Ganguli, S. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1312.6120 · Cited by 931
LeCun, Y., Bottou, L., Orr, G. B., & Müller, K. R. (1998) · Lecture Notes in Computer Science · doi:10.1007/3-540-49430-8_2 · Cited by 919
Weight Normalization: A Simple Reparameterization to Accelerate Training of Deep Neural Networks
Salimans, T., & Kingma, D. P. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1602.07868 · Cited by 883
Neural Style Transfer: A Review
Jing, Y., Yang, Y., Feng, Z., Ye, J., Yu, Y., & Song, M. (2020) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2019.2921336 · Cited by 856
A feasible method for optimization with orthogonality constraints
Wen, Z., & Yin, W. (2013) · Mathematical Programming · doi:10.1007/s10107-012-0584-1 · Cited by 849
On the Number of Linear Regions of Deep Neural Networks
Montúfar, G., Pascanu, R., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1402.1869 · Cited by 840
A Learned Representation For Artistic Style
Dumoulin, V., Shlens, J., & Kudlur, M. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1610.07629 · Cited by 763
Adaptive batch normalization for practical domain adaptation
Li, Y., Wang, N., Shi, J., Hou, X., & Liu, J. (2018) · Pattern Recognition · doi:10.1016/j.patcog.2018.03.005 · Cited by 601
Learning to Find Good Correspondences
Yi, K. M., Trulls, E., Ono, Y., Lepetit, V., Salzmann, M., & Fua, P. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00282 · Cited by 552
Deep Neural Networks as Gaussian Processes
Lee, J., Bahri, Y., Novak, R., Schoenholz, S. S., Pennington, J., & Sohl‐Dickstein, J. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.00165 · Cited by 529
Optimal Whitening and Decorrelation
Kessy, A., Lewin, A., & Strimmer, K. (2018) · The American Statistician · doi:10.1080/00031305.2016.1277159 · Cited by 506
Adversarial Examples Improve Image Recognition
Xie, C., Tan, M., Gong, B., Wang, J., Yuille, A. L., & Le, Q. V. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00090 · Cited by 488
Large Batch Training of Convolutional Networks
You, Y., Gitman, I., & Ginsburg, B. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1708.03888 · Cited by 463
GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networks
Chen, Z., Badrinarayanan, V., Lee, C., & Rabinovich, A. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.02257 · Cited by 443
Style Normalization and Restitution for Generalizable Person Re-Identification
Jin, X., Lan, C., Zeng, W., Chen, Z., & Zhang, L. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00321 · Cited by 426
Domain-Specific Batch Normalization for Unsupervised Domain Adaptation
Chang, W.-G., You, T., Seo, S., Kwak, S., & Han, B. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00753 · Cited by 424
Optimizing Neural Networks with Kronecker-factored Approximate Curvature
Martens, J., & Grosse, R. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1503.05671 · Cited by 414
Sorting out Lipschitz function approximationPreprint
Anil, C., Lucas, J., & Grosse, R. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.05381 · Cited by 403
cGANs with Projection Discriminator
Miyato, T., & Koyama, M. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.05637 · Cited by 382
Evaluating Prediction-Time Batch Normalization for Robustness under Covariate ShiftPreprint
Nado, Z., Padhy, S., Sculley, D., D’Amour, A., Lakshminarayanan, B., & Snoek, J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.10963 · Cited by 373
Modulating early visual processing by language
de Vries, H., Strub, F., Mary, J., Larochelle, H., Pietquin, O., & Courville, A. C. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1707.00683 · Cited by 369
Kim, J., Kim, M., Kang, H.-W., & Lee, K.-H. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1907.10830 · Cited by 368
Batch Renormalization: Towards Reducing Minibatch Dependence in Batch-Normalized Models
Ioffe, S. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1702.03275 · Cited by 367
Universal Style Transfer via Feature Transforms
Li, Y., Chen, F., Yang, J., Wang, Z., Lü, X., & Yang, M. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.08086 · Cited by 346
Revisiting Batch Normalization For Practical Domain Adaptation
Li, Y., Wang, N., Shi, J., Liu, J., & Hou, X. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1603.04779 · Cited by 340
Choi, S., Jung, S., Yun, H., Kim, J. T., Kim, S., & Choo, J. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01141 · Cited by 329
Understanding the Disharmony Between Dropout and Batch Normalization by Variance Shift
Li, X., Chen, S., Hu, X., & Yang, J. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00279 · Cited by 318
Avatar-Net: Multi-scale Zero-Shot Style Transfer by Feature Decoration
Sheng, L., Lin, Z., Shao, J., & Wang, X. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00860 · Cited by 313
MegDet: A Large Mini-Batch Object Detector
Peng, C., Xiao, T., Li, Z., Jiang, Y., Zhang, X., Jia, K., Yu, G., & Sun, J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00647 · Cited by 311
AutoDIAL: Automatic Domain Alignment Layers
Carlucci, F. M., Porzi, L., Caputo, B., Ricci, E., & Bulo, S. R. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.542 · Cited by 311
A PAC-Bayesian Approach to Spectrally-Normalized Margin Bounds for Neural Networks
Neyshabur, B., Bhojanapalli, S., & Srebro, N. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1707.09564 · Cited by 283
Optimization for deep learning: theory and algorithmsPreprint
Sun, R. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.08957 · Cited by 283
New insights and perspectives on the natural gradient method
Martens, J. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1412.1193 · Cited by 283
Norm-Based Capacity Control in Neural Networks
Neyshabur, B., Tomioka, R., & Srebro, N. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1503.00036 · Cited by 279
Concept whitening for interpretable image recognition
Chen, Z., Bei, Y., & Rudin, C. (2020) · Nature Machine Intelligence · doi:10.1038/s42256-020-00265-z · Cited by 279
Tent: Fully Test-time Adaptation by Entropy Minimization
Wang, D., Shelhamer, E., Liu, S., Olshausen, B. A., & Darrell, T. J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.10726 · Cited by 278
Neural Photo Editing with Introspective Adversarial Networks
Brock, A., Lim, T., Ritchie, J. M., & Weston, N. (2016) · arXiv · doi:10.48550/arxiv.1609.07093 · Cited by 275
Root Mean Square Layer Normalization
Zhang, B., & Sennrich, R. (2019) · arXiv · doi:10.48550/arxiv.1910.07467 · Cited by 262
Transformers without Tears: Improving the Normalization of Self-AttentionPreprint
Nguyen, T. Q., & Salazar, J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.05895 · Cited by 230
PyHessian: Neural Networks Through the Lens of the Hessian
Yao, Z., Gholami, A., Keutzer, K., & Mahoney, M. W. (2020) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata50022.2020.9378171 · Cited by 227
Yu, J., Yang, L., Xu, N., Yang, J., & Huang, T. S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1812.08928 · Cited by 216
Learning to Optimize Domain Specific Normalization for Domain Generalization
Seo, S., Suh, Y., Kim, D., Kim, G., Han, J., & Han, B. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58542-6_5 · Cited by 214
Cosine Normalization: Using Cosine Similarity Instead of Dot Product in Neural Networks
Luo, C., Zhan, J., Xue, X., Wang, L., Ren, R., & Yang, Q. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01418-6_38 · Cited by 211
Unitary Evolution Recurrent Neural Networks
Arjovsky, M., Shah, A., & Bengio, Y. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1511.06464 · Cited by 210
Training Deep and Recurrent Networks with Hessian-Free Optimization
Martens, J., & Sutskever, I. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-35289-8_27 · Cited by 202
Batch normalized recurrent neural networks
Laurent, C., Pereyra, G., Brakel, P., Zhang, Y., & Bengio, Y. (2016) · 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2016.7472159 · Cited by 201
Norm matters: efficient and accurate normalization schemes in deep networksPreprint
Hoffer, E., Banner, R., Golan, I., & Soudry, D. (2018) · Neural Information Processing Systems · doi:10.48550/arxiv.1803.01814 · Cited by 200
Bayesian Uncertainty Estimation for Batch Normalized Deep NetworksPreprint
Teye, M., Azizpour, H., & Smith, K. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.06455 · Cited by 199
Gradient Centralization: A New Optimization Technique for Deep Neural Networks
Yong, H., Huang, J., Hua, X., & Zhang, L. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58452-8_37 · Cited by 196
Scalable Methods for 8-bit Training of Neural Networks
Banner, R., Hubara, I., Hoffer, E., & Soudry, D. (2018) · arXiv · doi:10.48550/arxiv.1805.11046 · Cited by 193
Orthogonal Convolutional Neural Networks
Wang, J., Chen, Y., Chakraborty, R., & Yu, S. X. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01152 · Cited by 190
Dynamic Instance Normalization for Arbitrary Style Transfer
Jing, Y., Liu, X., Ding, Y., Wang, X., Ding, E., Song, M., & Wen, S. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i04.5862 · Cited by 189
L2 Regularization versus Batch and Weight Normalization
van Laarhoven, T. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1706.05350 · Cited by 188
Rethinking the Distribution Gap of Person Re-identification with Camera-Based Batch Normalization
Zhuang, Z., Wei, L., Xie, L., Zhang, T., Zhang, H., Wu, H., Ai, H., & Tian, Q. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58610-2_9 · Cited by 187
Domain-Invariant Stereo Matching Networks
Zhang, F., Qi, X., Yang, R., Prisacariu, V., Wah, B., & Torr, P. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58536-5_25 · Cited by 186
Decorrelated Batch Normalization
Huangi, L., Huangi, L., Yang, D., Lang, B., & Deng, J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00089 · Cited by 185
On orthogonality and learning recurrent networks with long term dependenciesPreprint
Vorontsov, E., Trabelsi, C., Kadoury, S., & Pal, C. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1702.00071 · Cited by 184
Singh, S., & Krishnan, S. M. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.01125 · Cited by 183
Orthogonal Deep Neural Networks
Li, S., Jia, K., Wen, Y., Liu, T., & Tao, D. (2019) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2948352 · Cited by 180
Huang, L., Liu, X., Lang, B., Yu, A., Wang, Y., & Li, B. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11768 · Cited by 179
Self-citationSwitchable Whitening for Deep Representation Learning
Pan, X., Zhan, X., Shi, J., Tang, X., & Luo, P. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00195 · Cited by 175
ACNe: Attentive Context Normalization for Robust Permutation-Equivariant Learning
Sun, W., Jiang, W., Trulls, E., Tagliasacchi, A., & Yi, K. M. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01130 · Cited by 174
Deep Learning Made Easier by Linear Transformations in Perceptrons
Raiko, T., Harri Valpola, B., & LeCun, Y. (2012) · Cited by 171
Transferable Normalization: Towards Improving Transferability of Deep Neural Networks
Wang, X., Jin, Y., Long, M., Wang, J., & Jordan, M. I. (2019) · neural information processing systems · Cited by 168
$L1$ -Norm Batch Normalization for Efficient Training of Deep Neural Networks
Wu, S., Li, G., Deng, L., Liu, L., Wu, D., Xie, Y., & Shi, L. (2019) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2018.2876179 · Cited by 166
Self-citationUnsupervised Domain Adaptation Using Feature-Whitening and Consensus Loss
Roy, S., Siarohin, A., Sangineto, E., Bulò, S. R., Sebe, N., & Ricci, E. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00970 · Cited by 166
Region Normalization for Image Inpainting
Yu, T., Guo, Z., Jin, X., Wu, S., Chen, Z., Li, W., Zhang, Z., & Liu, S. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i07.6967 · Cited by 166
Understanding and Improving Layer Normalization
Xu, J., Sun, X., Zhang, Z., Zhao, G., & Lin, J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.07013 · Cited by 163
EagleEye: Fast Sub-net Evaluation for Efficient Neural Network Pruning
Li, B., Wu, B., Su, J., & Wang, G. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58536-5_38 · Cited by 163
On Feature Normalization and Data Augmentation
Li, B., Wu, F., Lim, S.-N., Belongie, S., & Weinberger, K. Q. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01220 · Cited by 160
Differentiable Learning-to-Normalize via Switchable NormalizationPreprint
Luo, P., Ren, J., Peng, Z., Zhang, R., & Li, J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.10779 · Cited by 159
Iterative Normalization: Beyond Standardization Towards Efficient Whitening
Huang, L., Zhou, Y., Zhu, F., Liu, L., & Shao, L. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00501 · Cited by 159
Self-citationParseval Networks: Improving Robustness to Adversarial Examples
Cissé, M., Bojanowski, P., Grave, É., Dauphin, Y., & Usunier, N. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1704.08847 · Cited by 158
Image-To-Image Translation via Group-Wise Deep Whitening-And-Coloring Transformation
Cho, W., Choi, S., Park, D. K., Shin, I., & Choo, J. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.01089 · Cited by 154 · cited 1×
“[247] proposed the group-wise deep whitening-and-coloring transformation (GDWCT) by matching higher-order statistics, such as covariance, for image-to-image translation tasks.”
BackgroundNonlinear image representation using divisive normalization
Lyu, S., & Simoncelli, E. P. (2008) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2008.4587821 · Cited by 153
Natural Neural NetworksPreprint
Desjardins, G., Simonyan, K., Pascanu, R., & Kavukcuoglu, K. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1507.00210 · Cited by 147
Tsuzuku, Y., Sato, I., & Sugiyama, M. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.04034 · Cited by 147
An Investigation into Neural Net Optimization via Hessian Eigenvalue Density
Ghorbani, B., Krishnan, S., & Xiao, Y. (2019) · arXiv · doi:10.48550/arxiv.1901.10159 · Cited by 144
Universal representations:The missing link between faces, text, planktons, and cat breedsPreprint
Bilen, H., & Vedaldi, A. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1701.07275 · Cited by 144
Knowledge Matters: Importance of Prior Information for OptimizationPreprint
Gülçehre, Ç., & Bengio, Y. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1301.4083 · Cited by 140
Arpit, D., Zhou, Y., Kota, B. U., & Govindaraju, V. (2016) · International Conference on Machine Learning · doi:10.48550/arxiv.1603.01431 · Cited by 135
Self-citationA Mean Field Theory of Batch NormalizationPreprint
Yang, G., Pennington, J., Rao, V., Sohl‐Dickstein, J., & Schoenholz, S. S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.08129 · Cited by 134
Gated Orthogonal Recurrent Units: On Learning to Forget
Jing, L., Gülçehre, Ç., Peurifoy, J., Shen, Y., Tegmark, M., Marin Soljacic, A., & Bengio, Y. (2019) · Neural Computation · doi:10.1162/neco_a_01174 · Cited by 129
Training BatchNorm and Only BatchNorm: On the Expressive Power of Random Features in CNNsPreprint
Frankle, J., Schwab, D. J., & Morcos, A. S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.00152 · Cited by 129
GraphNorm: A Principled Approach to Accelerating Graph Neural Network TrainingPreprint
Cai, T., Luo, S., Xu, K., He, D., Liu, T., & Wang, L. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2009.03294 · Cited by 126
Micro-Batch Training with Batch-Channel Normalization and Weight Standardization
Qiao, S., Wang, H., Liu, C., Shen, W., & Yuille, A. (2019) · arXiv · doi:10.48550/arxiv.1903.10520 · Cited by 126
Meta-Learning Probabilistic Inference For Prediction
Gordon, J., Bronskill, J., Bauer, M., Nowozin, S., & Turner, R. E. (2018) · Apollo - University of Cambridge Repository · doi:10.17863/cam.47572 · Cited by 121
Cross-Iteration Batch Normalization
Yao, Z., Cao, Y., Zheng, S., Huang, G., & Lin, S. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01215 · Cited by 112
Evolving Normalization-Activation LayersPreprint
Liu, H., Brock, A., Simonyan, K., & Le, Q. V. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.02967 · Cited by 110
WNGrad: Learn the Learning Rate in Gradient DescentPreprint
Wu, X., Ward, R., & Bottou, L. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1803.02865 · Cited by 106
Learning values across many orders of magnitudePreprint
van Hasselt, H. P., Guez, A., Hessel, M., Mnih, V., & Silver, D. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1602.07714 · Cited by 104
On the Effects of Batch and Weight Normalization in Generative Adversarial NetworksPreprint
Xiang, S., & Li, H. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1704.03971 · Cited by 102
Deep Boltzmann Machines and the Centering Trick
Montavon, G., & Müller, K. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-35289-8_33 · Cited by 102
Improving Training of Deep Neural Networks via Singular Value Bounding
Jia, K., Tao, D., Gao, S., & Xu, X. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.425 · Cited by 95
On the Number of Linear Regions of Convolutional Neural NetworksPreprint
Xiong, H., Huang, L., Yu, M., Liu, L., Zhu, F., & Shao, L. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.00978 · Cited by 95
Self-citationOn Self Modulation for Generative Adversarial NetworksPreprint
Chen, T., Lučić, M., Houlsby, N., & Gelly, S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.01365 · Cited by 91
Dynamic Layer Normalization for Adaptive Neural Acoustic Modeling in Speech Recognition
Kim, T., Song, I., & Bengio, Y. (2017) · Interspeech (USB) · doi:10.21437/interspeech.2017-556 · Cited by 83
Batch Normalization Provably Avoids Rank Collapse for Randomly Initialised Deep NetworksPreprint
Daneshmand, H., Köhler, J., Bach, F., Hofmann, T., & Lucchi, A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.01652 · Cited by 78
Empirical Arithmetic Averaging Over the Compact Stiefel Manifold
Kaneko, T., Fiori, S., & Tanaka, T. (2012) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2012.2226167 · Cited by 77
Normalizing the Normalizers: Comparing and Extending Network Normalization SchemesPreprint
Ren, M., Liao, R., Urtasun, R., Sinz, F. H., & Zemel, R. S. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.04520 · Cited by 77
Second Order Properties of Error Surfaces.
LeCun, Y., Kanter, I., & Solla, S. A. (1990) · neural information processing systems · Cited by 76
Gitman, I., & Ginsburg, B. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1709.08145 · Cited by 75
Centered Weight Normalization in Accelerating Training of Deep Neural Networks
Huang, L., Liu, X., Liu, Y., Lang, B., & Tao, D. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.305 · Cited by 71
Self-citationTheoretical Analysis of Auto Rate-Tuning by Batch NormalizationPreprint
Arora, S., Li, Z., & Lyu, K. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1812.03981 · Cited by 71
Riemannian batch normalization for SPD neural networksPreprint
Brooks, D., Schwander, O., Barbaresco, F., Schneider, J.-Y., & Cord, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.02414 · Cited by 71
Revisiting Batch Normalization for Improving Corruption Robustness
Benz, P., Zhang, C., Karjauv, A., & Kweon, I. S. (2021) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv48630.2021.00054 · Cited by 67
Scaling up Natural Gradient by Sparsely Factorizing the Inverse Fisher Matrix
Grosse, R., & Salakhudinov, R. (2015) · International Conference on Machine Learning · Cited by 66
Distributed Second-Order Optimization using Kronecker-Factored Approximations
Ba, J. L., Grosse, R., & Martens, J. (2017) · International Conference on Learning Representations · Cited by 66
Exact natural gradient in deep linear networks and its application to the nonlinear case
Bernacchia, A., Lengyel, M., & Hennequin, G. (2018) · neural information processing systems · Cited by 66
Mean-normalized stochastic gradient for large-scale deep learning
Wiesler, S., Richard, A., Schlüter, R., & Ney, H. (2014) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2014.6853582 · Cited by 65
Generalized BackPropagation, Étude De Cas: OrthogonalityPreprint
Harandi, M. T., & Fernando, B. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.05927 · Cited by 63
CapProNet: Deep Feature Learning via Orthogonal Projections onto Capsule SubspacesPreprint
Zhang, L., Edraki, M., & Qi, G.-J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.07621 · Cited by 61
Training Faster by Separating Modes of Variation in Batch-Normalized Models
Kalayeh, M. M., & Shah, M. (2019) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2895781 · Cited by 59
Riemannian approach to batch normalizationPreprint
Cho, M., & Lee, J. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1709.09603 · Cited by 58
Weight StandardizationPreprint
Qiao, S., Wang, H., Liu, C., Shen, W., & Yuille, A. (2019) · arXiv (Cornell University) · Cited by 57
SSN: Learning Sparse Switchable Normalization via SparsestMax
Shao, W., Meng, T., Li, J., Zhang, R., Li, Y., Wang, X., & Luo, P. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00053 · Cited by 55
Instance Enhancement Batch Normalization: An Adaptive Regulator of Batch Noise
Liang, S., Huang, Z., Liang, M., & Yang, H. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i04.5917 · Cited by 55
New Interpretations of Normalization Methods in Deep Learning
Sun, J., Cao, X., Liang, H., Huang, W., Chen, Z., & Li, Z. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i04.6046 · Cited by 55
Attentive Normalization for Conditional Image Generation
Wang, Y., Chen, Y.-C., Zhang, X., Sun, J., & Jia, J. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00514 · Cited by 51
Uncertainty Estimation via Stochastic Batch Normalization
Atanov, A., Ashukha, A., Molchanov, D., Neklyudov, K., & Vetrov, D. (2019) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-22796-8_28 · Cited by 50
Four Things Everyone Should Know to Improve Batch NormalizationPreprint
Summers, C., & Dinneen, M. J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.03548 · Cited by 50
Towards Stabilizing Batch Statistics in Backward Propagation of Batch NormalizationPreprint
Yan, J., Wan, R., Zhang, X., Zhang, W., Wei, Y., & Sun, J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2001.06838 · Cited by 49
EvalNorm: Estimating Batch Normalization Statistics for Evaluation
Singh, S., & Shrivastava, A. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00373 · Cited by 47
Online Normalization for Training Neural NetworksPreprint
Chiley, V., Sharapov, I., Kosson, A., Köster, U., Reece, R., de la Fuente, S. S., Subbiah, V., & James, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.05894 · Cited by 45
Optimization on Submanifolds of Convolution Kernels in CNNsPreprint
Özay, M., & Okatani, T. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1610.07008 · Cited by 44
TaskNorm: Rethinking Batch Normalization for Meta-LearningPreprint
Bronskill, J. F., Gordon, J. A. R., Requeima, J., Nowozin, S., & Turner, R. E. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.03284 · Cited by 44
Ishii, M., & Sugiyama, M. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2101.10842 · Cited by 44
L2-Nonexpansive Neural NetworksPreprint
Qian, H., & Wegman, M. N. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.07896 · Cited by 42
Efficient Multi-Domain Learning by Covariance Normalization
Li, Y., & Vasconcelos, N. M. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00557 · Cited by 41
Controllable Orthogonalization in Training DNNs
Huang, L., Liu, L., Zhu, F., Wan, D., Yuan, Z., Li, B., & Shao, L. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00646 · Cited by 41
Self-citationRegularizing Deep Convolutional Neural Networks with a Structured Decorrelation Constraint
Xiong, W., Du, B., Zhang, L., Hu, R., & Tao, D. (2016) · Proceedings · doi:10.1109/icdm.2016.0063 · Cited by 40
FET-GAN: Font and Effect Transfer via K-shot Adaptive Instance Normalization
Li, W., He, Y., Qi, Y., et al. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i02.5535 · Cited by 40
Understanding Generalized Whitening and Coloring Transform for Universal Style Transfer
Chiu, T.-Y. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00455 · Cited by 39
OOGAN: Disentangling GAN with One-Hot Sampling and Orthogonal Regularization
Liu, B., Zhu, Y., Fu, Z., de Melo, G., & Elgammal, A. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i04.5919 · Cited by 39
How Does Batch Normalization Help Binary Training?Preprint
Sari, E., Belbahri, M., & Nia, V. P. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.09139 · Cited by 38
Deep Isometric Learning for Visual RecognitionPreprint
Qi, H., You, C., Wang, X., Ma, Y., & Malik, J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.16992 · Cited by 38
Momentum Batch Normalization for Deep Learning with Small Batch Size
Yong, H.-W., Huang, J.-Q., Meng, D.-Y., Hua, X.-S., & Zhang, L. (2020) · European Conference on Computer Vision · doi:10.1007/978-3-030-58610-2_14 · Cited by 38
Cited by
A review of convolutional neural networks in computer vision
Zhao, X., Wang, L., Zhang, Y., Han, X., Deveci, M., & Parmar, M. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-024-10721-6 · Cited by 1,148
Survey on Machine Learning Biases and Mitigation Techniques
Siddique, S., Haque, M. A., George, R., Gupta, K. D., Gupta, D., & Faruk, M. J. H. (2023) · Digital · doi:10.3390/digital4010001 · Cited by 78
Lamaakal, I., Ouahbi, I., El Makkaoui, K., Maleh, Y., Pławiak, P., & Alblehai, F. (2024) · IEEE Access · doi:10.1109/access.2024.3406631 · Cited by 34
Metadata
- Authors
- Huang, L., Qin, J., Zhou, Y., Zhu, F., Liu, L., & Shao, L.
- Venue
- IEEE Transactions on Pattern Analysis and Machine Intelligence
- Published
- 2023-08
- Volume
- 45
- Issue
- 8
- Pages
- 10173-10196
- ISSN
- 0162-8828 (ISSN-L)
- 2160-9292
- 1939-3539
- OpenAlex
- W4287660544
- Cited by
- 537
- References
- 250
- FWCI
- 45.07
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.