Group Normalization
Open accessThe DOI matches its Crossref record, with a small difference in title or year; 59 of the 66 references it cites were found in a registry or an index, and 7 are not checked yet.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Replace BatchNorm with GroupNorm Wu and He (2018) for stable training. This is important when the normalization layer is used in conjunction with Exponential Moving Average He et al. (2020) (commonly used in DDPMs).”
From Diffusion policy: Visuomotor policy learning via action diffusion (2025) · cites this paper as Wu and He (2018)
The cited source says what the claim attributes to it.
In the citing paper
“In particular, Sun et al. [44] elect to use group normalization [49] rather than BN, thus single point BN adaptation is not applicable for this model architecture.”
From MEMO: Test Time Robustness Via Adaptation and Augmentation (2022) · cites this paper as [49]
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Ensembling is also helpful to boost the performance, and to gain robustness against possible misalignments [8].”
From Audio-Visual Person-of-Interest DeepFake Detection (2023) · cites this paper as [8]
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“prevents loss of information in the representation learning process [50]”
From Robust Multimodal Learning with Missing Modalities via Parameter-Efficient Adaptation (2025) · cites this paper as [50]
Experimental — accuracy not yet measuredIn the citing paper
“For normalization, we use Group Normalization (GN) [22], which shows better than BatchNorm performance when batch size is small (bath size of 1 in our case).”
From 3D MRI Brain Tumor Segmentation Using Autoencoder Regularization (2019) · cites this paper as [22]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · cited 10×
“28 M training images and evaluate on the 50,000 validation images, using the ResNet models (He et al. 2016).”
BackgroundMethodInfluentialSelf-citationVery Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148 · cited 4×
“This possibly implies that VGG-16 benefits less from BN’s regularization effect, and GN (that leads to lower training error) is superior to BN in this case.”
BackgroundInfluentialImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · cited 6×
“Local Response Normalization (LRN) (Lyu and Simoncelli 2008; Jarrett et al. 2009; Krizhevsky et al. 2012) was a component in AlexNet (Krizhevsky et al.”
BackgroundMethodInfluentialHochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · cited 1×
“These methods are effective for training sequential models (RNN/LSTM [49, 22]) or generative models (GANs [15, 27]).”
MethodFaster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2577031 · Cited by 73,472
Self-citationFaster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2015) · arXiv · doi:10.48550/arxiv.1506.01497 · Cited by 70,812
Self-citationDistinctive Image Features from Scale-Invariant Keypoints
Lowe, D. G. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000029664.99615.94 · Cited by 55,589 · cited 5×
“Classical features of SIFT [39], HOG [9], and GIST [41] are group-wise representations by design, where each group of channels is constructed by some kind of histogram.”
BackgroundMethodInfluentialDensely Connected Convolutional Networks
Huang, G., Liu, Z., Van Der Maaten, L., & Weinberger, K. Q. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.243 · Cited by 48,566
Going deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562
Microsoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380 · cited 10×
“The effectiveness of GN in ImageNet, COCO, and Kinetics demonstrates that GN is a competitive alternative to BN that has been dominant in these tasks.”
BackgroundMethodInfluentialFully convolutional networks for semantic segmentation
Long, J., Shelhamer, E., & Darrell, T. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298965 · Cited by 38,834
Histograms of Oriented Gradients for Human Detection
Dalal, N., & Triggs, B. (2005) · 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) · doi:10.1109/cvpr.2005.177 · Cited by 33,584 · cited 6×
“Classical features of SIFT [39], HOG [9], and GIST [41] are group-wise representations by design, where each group of channels is constructed by some kind of histogram.”
BackgroundMethodInfluentialRethinking the Inception Architecture for Computer Vision
Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., & Wojna, Z. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.308 · Cited by 32,138
Learning representations by back-propagating errors
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986) · Nature · doi:10.1038/323533a0 · Cited by 31,892 · cited 1×
“These methods are effective for training sequential models (RNN/LSTM [49, 22]) or generative models (GANs [15, 27]).”
MethodFeature Pyramid Networks for Object Detection
Lin, T.-Y., Dollar, P., Girshick, R., He, K., Hariharan, B., & Belongie, S. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.106 · Cited by 30,939
Self-citationFocal Loss for Dense Object Detection
Lin, T.-Y., Goyal, P., Girshick, R., He, K., & Dollar, P. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.324 · Cited by 30,581 · cited 1×
“These computer vision tasks in general benefit from higher-resolution input, so the batch size tends to be small in common practice (1 or 2 images/GPU [12, 47, 18, 36]).”
BackgroundSelf-citationHe, K., Gkioxari, G., Dollar, P., & Girshick, R. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.322 · Cited by 29,137
Self-citationGirshick, R. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.169 · Cited by 28,105
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S., & Szegedy, C. (2015) · arXiv · doi:10.48550/arxiv.1502.03167 · Cited by 27,818 · cited 9×
“Unlike recent meth-ods [26, 3, 61], LRN computes the statistics in a small neighborhood for each pixel.”
BackgroundMethodInfluentialImage-to-Image Translation with Conditional Adversarial Networks
Isola, P., Zhu, J.-Y., Zhou, T., & Efros, A. A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.632 · Cited by 22,930 · cited 1×
“1986; Hochreiter and Schmidhuber 1997) or generativemodels (GANs; Goodfellow et al. 2014; Isola et al. 2017).”
BackgroundXception: Deep Learning with Depthwise Separable Convolutions
Chollet, F. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.195 · Cited by 20,380 · cited 1×
“2017) andXception (Chollet 2017) exploit channel-wise (also called “depth-wise”) convolutions, which are group convolutions with a group number equal to the channel number.”
BackgroundDelving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification
He, K., Zhang, X., Ren, S., & Sun, J. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.123 · Cited by 19,741 · cited 1×
“To normalize hidden features, initialization methods (LeCun et al. 1998; Glorot and Bengio 2010; He et al. 2015) have been derived based on strong assumptions of feature distributions, which can become invalid when training evolves.”
MethodSelf-citationGAN(Generative Adversarial Nets)
(2017) · Journal of Japan Society for Fuzzy Theory and Intelligent Informatics · doi:10.3156/jsoft.29.5_177_2 · Cited by 19,270
MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
Howard, A. G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., & Adam, H. (2017) · arXiv · doi:10.48550/arxiv.1704.04861 · Cited by 19,015 · cited 1×
“MobileNet (Howard et al. 2017) andXception (Chollet 2017) exploit channel-wise (also called “depth-wise”) convolutions, which are group convolutions with a group number equal to the channel number.”
BackgroundVisualizing and understanding convolutional networks
Zeiler, M. D., & Fergus, R. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10590-1_53 · Cited by 15,839
(2010) · Cited by 14,134
Aggregated Residual Transformations for Deep Neural Networks
Xie, S., Girshick, R., Dollar, P., Tu, Z., & He, K. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.634 · Cited by 12,465 · cited 3×
“The work of ResNeXt (Xie et al. 2017) investigates the trade-off between depth, width, and groups, and it suggests that a larger number of groups can improve accuracy under similar computational cost.”
BackgroundSelf-citationTensorFlow: A system for large-scale machine learning
Abadi, M., Barham, P., Chen, J., Chen, Z., Davis, A., Dean, J., Devin, M., Ghemawat, S., Irving, G., Isard, M., et al. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1605.08695 · Cited by 11,822
(2017) · Cited by 11,625
Wang, X., Girshick, R., Gupta, A., & He, K. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00813 · Cited by 11,596
Self-citationShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices
Zhang, X., Zhou, X., Lin, M., & Sun, J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00716 · Cited by 9,982
ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices
Zhang, X., Zhou, X., Lin, M., & Sun, J. (2017) · arXiv · doi:10.48550/arxiv.1707.01083 · Cited by 9,982 · cited 1×
“ShuffleNet (Zhang et al. 2018) proposes a channel shuffle operation that permutes the axes of grouped features.”
BackgroundLearning Spatiotemporal Features with 3D Convolutional Networks
Tran, D., Bourdev, L., Fergus, R., Torresani, L., & Paluri, M. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.510 · Cited by 9,593 · cited 3×
“For example, the Fast/er and Mask R-CNN frameworks [12, 47, 18] use a batch size of 1 or 2 images because of higher resolution, where BN is “frozen” by transforming to a linear layer [20]; in video classification with 3D convolutions [60, 6], the presence of spatial-temporal features introduces a trade-off between the…”
BackgroundMethodQuo Vadis, Action Recognition? A New Model and the Kinetics Dataset
Carreira, J., & Zisserman, A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.502 · Cited by 9,570
Mastering the game of Go without human knowledge
Silver, D., Schrittwieser, J., Simonyan, K., Antonoglou, I., Huang, A., Guez, A., Hubert, T., Baker, L., Lai, M., Bolton, A., et al. (2017) · Nature · doi:10.1038/nature24270 · Cited by 9,084 · cited 2×
“We will also investigate GN’s performance on learning representations for reinforcement learning (RL) tasks, e.g ., [54], where BN is playing an important role for training very deep models [20].”
BackgroundModeling the Shape of the Scene: A Holistic Representation of the Spatial Envelope
Oliva, A., & Torralba, A. (2001) · International Journal of Computer Vision · doi:10.1023/a:1011139631724 · Cited by 6,319 · cited 1×
“Classical features of SIFT [39], HOG [9], and GIST [41] are group-wise representations by design, where each group of channels is constructed by some kind of histogram.”
MethodInception-v4, Inception-ResNet and the Impact of Residual Connections on Learning
Szegedy, C., Ioffe, S., Vanhoucke, V., & Alemi, A. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.11231 · Cited by 4,636 · cited 1×
“As a result, many recent models [2,3,5,6,7] are trained with non-trivial batch sizes that are memory-consuming.”
BackgroundLarge Scale Distributed Deep Networks
Dean, J. B., Corrado, G. S., Monga, R., Chen, K., Devin, M., Mao, M., Ranzato, M., Senior, A., Tucker, P. A., Yang, K., et al. (2012) · neural information processing systems · Cited by 4,489
Instance Normalization: The Missing Ingredient for Fast Stylization
Ulyanov, D., Vedaldi, A., & Lempitsky, V. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1607.08022 · Cited by 2,775 · cited 9×
“There have been existing methods, such as Layer Normalization (LN) [3] and Instance Normalization (IN) [61] (Figure 2), that also avoid normalizing along the batch dimension.”
BackgroundMethodInfluentialAggregating local descriptors into a compact image representation
Jegou, H., Douze, M., Schmid, C., & Perez, P. (2010) · 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2010.5540039 · Cited by 2,692 · cited 2×
“Higher-level features such as VLAD [29] and Fisher Vectors (FV) [44] are also group-wise features where a group can be thought of as the sub-vector computed with respect to a cluster.”
MethodOverFeat: Integrated Recognition, Localization and Detection using Convolutional Networks
Sermanet, P., Eigen, D., Zhang, X., Mathieu, M., Fergus, R., & LeCun, Y. (2013) · arXiv · doi:10.48550/arxiv.1312.6229 · Cited by 2,680
The Kinetics Human Action Video Dataset
Kay, W., Carreira, J., Simonyan, K., Zhang, B., Hillier, C., Vijayanarasimhan, S., Viola, F., Green, T., Back, T., Natsev, P., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.06950 · Cited by 2,593 · cited 2×
“2014), and on 3D convolutional networks for Kinetics video classification (Kay et al. 2017).”
BackgroundMethodNatural Image Statistics and Neural Representation
Simoncelli, E. P., & Olshausen, B. A. (2001) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.24.1.1193 · Cited by 2,460 · cited 1×
“In fact, a well-accepted computational model in neuroscience is to normalize across the cell responses [21, 52, 55, 5], “with various receptive-field centers (covering the visual field) and with various spatiotemporal frequency tunings” (p183, [21]); this can happen not only in the primary visual cortex, but also “thr…”
BackgroundAccurate, Large Minibatch SGD: Training ImageNet in 1 Hour
Goyal, P., Dollár, P., Girshick, R., Noordhuis, P., Wesolowski, L., Kyrola, A., Tulloch, A., Jia, Y., & He, K. (2017) · arXiv · doi:10.48550/arxiv.1706.02677 · Cited by 2,294 · cited 8×
“BN is also shown with 1024 GPUs (a total batch size of 32768), when it starts to show degradation (Goyal et al. 2017)”
BackgroundMethodInfluentialSelf-citationWhat is the best multi-stage architecture for object recognition?
Jarrett, K., Kavukcuoglu, K., Ranzato, M. A., & LeCun, Y. (2009) · 2009 IEEE 12th International Conference on Computer Vision · doi:10.1109/iccv.2009.5459469 · Cited by 2,153 · cited 1×
“Normalization layers in deep networks had been widely used before the development of BN. Local Response Nor-malization (LRN) [40, 28, 32] was a component in AlexNet [32] and following models [64, 53, 58].”
MethodNormalization as a canonical neural computation
Carandini, M., & Heeger, D. J. (2012) · Nature Reviews Neuroscience · doi:10.1038/nrn3136 · Cited by 1,906 · cited 1×
“In fact, a well-accepted computational model in neuroscience is to normalize across the cell responses [21, 52, 55, 5], “with various receptive-field centers (covering the visual field) and with various spatiotemporal frequency tunings” (p183, [21]); this can happen not only in the primary visual cortex, but also “thr…”
BackgroundNormalization of cell responses in cat striate cortex
Heeger, D. J. (1992) · Visual Neuroscience · doi:10.1017/s0952523800009640 · Cited by 1,835
Fisher Kernels on Visual Vocabularies for Image Categorization
Perronnin, F., & Dance, C. (2007) · 2007 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2007.383266 · Cited by 1,641 · cited 1×
“2010) and Fisher Vectors (FV) (Perronnin and Dance 2007) are also group-wise features where a group can be thought of as the sub-vector computed with respect to a cluster.”
MethodOne weird trick for parallelizing convolutional neural networks
Krizhevsky, A. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1404.5997 · Cited by 919
Weight Normalization: A Simple Reparameterization to Accelerate Training of Deep Neural Networks
Salimans, T., & Kingma, D. P. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1602.07868 · Cited by 883 · cited 4×
“4 For completeness, we have also trained ResNet-50 with WN (Salimans and Kingma 2016), which is filter (instead of feature) normalization.”
BackgroundMethodInfluentialNatural signal statistics and sensory gain control
Schwartz, O., & Simoncelli, E. P. (2001) · Nature Neuroscience · doi:10.1038/90526 · Cited by 835 · cited 1×
“In fact, a well-accepted computational model in neuroscience is to normalize across the cell responses [21, 52, 55, 5], “with various receptive-field centers (covering the visual field) and with various spatiotemporal frequency tunings” (p183, [21]); this can happen not only in the primary visual cortex, but also “thr…”
BackgroundLearning multiple visual domains with residual adapters
Rebuffi, S.-A., Bilen, H., & Vedaldi, A. (2017) · arXiv · doi:10.48550/arxiv.1705.08045 · Cited by 605
Group Equivariant Convolutional Networks
Cohen, T., & Welling, M. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1602.07576 · Cited by 524 · cited 2×
“However, in addition to orientations (SIFT [14], HOG [15], or [44,45]), there are many factors that could lead to grouping, e.”
BackgroundObject Detection Networks on Convolutional Feature Maps
Ren, S., He, K., Girshick, R., Zhang, X., & Sun, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2601099 · Cited by 454
Self-citationObject Detection Networks on Convolutional Feature Maps
Ren, S., He, K., Girshick, R., Zhang, X., & Sun, J. (2015) · arXiv · doi:10.48550/arxiv.1504.06066 · Cited by 454 · cited 1×
“We find that replacing the 2fc box head with 4conv1fc (similar to [48]) can better leverage GN.”
BackgroundSelf-citationVisualizing and Understanding Convolutional Neural NetworksPreprint
Zeiler, M. D., & Fergus, R. (2013) · arXiv (Cornell University) · Cited by 442
Ba, J. L., Kiros, J., & Hinton, G. E. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1607.06450 · Cited by 440 · cited 9×
“Several normalization methods [3, 61, 51, 2, 46] have been proposed to avoid exploiting the batch dimension.”
BackgroundMethodInfluentialBatch Renormalization: Towards Reducing Minibatch Dependence in Batch-Normalized Models
Ioffe, S. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1702.03275 · Cited by 367 · cited 3×
“Comparison with Batch Renorm (BR) BR (Ioffe 2017) introduces two extra parameters [r and d in Ioffe (2017)] that constrain the estimated mean and variance of BN.”
BackgroundInfluentialDetNet: Design Backbone for Object Detection
Li, Z., Peng, C., Yu, G., Zhang, X., Deng, Y., & Sun, J. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01240-3_21 · Cited by 337
MegDet: A Large Mini-Batch Object Detector
Peng, C., Xiao, T., Li, Z., Jiang, Y., Zhang, X., Jia, K., Yu, G., & Sun, J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00647 · Cited by 311
DetNet: A Backbone network for Object Detection
Li, Z., Peng, C., Yu, G., Zhang, X., Deng, Y. S., & Sun, J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1804.06215 · Cited by 216 · cited 2×
“Here the BN results are from [34], and BN is synced across GPUs [43] and is not frozen.”
BackgroundExploiting Cyclic Symmetry in Convolutional Neural Networks
Dieleman, S., Fauw, J. D., & Kavukcuoglu, K. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1602.02660 · Cited by 149 · cited 2×
“However, in addition to orientations (SIFT [14], HOG [15], or [44,45]), there are many factors that could lead to grouping, e.”
BackgroundNonlinear image representation using divisive normalization
Lyu, S., & Simoncelli, E. P. (2008) · 2008 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2008.4587821 · Cited by 146 · cited 3×
“Normalization layers in deep networks had been widely used before the development of BN. Local Response Nor-malization (LRN) [40, 28, 32] was a component in AlexNet [32] and following models [64, 53, 58].”
Arpit, D., Zhou, Y., Kota, B. U., & Govindaraju, V. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1603.01431 · Cited by 81 · cited 2×
“moments(x, [2, 3, 4], keep dims=True) x = (x − mean) / tf.”
MethodNormalizing the Normalizers: Comparing and Extending Network Normalization Schemes
Ren, M., Liao, R., Urtasun, R., Sinz, F. H., & Zemel, R. S. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.04520 · Cited by 66 · cited 1×
“Several normalization methods [17,18,19,33,34] have been proposed to avoid exploiting the batch dimension.”
Method
Research works printed but not linked: 6
- Bottou, L., Curtis, F. E., & Nocedal, J. (2016). Optimization methods for large-scale machine learning. arXiv:1606.04838 . · arXiv
- Gitman, Y. Y. I., & Ginsburg, B. (2017). Scaling SGD batch size to 32 k for imagenet training. arXiv:1708.03888 . · arXiv
- Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., & Bengio, Y. (2014). Generative adversarial nets. In Neural information processing systems (NeurIPS).
- He, K., Girshick, R., & Dollár, P. (2018). Rethinking imagenet pre-training. arXiv:1811.08883 . · arXiv
- Shillingford, B., Assael, Y., Hoffman, M. W., Paine, T., Hughes, C., Prabhu, U., et al. (2018). Large-scale visual speech recognition. arXiv:1807.05162 . · arXiv
- Wu, Y., & He, K. (2018). Group normalization. In European conference on computer vision (ECCV). · doi:10.1007/978-3-030-01261-8_1 · Paper page
Cited by
nnU-Net: a self-configuring method for deep learning-based biomedical image segmentation
Isensee, F., Jaeger, P. F., Kohl, S. A. A., Petersen, J., & Maier-Hein, K. H. (2021) · Nature Methods · doi:10.1038/s41592-020-01008-z · Cited by 10,477
“Batch normalization 43 , which is often used to speed up or stabilize training, does not perform well with small batch sizes 47,48 .”
MethodDenoising Diffusion Probabilistic Models
Ho, J., Jain, A., & Abbeel, P. (2020) · arXiv · doi:10.48550/arxiv.2006.11239 · Cited by 8,712
“To represent the reverse process, we use a U-Net backbone similar to an unmasked PixelCNN++ [49, 45] with group normalization throughout [62].”
MethodLiu, Z., Mao, H., Wu, C.-Y., Feichtenhofer, C., Darrell, T., & Xie, S. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01167 · Cited by 8,494
“There have been numerous attempts at developing alternative normalization [60,75,83] techniques, but BN has remained the preferred option in most vision tasks.”
BackgroundMethodFCOS: Fully Convolutional One-Stage Object Detection
Tian, Z., Shen, C., Chen, H., & He, T. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00972 · Cited by 7,013
Diffusion Models Beat GANs on Image Synthesis
Dhariwal, P., & Nichol, A. (2021) · arXiv · doi:10.48550/arxiv.2105.05233 · Cited by 3,362
“We also experiment with a layer [43] that we refer to as adaptive group normalization (AdaGN), which incorporates the timestep and class embedding into each residual block after a group normalization operation [69], similar to adaptive instance norm [27] and FiLM [48].”
Method
Metadata
- Authors
- Wu, Y., & He, K.
- Affiliations
- Meta (United States) (US)
- Corresponding author
- He, Kaiming
- ORCID
- Venue
- International Journal of Computer Vision
- Published
- 2019-07-22
- Volume
- 128
- Issue
- 3
- Pages
- 742-755
- ISSN
- 0302-9743 (ISSN-L)
- 0920-5691
- 1573-1405
- OpenAlex
- W2795783309
- Cited by
- 3,601
- References
- 61
- Citations by year
- 2026: 399 · 2025: 578 · 2024: 599 · 2023: 445 · 2022: 497 · 2021: 502 · 2020: 388 · 2019: 183
- FWCI
- 81.61
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- www.springer.com/tdm (tdm)
- www.springer.com/tdm (vor)
- Locations
- Lecture notes in computer science (publishedVersion)
- International Journal of Computer Vision (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Stochastic Gradient Optimization Techniques
- Keywords
- group normalization, batch normalization, small batch sizes, pre-training and fine-tuning, object detection, semantic segmentation, video classification, ResNet50
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.