Normalizing the Normalizers: Comparing and Extending Network Normalization Schemes
Open accessThe DOI matches its OpenAlex record; 9 of the 9 references it cites were found in a registry or an index.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
Image Super-Resolution Using Deep Convolutional Networks
Dong, C., Loy, C. C., He, K., & Tang, X. (2016) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2015.2439281 · Cited by 10,105
Image Style Transfer Using Convolutional Neural Networks
Gatys, L. A., Ecker, A. S., & Bethge, M. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.265 · Cited by 6,025
Normalization of cell responses in cat striate cortex
Heeger, D. J. (1992) · Visual Neuroscience · doi:10.1017/s0952523800009640 · Cited by 1,835 · cited 2×
“Divisive Normalization (DN) on the other hand modulates the neural activity by the activity of a pool of neighboring neurons (Heeger, 1992; Bonds, 1989).”
BackgroundThe Normalization Model of Attention
Reynolds, J. H., & Heeger, D. J. (2009) · Neuron · doi:10.1016/j.neuron.2009.01.002 · Cited by 1,457 · cited 2×
“…(Simoncelli & Heeger, 1998), stimulus decoding (Ringach, 2009; Froudarakis et al., 2014), winner-take-all mechanisms (Busse et al., 2009), attention (Reynolds & Heeger, 2009), redundancy reduction (Schwartz & Simoncelli, 2001; Sinz & Bethge, 2008; Lyu & Simoncelli, 2008; Sinz & Bethge, 2013),…”
A model of neuronal responses in visual area MT
Simoncelli, E. P., & Heeger, D. J. (1998) · Vision Research · doi:10.1016/s0042-6989(97)00183-1 · Cited by 941 · cited 2×
“In this section we first review related work on normalization, followed by a brief description of regularization in neural networks.”
BackgroundNatural signal statistics and sensory gain control
Schwartz, O., & Simoncelli, E. P. (2001) · Nature Neuroscience · doi:10.1038/90526 · Cited by 835
Divisive Normalization in Olfactory Population Codes
Olsen, S. R., Bhandawat, V., & Wilson, R. I. (2010) · Neuron · doi:10.1016/j.neuron.2010.04.009 · Cited by 500 · cited 2×
“weights. Previous theoretical studies have outlined several potential computational roles for divisive normalization such as sensitivity maximization (Carandini & Heeger, 2012), invariant coding (Olsen et al., 2010), density modelling (Balle et al., 2016), image compression (Malo et al., 2006), distributed´ neural…”
BackgroundRepresentation of Concurrent Stimuli by Population Activity in Visual Cortex
Busse, L., Wade, A. R., & Carandini, M. (2009) · Neuron · doi:10.1016/j.neuron.2009.11.004 · Cited by 247 · cited 2×
“…neural representations (Simoncelli & Heeger, 1998), stimulus decoding (Ringach, 2009; Froudarakis et al., 2014), winner-take-all mechanisms (Busse et al., 2009), attention (Reynolds & Heeger, 2009), redundancy reduction (Schwartz & Simoncelli, 2001; Sinz & Bethge, 2008; Lyu & Simoncelli,…”
Cited by
Arbitrary Style Transfer in Real-Time with Adaptive Instance Normalization
Huang, X., & Belongie, S. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.167 · Cited by 5,076
Wu, Y., & He, K. (2020) · International Journal of Computer Vision · doi:10.1007/s11263-019-01198-w · Cited by 3,601
Normalization Techniques in Training DNNs: Methodology, Analysis and Application
Huang, L., Qin, J., Zhou, Y., Zhu, F., Liu, L., & Shao, L. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3250241 · Cited by 537
Generalisation in humans and deep neural networks
Geirhos, R., Temme, C. R. M., Rauber, J., Schütt, H. H., Bethge, M., & Wichmann, F. A. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1808.08750 · Cited by 399
Micro-Batch Training with Batch-Channel Normalization and Weight Standardization
Qiao, S., Wang, H., Liu, C., Shen, W., & Yuille, A. (2019) · arXiv · doi:10.48550/arxiv.1903.10520 · Cited by 126
“To explore other dimensions [2, 33], Layer Normalization [3] normalizes data on the channel dimension, Instance Normalization [40] performs BN for each sample individually, and Group Nor-malization [42] finds a better middle point between Layer Normalization and Instance Normalization.”
Method
Metadata
- Authors
- Ren, M., Liao, R., Urtasun, R., Sinz, F. H., & Zemel, R. S.
- Venue
- arXiv (Cornell University)
- Publisher
- Cornell University
- Published
- 2016-11-14
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1611.04520
- OpenAlex
- W2568343048
- Cited by
- 66
- References
- 9
- Type
- posted content
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Normalizing the Normalizers: Comparing and Extending Network Normalization Schemes. https://app.cochat.ai/papers/10.48550/arxiv.1611.04520
- The paper
- Ren, M., Liao, R., Urtasun, R., Sinz, F. H., & Zemel, R. S. (2016). Normalizing the Normalizers: Comparing and Extending Network Normalization Schemes. arXiv (Cornell University). https://doi.org/10.48550/arxiv.1611.04520
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.