Improved regularization of convolutional neural networks with cutout
Open accessAbstract
Convolutional neural networks are capable of learning powerful representational spaces, which are necessary for tackling complex learning tasks. However, due to the model capacity required to capture such representations, they are often susceptible to overfitting and therefore require proper regularization in order to generalize well. In this paper, we show that the simple regularization technique of randomly masking out square regions of input during training, which we call cutout, can be used to improve the robustness and overall performance of convolutional neural networks. Not only is this method extremely easy to implement, but we also demonstrate that it can be used in conjunction with existing forms of data augmentation and other regularizers to further improve model performance. We evaluate this method by applying it to current state-of-the-art architectures on the CIFAR-10, CIFAR-100, and SVHN datasets, yielding new state-of-the-art results of 2.56%, 15.20%, and 1.30% test error respectively. Code is available at https://github.com/uoguelph-mlrg/Cutout
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The mask strategy leads to better performance than the well-known Cutout (1.46%).
“Besides, the mask strategy leads to better performance than the well-known Cutout ( 1 . 46% ).”
From TALL: Thumbnail Layout for Deepfake Video Detection (2023) · cites this paper as [8]
Section 3 (p. 3) identifies and describes Cutout as an image-masking technique. The cited paper does not compare it with the citing authors’ mask strategy or establish the stated 1.46% figure; Section 4.2 (p. 6) instead reports 1.30% test error for Cutout on SVHN. (LLM, conf=0.82)
Experimental — accuracy not yet measuredThe performance of a random Cutout on four sub-images is better than on one sub-image.
“The performance of a random Cutout [8] on four sub-images is better than on one subimage.”
From TALL: Thumbnail Layout for Deepfake Video Detection (2023) · cites this paper as [8]
The Implementation Details (p. 4) describe applying one randomly located mask to each input image. The passages found when searching for four versus one sub-image and multiple patches contain no performance comparison between those arrangements. (LLM, conf=0.76)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
No citation lists for this paper
No citation lists for this paper yet — checked again daily.
References 19 · Cited by 2,776
Research works printed but not linked: 12
- [1][1] Y. Bengio, A. Bergeron, N. Boulanger-Lewandowski, T. Breuel, Y. Chherawala, et al. Deep learners benefit more from out-of-distribution examples. In Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics , pages 164-172, 2011. -
- [2][2] A. Canziani, A. Paszke, and E. Culurciello. An analysis of deep neural network models for practical applications. In IEEE International Symposium on Circuits & Systems , 2016. -
- [3][3] A. Coates, A. Ng, and H. Lee. An analysis of single-layer networks in unsupervised feature learning. In Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics , pages 215-223, 2011. -
- [8][8] A. Krizhevsky, I. Sutskever, and G. E. Hinton. Imagenet classification with deep convolutional neural networks. In Advances in Neural Information Processing Systems , pages 1097-1105, 2012. -
- [9][9] Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner. Gradientbased learning applied to document recognition. Proceedings of the IEEE , 86(11):2278-2324, 1998. -
- [10][10] J. Lemley, S. Bazrafkan, and P. Corcoran. Smart augmentation-learning an optimal data augmentation strategy. IEEE Access , 2017. -
- [11][11] J. Long, E. Shelhamer, and T. Darrell. Fully convolutional networks for semantic segmentation. In CVPR , pages 34313440, 2015. -
- [12][12] Y. Netzer, T. Wang, A. Coates, A. Bissacco, B. Wu, and A. Y. Ng. Reading digits in natural images with unsupervised feature learning. In NIPS Workshop on Deep Learning and Unsupervised Feature Learning , volume 2011, page 5, 2011. -
- [13][13] S. Park and N. Kwak. Analysis on the dropout effect in convolutional neural networks. In Asian Conference on Computer Vision , pages 189-204. Springer, 2016. -
- [15][15] N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov. Dropout: A simple way to prevent neural networks from overfitting. The Journal of Machine Learning Research , 15(1):1929-1958, 2014. -
- [18][18] P. Vincent, H. Larochelle, I. Lajoie, Y. Bengio, and P.A. Manzagol. Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion. Journal of Machine Learning Research , 11(Dec):3371-3408, 2010. -
- [22][22] S. Zagoruyko and N. Komodakis. Wide residual networks. British Machine Vision Conference (BMVC) , 2016.
- Authors
- DeVries, T., & Taylor, G. W.
- ORCID
- Taylor, Graham W.
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2017
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1708.04552
- OpenAlex
- W2746314669
- Cited by
- 2,776
- References
- 19
- Citations by year
- 2026: 14 · 2025: 226 · 2024: 465 · 2023: 535 · 2022: 379 · 2021: 555 · 2020: 370 · 2019: 180 · 2018: 49 · 2017: 2
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Data Classification
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- cutouts, input masking, regularization, convolutional neural networks, overfitting, data augmentation, robustness, image classification, CIFAR-10, CIFAR-100, SVHN
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
