CoChat
Sign Up
PreprintCited by 5,439Advanced Neural Network ApplicationsDomain Adaptation and Few-Shot LearningMachine Learning and Data Classification

CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Features

Open access
Yun, Sangdoo, Han, Dongyoon, Oh, Seong Joon, Chun, Sanghyuk, Choe, Junsuk, Yoo, Youngjoon·2019·arXiv
DOI 10.48550/arxiv.1905.04899arXiv 1905.04899
Open this paper in CoChat — free accountOpen PDF

Abstract

Regional dropout strategies have been proposed to enhance the performance of convolutional neural network classifiers. They have proved to be effective for guiding the model to attend on less discriminative parts of objects (e.g. leg as opposed to head of a person), thereby letting the network generalize better and have better object localization capabilities. On the other hand, current methods for regional dropout remove informative pixels on training images by overlaying a patch of either black pixels or random noise. Such removal is not desirable because it leads to information loss and inefficiency during training. We therefore propose the CutMix augmentation strategy: patches are cut and pasted among training images where the ground truth labels are also mixed proportionally to the area of the patches. By making efficient use of training pixels and retaining the regularization effect of regional dropout, CutMix consistently outperforms the state-of-the-art augmentation strategies on CIFAR and ImageNet classification tasks, as well as on the ImageNet weakly-supervised localization task. Moreover, unlike previous augmentation methods, our CutMix-trained ImageNet classifier, when used as a pretrained model, results in consistent performance gains in Pascal detection and MS-COCO image captioning benchmarks. We also show that CutMix improves the model robustness against input corruptions and its out-of-distribution detection performances. Source code and pretrained models are available at https://github.com/clovaai/CutMix-PyTorch .

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.1905.04899

Cites

2 of 2 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 48

  1. Liang-Chieh Chen, George Papandreou, Iasonas Kokkinos, Kevin Murphy, and Alan L Yuille. Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolu- tion, and fully connected crfs. IEEE transactions on pattern analysis and machine intelligence, 40(4):834-848, 2018.
  2. Junsuk Choe and Hyunjung Shim. Attention-based dropout layer for weakly supervised object localization. In Proceed- ings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 2219-2228, 2019.
  3. Terrance DeVries and Graham W Taylor. Improved regular- ization of convolutional neural networks with cutout. arXiv preprint arXiv:1708.04552, 2017. · arXiv
  4. Nikita Dvornik, Julien Mairal, and Cordelia Schmid. Mod- eling visual context is key to augmenting object detection datasets. In Proceedings of the European Conference on Computer Vision (ECCV), pages 364-380, 2018.
  5. Debidatta Dwibedi, Ishan Misra, and Martial Hebert. Cut, paste and learn: Surprisingly easy synthesis for instance de- tection. In Proceedings of the IEEE International Confer- ence on Computer Vision, pages 1301-1310, 2017.
  6. M. Everingham, L. Van Gool, C. K. I. Williams, J. Winn, and A. Zisserman. The pascal visual object classes (voc) chal- lenge. International Journal of Computer Vision, 88(2):303- 338, June 2010.
  7. Robert Geirhos, Patricia Rubisch, Claudio Michaelis, Matthias Bethge, Felix A Wichmann, and Wieland Brendel. Imagenet-trained cnns are biased towards texture; increasing shape bias improves accuracy and robustness. arXiv preprint arXiv:1811.12231, 2018. · arXiv
  8. Golnaz Ghiasi, Tsung-Yi Lin, and Quoc V Le. Dropblock: A regularization method for convolutional networks. In Advances in Neural Information Processing Systems, pages 10750-10760, 2018.
  9. Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. In Interna- tional Conference on Learning Representations, 2015.
  10. Hongyu Guo, Yongyi Mao, and Richong Zhang. Mixup as locally linear out-of-manifold regularization. arXiv preprint arXiv:1809.02499, 2018. · arXiv
  11. Dongyoon Han, Jiwhan Kim, and Junmo Kim. Deep pyra- midal residual networks. In CVPR, 2017.
  12. Dan Hendrycks and Kevin Gimpel. A baseline for detect- ing misclassified and out-of-distribution examples in neural networks. In International Conference on Learning Repre- sentations, 2017.
  13. Jie Hu, Li Shen, Samuel Albanie, Gang Sun, and Andrea Vedaldi. Gather-excite: Exploiting feature context in convo- lutional neural networks. In Advances in Neural Information Processing Systems, pages 9423-9433, 2018.
  14. Jie Hu, Li Shen, and Gang Sun. Squeeze-and-excitation net- works. In arXiv:1709.01507, 2017. · arXiv
  15. Gao Huang, Zhuang Liu, and Kilian Q Weinberger. Densely connected convolutional networks. In CVPR, 2017.
  16. Gao Huang, Yu Sun, Zhuang Liu, Daniel Sedra, and Kilian Weinberger. Deep networks with stochastic depth. In ECCV, 2016.
  17. Sergey Ioffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal co- variate shift. In ICML, 2015.
  18. Hanjoo Kim, Minkyu Kim, Dongjoo Seo, Jinwoong Kim, Heungseok Park, Soeun Park, Hyunwoo Jo, KyungHyun Kim, Youngil Yang, Youngkwan Kim, Nako Sung, and Jung- Woo Ha. NSML: meet the mlaas platform with a real-world case study. CoRR, abs/1810.09957, 2018. · arXiv
  19. Kimin Lee, Kibok Lee, Honglak Lee, and Jinwoo Shin. A simple unified framework for detecting out-of-distribution samples and adversarial attacks. In Advances in Neural In- formation Processing Systems, pages 7167-7177, 2018.
  20. Shiyu Liang, Yixuan Li, and R Srikant. Enhancing the re- liability of out-of-distribution image detection in neural net- works. In International Conference on Learning Represen- tations, 2018.
  21. Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In European conference on computer vision, pages 740-755. Springer, 2014.
  22. Wei Liu, Dragomir Anguelov, Dumitru Erhan, Christian Szegedy, Scott Reed, Cheng-Yang Fu, and Alexander C Berg. Ssd: Single shot multibox detector. In European con- ference on computer vision, pages 21-37. Springer, 2016.
  23. Jonathan Long, Evan Shelhamer, and Trevor Darrell. Fully convolutional networks for semantic segmentation. In The IEEE Conference on Computer Vision and Pattern Recogni- tion (CVPR), June 2015.
  24. Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learn- ing models resistant to adversarial attacks. arXiv preprint arXiv:1706.06083, 2017. · arXiv
  25. Dhruv Mahajan, Ross Girshick, Vignesh Ramanathan, Kaiming He, Manohar Paluri, Yixuan Li, Ashwin Bharambe, and Laurens van der Maaten. Exploring the limits of weakly supervised pretraining. In Proceedings of the European Con- ference on Computer Vision (ECCV), pages 181-196, 2018.
  26. Hyeonseob Nam and Bohyung Han. Learning multi-domain convolutional neural networks for visual tracking. In Pro- ceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 4293-4302, 2016.
  27. Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. Faster r-cnn: Towards real-time object detection with region proposal networks. In NIPS, 2015.
  28. Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, San- jeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Li Fei-Fei. Imagenet large scale visual recognition challenge. International Journal of Computer Vision, 115(3):211-252, 2015.
  29. Karen Simonyan and Andrew Zisserman. Two-stream con- volutional networks for action recognition in videos. In Ad- vances in neural information processing systems, pages 568- 576, 2014.
  30. Krishna Kumar Singh and Yong Jae Lee. Hide-and-seek: Forcing a network to be meticulous for weakly-supervised object and action localization. In 2017 IEEE International Conference on Computer Vision (ICCV), pages 3544-3553. IEEE, 2017.
  31. Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. Dropout: A simple way to prevent neural networks from overfitting. Journal of Machine Learning Research, 15:1929-1958, 2014.
  32. Cecilia Summers and Michael J Dinneen. Improved mixed- example data augmentation. In 2019 IEEE Winter Confer- ence on Applications of Computer Vision (WACV), pages 1262-1270. IEEE, 2019.
  33. Christian Szegedy, Sergey Ioffe, and Vincent Vanhoucke. Inception-v4, inception-resnet and the impact of residual connections on learning. In ICLR Workshop, 2016.
  34. Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich. Going deeper with convolutions. In CVPR, 2015.
  35. Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jon Shlens, and Zbigniew Wojna. Rethinking the inception archi- tecture for computer vision. In Proceedings of the IEEE con- ference on computer vision and pattern recognition, pages 2818-2826, 2016.
  36. Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. arXiv preprint arXiv:1312.6199, 2013. · arXiv
  37. Ryo Takahashi, Takashi Matsubara, and Kuniaki Uehara. Ri- cap: Random image cropping and patching data augmenta- tion for deep cnns. In Asian Conference on Machine Learn- ing, pages 786-798, 2018.
  38. Yuji Tokozume, Yoshitaka Ushiku, and Tatsuya Harada. Between-class learning for image classification. In Proceed- ings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 5486-5494, 2018.
  39. Vikas Verma, Alex Lamb, Christopher Beckham, Amir Na- jafi, Ioannis Mitliagkas, David Lopez-Paz, and Yoshua Ben- gio. Manifold mixup: Better representations by interpolat- ing hidden states. In International Conference on Machine Learning, pages 6438-6447, 2019.
  40. Oriol Vinyals, Alexander Toshev, Samy Bengio, and Du- mitru Erhan. Show and tell: A neural image caption gen- erator. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 3156-3164, 2015.
  41. Saining Xie, Ross Girshick, Piotr Dollár, Zhuowen Tu, and Kaiming He. Aggregated residual transformations for deep neural networks. In CVPR, 2017.
  42. Yoshihiro Yamada, Masakazu Iwamura, Takuya Akiba, and Koichi Kise. Shakedrop regularization for deep residual learning. arXiv preprint arXiv:1802.02375, 2018. · arXiv
  43. Fisher Yu, Ari Seff, Yinda Zhang, Shuran Song, Thomas Funkhouser, and Jianxiong Xiao. Lsun: Construction of a large-scale image dataset using deep learning with humans in the loop. arXiv preprint arXiv:1506.03365, 2015. · arXiv
  44. Hongyi Zhang, Moustapha Cisse, Yann N Dauphin, and David Lopez-Paz. mixup: Beyond empirical risk minimiza- tion. arXiv preprint arXiv:1710.09412, 2017. · arXiv
  45. Xiaolin Zhang, Yunchao Wei, Jiashi Feng, Yi Yang, and Thomas S Huang. Adversarial complementary learning for weakly supervised object localization. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recogni- tion, pages 1325-1334, 2018.
  46. Xiaolin Zhang, Yunchao Wei, Guoliang Kang, Yi Yang, and Thomas Huang. Self-produced guidance for weakly- supervised object localization. In Proceedings of the Euro- pean Conference on Computer Vision (ECCV), pages 597- 613, 2018.
  47. Zhun Zhong, Liang Zheng, Guoliang Kang, Shaozi Li, and Yi Yang. Random erasing data augmentation. arXiv preprint arXiv:1708.04896, 2017. · arXiv
  48. Bolei Zhou, Aditya Khosla, Agata Lapedriza, Aude Oliva, and Antonio Torralba. Learning deep features for discrimina- tive localization. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2921-2929, 2016.

Cited by

Sorted by citations

Authors
Yun, S., Han, D., Oh, S. J., Chun, S., Choe, J., & Yoo, Y.
Affiliations
  • NAVER Cloud (South Korea) (KR)
  • Yonsei University (KR)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2019
Pages
6022-6031
DOI
10.48550/arxiv.1905.04899
arXiv
1905.04899
OpenAlex
W2992308087
Cited by
5,439
References
2
Citations by year
2026: 523 · 2025: 1027 · 2024: 1113 · 2023: 1143 · 2022: 778 · 2021: 632 · 2020: 209 · 2019: 14
FWCI
131.59
Type
conference paper
Language
en
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • IEEE/CVF International Conference on Computer Vision (ICCV) (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
  • Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
  • Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Data Classification
Keywords
CutMix, data augmentation, regularization, strong classifier, object localization, weakly supervised localization, image captioning, out-of-distribution detection, ImageNet classification

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review