Data Augmentation Using Random Image Cropping and Patching for Deep CNNs
Open accessThe DOI matches its Crossref record; 54 of the 54 references it cites were found in a registry or an index.
Abstract
Deep convolutional neural networks (CNNs) have achieved remarkable results in image processing tasks. However, their high expression ability risks overfitting. Consequently, data augmentation techniques have been proposed to prevent overfitting while enriching datasets. Recent CNN architectures with more parameters are rendering traditional data augmentation techniques insufficient. In this study, we propose a new data augmentation technique called random image cropping and patching (RICAP) which randomly crops four images and patches them to create a new training image. Moreover, RICAP mixes the class labels of the four images, resulting in an advantage of the soft labels. We evaluated RICAP with current state-of-the-art CNNs (e.g., the shake-shake regularization model) by comparison with competitive data augmentation techniques such as cutout and mixup. RICAP achieves a new state-of-the-art test error of 2.19% on CIFAR-10. We also confirmed that deep CNNs with RICAP achieve better results on classification tasks using CIFAR-100 and ImageNet, an image-caption retrieval task using Microsoft COCO, and other computer vision tasks.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“However, inserting a dropout between the convolutional layers (as opposed to within the residual block) made the learning more effective in WideResNet [121, 122].”
From Review of deep learning: concepts, CNN architectures, challenges, applications, future directions (2021) · cites this paper as [122]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · cited 5×
“The backbone model was the 50 layer ResNet [11] pre-trained on ImageNet.”
BackgroundMethodInfluentialVery Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · cited 4×
“AlexNet also performed principal component analysis (PCA) on a set of RGB values to alter the intensities of the RGB channels for evaluation on the ImageNet dataset [24].”
MethodInfluentialAdam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · cited 2×
“The weight parameters were updated using the Adam optimizer [42] over 100 epochs with a mini-batch size of 16.”
MethodYou Only Look Once: Unified, Real-Time Object Detection
Redmon, J., Divvala, S., Girshick, R., & Farhadi, A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.91 · Cited by 53,803 · cited 1×
“YOLO [46] is an end-to-end model achieving faster inference than previous methods.”
MethodDensely Connected Convolutional Networks
Huang, G., Liu, Z., Van Der Maaten, L., & Weinberger, K. Q. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.243 · Cited by 48,566
Going deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562
Microsoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380 · cited 16×
“We apply RICAP to existing deep CNNs and evaluate them on the classification tasks using the CIFAR-10, CIFAR-100 [8], and ImageNet [24] datasets in Sections IV-A, IV-B and IV-C.”
MethodInfluentialRethinking the Inception Architecture for Computer Vision
Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., & Wojna, Z. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.308 · Cited by 32,138
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S., & Szegedy, C. (2015) · arXiv · doi:10.48550/arxiv.1502.03167 · Cited by 27,818 · cited 1×
“Batch normalization [36] and ReLU activation function [37] were used.”
MethodLearning Multiple Layers of Features from Tiny Images
Krizhevsky, A. (2024) · doi:10.57702/zp44cu3g · Cited by 23,395
Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification
He, K., Zhang, X., Ren, S., & Sun, J. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.123 · Cited by 19,741 · cited 1×
“The weight parameters were initialized following the He algorithm [38].”
MethodYOLOv3: An Incremental Improvement
Redmon, J., & Farhadi, A. (2018) · arXiv · doi:10.48550/arxiv.1804.02767 · Cited by 17,141 · cited 1×
“YOLO was updated to version 3 (YOLOv3) [47] by the original authors, and we used it as a baseline of our experiments.”
MethodVisualizing and understanding convolutional networks
Zeiler, M. D., & Fergus, R. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10590-1_53 · Cited by 15,839 · cited 2×
“Conversely, without enough training samples, CNNs with numerous parameters have a risk of overfitting because they memorize detailed features of training images that cannot be generalized [2], [5].”
BackgroundDistilling the Knowledge in a Neural Network
Hinton, G., Vinyals, O., & Dean, J. (2015) · arXiv · doi:10.48550/arxiv.1503.02531 · Cited by 14,204 · cited 2×
“For classification tasks, the ground truth is typically given as probabilities of 0 and 1, called the hard labels [32].”
BackgroundA Multi-Modal Distributed Real-Time IoT System for Urban Traffic Control (Invited Paper)
Khanam, Z., Achari, V. P. S., Boukhennoufa, I., Jindal, A., & Singh, A. K. (2024) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/oasics.ng-res.2024.2 · Cited by 14,040
(2010) · International Conference on Machine Learning · Cited by 13,995
Aggregated Residual Transformations for Deep Neural Networks
Xie, S., Girshick, R., Dollar, P., Tu, Z., & He, K. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.634 · Cited by 12,465 · cited 2×
“However, the number of parameters is ever-growing and the risk of overfitting is also ever-increasing as many studies propose new network architectures [15]–[19] following ResNet [11].”
BackgroundBackpropagation Applied to Handwritten Zip Code Recognition
LeCun, Y., Boser, B., Denker, J. S., Henderson, D., Howard, R. E., Hubbard, W., & Jackel, L. D. (1989) · Neural Computation · doi:10.1162/neco.1989.1.4.541 · Cited by 11,891 · cited 2×
“cat ((cropped_images[0] , cropped_images[1]) , 2) torch .”
BackgroundLearning Deep Features for Discriminative Localization
Zhou, B., Khosla, A., Lapedriza, A., Oliva, A., & Torralba, A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.319 · Cited by 11,428 · cited 6×
“Class Activation Mapping (CAM) [25] of WideResNet 28-10 using the images that RICAP cropped and patched.”
BackgroundMethodInfluentialIdentity mappings in deep residual networks
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46493-0_38 · Cited by 10,741
Identity Mappings in Deep Residual Networks
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · arXiv · doi:10.48550/arxiv.1603.05027 · Cited by 10,665 · cited 2×
“However, the number of parameters is ever-growing and the risk of overfitting is also ever-increasing as many studies propose new network architectures [15]–[19] following ResNet [11].”
BackgroundExplaining and Harnessing Adversarial Examples
Goodfellow, I. J., Shlens, J., & Szegedy, C. (2014) · arXiv · doi:10.48550/arxiv.1412.6572 · Cited by 10,381 · cited 1×
“the variety of training images but also works like adversarial perturbation [29].”
BackgroundZagoruyko, S., & Komodakis, N. (2016) · Procedings of the British Machine Vision Conference 2016 · doi:10.5244/c.30.87 · Cited by 7,244
Improving neural networks by preventing co-adaptation of feature detectors
Hinton, G. E., Srivastava, N., Krizhevsky, A., Sutskever, I., & Salakhutdinov, R. R. (2012) · arXiv · doi:10.48550/arxiv.1207.0580 · Cited by 6,382 · cited 3×
“We also summarized the results of RICAP in Table I as well as the results of competitive methods: input dropout [13], cutout [20], random erasing [21], and mixup [22].”
BackgroundMethodmixup: Beyond Empirical Risk Minimization
Zhang, H., Cisse, M., Dauphin, Y. N., & Lopez-Paz, D. (2017) · arXiv · doi:10.48550/arxiv.1710.09412 · Cited by 5,831 · cited 7×
“Mixup alpha-blends two images to construct a new training image [22].”
BackgroundMethodInfluentialScalable Person Re-identification: A Benchmark
Zheng, L., Shen, L., Tian, L., Wang, S., Wang, J., & Tian, Q. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.133 · Cited by 4,506 · cited 1×
“We used the Mar-ket1501 dataset [45], consisting of 32 , 668 images of 1 , 501 identities shoot by 6 cameras.”
MethodNeural Architecture Search with Reinforcement Learning
Zoph, B., & Le, Q. V. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.01578 · Cited by 3,974 · cited 1×
“work exploring the best hyperparameters of existing data augmentations using reinforcement learning [31].”
BackgroundMulti-column deep neural networks for image classification
Ciresan, D., Meier, U., & Schmidhuber, J. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6248110 · Cited by 3,720
Random Erasing Data Augmentation
Zhong, Z., Zheng, L., Kang, G., Li, S., & Yang, Y. (2017) · arXiv · doi:10.48550/arxiv.1708.04896 · Cited by 3,617 · cited 3×
“We also summarized the results of RICAP in Table I as well as the results of competitive methods: input dropout [13], cutout [20], random erasing [21], and mixup [22].”
BackgroundMethodRandom Erasing Data Augmentation
Zhong, Z., Zheng, L., Kang, G., Li, S., & Yang, Y. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i07.7000 · Cited by 3,617
Striving for Simplicity: The All Convolutional Net
Springenberg, J. T., Dosovitskiy, A., Brox, T., & Riedmiller, M. (2014) · arXiv · doi:10.48550/arxiv.1412.6806 · Cited by 3,447 · cited 1×
“studies [33]–[35], we normalized each channel of all images”
MethodFitNets: Hints for Thin Deep Nets
Romero, A., Ballas, N., Kahou, S. E., Chassang, A., Gatta, C., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1412.6550 · Cited by 2,859
Improved regularization of convolutional neural networks with cutout
DeVries, T., & Taylor, G. W. (2017) · arXiv · doi:10.48550/arxiv.1708.04552 · Cited by 2,776 · cited 6×
“Cutout [20] randomly masks a square region in an image at every training step and thus changes the apparent features.”
BackgroundMethodInfluentialOverFeat: Integrated Recognition, Localization and Detection using Convolutional Networks
Sermanet, P., Eigen, D., Zhang, X., Mathieu, M., Fergus, R., & LeCun, Y. (2013) · arXiv · doi:10.48550/arxiv.1312.6229 · Cited by 2,680 · cited 2×
“DEEP convolutional neural networks (CNNs) [1] have led to significant achievement in the fields of image classification and image processing owing to their numerous parameters and rich expression ability [2], [3].”
BackgroundBeyond Part Models: Person Retrieval with Refined Part Pooling (and A Strong Convolutional Baseline)
Sun, Y., Zheng, L., Yang, Y., Tian, Q., & Wang, S. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01225-0_30 · Cited by 2,522
Revisiting Unreasonable Effectiveness of Data in Deep Learning Era
Sun, C., Shrivastava, A., Singh, S., & Gupta, A. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.97 · Cited by 2,484 · cited 1×
“A recent study demonstrated that the performance of CNNs is logarithmically proportional to the number of training samples [4].”
Background(2011) · doi:10.5591/978-1-57735-516-8/ijcai11-210 · Cited by 1,229
Unifying Visual-Semantic Embeddings with Multimodal Neural Language Models
Kiros, R., Salakhutdinov, R., & Zemel, R. S. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1411.2539 · Cited by 1,187 · cited 9×
“Experiments and Results : We used the same experimental settings as in the original study of VSE++ [41].”
BackgroundMethodInfluentialAutoAugment: Learning Augmentation Policies from Data
Cubuk, E. D., Zoph, B., Mane, D., Vasudevan, V., & Le, Q. V. (2018) · arXiv · doi:10.48550/arxiv.1805.09501 · Cited by 1,106 · cited 1×
“4) Auto-adjustment Method: AutoAugment [30] is a framework exploring the best hyperparameters of existing data aug-mentations using reinforcement learning [31].”
MethodLee, C., Xie, S., Gallagher, P., Zhang, Z., & Zhuowen Tu, A. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1409.5185 · Cited by 993 · cited 1×
“studies [33]–[35], we normalized each channel of all images”
MethodPerson Re-identification: Past, Present and Future
Zheng, L., Yang, Y. P., & Hauptmann, A. G. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1610.02984 · Cited by 922 · cited 4×
“RICAP also handles partial features and we evaluate RICAP with the IDE. Application RICAP to IDE : Unlike natural images such as images in CIFAR and ImageNet datasets, typical images for person re-identification are already aligned and cropped to depict persons at the center.”
MethodInfluentialMixup: Beyond empirical risk minimization
Zhang, H. (2024) · arXiv (Cornell University) · doi:10.57702/dcy1c3gw · Cited by 773
Deep Pyramidal Residual Networks
Han, D., Kim, J., & Kim, J. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.668 · Cited by 715 · cited 3×
“CIFAR-10 classification among the studies under the same conditions [17], [18], [20]–[22], [39].”
BackgroundMethodResultCreating artificial neural networks that generalize
Sietsma, J., & Dow, R. J. F. (1991) · Neural Networks · doi:10.1016/0893-6080(91)90033-2 · Cited by 597 · cited 1×
“Pixel dropping functions as injection of noise into an image [28].”
BackgroundImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2014) · arXiv · doi:10.48550/arxiv.1409.0575 · Cited by 567 · cited 16×
“We apply RICAP to existing deep CNNs and evaluate them on the classification tasks using the CIFAR-10, CIFAR-100 [8], and ImageNet [24] datasets in Sections IV-A, IV-B and IV-C.”
MethodInfluentialVSE++: Improving Visual-Semantic Embeddings with Hard Negatives
Faghri, F., Fleet, D. J., Kiros, J., & Fidler, S. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1707.05612 · Cited by 543
Visualizing Deep Neural Network Decisions: Prediction Difference Analysis
Zintgraf, L. M., Cohen, T. S., Adel, T., & Welling, M. (2017) · arXiv · doi:10.48550/arxiv.1702.04595 · Cited by 487 · cited 1×
“memorize detailed features of training images that cannot be generalized [2], [5].”
BackgroundRevisiting Unreasonable Effectiveness of Data in Deep Learning EraPreprint
Sun, C., Shrivastava, A., Singh, S., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1707.02968 · Cited by 300 · cited 1×
“A recent study demonstrated that the performance of CNNs is logarithmically proportional to the number of training samples [4].”
BackgroundGastaldi, X. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.07485 · Cited by 276 · cited 3×
“CIFAR-10 classification among the studies under the same conditions [17], [18], [20]–[22], [39].”
BackgroundMethodResultVSE++: Improving Visual-Semantic Embeddings with Hard Negatives.
Faghri, F., Fleet, D. J., Kiros, J., & Fidler, S. (2018) · Cited by 184
RICAP: Random Image Cropping and Patching Data Augmentation for Deep CNNs
Takahashi, R., Matsubara, T., & Uehara, K. (2018) · Asian Conference on Machine Learning · Cited by 160
Self-citationEquivalence between dropout and data augmentation: A mathematical check
Zhao, D., Yu, G., Xu, P., & Luo, M. (2019) · Neural Networks · doi:10.1016/j.neunet.2019.03.013 · Cited by 39 · cited 1×
“Dropout on the input layer [13] is a data augmentation technique [14] that disturbs and masks the original information of given data by dropping pixels.”
Background
Cited by
A survey on Image Data Augmentation for Deep Learning
Shorten, C., & Khoshgoftaar, T. M. (2019) · Journal of Big Data · doi:10.1186/s40537-019-0197-0 · Cited by 13,203
Review of deep learning: concepts, CNN architectures, challenges, applications, future directions
Alzubaidi, L., Zhang, J., Humaidi, A. J., Al-Dujaili, A., Duan, Y., Al-Shamma, O., Santamaría, J., Fadhel, M. A., Al-Amidie, M., & Farhan, L. (2021) · Journal of Big Data · doi:10.1186/s40537-021-00444-8 · Cited by 7,906
A review of medical image data augmentation techniques for deep learning applications
Chlap, P., Min, H., Vandenberg, N., Dowling, J., Holloway, L., & Haworth, A. (2021) · Journal of Medical Imaging and Radiation Oncology · doi:10.1111/1754-9485.13261 · Cited by 956
“also utilised Random Image Cropping And Patching (RICAP), where, rather than blending the images, four images are randomly selected, cropped and then patched together.(89,110) Unlike other augmentation techniques, basic augmentation methods do not aim to produce ‘realistic’ images.”
MethodData augmentation: A comprehensive survey of modern approaches
Mumuni, A., & Mumuni, F. (2022) · Array · doi:10.1016/j.array.2022.100258 · Cited by 891
AugMix: A Simple Data Processing Method to Improve Robustness and Uncertainty
Hendrycks, D., Mu, N., Cubuk, E. D., Zoph, B., Gilmer, J., & Lakshminarayanan, B. (2019) · arXiv · doi:10.48550/arxiv.1912.02781
“Rather than occluding a portion of an image, CutMix replaces a portion of an image with a portion of a different image (Yun et al., 2019; Takahashi et al., 2019).”
Background
Metadata
- Authors
- Takahashi, R., Matsubara, T., & Uehara, K.
- Affiliations
- Kobe University (JP)
- Venue
- IEEE Transactions on Circuits and Systems for Video Technology
- Published
- 2020-09
- Volume
- 30
- Issue
- 9
- Pages
- 2917-2931
- ISSN
- 1051-8215 (ISSN-L)
- 1558-2205
- OpenAlex
- W2900595477
- Cited by
- 477
- References
- 54
- Citations by year
- 2026: 30 · 2025: 68 · 2024: 73 · 2023: 89 · 2022: 100 · 2021: 72 · 2020: 34
- FWCI
- 24.17
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- IEEE Transactions on Circuits and Systems for Video Technology (publishedVersion)
- arXiv (Cornell University) (acceptedVersion)
- Institutional Repositories DataBase (IRDB) (acceptedVersion)
- Kobe University Repository Kernel (Kobe University) (submittedVersion, other-oa)
- arXiv (Cornell University)
- Funders
- Strategic Information and Communications R&D Promotion Programme of Ministry of Internal Affairs and Communications, Japan (172107101)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Data Classification
- Keywords
- data augmentation, overfitting, convolutional neural networks, soft labels, cutouts, mixup, image classification
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.