The DOI matches its Crossref record; 77 of the 77 references it cites were found in a registry or an index.
A Comprehensive Survey of Image Augmentation Techniques for Deep Learning
Open accessAbstract
Although deep learning has achieved satisfactory performance in computer vision, a large volume of images is required. However, collecting images is often expensive and challenging. Many image augmentation algorithms have been proposed to alleviate this issue. Understanding existing algorithms is, therefore, essential for finding suitable and developing novel methods for a given task. In this study, we perform a comprehensive survey of image augmentation for deep learning using a novel informative taxonomy. To examine the basic objective of image augmentation, we introduce challenges in computer vision tasks and vicinity distribution. The algorithms are then classified among three categories: model-free, model-based, and optimizing policy-based. The model-free category employs the methods from image processing, whereas the model-based approach leverages image generation models to synthesize images. In contrast, the optimizing policy-based approach aims to find an optimal combination of operations. Based on this analysis, we believe that our survey enhances the understanding necessary for choosing suitable methods and designing novel algorithms.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2577031 · Cited by 73,472
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2015) · arXiv · doi:10.48550/arxiv.1506.01497 · Cited by 70,812
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
Densely Connected Convolutional Networks
Huang, G., Liu, Z., Van Der Maaten, L., & Weinberger, K. Q. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.243 · Cited by 48,566
Microsoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460
Unpaired image-to-image translation using cycle-consistent adversarial networks
Zhu, J.-Y., Park, T., Isola, P., & Efros, A. A. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.244 · Cited by 23,602
SSD: Single Shot MultiBox Detector
Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.-Y., & Berg, A. C. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46448-0_2 · Cited by 22,128
Labaca-Castro, R. (2023) · Machine Learning under Malware Attack · doi:10.1007/978-3-658-40442-0_9 · Cited by 19,272
MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
Howard, A. G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., & Adam, H. (2017) · arXiv · doi:10.48550/arxiv.1704.04861 · Cited by 19,015
A survey on deep learning in medical image analysis
Litjens, G., Kooi, T., Bejnordi, B. E., Setio, A. A. A., Ciompi, F., Ghafoorian, M., van der Laak, J. A. W. M., van Ginneken, B., & Sánchez, C. I. (2017) · Medical Image Analysis · doi:10.1016/j.media.2017.07.005 · Cited by 15,332
A survey on Image Data Augmentation for Deep Learning
Shorten, C., & Khoshgoftaar, T. M. (2019) · Journal of Big Data · doi:10.1186/s40537-019-0197-0 · Cited by 13,203 · cited 3×
“Finally, unlike two related studies [6, 7], our survey encompasses more recent yet effective image augmentation algorithms such as instance level multiple image mixing, as well as comprehensive analysis of model-based methods.”
BackgroundMethodProceedings of IEEE International Conference on Computer Vision
(1995) · Proceedings of IEEE International Conference on Computer Vision · doi:10.1109/iccv.1995.466933 · Cited by 12,860
The Cityscapes Dataset for Semantic Urban Scene Understanding
Cordts, M., Omran, M., Ramos, S., Rehfeld, T., Enzweiler, M., Benenson, R., Franke, U., Roth, S., & Schiele, B. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.350 · Cited by 12,433 · cited 1×
“For instance, the Cityscapes [81] dataset can be augmented horizontally but not vertically.”
BackgroundUnsupervised representation learning with deep convolutional generative adversarial networks
Radford, A., Metz, L., & Chintala, S. (2015) · arXiv · doi:10.48550/arxiv.1511.06434 · Cited by 12,235
The Nature of Statistical Learning Theory
Vapnik, V. N. (2000) · Springer eBooks · doi:10.1007/978-1-4757-3264-1 · Cited by 10,696
Proximal Policy Optimization Algorithms
Schulman, J., Wolski, F., Dhariwal, P., Radford, A., & Klimov, O. (2017) · arXiv · doi:10.48550/arxiv.1707.06347 · Cited by 10,625 · cited 1×
“SPA [74] 2021 Select suitable samples to perform image augmentation. instead of Proximal Policy Optimization [97], to optimize the controller in Au-toAugment.”
YOLOv4: Optimal Speed and Accuracy of Object Detection
Bochkovskiy, A., Wang, C.-Y., & Liao, H.-Y. M. (2020) · arXiv · doi:10.48550/arxiv.2004.10934 · Cited by 10,307
Conditional Generative Adversarial Nets
Mirza, M., & Osindero, S. (2014) · arXiv · doi:10.48550/arxiv.1411.1784 · Cited by 8,863
International Conference on Learning Representations (ICLR 2013)
松尾, 豊. (2013) · 人工知能学会誌 = Journal of Japanese Society for Artificial Intelligence · Cited by 6,286
mixup: Beyond Empirical Risk Minimization
Zhang, H., Cisse, M., Dauphin, Y. N., & Lopez-Paz, D. (2017) · arXiv · doi:10.48550/arxiv.1710.09412 · Cited by 5,831
2009 IEEE Conference on Computer Vision and Pattern Recognition
(2009) · doi:10.1109/cvpr14067.2009 · Cited by 5,550
Arbitrary Style Transfer in Real-Time with Adaptive Instance Normalization
Huang, X., & Belongie, S. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.167 · Cited by 5,076
Bootstrap Your Own Latent: A New Approach to Self-Supervised Learning
Grill, J.-B., Strub, F., Altché, F., Tallec, C., Richemond, P. H., Buchatskaya, E., Doersch, C., Pires, B. A., Guo, Z. D., Azar, M. G., et al. (2020) · arXiv · doi:10.48550/arxiv.2006.07733 · Cited by 3,846 · cited 1×
“Second, image augmentation can be leveraged to generate positive samples for contrast learning under the assumption that an augmented image is similar to the corresponding original image [107, 108, 109].”
BackgroundRandom Erasing Data Augmentation
Zhong, Z., Zheng, L., Kang, G., Li, S., & Yang, Y. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i07.7000 · Cited by 3,617
Random Erasing Data Augmentation
Zhong, Z., Zheng, L., Kang, G., Li, S., & Yang, Y. (2017) · arXiv · doi:10.48550/arxiv.1708.04896 · Cited by 3,617 · cited 6×
“Occlusion: Hide-and-Seek [23], Cutout [24], Random Eras-ing [25], GridMask [26].”
BackgroundMethodInfluentialRandAugment: Practical automated data augmentation with a reduced search space
Cubuk, E. D., Zoph, B., Shlens, J., & Le, Q. V. (2019) · arXiv · doi:10.48550/arxiv.1909.13719 · Cited by 3,223
2017 IEEE International Conference on Computer Vision (ICCV)
(2017) · doi:10.1109/iccv37128.2017 · Cited by 3,056
Semantic Image Synthesis With Spatially-Adaptive Normalization
Park, T., Liu, M.-Y., Wang, T.-C., & Zhu, J.-Y. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00244 · Cited by 2,974
Understanding deep learning requires rethinking generalization
Zhang, C., Bengio, S., Hardt, M., Recht, B., & Vinyals, O. (2016) · arXiv · doi:10.48550/arxiv.1611.03530 · Cited by 2,838
Improved regularization of convolutional neural networks with cutout
DeVries, T., & Taylor, G. W. (2017) · arXiv · doi:10.48550/arxiv.1708.04552 · Cited by 2,776
Unsupervised Visual Representation Learning by Context Prediction
Doersch, C., Gupta, A., & Efros, A. A. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.167 · Cited by 2,629
Unsupervised Representation Learning by Predicting Image Rotations
Gidaris, S., Singh, P., & Komodakis, N. (2018) · arXiv · doi:10.48550/arxiv.1803.07728 · Cited by 2,477 · cited 1×
“First, a pretext task can be created via image augmentation, such as predicting the degrees of rotation [105] and relative positions of image patches [106].”
BackgroundFrid-Adar, M., Diamant, I., Klang, E., Amitai, M., Goldberger, J., & Greenspan, H. (2018) · Neurocomputing · doi:10.1016/j.neucom.2018.09.013 · Cited by 1,938
ResNeSt: Split-Attention Networks
Zhang, H., Wu, C., Zhang, Z., Zhu, Y., Lin, H., Zhang, Z., Sun, Y., He, T., Mueller, J., Manmatha, R., et al. (2020) · arXiv · doi:10.48550/arxiv.2004.08955 · Cited by 1,809
Unsupervised Data Augmentation for Consistency Training
Xie, Q., Dai, Z., Hovy, E., Luong, M.-T., & Le, Q. V. (2019) · arXiv · doi:10.48550/arxiv.1904.12848 · Cited by 1,517
Proceedings of IEEE Conference on Computer Vision and Pattern Recognition
(1994) · 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.1994.323798 · Cited by 1,477
Proceedings of the 28th ACM International Conference on Multimedia
(2020) · doi:10.1145/3394171 · Cited by 1,213
AutoAugment: Learning Augmentation Policies from Data
Cubuk, E. D., Zoph, B., Mane, D., Vasudevan, V., & Le, Q. V. (2018) · arXiv · doi:10.48550/arxiv.1805.09501 · Cited by 1,106
Understanding Data Augmentation for Classification: When to Warp?
Wong, S. C., Gatt, A., Stamatescu, V., & McDonnell, M. D. (2016) · 2016 International Conference on Digital Image Computing: Techniques and Applications (DICTA) · doi:10.1109/dicta.2016.7797091 · Cited by 1,007
Improving Deep Learning with Generic Data Augmentation
Taylor, L., & Nitschke, G. (2018) · 2018 IEEE Symposium Series on Computational Intelligence (SSCI) · doi:10.1109/ssci.2018.8628742 · Cited by 796 · cited 1×
“To address this issue, image augmentation has been confirmed to be an effective and efficient strategy [3, 4].”
MethodA Learned Representation For Artistic Style
Dumoulin, V., Shlens, J., & Kudlur, M. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1610.07629 · Cited by 763
Data Augmentation Generative Adversarial Networks
Antoniou, A., Storkey, A., & Edwards, H. S. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.04340 · Cited by 747
Singh, K. K., & Lee, Y. J. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.381 · Cited by 724 · cited 8×
“Occlusion: Hide-and-Seek [23], Cutout [24], Random Eras-ing [25], GridMask [26].”
BackgroundMethodInfluentialPrinciples of Risk Minimization for Learning Theory
Vapnik, V. N. (1991) · neural information processing systems · Cited by 692
2020 25th International Conference on Pattern Recognition (ICPR)
(2021) · doi:10.1109/icpr48806.2021 · Cited by 649
Effective data generation for imbalanced learning using conditional generative adversarial networks
Douzas, G., & Bacao, F. (2018) · Expert Systems with Applications · doi:10.1016/j.eswa.2017.09.030 · Cited by 601
Medical Imaging 2012: Image Processing
Haynor, D. R., & Sebastien Ourselin, A. (2012) · SPIE eBooks · Cited by 563
DRIT++: Diverse Image-to-Image Translation via Disentangled Representations
Lee, H.-Y., Tseng, H.-Y., Mao, Q., Huang, J.-B., Lu, Y.-D., Singh, M., & Yang, M.-H. (2020) · International Journal of Computer Vision · doi:10.1007/s11263-019-01284-z · Cited by 486
Smart Augmentation Learning an Optimal Data Augmentation Strategy
Lemley, J., Bazrafkan, S., & Corcoran, P. (2017) · IEEE Access · doi:10.1109/access.2017.2696121 · Cited by 428
A comprehensive survey of recent trends in deep learning for digital images augmentation
Khalifa, N. E., Loey, M., & Mirjalili, S. (2022) · Artificial Intelligence Review · doi:10.1007/s10462-021-10066-4 · Cited by 381
Chapelle, O., Weston, J., Bottou, L., & Vapnik, V. N. (2000) · MPG.PuRe (Max Planck Society) · Cited by 373
Data Augmentation by Pairing Samples for Images Classification
Inoue, H. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1801.02929 · Cited by 319
Delta-encoder: an effective sample synthesis method for few-shot object recognition
Schwartz, E., Karlinsky, L., Shtok, J., Harary, S., Marder, M., Feris, R., Kumar, A., Giryes, R., & Bronstein, A. M. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.04734 · Cited by 266
2019 IEEE Symposium Series on Computational Intelligence (SSCI)
(2019) · doi:10.1109/ssci44817.2019 · Cited by 261
A Group-Theoretic Framework for Data AugmentationPreprint
Chen, S., Dobriban, E., & Lee, J. H. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1907.10905 · Cited by 259
BAGAN: Data Augmentation with Balancing GAN
Mariani, G., Scheidegger, F., Istrate, R., Bekas, C., & Malossi, A. C. I. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1803.09655 · Cited by 239
Synthesizing Training Data for Object Detection in Indoor Scenes
Georgakis, G., Mousavian, A., Berg, A., & Kosecka, J. (2017) · Robotics: Science and Systems XIII · doi:10.15607/rss.2017.xiii.043 · Cited by 225
Chen, P., Liu, S., Zhao, H., Wang, X., & Jia, J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2001.04086 · Cited by 221
Ali-Gombe, A., & Elyan, E. (2019) · Neurocomputing · doi:10.1016/j.neucom.2019.06.043 · Cited by 181
A Bayesian Data Augmentation Approach for Learning Deep Models
Tran, T. B., Pham, T. T., Carneiro, G., Palmer, L. J., & Reid, I. D. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1710.10564 · Cited by 180
Learning to Compose Domain-Specific Transformations for Data Augmentation
Ratner, A., Ehrenberg, H. R., Hussain, Z., Dunnmon, J., & Ré, C. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1709.01643 · Cited by 180
Adversarial AutoAugmentPreprint
Zhang, X., Wang, Q., Zhang, J., & Zhong, Z. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.11188 · Cited by 179
Chest x-ray generation and data augmentation for cardiovascular abnormality classification
Moradi, M., Madani, A., Karargyris, A., & Syeda-Mahmood, T. F. (2018) · Medical Imaging 2018: Image Processing · doi:10.1117/12.2293971 · Cited by 166
Regularizing Deep Networks with Semantic Data Augmentation
Wang, Y., Huang, G., Song, S., Pan, X., Xia, Y., & Wu, C. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3052951 · Cited by 163
MDFN: Multi-scale deep feature learning network for object detection
Ma, W., Wu, Y., Cen, F., & Wang, G. (2020) · Pattern Recognition · doi:10.1016/j.patcog.2019.107149 · Cited by 161
A survey on face data augmentation for the training of deep neural networks
Wang, X., Wang, K., & Lian, S. (2020) · Neural Computing and Applications · doi:10.1007/s00521-020-04748-3 · Cited by 141 · cited 1×
“First, we do not confine ourselves to a specific type of image, such as facial images [8].”
BackgroundDeep feature augmentation for occluded image classification
Cen, F., Zhao, X., Wu-zhuang, L., & Wang, G. (2020) · Pattern Recognition · doi:10.1016/j.patcog.2020.107737 · Cited by 80
Xu, M., Yoon, S., Fuentes, A., Yang, J., & Park, D. S. (2022) · Frontiers in Plant Science · doi:10.3389/fpls.2021.773142 · Cited by 78
Self-citationSimplified unsupervised image translation for semantic segmentation adaptation
Li, R., Cao, W., Jiao, Q., Wu, S., & Wong, H. (2020) · Pattern Recognition · doi:10.1016/j.patcog.2020.107343 · Cited by 65
Toward learning a unified many-to-many mapping for diverse image translation
Xu, W., Shawn, K., & Wang, G. (2019) · Pattern Recognition · doi:10.1016/j.patcog.2019.05.017 · Cited by 60
GridMix: Strong regularization through local context mapping
Baek, K., Bang, D., & Shim, H. (2020) · Pattern Recognition · doi:10.1016/j.patcog.2020.107594 · Cited by 48
MetalGAN: Multi-domain label-less image synthesis using cGANs and meta-learning
Fontanini, T., Iotti, E., Donati, L., & Prati, A. (2020) · Neural Networks · doi:10.1016/j.neunet.2020.07.031 · Cited by 30
Zheng, Z., Yu, Z., Wu, Y., Zheng, H., Zheng, B., & Lee, M. (2021) · Neural Networks · doi:10.1016/j.neunet.2021.04.013 · Cited by 29
Self-paced data augmentation for training neural networks
Takase, T., Karakida, R., & Asoh, H. (2021) · Neurocomputing · doi:10.1016/j.neucom.2021.02.080 · Cited by 23
Data Augmentation with norm-VAE for Unsupervised Domain AdaptationPreprint
Wang, Q., Meng, F., & Breckon, T. P. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2012.00848 · Cited by 9
Cited by
Deep learning: systematic review, models, challenges, and research directions
Talaei Khoei, T., Ould Slimane, H., & Kaabouch, N. (2023) · Neural Computing and Applications · doi:10.1007/s00521-023-08957-4 · Cited by 411
Image Data Augmentation Approaches: A Comprehensive Survey and Future Directions
Kumar, T., Brennan, R., Mileo, A., & Bendechache, M. (2024) · IEEE Access · doi:10.1109/access.2024.3470122 · Cited by 287
Shobayo, O., & Saatchi, R. (2025) · Diagnostics · doi:10.3390/diagnostics15091072 · Cited by 53
A Unified Multimodal De- and Re-Coupling Framework for RGB-D Motion Recognition
Zhou, B., Wang, P., Wan, J., Liang, Y., & Wang, F. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3274783 · Cited by 38
Metadata
- Authors
- Xu, M., Yoon, S., Fuentes, A., & Park, D. S.
- Venue
- Pattern Recognition
- Publisher
- Elsevier BV
- Published
- 2023-05
- Volume
- 137
- Pages
- 109347
- ISSN
- 0031-3203 (ISSN-L)
- OpenAlex
- W4320525814
- Cited by
- 634
- References
- 77
- FWCI
- 57.10
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.