SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition
Open accessThe DOI matches its Crossref record; 51 of the 51 references it cites were found in a registry or an index.
Abstract
We present SpecAugment, a simple data augmentation method for speech recognition. SpecAugment is applied directly to the feature inputs of a neural network (i.e., filter bank coefficients). The augmentation policy consists of warping the features, masking blocks of frequency channels, and masking blocks of time steps. We apply SpecAugment on Listen, Attend and Spell networks for end-to-end speech recognition tasks. We achieve state-of-the-art performance on the LibriSpeech 960h and Swichboard 300h tasks, outperforming all prior work. On LibriSpeech, we achieve 6.8% WER on test-other without the use of a language model, and 5.8% WER with shallow fusion with a language model. This compares to the previous state-of-the-art hybrid system of 7.5% WER. For Switchboard, we achieve 7.2%/14.6% on the Switchboard/CallHome portion of the Hub5'00 test set without the use of a language model, and 6.8%/14.1% with shallow fusion, which compares to the previous state-of-the-art hybrid system at 8.3%/17.3% WER.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“SpecAugment is a spectrum augmentation method, which is only suitable for the audio deepfake model based on front-end feature extraction, and it is not easy to operate on the audio waveform.”
From Voice Deepfake Detection Using the Self-Supervised Pre-Training Model HuBERT (2023) · cites this paper as [19]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“In addition to balanced sampling, we also apply SpecAugment [23] and random time shift to alleviate the class-imbalance issue.”
From Vocalsound: A Dataset for Improving Human Vocal Sounds Recognition (2022) · cites this paper as [23]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Rethinking the Inception Architecture for Computer Vision
Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., & Wojna, Z. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.308 · Cited by 32,138
Speech recognition with deep recurrent neural networks
Graves, A., Mohamed, A.-R., & Hinton, G. (2013) · 2013 IEEE International Conference on Acoustics, Speech and Signal Processing · doi:10.1109/icassp.2013.6638947 · Cited by 8,893 · cited 1×
“Deep Learning has been applied successfully to Automatic Speech Recognition (ASR) [1], where the main focus of research has been designing better network architectures, for example, DNNs [2], CNNs [3], RNNs [4] and end-to-end models [5, 6, 7].”
Method“Librispeech: an ASR corpus based on public domain audio books,”
Panayotov, V., Chen, G., Povey, D., & Khudanpur, S. (2015) · 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2015.7178964 · Cited by 6,072
Kaldi Speech Recognition Toolkit
Povey, D. (2024) · Infoscience (Ecole Polytechnique Fédérale de Lausanne) · doi:10.57702/jb3fvbn9 · Cited by 4,609
Context-dependent pre-trained deep neural networks for large-vocabulary speech recognition
Dahl, G. E., Yu, D., Deng, L., & Acero, A. (2012) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/tasl.2011.2134090 · Cited by 3,011 · cited 1×
“Deep Learning has been applied successfully to Automatic Speech Recognition (ASR) [1], where the main focus of research has been designing better network architectures, for example, DNNs [2], CNNs [3], RNNs [4] and end-to-end models [5, 6, 7].”
MethodImproved regularization of convolutional neural networks with cutout
DeVries, T., & Taylor, G. W. (2017) · arXiv · doi:10.48550/arxiv.1708.04552 · Cited by 2,776
AutoAugment: Learning Augmentation Strategies From Data
Cubuk, E. D., Zoph, B., Mané, D., Vasudevan, V., & Le, Q. V. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00020 · Cited by 2,733
Self-citationTowards End-To-End Speech Recognition with Recurrent Neural Networks
Graves, A., & Jaitly, N. (2014) · Cited by 2,557
Listen, attend and spell: A neural network for large vocabulary conversational speech recognition
Chan, W., Jaitly, N., Le, Q., & Vinyals, O. (2016) · 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2016.7472621 · Cited by 2,226
Self-citationSWITCHBOARD: telephone speech corpus for research and development
Godfrey, J. J., Holliman, E. C., & McDaniel, J. (1992) · [Proceedings] ICASSP-92: 1992 IEEE International Conference on Acoustics, Speech, and Signal Processing · doi:10.1109/icassp.1992.225858 · Cited by 2,134 · cited 2×
“On Switchboard 300h (LDC97S62) [22], we obtain 7.2% WER on the Switchboard portion of the Hub5’00 (LDC2002S09, LDC2003T02) test set, and 14.6% on the Call-Home portion, without using an LM.”
BackgroundInfluentialDeep Neural Networks for Acoustic Modeling in Speech Recognition
Hinton, G. E., Deng, L., Yu, D., Dahl, G. E., Mohamed, A., Jaitly, N., Senior, A., Vanhoucke, V., Nguyen, P., Sainath, T. N., et al. (2012) · IEEE Signal Processing Magazine · Cited by 2,115
AutoAugment: Learning Augmentation Policies from Data
Cubuk, E. D., Zoph, B., Mane, D., Vasudevan, V., & Le, Q. V. (2018) · arXiv · doi:10.48550/arxiv.1805.09501 · Cited by 1,106
Self-citationAudio augmentation for speech recognition
Ko, T., Peddinti, V., Povey, D., & Khudanpur, S. (2015) · Interspeech 2015 · doi:10.21437/interspeech.2015-711 · Cited by 1,104
End-to-end attention-based large vocabulary speech recognition
Bahdanau, D., Chorowski, J., Serdyuk, D., Brakel, P., & Bengio, Y. (2016) · 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2016.7472618 · Cited by 1,096
Practical Variational Inference for Neural Networks
Graves, A. (2011) · Cited by 1,078
Deep convolutional neural networks for LVCSR
Sainath, T. N., Mohamed, A.-R., Kingsbury, B., & Ramabhadran, B. (2013) · 2013 IEEE International Conference on Acoustics, Speech and Signal Processing · doi:10.1109/icassp.2013.6639347 · Cited by 1,053
State-of-the-Art Speech Recognition with Sequence-to-Sequence Models
Chiu, C.-C., Sainath, T. N., Wu, Y., Prabhavalkar, R., Nguyen, P., Chen, Z., Kannan, A., Weiss, R. J., Rao, K., Gonina, E., et al. (2018) · 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2018.8462105 · Cited by 1,043
Self-citationJapanese and Korean voice search
Schuster, M., & Nakajima, K. (2012) · 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2012.6289079 · Cited by 862
Purely Sequence-Trained Neural Networks for ASR Based on Lattice-Free MMI
Povey, D., Peddinti, V., Galvez, D., Ghahremani, P., Manohar, V., Na, X., Wang, Y., & Khudanpur, S. (2016) · Interspeech 2016 · doi:10.21437/interspeech.2016-595 · Cited by 740
Sequence-discriminative training of deep neural networks
Veselý, K., Ghoshal, A., Burget, L., & Povey, D. (2013) · Interspeech 2013 · doi:10.21437/interspeech.2013-548 · Cited by 657
On Using Monolingual Corpora in Neural Machine Translation
Gülçehre, Ç., Fırat, O., Xu, K., Cho, K., Barrault, L., Lin, H.-C., Bougares, F., Schwenk, H., & Bengio, Y. (2015) · HAL (Le Centre pour la Communication Scientifique Directe) · doi:10.48550/arxiv.1503.03535 · Cited by 466
The Fisher Corpus: a Resource for the Next Generations of Speech-to-Text
Cieri, C., Miller, D. A. B., & Walker, K. P. (2004) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/5q862povzthr · Cited by 464
Towards Better Decoding and Language Model Integration in Sequence to Sequence Models
Chorowski, J., & Jaitly, N. (2017) · Interspeech 2017 · doi:10.21437/interspeech.2017-343 · Cited by 358 · cited 1×
“We also use a coverage penalty c [29].”
MethodVocal Tract Length Perturbation (VTLP) improves speech recognition
Jaitly, N., & Hinton, E. (2013) · Cited by 348
Improved Training of End-to-end Attention Models for Speech Recognition
Zeyer, A., Irie, K., Schlüter, R., & Ney, H. (2018) · Interspeech 2018 · doi:10.21437/interspeech.2018-1616 · Cited by 257
Jasper: An End-to-End Convolutional Neural Acoustic Model
Li, J., Lavrukhin, V., Ginsburg, B., Leary, R., Kuchaiev, O., Cohen, J. M., Nguyen, H., & Gadde, R. T. (2019) · Interspeech 2019 · doi:10.21437/interspeech.2019-1819 · Cited by 243
Wav2Letter: an End-to-End ConvNet-based Speech Recognition System
Collobert, R., Puhrsch, C., & Synnaeve, G. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1609.03193 · Cited by 230
Kim, C., Misra, A., Chin, K., Hughes, T., Narayanan, A., Sainath, T. N., & Bacchiani, M. (2017) · Interspeech 2017 · doi:10.21437/interspeech.2017-1510 · Cited by 218
Data augmentation for low resource languages
Ragni, A., Knill, K., Rath, S. P., & Gales, M. (2014) · Interspeech (USB) · doi:10.21437/interspeech.2014-207 · Cited by 175
End-to-end Speech Recognition Using Lattice-free MMI
Hadian, H., Sameti, H., Povey, D., & Khudanpur, S. (2018) · Interspeech 2018 · doi:10.21437/interspeech.2018-1423 · Cited by 137
Direct Acoustics-to-Word Models for English Conversational Speech Recognition
Audhkhasi, K., Ramabhadran, B., Saon, G., Picheny, M., & Nahamoo, D. (2017) · Interspeech (USB) · doi:10.21437/interspeech.2017-546 · Cited by 128
Lu, L., Zhang, X., & Renals, S. J. (2016) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2016.7472641 · Cited by 125
Multitask Learning with Low-Level Auxiliary Tasks for Encoder-Decoder Based Speech Recognition
Toshniwal, S., Tang, H., Lu, L., & Livescu, K. (2017) · Interspeech 2017 · doi:10.21437/interspeech.2017-1118 · Cited by 119
Elastic spectral distortion for low resource speech recognition with deep neural networks
Kanda, N., Takeda, R., & Obuchi, Y. (2013) · 2013 IEEE Workshop on Automatic Speech Recognition and Understanding · doi:10.1109/asru.2013.6707748 · Cited by 113
Prabhavalkar, R., Álvarez, R., Parada, C., Nakkiran, P., & Sainath, T. N. (2015) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2015.7178863 · Cited by 112
Building Competitive Direct Acoustics-to-Word Models for English Conversational Speech Recognition
Audhkhasi, K., Kingsbury, B. E. D., Ramabhadran, B., Saon, G., & Picheny, M. (2018) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2018.8461935 · Cited by 103
Fully Convolutional Speech RecognitionPreprint
Zeghidour, N., Xu, Q., Liptchinsky, V., Usunier, N., Synnaeve, G., & Collobert, R. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1812.06864 · Cited by 97
Advances in all-neural speech recognition
Zweig, G., Yu, C., Droppo, J., & Stolcke, A. (2017) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2017.7953069 · Cited by 91
The CAPIO 2017 Conversational Speech Recognition System
Han, K. J., Chandrashekaran, A., Kim, J., & Lane, I. R. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1801.00059 · Cited by 73
Letter-Based Speech Recognition with Gated ConvNetsPreprint
Liptchinsky, V., Synnaeve, G., & Collobert, R. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1712.09444 · Cited by 63
Weng, C., Cui, J., Wang, G., Wang, J., Yu, C., Su, D., & Yu, D. (2018) · Interspeech (USB) · doi:10.21437/interspeech.2018-1030 · Cited by 58
Kovács, G., Tóth, L., Compernolle, D. V., & Ganapathy, S. (2017) · Pattern Recognition Letters · doi:10.1016/j.patrec.2017.09.023 · Cited by 44
Improving End-to-End Speech Recognition with Policy Learning
Zhou, Y., Xiong, C., & Socher, R. (2018) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2018.8462361 · Cited by 43
Optimal Completion Distillation for Sequence LearningPreprint
Sabour, S., Chan, W., & Norouzi, M. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.01398 · Cited by 42
Self-citationData Augmentation for Robust Keyword Spotting under Playback InterferencePreprint
Raju, A., Panchapagesan, S., Liu, X., Mandal, A., & Ström, N. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1808.00563 · Cited by 36
Novel neural network based fusion for multistream ASR
Mallidi, S. H., & Heřmanský, H. (2016) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2016.7472765 · Cited by 29
A Perceptually Inspired Data Augmentation Method for Noise Robust CNN Acoustic Models
Tóth, L., Kovács, G., & Compernolle, D. V. (2018) · Lecture notes in computer science · doi:10.1007/978-3-319-99579-3_71 · Cited by 27
A comprehensive analysis on attention models
Zeyer, A., Merboldt, A., Schlüter, R., & Ney, H. (2019) · RWTH Publications (RWTH Aachen) · doi:10.18154/rwth-2019-03648 · Cited by 16
Model Unit Exploration for Sequence-to-Sequence Speech Recognition.Preprint
Irie, K., Prabhavalkar, R., Kannan, A., Bruguier, A., Rybach, D., & Nguyen, P. (2019) · arXiv (Cornell University) · Cited by 13
A novel pyramidal-FSMN architecture with lattice-free MMI for speech recognitionPreprint
Yang, X., Li, J., & Zhou, X. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.11352 · Cited by 11
(2014) · doi:10.4230/oasics.slate.2021.16 · Cited by 1
Research works printed but not linked: 2
- [41][41] K. Vesely, A. Ghoshal, L. Burger, and D. Povey, 'Sequencediscriminative training of deep neural networks,' in INTERSPEECH , 2013. -
- [43][43] G. Zweig, C. Yu, J. Droppo, and A. Stolcke, 'Advances in AllNeural Speech Recognition,' in ICASSP , 2017. -
Cited by
Randaugment: Practical automated data augmentation with a reduced search space
Cubuk, E. D., Zoph, B., Shlens, J., & Le, Q. V. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw50498.2020.00359 · Cited by 3,223
Self-citationRandAugment: Practical automated data augmentation with a reduced search space
Cubuk, E. D., Zoph, B., Shlens, J., & Le, Q. V. (2019) · arXiv · doi:10.48550/arxiv.1909.13719 · Cited by 3,223
“Although this assumption is sufficient for identifying learned augmentation policies to improve performance [5, 56, 35, 24, 19], it is unclear if this assumption is overly stringent and may lead to sub-optimal data augmentation policies.”
BackgroundMethodSelf-citationConformer: Convolution-augmented Transformer for Speech Recognition
Gulati, A., Qin, J., Chiu, C.-C., Parmar, N., Zhang, Y., Yu, J., Han, W., Wang, S., Zhang, Z., Wu, Y., et al. (2020) · Interspeech 2020 · doi:10.21437/interspeech.2020-3015 · Cited by 3,150
Self-citationConformer: Convolution-augmented Transformer for Speech Recognition
Gulati, A., Qin, J., Chiu, C.-C., Parmar, N., Zhang, Y., Yu, J., Han, W., Wang, S., Zhang, Z., Wu, Y., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.08100 · Cited by 3,150
“We use SpecAugment [27, 28] with mask parameter (F = 27), and ten time masks with maximum time-mask ratio (pS = 0.05), where the maximum-size of the time mask is set to pS times the length of the utterance.”
MethodSelf-citationFixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence
Sohn, K., Berthelot, D., Li, C.-L., Zhang, Z., Carlini, N., Cubuk, E. D., Kurakin, A., Zhang, H., & Raffel, C. (2020) · arXiv · doi:10.48550/arxiv.2001.07685 · Cited by 2,892
Self-citation
Metadata
- Authors
- Park, D. S., Chan, W., Zhang, Y., Chiu, C.-C., Zoph, B., Cubuk, E. D., & Le, Q. V.
- Affiliations
- Google (United States) (US)
- Venue
- Interspeech 2019
- Publisher
- ISCA
- Published
- 2019-09-15
- Pages
- 2613-2617
- ISSN
- 2308-457X (ISSN-L)
- OpenAlex
- W2936774411
- Cited by
- 3,558
- References
- 51
- Citations by year
- 2026: 239 · 2025: 415 · 2024: 450 · 2023: 649 · 2022: 603 · 2021: 738 · 2020: 391 · 2019: 69
- FWCI
- 172.29
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- Interspeech (USB) (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
- Physical Sciences › Computer Science › Signal Processing › Speech and Audio Processing
- Physical Sciences › Computer Science › Signal Processing › Music and Audio Processing
- Keywords
- SpecAugment, data augmentation, speech recognition, frequency masking, feature warping, end-to-end speech recognition, LibriSpeech, switchboards, CALLHOME, word error rate
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.