CoChat
Sign Up
Cited by 104Speech and Audio ProcessingSpeech Recognition and SynthesisImage and Signal Denoising Methods

Improving Noise Robust Automatic Speech Recognition with Single-Channel Time-Domain Enhancement Network

Open access
Kinoshita, Keisuke, Ochiai, Tsubasa, Delcroix, Marc, Nakatani, Tomohiro·2020·ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
DOI 10.1109/icassp40776.2020.9053266
Open this paper in CoChat — free accountOpen PDF

Abstract

With the advent of deep learning, research on noise-robust automatic speech recognition (ASR) has progressed rapidly. However, ASR performance in noisy conditions of single-channel systems remains unsatisfactory. Indeed, most single-channel speech enhancement (SE) methods (denoising) have brought only limited performance gains over state-of-the-art ASR back-end trained on multi-condition training data. Recently, there has been much research on neural network-based SE methods working in the time-domain showing levels of performance never attained before. However, it has not been established whether the high enhancement performance achieved by such time-domain approaches could be translated into ASR. In this paper, we show that a single-channel time-domain denoising approach can significantly improve ASR performance, providing more than 30 % relative word error reduction over a strong ASR back-end on the real evaluation data of the single-channel track of the CHiME-4 dataset. These positive results demonstrate that single-channel noise reduction can still improve ASR performance, which should open the door to more research in that direction.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1109/icassp40776.2020.9053266

Cites

2 of 2 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • The quality of ASR in poor acoustic environments has greatly improved, e.g. accuracy in noisy environments.

    “The quality of ASR in poor acoustic environments has greatly improved, e.g. the accuracy in noisy environments [Kinoshita et al. 2020].”

    From Measuring the Accuracy of Automatic Speech Recognition Solutions (2023) · cites this paper as Kinoshita et al. 2020

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 27

  1. G. Saon, G. Kurata, T. Sercu, K. Audhkhasi, S. Thomas, D. Dimitriadis, X. Cui, B. Ramabhadran, M. Picheny, L.-L. Lim, B. Roomi, and P. Hall, "English conversational telephone speech recognition by humans and machines," in Proc. of Interspeech'17, 2017, pp. 132-136.
  2. J. Heymann, L. Drude, C. Boeddeker, P. Hanebrink, and R. Haeb- Umbach, "Beamnet: End-to-end training of a beamformer-supported multi-channel asr system," in Proc. of ICASSP'17, 2017, pp. 5325- 5329.
  3. M. Kolbaek, "Single-microphone speech enhancement and separation using deep learning," Ph.D. dissertation, Aalborg Universitetsforlag, 2018.
  4. S.-W. Fu, Y. Tsao, X. Lu, and H. Kawai, "Raw waveform-based speech enhancement by fully convolutional networks," in Proc. of APSIPA'17, 2017, pp. 006-012.
  5. J. Chen and D. Wang, "Long short-term memory for speaker general- ization in supervised speech separation," The Journal of the Acoustical Society of America, vol. 141, no. 6, pp. 4705-4714, 2017.
  6. Y. Wang, A. Narayanan, and D. Wang, "On training targets for super- vised speech separation," IEEE/ACM transactions on audio, speech, and language processing, vol. 22, no. 12, pp. 1849-1858, 2014.
  7. H. Erdogan, J. R. Hershey, S. Watanabe, and J. L. Roux, "Phase- sensitive and recognition-boosted speech separation using deep recur- rent neural networks," in Proc. of ICASSP'15, 2015, pp. 708-712.
  8. D. S. Williamson, Y. Wang, and D. Wang, "Complex ratio masking for monaural speech separation," IEEE/ACM Transactions on Audio, Speech and Language Processing (TASLP), vol. 24, no. 3, pp. 483-492, 2016.
  9. T. Yoshioka, N. Ito, M. Delcroix, A. Ogawa, K. Kinoshita, M. Fuji- moto, C. Yu, W. J. Fabian, M. Espi, T. Higuchi, et al., "The NTT chime-3 system: Advances in speech enhancement and recognition for mobile multi-microphone devices," in 2015 IEEE Workshop on Auto- matic Speech Recognition and Understanding (ASRU). IEEE, 2015, pp. 436-443.
  10. S.-J. Chen, A. S. Subramanian, H. Xu, and S. Watanabe, "Build- ing state-of-the-art distant speech recognition using the chime-4 chal- lenge with a setup of speech enhancement baseline," in Proc. of Inter- speech'18, 2018, pp. 1571-1575.
  11. M. Fujimoto and H. Kawai, "One-pass single-channel noisy speech recognition using a combination of noisy and enhanced features," in Proc. of Interspeech'19, 2019, pp. 486-490.
  12. S. Pascual, A. Bonafonte, and J. Serrà, "SEGAN: Speech enhancement generative adversarial network," in Proc. of Interspeech'17, 2017, pp. 3642-3646.
  13. D. Rethage, J. Pons, and X. Serra, "A Wavenet for speech denoising," in Proc. of ICASSP'18, 2018, pp. 5069-5073.
  14. S.-W. Fu, T.-W. Wang, Y. Tsao, X. Lu, and H. Kawai, "End-to-end waveform utterance enhancement for direct evaluation metrics opti- mization by fully convolutional neural networks," IEEE/ACM Trans. ASLP, vol. 26, no. 9, pp. 1570-1584, 2018.
  15. Y. Luo and N. Mesgarani, "TasNet: Surpassing ideal time-frequency masking for speech separation," in Proc. of ICASSP'18, 2018.
  16. Y. Luo and N. Mesgarani, "Conv-TasNet: Surpassing ideal time- frequency magnitude masking for speech separation," IEEE/ACM Trans. ASLP, vol. 27, no. 8, pp. 1256-1266, 2019.
  17. S. Bai, J. Z. Kolter, and V. Koltun, "An empirical evaluation of generic convolutional and recurrent networks for sequence modeling," arXiv preprint arXiv:1803.01271, 2018. · arXiv
  18. J. Le Roux, S. Wisdom, H. Erdogan, and J. R. Hershey, "SDR-half- baked or well done?," in Proc. of ICASSP'19, 2019, pp. 626-630.
  19. H. Erdogan and T. Yoshioka, "Investigations on data augmentation and loss functions for deep learning based speech-background separation," in Proc. of Interspeech'18, 2018, pp. 3499-3503.
  20. M. Kolbæk, D. Yu, Z.-H. Tan, and J. Jensen, "Multitalker speech sep- aration with utterance-level permutation invariant training of deep re- current neural networks," IEEE/ACM Trans. ASLP, vol. 25, no. 10, pp. 1901-1913, 2017.
  21. C. Donahue, B. Li, and R. Prabhavalkar, "Exploring speech enhance- ment with generative adversarial networks for robust speech recogni- tion," in Proc. of ICASSP'18, 2018, pp. 5024-5028.
  22. J. S. Lim and A. V. Oppenheim, "Enhancement and bandwidth com- pression of noisy speech," Proceedings of the IEEE, vol. 67, no. 12, pp. 1586-1604, 1979.
  23. T. Menne, R. Schlüter, and H. Ney, "Investigation into joint optimiza- tion of single channel speech enhancement and acoustic modeling for robust asr," in Proc. of ICASSP'19, 2019, pp. 6660-6664.
  24. E. Vincent, R. Gribonval, and C. Févotte, "Performance measurement in blind audio source separation," IEEE trans. ASLP, vol. 14, no. 4, pp. 1462-1469, 2006.
  25. J. Barker, R. Marxer, E. Vincent, and S. Watanabe, "The third CHiME speech separation and recognition challenge: Dataset, task and base- lines," in Proc. of ASRU'15, 2015, pp. 504-511.
  26. J. B. Allen and D. A. Berkley, "Image method for efficiently simulat- ing small-room acoustics," The Journal of the Acoustical Society of America, vol. 65, no. 4, pp. 943-950, 1979.
  27. S. Watanabe, T. Hori, S. Karita, T. Hayashi, J. Nishitoba, Y. Unno, N. Enrique Yalta Soplin, J. Heymann, M. Wiesner, N. Chen, A. Ren- duchintala, and T. Ochiai, "ESPnet: End-to-End Speech Processing Toolkit," in Proc. Interspeech'18, 2018, pp. 2207-2211.

Cited by

Sorted by citations

Authors
Kinoshita, K., Ochiai, T., Delcroix, M., & Nakatani, T.
Affiliations
NTT (Japan) (JP)
ORCID
Venue
ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
Publisher
IEEE
Published
2020-05
Pages
7009-7013
ISSN
1520-6149 (ISSN-L)
DOI
10.1109/icassp40776.2020.9053266
OpenAlex
W3015312544
Cited by
104
References
2
Citations by year
2026: 3 · 2025: 15 · 2024: 21 · 2023: 22 · 2022: 19 · 2021: 19 · 2020: 5
FWCI
10.16
Type
conference paper
Language
en
Open access
Licence
Locations
  • IEEE International Conference on Acoustics Speech and Signal Processing (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Signal Processing › Speech and Audio Processing
  • Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Image and Signal Denoising Methods
Keywords
noise-robust speech recognition, single-channel speech enhancement, speech recognition, CHiME-4, word error rate reduction, noisy speech

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review