CoChat
Sign Up
Cited by 537Speech and Audio ProcessingSpeech Recognition and SynthesisMusic and Audio Processing

The DOI matches its Crossref record, with a small difference in title or year; 4 of the 4 references it cites were found in a registry or an index.

Real Time Speech Enhancement in the Waveform Domain

Open access
Défossez, Alexandre, Synnaeve, Gabriel, Adi, Yossi·2020·Interspeech 2020
DOI 10.21437/interspeech.2020-2409
Open this paper in CoChat — free accountOpen PDF

Abstract

We present a causal speech enhancement model working on the raw waveform that runs in real-time on a laptop CPU. The proposed model is based on an encoder-decoder architecture with skip-connections. It is optimized on both time and frequency domains, using multiple loss functions. Empirical evidence shows that it is capable of removing various kinds of background noise including stationary and non-stationary noises, as well as room reverb. Additionally, we suggest a set of data augmentation techniques applied directly on the raw waveform which further improve model performance and its generalization abilities. We perform evaluations on several standard benchmarks, both using objective metrics and human judgements. The proposed model matches state-of-the-art performance of both causal and non causal methods while working directly on the raw waveform.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.21437/interspeech.2020-2409

Cites

4 of 4 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Flagged first, then by citations

Research works printed but not linked: 38

  1. C. K. Reddy et al., "A scalable noisy speech dataset and online subjective test framework," preprint arXiv:1909.08050, 2019. · arXiv
  2. C. K. A. Reddy et al., "An individualized super-gaussian sin- gle microphone speech enhancement for hearing aid users with smartphone as an assistive device," IEEE signal processing let- ters, vol. 24, no. 11, pp. 1601-1605, 2017.
  3. C. Zorila, C. Boeddeker, R. Doddipatla, and R. Haeb-Umbach, "An investigation into the effectiveness of enhancement in asr training and test for chime-5 dinner party transcription," preprint arXiv:1909.12208, 2019. · arXiv
  4. J. S. Lim and A. V. Oppenheim, "Enhancement and bandwidth compression of noisy speech," Proceedings of the IEEE, vol. 67, no. 12, pp. 1586-1604, 1979.
  5. Y. Ephraim and D. Malah, "Speech enhancement using a minimum-mean square error short-time spectral amplitude esti- mator," IEEE Transactions on acoustics, speech, and signal pro- cessing, vol. 32, no. 6, pp. 1109-1121, 1984.
  6. N. Krishnamurthy and J. H. Hansen, "Babble noise: modeling, analysis, and applications," IEEE transactions on audio, speech, and language processing, vol. 17, no. 7, pp. 1394-1407, 2009.
  7. S. Pascual, A. Bonafonte, and J. Serra, "Segan: Speech enhance- ment generative adversarial network," preprint arXiv:1703.09452, 2017. · arXiv
  8. H. Phan et al., "Improving gans for speech enhancement," preprint arXiv:2001.05532, 2020. · arXiv
  9. Y. Luo and N. Mesgarani, "Conv-TASnet: Surpassing ideal time- frequency magnitude masking for speech separation," IEEE/ACM transactions on audio, speech, and language processing, vol. 27, no. 8, pp. 1256-1266, 2019.
  10. E. Nachmani, Y. Adi, and L. Wolf, "Voice separation with an un- known number of multiple speakers," arXiv:2003.01531, 2020. · arXiv
  11. A. Dfossez et al., "Music source separation in the waveform do- main," 2019, preprint arXiv:1911.13254. · arXiv
  12. R. Yamamoto, E. Song, and J.-M. Kim, "Parallel wavegan: A fast waveform generation model based on generative ad- versarial networks with multi-resolution spectrogram," preprint arXiv:1910.11480, 2019. · arXiv
  13. --, "Probability density distillation with generative adversarial networks for high-quality parallel waveform generation," preprint arXiv:1904.04472, 2019. · arXiv
  14. Y. N. Dauphin et al., "Language modeling with gated convolu- tional networks," in ICML, 2017.
  15. J. Smith and P. Gossett, "A flexible sampling-rate conversion method," in ICASSP, vol. 9. IEEE, 1984, pp. 112-115.
  16. C. Macartney and T. Weyde, "Improved speech enhancement with the wave-u-net," preprint arXiv:1811.11307, 2018. · arXiv
  17. M. H. Soni, N. Shah, and H. A. Patil, "Time-frequency masking- based speech enhancement using generative adversarial network," in ICASSP. IEEE, 2018, pp. 5039-5043.
  18. S.-W. Fu et al., "Metricgan: Generative adversarial networks based black-box metric scores optimization for speech enhance- ment," in ICML, 2019.
  19. Q. Zhang et al., "Deepmmse: A deep learning approach to mmse- based noise power spectral density estimation," IEEE/ACM Trans- actions on Audio, Speech, and Language Processing, 2020.
  20. C. H. Taal et al., "An algorithm for intelligibility prediction of time-frequency weighted noisy speech," IEEE Transactions on Audio, Speech, and Language Processing, vol. 19, no. 7, pp. 2125-2136, 2011.
  21. Y. Hu and P. C. Loizou, "Evaluation of objective quality measures for speech enhancement," IEEE Transactions on audio, speech, and language processing, vol. 16, no. 1, pp. 229-238, 2007.
  22. F. Protasio Ribeiro et al., "Crowdmos: An approach for crowd- sourcing mean opinion score studies," in ICASSP. IEEE, 2011.
  23. D. S. Park et al., "Specaugment: A simple data augmentation method for automatic speech recognition," in Interspeech, 2019.
  24. Y. Xia et al., "Weighted speech distortion losses for neural- network-based real-time speech enhancement," preprint arXiv:2001.10601, 2020. · arXiv
  25. G. Synnaeve et al., "End-to-end asr: from supervised to semi-supervised learning with modern architectures," preprint arXiv:1911.08460, 2019. · arXiv
  26. Y. Hu and P. C. Loizou, "Subjective comparison of speech en- hancement algorithms," in ICASSP, vol. 1. IEEE, 2006, pp. I-I.
  27. Y. Wang and D. Wang, "A deep neural network for time-domain signal reconstruction," in ICASSP. IEEE, 2015, pp. 4390-4394.
  28. F. Weninger, H. Erdogan, S. Watanabe, E. Vincent, J. Le Roux, J. R. Hershey, and B. Schuller, "Speech enhancement with lstm recurrent neural networks and its application to noise-robust asr," in International Conference on Latent Variable Analysis and Sig- nal Separation. Springer, 2015, pp. 91-99.
  29. Y. Xu et al., "A regression approach to speech enhancement based on deep neural networks," IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 23, no. 1, pp. 7-19, 2014.
  30. D. Rethage, J. Pons, and X. Serra, "A wavenet for speech denois- ing," in ICASSP. IEEE, 2018, pp. 5069-5073.
  31. A. Nicolson and K. K. Paliwal, "Deep learning for minimum mean-square error approaches to speech enhancement," Speech Communication, vol. 111, pp. 44-55, 2019.
  32. F. G. Germain, Q. Chen, and V. Koltun, "Speech denoising with deep feature losses," preprint arXiv:1806.10522, 2018. · arXiv
  33. M. Nikzad, A. Nicolson, Y. Gao, J. Zhou, K. K. Paliwal, and F. Shang, "Deep residual-dense lattice network for speech en- hancement," preprint arXiv:2002.12794, 2020. · arXiv
  34. K. Wang, J. Zhang, S. Sun, Y. Wang, F. Xiang, and L. Xie, "In- vestigating generative adversarial networks based speech derever- beration for robust speech recognition," arXiv:1803.10132, 2018. · arXiv
  35. D. Baby and S. Verhulst, "Sergan: Speech enhancement using relativistic generative adversarial networks with gradient penalty," in ICASSP. IEEE, 2019, pp. 106-110.
  36. Y. Xu et al., "Multi-objective learning and mask-based post- processing for deep neural network based speech enhancement," preprint arXiv:1703.07172, 2017. · arXiv
  37. F. Weninger et al., "Discriminatively trained recurrent neural networks for single-channel speech separation," in GlobalSIP. IEEE, 2014, pp. 577-581.
  38. K. Tan and D. Wang, "A convolutional recurrent neural network for real-time speech enhancement." in Interspeech, vol. 2018, 2018, pp. 3229-3233.

Cited by

Sorted by citations

Metadata

Authors
Défossez, A., Synnaeve, G., & Adi, Y.
Affiliations
Meta (United Kingdom) (GB)
ORCID
Venue
Interspeech 2020
Publisher
ISCA
Published
2020-10-25
Pages
3291-3295
ISSN
2308-457X (ISSN-L)
DOI
10.21437/interspeech.2020-2409
OpenAlex
W3036367741
Cited by
537
References
4
Citations by year
2026: 54 · 2025: 121 · 2024: 121 · 2023: 107 · 2022: 85 · 2021: 48
FWCI
37.50
Type
conference paper
Language
en
Open access
Licence
public-domain
Locations
  • Interspeech (USB) (publishedVersion)
  • arXiv (Cornell University) (submittedVersion, public-domain)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Signal Processing › Speech and Audio Processing
  • Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
  • Physical Sciences › Computer Science › Signal Processing › Music and Audio Processing
Keywords
speech enhancement, real-time speech enhancement, encoder-decoder architecture, skip connections, background noise suppression, room reverberation, data augmentation, CPU-only inference

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review