The DOI matches its Crossref record; 43 of the 43 references it cites were found in a registry or an index.
A New Framework for CNN-Based Speech Enhancement in the Time Domain
Open accessAbstract
This paper proposes a new learning mechanism for a fully convolutional neural network (CNN) to address speech enhancement in the time domain. The CNN takes as input the time frames of noisy utterance and outputs the time frames of the enhanced utterance. At the training time, we add an extra operation that converts the time domain to the frequency domain. This conversion corresponds to simple matrix multiplication, and is hence differentiable implying that a frequency domain loss can be used for training in the time domain. We use mean absolute error loss between the enhanced short-time Fourier transform (STFT) magnitude and the clean STFT magnitude to train the CNN. This way, the model can exploit the domain knowledge of converting a signal to the frequency domain for analysis. Moreover, this approach avoids the well-known invalid STFT problem since the proposed CNN operates in the time domain. Experimental results demonstrate that the proposed method substantially outperforms the other methods of speech enhancement. The proposed method is easy to implement and applicable to related speech processing tasks that require time-frequency masking or spectral mapping.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
U-Net: Convolutional Networks for Biomedical Image Segmentation
Ronneberger, O., Fischer, P., & Brox, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_28 · Cited by 99,404
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
(2014) · Cited by 33,419
Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification
He, K., Zhang, X., Ren, S., & Sun, J. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.123 · Cited by 19,741
Labaca-Castro, R. (2023) · Machine Learning under Malware Attack · doi:10.1007/978-3-658-40442-0_9 · Cited by 19,272
(2010) · Cited by 14,134
WaveNet: a generative model for raw audio
Oord, A. V. D., Dieleman, S., Zen, H., Simonyan, K., Vinyals, O., Graves, A., Kalchbrenner, N., Senior, A., & Kavukcuoglu, K. (2016) · arXiv · doi:10.48550/arxiv.1609.03499 · Cited by 4,911
Learning Deconvolution Network for Semantic Segmentation
Noh, H., Hong, S., & Han, B. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.178 · Cited by 4,422
Rix, A. W., Beerends, J. G., Hollier, M. P., & Hekstra, A. P. (2002) · 2001 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings (Cat. No.01CH37221) · doi:10.1109/icassp.2001.941023 · Cited by 3,149
Performance measurement in blind audio source separation
Vincent, E., Gribonval, R., & Fevotte, C. (2006) · IEEE Transactions on Audio, Speech and Language Processing · doi:10.1109/tsa.2005.858005 · Cited by 2,889
TIMIT Acoustic-Phonetic Continuous Speech Corpus
Garofolo, J. S., Lamel, L. F., Fisher, W. M., Pallett, D. S., Dahlgren, N. L., Zue, V. W., & Fiscus, J. G. (2020) · Americanae (AECID Library) · doi:10.35111/17gk-bn40 · Cited by 2,508
An Algorithm for Intelligibility Prediction of Time–Frequency Weighted Noisy Speech
Taal, C. H., Hendriks, R. C., Heusdens, R., & Jensen, J. (2011) · IEEE Transactions on Audio, Speech, and Language Processing · doi:10.1109/tasl.2011.2114881 · Cited by 2,254
Varga, A., & Steeneken, H. J. M. (1993) · Speech Communication · doi:10.1016/0167-6393(93)90095-3 · Cited by 2,105
Signal estimation from modified short-time Fourier transform
Griffin, D., & Lim, J. (1984) · IEEE Transactions on Acoustics, Speech, and Signal Processing · doi:10.1109/tassp.1984.1164317 · Cited by 2,095
Conv-TasNet: Surpassing Ideal Time–Frequency Magnitude Masking for Speech Separation
Luo, Y., & Mesgarani, N. (2019) · IEEE/ACM Transactions on Audio, Speech, and Language Processing · doi:10.1109/taslp.2019.2915167 · Cited by 2,042
Speech enhancement theory and practice
Loizou, P. C. (2007) · Cited by 1,823
Supervised Speech Separation Based on Deep Learning: An Overview
Wang, D., & Chen, J. (2018) · IEEE/ACM Transactions on Audio, Speech, and Language Processing · doi:10.1109/taslp.2018.2842159 · Cited by 1,524
Self-citationA Regression Approach to Speech Enhancement Based on Deep Neural Networks
Xu, Y., Du, J., Dai, L.-R., & Lee, C.-H. (2015) · IEEE/ACM Transactions on Audio, Speech, and Language Processing · doi:10.1109/taslp.2014.2364452 · Cited by 1,366
SEGAN: Speech Enhancement Generative Adversarial Network
Pascual, S., Bonafonte, A., & Serrà, J. (2017) · Interspeech 2017 · doi:10.21437/interspeech.2017-1428 · Cited by 1,297
DARPA TIMIT: acoustic-phonetic continuous speech corpus CD-ROM, NIST speech disc 1-1.1
Garofolo, J. S., Lamel, L. F., Fisher, W. M., Fiscus, J. G., Pallett, D. S., & Dahlgren, N. L. (1993) · doi:10.6028/nist.ir.4930 · Cited by 1,267
Gradient Flow in Recurrent Nets: The Difficulty of Learning LongTerm Dependencies
(2009) · A Field Guide to Dynamical Recurrent Networks · doi:10.1109/9780470544037.ch14 · Cited by 1,246
IEEE Recommended Practice for Speech Quality Measurements
(1969) · IEEE Transactions on Audio and Electroacoustics · doi:10.1109/tau.1969.1162058 · Cited by 1,232
The design for the wall street journal-based CSR corpus
Paul, D. B., & Baker, J. M. (1992) · Proceedings of the workshop on Speech and Natural Language - HLT '91 · doi:10.3115/1075527.1075614 · Cited by 1,099
On Training Targets for Supervised Speech Separation
Wang, Y., Narayanan, A., & Wang, D. (2014) · IEEE/ACM Transactions on Audio, Speech, and Language Processing · doi:10.1109/taslp.2014.2352935 · Cited by 1,067
Self-citationSpeech enhancement based on deep denoising autoencoder
Lu, X., Tsao, Y., Matsuda, S., & Hori, C. (2013) · Interspeech 2013 · doi:10.21437/interspeech.2013-130 · Cited by 834
Complex Ratio Masking for Monaural Speech Separation
Williamson, D. S., Wang, Y., & Wang, D. (2016) · IEEE/ACM Transactions on Audio, Speech, and Language Processing · doi:10.1109/taslp.2015.2512042 · Cited by 797
Self-citationSpeech Enhancement: Theory and Practice
Loizou, P. C. (2013) · CRC Press eBooks · doi:10.1201/b14529 · Cited by 759
Computational Auditory Scene Analysis: Principles, Algorithms and Applications
Darwin, C. (2008) · The Journal of the Acoustical Society of America · doi:10.1121/1.2920958 · Cited by 749
Phase-sensitive and recognition-boosted speech separation using deep recurrent neural networks
Erdogan, H., Hershey, J. R., Watanabe, S., & Le Roux, J. (2015) · 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2015.7178061 · Cited by 646
A Wavenet for Speech Denoising
Rethage, D., Pons, J., & Serra, X. (2018) · 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2018.8462417 · Cited by 457
Towards Scaling Up Classification-Based Speech Separation
Wang, Y., & Wang, D. (2013) · IEEE Transactions on Audio, Speech, and Language Processing · doi:10.1109/tasl.2013.2250961 · Cited by 430
Self-citationA Fully Convolutional Neural Network for Speech Enhancement
Park, S. R., & Lee, J. W. (2017) · Interspeech 2017 · doi:10.21437/interspeech.2017-1465 · Cited by 409
Fu, S.-W., Wang, T.-W., Tsao, Y., Lu, X., & Kawai, H. (2018) · IEEE/ACM Transactions on Audio, Speech, and Language Processing · doi:10.1109/taslp.2018.2821903 · Cited by 314
Learning Spectral Mapping for Speech Dereverberation and Denoising
Han, K., Wang, Y., Wang, D., Woods, W. S., Merks, I., & Zhang, T. (2015) · IEEE/ACM Transactions on Audio, Speech, and Language Processing · doi:10.1109/taslp.2015.2416653 · Cited by 260
Self-citationLong short-term memory for speaker generalization in supervised speech separation
Chen, J., & Wang, D. (2017) · The Journal of the Acoustical Society of America · doi:10.1121/1.4986931 · Cited by 244
Self-citationRaw waveform-based speech enhancement by fully convolutional networks
Fu, S.-W., Tsao, Y., Lu, X., & Kawai, H. (2017) · 2017 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC) · doi:10.1109/apsipa.2017.8281993 · Cited by 221
Complex spectrogram enhancement by convolutional neural network with multi-metrics learning
Fu, S.-W., Hu, T.-Y., Tsao, Y., & Lu, X. (2017) · 2017 IEEE 27th International Workshop on Machine Learning for Signal Processing (MLSP) · doi:10.1109/mlsp.2017.8168119 · Cited by 173
End-to-End Speech Separation with Unfolded Iterative Phase Reconstruction
Wang, Z.-Q., Roux, J. L., Wang, D., & Hershey, J. R. (2018) · Interspeech (USB) · doi:10.21437/interspeech.2018-1629 · Cited by 157
Self-citationA deep neural network for time-domain signal reconstruction
Wang, Y., & Wang, D. (2015) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2015.7178800 · Cited by 100
Self-citationOn Adversarial Training and Loss Functions for Speech Enhancement
Pandey, A., & Wang, D. (2018) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2018.8462614 · Cited by 85
Self-citationSpeech Enhancement Using Bayesian Wavenet
Qian, K., Zhang, Y., Chang, S., Yang, X., Florencio, D. A. F., & Hasegawa‐Johnson, M. (2017) · Interspeech (USB) · doi:10.21437/interspeech.2017-1672 · Cited by 82
TasNet: Surpassing Ideal Time-Frequency Masking for Speech Separation.Preprint
Luo, Y., & Mesgarani, N. (2018) · arXiv (Cornell University) · Cited by 73
Gated Residual Networks with Dilated Convolutions for Supervised Speech Separation
Tan, K., Chen, J., & Wang, D. (2018) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2018.8461819 · Cited by 52
Self-citation
Cited by
Wang, N. Y.-H., Wang, H.-L. S., Wang, T.-W., Fu, S.-W., Lu, X., Wang, H.-M., & Tsao, Y. (2021) · IEEE Transactions on Neural Systems and Rehabilitation Engineering · doi:10.1109/tnsre.2020.3042655 · Cited by 24
Metadata
- Authors
- Pandey, A., & Wang, D.
- Venue
- IEEE/ACM Transactions on Audio, Speech, and Language Processing
- Published
- 2019-07
- Volume
- 27
- Issue
- 7
- Pages
- 1179-1188
- ISSN
- 2329-9290 (ISSN-L)
- 2329-9304
- OpenAlex
- W2943554574
- Cited by
- 299
- References
- 43
- FWCI
- 24.58
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.