The DOI matches its Crossref record; 61 of the 61 references it cites were found in a registry or an index.
End-to-End Waveform Utterance Enhancement for Direct Evaluation Metrics Optimization by Fully Convolutional Neural Networks
Open accessAbstract
Speech enhancement model is used to map a noisy speech to a clean speech. In the training stage, an objective function is often adopted to optimize the model parameters. However, in the existing literature, there is an inconsistency between the model optimization criterion and the evaluation criterion for the enhanced speech. For example, in measuring speech intelligibility, most of the evaluation metric is based on a short-time objective intelligibility (STOI) measure, while the frame based mean square error (MSE) between estimated and clean speech is widely used in optimizing the model. Due to the inconsistency, there is no guarantee that the trained model can provide optimal performance in applications. In this study, we propose an end-to-end utterance-based speech enhancement framework using fully convolutional neural networks (FCN) to reduce the gap between the model optimization and the evaluation criterion. Because of the utterance-based optimization, temporal correlation information of long speech segments, or even at the entire utterance level, can be considered to directly optimize perception-based objective functions. As an example, we implemented the proposed FCN enhancement framework to optimize the STOI measure. Experimental results show that the STOI of a test speech processed by the proposed approach is better than conventional MSE-optimized speech due to the consistency between the training and the evaluation targets. Moreover, by integrating the STOI into model optimization, the intelligibility of human subjects and automatic speech recognition system on the enhanced speech is also substantially improved compared to those generated based on the minimum MSE criterion.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Fully convolutional networks for semantic segmentation
Long, J., Shelhamer, E., & Darrell, T. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298965 · Cited by 38,834
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S., & Szegedy, C. (2015) · arXiv · doi:10.48550/arxiv.1502.03167 · Cited by 27,818
Discrete-time Signal Processing
Williamson, D. (1999) · Advanced Textbooks in Control and Signal Processing · doi:10.1007/978-1-4471-0541-1 · Cited by 4,858
Speech enhancement using a minimum mean-square error log-spectral amplitude estimator
Ephraim, Y., & Malah, D. (1985) · IEEE Transactions on Acoustics, Speech, and Signal Processing · doi:10.1109/tassp.1985.1164550 · Cited by 3,723
Rix, A. W., Beerends, J. G., Hollier, M. P., & Hekstra, A. P. (2002) · 2001 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings (Cat. No.01CH37221) · doi:10.1109/icassp.2001.941023 · Cited by 3,149
Speech Recognition with Primarily Temporal Cues
Shannon, R. V., Zeng, F.-G., Kamath, V., Wygonski, J., & Ekelid, M. (1995) · Science · doi:10.1126/science.270.5234.303 · Cited by 3,024
Speech enhancement using a minimum-mean square error short-time spectral amplitude estimator
Ephraim, Y., & Malah, D. (1984) · IEEE Transactions on Acoustics, Speech, and Signal Processing · doi:10.1109/tassp.1984.1164453 · Cited by 2,863
An Algorithm for Intelligibility Prediction of Time–Frequency Weighted Noisy Speech
Taal, C. H., Hendriks, R. C., Heusdens, R., & Jensen, J. (2011) · IEEE Transactions on Audio, Speech, and Language Processing · doi:10.1109/tasl.2011.2114881 · Cited by 2,254
Speech enhancement theory and practice
Loizou, P. C. (2007) · Cited by 1,823
Prechelt, L. (1998) · Lecture Notes in Computer Science · doi:10.1007/3-540-49430-8_3 · Cited by 1,649
Static Analysis of Shape in TensorFlow Programs
The Theano Development Team, Al‐Rfou, R., Alain, G., Almahairi, A., Angermueller, C., Bahdanau, D., Ballas, N., Bastien, F., Bayer, J., Belikov, A., et al. (2016) · arXiv (Cornell University) · doi:10.4230/lipics.ecoop.2020.15 · Cited by 1,589
Supervised Speech Separation Based on Deep Learning: An Overview
Wang, D., & Chen, J. (2018) · IEEE/ACM Transactions on Audio, Speech, and Language Processing · doi:10.1109/taslp.2018.2842159 · Cited by 1,524
A Regression Approach to Speech Enhancement Based on Deep Neural Networks
Xu, Y., Du, J., Dai, L.-R., & Lee, C.-H. (2015) · IEEE/ACM Transactions on Audio, Speech, and Language Processing · doi:10.1109/taslp.2014.2364452 · Cited by 1,366
Deep clustering: Discriminative embeddings for segmentation and separation
Hershey, J. R., Chen, Z., Le Roux, J., & Watanabe, S. (2016) · 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2016.7471631 · Cited by 1,354
SEGAN: Speech Enhancement Generative Adversarial Network
Pascual, S., Bonafonte, A., & Serrà, J. (2017) · Interspeech 2017 · doi:10.21437/interspeech.2017-1428 · Cited by 1,297
On Training Targets for Supervised Speech Separation
Wang, Y., Narayanan, A., & Wang, D. (2014) · IEEE/ACM Transactions on Audio, Speech, and Language Processing · doi:10.1109/taslp.2014.2352935 · Cited by 1,067
An Experimental Study on Speech Enhancement Based on Deep Neural Networks
Xu, Y., Du, J., Dai, L.-R., & Lee, C.-H. (2014) · IEEE Signal Processing Letters · doi:10.1109/lsp.2013.2291240 · Cited by 924
Speech enhancement based on deep denoising autoencoder
Lu, X., Tsao, Y., Matsuda, S., & Hori, C. (2013) · Interspeech 2013 · doi:10.21437/interspeech.2013-130 · Cited by 834
Self-citationKolbaek, M., Yu, D., Tan, Z.-H., & Jensen, J. (2017) · IEEE/ACM Transactions on Audio, Speech, and Language Processing · doi:10.1109/taslp.2017.2726762 · Cited by 820
Complex Ratio Masking for Monaural Speech Separation
Williamson, D. S., Wang, Y., & Wang, D. (2016) · IEEE/ACM Transactions on Audio, Speech, and Language Processing · doi:10.1109/taslp.2015.2512042 · Cited by 797
Speech Enhancement: Theory and Practice
Loizou, P. C. (2013) · CRC Press eBooks · doi:10.1201/b14529 · Cited by 759
Darpa Timit Acoustic-Phonetic Continuous Speech Corpus CD-ROM {TIMIT} | NIST
Garofolo, J. S., Lamel, L. F., Fisher, W. M., Fiscus, J. G., Pallett, D. S., & Dahlgren, N. L. (1993) · Cited by 680
Phase-sensitive and recognition-boosted speech separation using deep recurrent neural networks
Erdogan, H., Hershey, J. R., Watanabe, S., & Le Roux, J. (2015) · 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2015.7178061 · Cited by 646
Speech Enhancement with LSTM Recurrent Neural Networks and its Application to Noise-Robust ASR
Weninger, F., Erdogan, H., Watanabe, S., Vincent, E., Le Roux, J., Hershey, J. R., & Schuller, B. (2015) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-22482-4_11 · Cited by 602
Joint Optimization of Masks and Deep Recurrent Neural Networks for Monaural Source Separation
Huang, P.-S., Kim, M., Hasegawa-Johnson, M., & Smaragdis, P. (2015) · IEEE/ACM Transactions on Audio, Speech, and Language Processing · doi:10.1109/taslp.2015.2468583 · Cited by 461
A Wavenet for Speech Denoising
Rethage, D., Pons, J., & Serra, X. (2018) · 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2018.8462417 · Cited by 457
Listening Effort and Fatigue in School-Age Children With and Without Hearing Loss
Hicks, C. B., & Tharpe, A. M. (2002) · Journal of Speech, Language, and Hearing Research · doi:10.1044/1092-4388(2002/046 · Cited by 410
The importance of phase in speech enhancement
Paliwal, K., Wójcicki, K., & Shannon, B. (2011) · Speech Communication · doi:10.1016/j.specom.2010.12.003 · Cited by 405
The second ‘chime’ speech separation and recognition challenge: Datasets, tasks and baselines
Vincent, E., Barker, J., Watanabe, S., Le Roux, J., Nesta, F., & Matassoni, M. (2013) · 2013 IEEE International Conference on Acoustics, Speech and Signal Processing · doi:10.1109/icassp.2013.6637622 · Cited by 306
Discriminatively trained recurrent neural networks for single-channel speech separation
Weninger, F., Hershey, J. R., Le Roux, J., & Schuller, B. (2014) · 2014 IEEE Global Conference on Signal and Information Processing (GlobalSIP) · doi:10.1109/globalsip.2014.7032183 · Cited by 283
Phase Processing for Single-Channel Speech Enhancement: History and recent advances
Gerkmann, T., Krawczyk-Becker, M., & Roux, J. L. (2015) · IEEE Signal Processing Magazine · doi:10.1109/msp.2014.2369251 · Cited by 244
Exploring Speech Enhancement with Generative Adversarial Networks for Robust Speech Recognition
Donahue, C., Li, B., & Prabhavalkar, R. (2018) · 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2018.8462581 · Cited by 237
Michelsanti, D., & Tan, Z.-H. (2017) · Interspeech 2017 · doi:10.21437/interspeech.2017-1620 · Cited by 233
Development of the Mandarin Hearing in Noise Test (MHINT)
Wong, L. L. N., Soli, S. D., Liu, S., Han, N., & Huang, M.-W. (2007) · Ear & Hearing · doi:10.1097/aud.0b013e31803154d0 · Cited by 231
Raw waveform-based speech enhancement by fully convolutional networks
Fu, S.-W., Tsao, Y., Lu, X., & Kawai, H. (2017) · 2017 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC) · doi:10.1109/apsipa.2017.8281993 · Cited by 221
Self-citationMultiple-target deep learning for LSTM-RNN based speech enhancement
Sun, L., Du, J., Dai, L.-R., & Lee, C.-H. (2017) · 2017 Hands-free Speech Communications and Microphone Arrays (HSCMA) · doi:10.1109/hscma.2017.7895577 · Cited by 207
Kolbæk, M., Tan, Z.-H., & Jensen, J. (2017) · IEEE/ACM Transactions on Audio, Speech, and Language Processing · doi:10.1109/taslp.2016.2628641 · Cited by 193
SNR-Aware Convolutional Neural Network Modeling for Speech Enhancement
Fu, S.-W., Tsao, Y., & Lu, X. (2016) · Interspeech 2016 · doi:10.21437/interspeech.2016-211 · Cited by 177
Self-citationComplex spectrogram enhancement by convolutional neural network with multi-metrics learning
Fu, S.-W., Hu, T.-Y., Tsao, Y., & Lu, X. (2017) · 2017 IEEE 27th International Workshop on Machine Learning for Signal Processing (MLSP) · doi:10.1109/mlsp.2017.8168119 · Cited by 173
Self-citationXia, B.-Y., & Bao, C. (2014) · Speech Communication · doi:10.1016/j.specom.2014.02.001 · Cited by 157
When the greedy algorithm fails
Bang‐Jensen, J., Gutin, G., & Yeo, A. (2004) · Discrete Optimization · doi:10.1016/j.disopt.2004.03.007 · Cited by 152
Chen, Z., Watanabe, S., Erdoğan, H., & Hershey, J. R. (2015) · Interspeech (USB) · doi:10.21437/interspeech.2015-659 · Cited by 149
On the optimality of ideal binary time–frequency masks
Li, Y., & Wang, D. (2008) · Speech Communication · doi:10.1016/j.specom.2008.09.001 · Cited by 144
Single channel audio source separation using convolutional denoising autoencoders
Grais, E. M., & Plumbley, M. D. (2017) · IEEE Global Conference on Signal and Information Processing · doi:10.1109/globalsip.2017.8309164 · Cited by 136
Lai, Y.-H., Chen, F., Wang, S., Lu, X., Tsao, Y., & Lee, C. (2016) · IEEE Transactions on Biomedical Engineering · doi:10.1109/tbme.2016.2613960 · Cited by 133
Self-citationXu, Y., Du, J., Huang, Z., Dai, L.-R., & Lee, C. (2015) · Interspeech (USB) · doi:10.21437/interspeech.2015-358 · Cited by 116
SNR-Based Progressive Learning of Deep Neural Network for Speech Enhancement
Gao, T., Du, J., Dai, L.-R., & Lee, C. (2016) · Interspeech (USB) · doi:10.21437/interspeech.2016-224 · Cited by 82
Speech Enhancement Using Bayesian Wavenet
Qian, K., Zhang, Y., Chang, S., Yang, X., Florencio, D. A. F., & Hasegawa‐Johnson, M. (2017) · Interspeech (USB) · doi:10.21437/interspeech.2017-1672 · Cited by 82
Complex ratio masking for joint enhancement of magnitude and phase
Williamson, D. S., Wang, Y., & Wang, D. (2016) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2016.7472673 · Cited by 77
Kolbæk, M., Tan, Z., & Jensen, J. (2018) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2018.8462040 · Cited by 68
Koizumi, Y., Niwa, K., Hioka, Y., Kobayashi, K., & Haneda, Y. (2017) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2017.7952122 · Cited by 54
Perception Optimized Deep Denoising AutoEncoders for Speech Enhancement
Shivakumar, P. G., & Georgiou, P. (2016) · Interspeech (USB) · doi:10.21437/interspeech.2016-1284 · Cited by 50
Fundamentals of Noise Reduction
Chen, J., Benesty, J., Huang, Y. A., & Diethorn, E. J. (2007) · Springer handbooks · doi:10.1007/978-3-540-49127-9_43 · Cited by 49
Two-Stage Single-Channel Audio Source Separation Using Deep Neural Networks
Grais, E. M., Roma, G., Simpson, A. J. R., & Plumbley, M. D. (2017) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2017.2716443 · Cited by 42
Moore, A. H., Parada, P. P., & Naylor, P. A. (2016) · Computer Speech & Language · doi:10.1016/j.csl.2016.11.003 · Cited by 40
Erdoğan, H., Hershey, J. R., Watanabe, S., & Roux, J. L. (2017) · Springer eBooks · doi:10.1007/978-3-319-64680-0_7 · Cited by 35
Multi-Resolution Fully Convolutional Neural Networks for Monaural Audio Source Separation
Grais, E. M., Wierstorf, H., Ward, D., & Plumbley, M. D. (2018) · Lecture notes in computer science · doi:10.1007/978-3-319-93764-9_32 · Cited by 23
Distant speech recognition for home automation: Preliminary experimental results in a smart home
Lecouteux, B., Vacher, M., & Portet, F. (2011) · International Conference on Speech Technology and Human-Computer Dialogue (SpeD) · doi:10.1109/sped.2011.5940728 · Cited by 15
Wang, S., Li, K., Huang, Z., Siniscalchi, S. M., & Lee, C. (2017) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2017.7953223 · Cited by 12
Deep Neural Network Approach for Single Channel Speech Enhancement Processing
Li, D. (2016) · uO Research (University of Ottawa) · doi:10.20381/ruor-5532 · Cited by 1
Cited by
Conv-TasNet: Surpassing Ideal Time–Frequency Magnitude Masking for Speech Separation
Luo, Y., & Mesgarani, N. (2019) · IEEE/ACM Transactions on Audio, Speech, and Language Processing · doi:10.1109/taslp.2019.2915167 · Cited by 2,042
Deep Learning for Audio Signal Processing
Purwins, H., Li, B., Virtanen, T., Schluter, J., Chang, S.-Y., & Sainath, T. (2019) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2019.2908700 · Cited by 855
Dual-Path RNN: Efficient Long Sequence Modeling for Time-Domain Single-Channel Speech Separation
Luo, Y., Chen, Z., & Yoshioka, T. (2020) · ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp40776.2020.9054266 · Cited by 780
DCCRN: Deep Complex Convolution Recurrent Network for Phase-Aware Speech Enhancement
Hu, Y., Liu, Y., Lv, S., Xing, M., Zhang, S., Fu, Y., Wu, J., Zhang, B., & Xie, L. (2020) · Interspeech 2020 · doi:10.21437/interspeech.2020-2537 · Cited by 737
Noise Reduction in ECG Signals Using Fully Convolutional Denoising Autoencoders
Chiang, H.-T., Hsieh, Y.-Y., Fu, S.-W., Hung, K.-H., Tsao, Y., & Chien, S.-Y. (2019) · IEEE Access · doi:10.1109/access.2019.2912036 · Cited by 363
Self-citation
Metadata
- Authors
- Fu, S.-W., Wang, T.-W., Tsao, Y., Lu, X., & Kawai, H.
- Venue
- IEEE/ACM Transactions on Audio, Speech, and Language Processing
- Published
- 2018-09
- Volume
- 26
- Issue
- 9
- Pages
- 1570-1584
- ISSN
- 2329-9290 (ISSN-L)
- 2329-9304
- OpenAlex
- W2963045393
- Cited by
- 314
- References
- 61
- FWCI
- 30.67
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.