The DOI matches its Crossref record; 45 of the 45 references it cites were found in a registry or an index.
Multitalker Speech Separation With Utterance-Level Permutation Invariant Training of Deep Recurrent Neural Networks
Open accessAbstract
In this paper, we propose the utterance-level permutation invariant training (uPIT) technique. uPIT is a practically applicable, end-to-end, deep-learning-based solution for speaker independent multitalker speech separation. Specifically, uPIT extends the recently proposed permutation invariant training (PIT) technique with an utterance-level cost function, hence eliminating the need for solving an additional permutation problem during inference, which is otherwise required by frame-level PIT. We achieve this using recurrent neural networks (RNNs) that, during training, minimize the utterance-level separation error, hence forcing separated frames belonging to the same speaker to be aligned to the same output stream. In practice, this allows RNNs, trained with uPIT, to separate multitalker mixed speech without any prior knowledge of signal duration, number of speakers, speaker identity, or gender. We evaluated uPIT on the WSJ0 and Danish two- and three-talker mixed-speech separation tasks and found that uPIT outperforms techniques based on nonnegative matrix factorization and computational auditory scene analysis, and compares favorably with deep clustering, and the deep attractor network. Furthermore, we found that models trained with uPIT generalize well to unseen speakers and languages. Finally, we found that a single model, trained with uPIT, can handle both two-speaker, and three-speaker speech mixtures.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Hinton, G., Deng, L., Yu, D., Dahl, G., Mohamed, A.-R., Jaitly, N., Senior, A., Vanhoucke, V., Nguyen, P., Sainath, T., et al. (2012) · IEEE Signal Processing Magazine · doi:10.1109/msp.2012.2205597 · Cited by 10,260
Self-citationALGORITHMS FOR NON-NEGATIVE MATRIX FACTORIZATION
Lee, D. D., & Seung, H. S. (2000) · neural information processing systems · Cited by 9,877
Bengio, Y., Louradour, J., Collobert, R., & Weston, J. (2009) · Proceedings of the 26th Annual International Conference on Machine Learning · doi:10.1145/1553374.1553380 · Cited by 5,366
Some Experiments on the Recognition of Speech, with One and with Two Ears
Cherry, E. C. (1953) · The Journal of the Acoustical Society of America · doi:10.1121/1.1907229 · Cited by 4,549
Rix, A. W., Beerends, J. G., Hollier, M. P., & Hekstra, A. P. (2002) · 2001 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings (Cat. No.01CH37221) · doi:10.1109/icassp.2001.941023 · Cited by 3,149
Context-dependent pre-trained deep neural networks for large-vocabulary speech recognition
Dahl, G. E., Yu, D., Deng, L., & Acero, A. (2012) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/tasl.2011.2134090 · Cited by 3,011
Self-citationPerformance measurement in blind audio source separation
Vincent, E., Gribonval, R., & Fevotte, C. (2006) · IEEE Transactions on Audio, Speech and Language Processing · doi:10.1109/tsa.2005.858005 · Cited by 2,889
Deep clustering: Discriminative embeddings for segmentation and separation
Hershey, J. R., Chen, Z., Le Roux, J., & Watanabe, S. (2016) · 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2016.7471631 · Cited by 1,354
Factorial Hidden Markov Models
Ghahramani, Z., & Jordan, M. I. (1997) · Machine Learning · doi:10.1023/a:1007425814087 · Cited by 1,157
On Training Targets for Supervised Speech Separation
Wang, Y., Narayanan, A., & Wang, D. (2014) · IEEE/ACM Transactions on Audio, Speech, and Language Processing · doi:10.1109/taslp.2014.2352935 · Cited by 1,067
An Experimental Study on Speech Enhancement Based on Deep Neural Networks
Xu, Y., Du, J., Dai, L.-R., & Lee, C.-H. (2014) · IEEE Signal Processing Letters · doi:10.1109/lsp.2013.2291240 · Cited by 924
Conversational speech transcription using context-dependent deep neural networks
Seide, F., Li, G., & Yu, D. (2011) · Interspeech 2011 · doi:10.21437/interspeech.2011-169 · Cited by 852
Self-citationPermutation invariant training of deep models for speaker-independent multi-talker speech separation
Yu, D., Kolbaek, M., Tan, Z.-H., & Jensen, J. (2017) · 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2017.7952154 · Cited by 839
Self-citationBronkhorst, A. W. (2000) · TNO Repository · Cited by 799
Complex Ratio Masking for Monaural Speech Separation
Williamson, D. S., Wang, Y., & Wang, D. (2016) · IEEE/ACM Transactions on Audio, Speech, and Language Processing · doi:10.1109/taslp.2015.2512042 · Cited by 797
A Theoretically Grounded Application of Dropout in Recurrent Neural Networks
Gal, Y., & Ghahramani, Z. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1512.05287 · Cited by 766
Computational Auditory Scene Analysis: Principles, Algorithms and Applications
Darwin, C. (2008) · The Journal of the Acoustical Society of America · doi:10.1121/1.2920958 · Cited by 749
Phase-sensitive and recognition-boosted speech separation using deep recurrent neural networks
Erdogan, H., Hershey, J. R., Watanabe, S., & Le Roux, J. (2015) · 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2015.7178061 · Cited by 646
Speech Enhancement with LSTM Recurrent Neural Networks and its Application to Noise-Robust ASR
Weninger, F., Erdogan, H., Watanabe, S., Vincent, E., Le Roux, J., Hershey, J. R., & Schuller, B. (2015) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-22482-4_11 · Cited by 602
Computational Auditory Scene Analysis
Wang, D., & Brown, G. J. (2006) · Institute of Electrical and Electronics Engineers eBooks · doi:10.1109/9780470043387 · Cited by 522
Haykin, S., & Chen, Z. (2005) · Neural Computation · doi:10.1162/0899766054322964 · Cited by 505
Joint Optimization of Masks and Deep Recurrent Neural Networks for Monaural Source Separation
Huang, P.-S., Kim, M., Hasegawa-Johnson, M., & Smaragdis, P. (2015) · IEEE/ACM Transactions on Audio, Speech, and Language Processing · doi:10.1109/taslp.2015.2468583 · Cited by 461
Achieving Human Parity in Conversational Speech Recognition
Xiong, W., Droppo, J., Huang, X., Seide, F. T. B., Seltzer, M., Stolcke, A., Yu, D., & Zweig, G. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1610.05256 · Cited by 451
Self-citationTowards Scaling Up Classification-Based Speech Separation
Wang, Y., & Wang, D. (2013) · IEEE Transactions on Audio, Speech, and Language Processing · doi:10.1109/tasl.2013.2250961 · Cited by 430
Deep attractor network for single-microphone speaker separation
Chen, Z., Luo, Y., & Mesgarani, N. (2017) · 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2017.7952155 · Cited by 397
Convolutive Speech Bases and Their Application to Supervised Speech Separation
Smaragdis, P. (2007) · IEEE Transactions on Audio, Speech and Language Processing · doi:10.1109/tasl.2006.876726 · Cited by 383
Single-channel speech separation using sparse non-negative matrix factorization
Schmidt, M. N., & Olsson, R. K. (2006) · Interspeech 2006 · doi:10.21437/interspeech.2006-655 · Cited by 365
English Conversational Telephone Speech Recognition by Humans and Machines
Saon, G., Kurata, G., Sercu, T., Audhkhasi, K., Thomas, S., Dimitriadis, D., Cui, X., Ramabhadran, B., Picheny, M., Lim, L.-L., et al. (2017) · Interspeech 2017 · doi:10.21437/interspeech.2017-405 · Cited by 349
An Introduction to Computational Networks and the Computational Network Toolkit
Yu, D., Eversole, A., Seltzer, M., Yao, K., Kuchaiev, O., Zhang, Y., Seide, F. T. B., Huang, Z., Guenter, B. K., Wang, H., et al. (2014) · Cited by 332
Self-citationPrediction-driven computational auditory scene analysis
Ellis, D. P. W. (1996) · DSpace@MIT (Massachusetts Institute of Technology) · doi:10.7916/d84j0n13 · Cited by 305
Toward Human Parity in Conversational Speech Recognition
Xiong, W., Droppo, J., Huang, X., Seide, F., Seltzer, M. L., Stolcke, A., Yu, D., & Zweig, G. (2017) · IEEE/ACM Transactions on Audio, Speech, and Language Processing · doi:10.1109/taslp.2017.2756440 · Cited by 245
Self-citationA Deep Ensemble Learning Method for Monaural Speech Separation
Zhang, X.-L., & Wang, D. (2016) · IEEE/ACM Transactions on Audio, Speech, and Language Processing · doi:10.1109/taslp.2016.2536478 · Cited by 219
Monaural speech separation and recognition challenge
Cooke, M., Hershey, J. R., & Rennie, S. J. (2010) · Computer Speech & Language · doi:10.1016/j.csl.2009.02.006 · Cited by 197
Kolbæk, M., Tan, Z.-H., & Jensen, J. (2017) · IEEE/ACM Transactions on Audio, Speech, and Language Processing · doi:10.1109/taslp.2016.2628641 · Cited by 193
Self-citationYu, D., Deng, L., & Dahl, G. E. (2010) · neural information processing systems · Cited by 189
Self-citationModelling Auditory Processing and Organisation
Cooke, M. P. (1993) · OpenGrey (Institut de l'Information Scientifique et Technique) · Cited by 172
Goehring, T., Bolner, F., Monaghan, J. J. M., van Dijk, B., Zarowski, A. J., & Bleeck, S. (2016) · Hearing Research · doi:10.1016/j.heares.2016.11.012 · Cited by 149
Deep Neural Networks for Single-Channel Multi-Talker Speech Recognition
Weng, C., Yu, D., Seltzer, M. L., & Droppo, J. (2015) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2015.2444659 · Cited by 90
Self-citationSuper-human multi-talker speech recognition: the IBM 2006 speech separation challenge system
Kristjansson, T. T., Hershey, J., Olsen, P. A., Rennie, S. J., & Gopinath, R. A. (2006) · Interspeech (USB) · doi:10.21437/interspeech.2006-25 · Cited by 87
Speech separation of a target speaker based on deep neural networks
Du, J., Tu, Y.-H., Xu, Y., Dai, L.-R., & Lee, C. (2014) · International Conference on Signal Processing · doi:10.1109/icosp.2014.7015050 · Cited by 83
Source–Filter-Based Single-Channel Speech Separation Using Pitch Information
Stark, M. M., Wohlmayr, M., & Pernkopf, F. (2010) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/tasl.2010.2047419 · Cited by 62
Erdoğan, H., Hershey, J. R., Watanabe, S., & Roux, J. L. (2017) · Springer eBooks · doi:10.1007/978-3-319-64680-0_7 · Cited by 35
Recurrent deep stacking networks for supervised speech separation
Wang, Z.-Q., & Wang, D. (2017) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2017.7952120 · Cited by 34
The Computational Network Toolkit [Best of the Web]
Yu, D., Yao, K., & Zhang, Y. (2015) · IEEE Signal Processing Magazine · doi:10.1109/msp.2015.2462371 · Cited by 16
Self-citation
Cited by
Conv-TasNet: Surpassing Ideal Time–Frequency Magnitude Masking for Speech Separation
Luo, Y., & Mesgarani, N. (2019) · IEEE/ACM Transactions on Audio, Speech, and Language Processing · doi:10.1109/taslp.2019.2915167 · Cited by 2,042
Supervised Speech Separation Based on Deep Learning: An Overview
Wang, D., & Chen, J. (2018) · IEEE/ACM Transactions on Audio, Speech, and Language Processing · doi:10.1109/taslp.2018.2842159 · Cited by 1,524
SDR – Half-baked or Well Done?
Roux, J. L., Wisdom, S., Erdogan, H., & Hershey, J. R. (2019) · ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2019.8683855 · Cited by 1,146
Dual-Path RNN: Efficient Long Sequence Modeling for Time-Domain Single-Channel Speech Separation
Luo, Y., Chen, Z., & Yoshioka, T. (2020) · ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp40776.2020.9054266 · Cited by 780
TaSNet: Time-Domain Audio Separation Network for Real-Time, Single-Channel Speech Separation
Luo, Y., & Mesgarani, N. (2018) · 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2018.8462116 · Cited by 686
Metadata
- Authors
- Kolbaek, M., Yu, D., Tan, Z.-H., & Jensen, J.
- Venue
- IEEE/ACM Transactions on Audio, Speech, and Language Processing
- Published
- 2017-10
- Volume
- 25
- Issue
- 10
- Pages
- 1901-1913
- ISSN
- 2329-9290 (ISSN-L)
- 2329-9304
- OpenAlex
- W2734774145
- Cited by
- 820
- References
- 45
- FWCI
- 66.08
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.