The DOI matches its Crossref record; 61 of the 61 references it cites were found in a registry or an index.
Conv-TasNet: Surpassing Ideal Time–Frequency Magnitude Masking for Speech Separation
Open accessAbstract
Single-channel, speaker-independent speech separation methods have recently seen great progress. However, the accuracy, latency, and computational cost of such methods remain insufficient. The majority of the previous methods have formulated the separation problem through the time-frequency representation of the mixed signal, which has several drawbacks, including the decoupling of the phase and magnitude of the signal, the suboptimality of time-frequency representation for speech separation, and the long latency of the entire system. To address these shortcomings, we propose a fully-convolutional time-domain audio separation network (Conv-TasNet), a deep learning framework for end-to-end time-domain speech separation. Conv-TasNet uses a linear encoder to generate a representation of the speech waveform optimized for separating individual speakers. Speaker separation is achieved by applying a set of weighting functions (masks) to the encoder output. The modified encoder representations are then inverted back to the waveforms using a linear decoder. The masks are found using a temporal convolutional network (TCN) consisting of stacked 1-D dilated convolutional blocks, which allows the network to model the long-term dependencies of the speech signal while maintaining a small model size. The proposed Conv-TasNet system significantly outperforms previous time-frequency masking methods in separating two- and three-speaker mixtures. Additionally, Conv-TasNet surpasses several ideal time-frequency magnitude masks in two-speaker speech separation as evaluated by both objective distortion measures and subjective quality assessment by human listeners. Finally, Conv-TasNet has a significantly smaller model size and a much shorter minimum latency, making it a suitable solution for both offline and real-time speech separation applications. This study therefore represents a major step toward the realization of speech separation systems for real-world speech processing technologies.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
U-Net: Convolutional Networks for Biomedical Image Segmentation
Ronneberger, O., Fischer, P., & Brox, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_28 · Cited by 99,404
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
MobileNetV2: Inverted Residuals and Linear Bottlenecks
Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., & Chen, L.-C. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00474 · Cited by 28,755
Xception: Deep Learning with Depthwise Separable Convolutions
Chollet, F. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.195 · Cited by 20,380
Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification
He, K., Zhang, X., Ren, S., & Sun, J. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.123 · Cited by 19,741
MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
Howard, A. G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., & Adam, H. (2017) · arXiv · doi:10.48550/arxiv.1704.04861 · Cited by 19,015
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S. (2024) · arXiv (Cornell University) · doi:10.57702/o9raffed · Cited by 14,214
WaveNet: a generative model for raw audio
Oord, A. V. D., Dieleman, S., Zen, H., Simonyan, K., Vinyals, O., Graves, A., Kalchbrenner, N., Senior, A., & Kavukcuoglu, K. (2016) · arXiv · doi:10.48550/arxiv.1609.03499 · Cited by 4,911
An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling
Bai, S., Kolter, J. Z., & Koltun, V. (2018) · arXiv · doi:10.48550/arxiv.1803.01271 · Cited by 4,655
A Statistical Method for Evaluating Systematic Relationships
Sokal, R. R., & Michener, C. D. (1958) · Biodiversity Heritage Library (Smithsonian Institution) · doi:10.5281/zenodo.16435756 · Cited by 4,416
Cited by
SDR – Half-baked or Well Done?
Roux, J. L., Wisdom, S., Erdogan, H., & Hershey, J. R. (2019) · ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2019.8683855 · Cited by 1,146
Dual-Path RNN: Efficient Long Sequence Modeling for Time-Domain Single-Channel Speech Separation
Luo, Y., Chen, Z., & Yoshioka, T. (2020) · ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp40776.2020.9054266 · Cited by 780
Self-citationDCCRN: Deep Complex Convolution Recurrent Network for Phase-Aware Speech Enhancement
Hu, Y., Liu, Y., Lv, S., Xing, M., Zhang, S., Fu, Y., Wu, J., Zhang, B., & Xie, L. (2020) · Interspeech 2020 · doi:10.21437/interspeech.2020-2537 · Cited by 737
Attention Is All You Need In Speech Separation
Subakan, C., Ravanelli, M., Cornell, S., Bronzi, M., & Zhong, J. (2021) · ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp39728.2021.9413901 · Cited by 635
Real Time Speech Enhancement in the Waveform Domain
Défossez, A., Synnaeve, G., & Adi, Y. (2020) · Interspeech 2020 · doi:10.21437/interspeech.2020-2409 · Cited by 537
SpeechBrain: A General-Purpose Speech Toolkit
Ravanelli, M., Parcollet, T., Plantinga, P., Rouhe, A., Cornell, S., Lugosch, L., Subakan, C., Dawalatabad, N., Heba, A., Zhong, J., et al. (2021) · arXiv · doi:10.48550/arxiv.2106.04624 · Cited by 512
A review of deep learning techniques for speech processing
Mehrish, A., Majumder, N., Bharadwaj, R., Mihalcea, R., & Poria, S. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.101869 · Cited by 389
Reddy, C. K. A., Gopal, V., Cutler, R., Beyrami, E., Cheng, R., Dubey, H., Matusevych, S., Aichner, R., Aazami, A., Braun, S., et al. (2020) · Interspeech 2020 · doi:10.21437/interspeech.2020-3038 · Cited by 363
A review of speaker diarization: Recent advances with deep learning
Park, T. J., Kanda, N., Dimitriadis, D., Han, K. J., Watanabe, S., & Narayanan, S. (2022) · Computer Speech & Language · doi:10.1016/j.csl.2021.101317 · Cited by 354
A survey of sound source localization with deep learning methods
Grumiaux, P.-A., Kitić, S., Girin, L., & Guérin, A. (2022) · The Journal of the Acoustical Society of America · doi:10.1121/10.0011809 · Cited by 347
PHASEN: A Phase-and-Harmonics-Aware Speech Enhancement Network
Yin, D., Luo, C., Xiong, Z., & Zeng, W. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i05.6489 · Cited by 323
Chen, J., Mao, Q., & Liu, D. (2020) · Interspeech 2020 · doi:10.21437/interspeech.2020-2205 · Cited by 320
An Overview of Deep-Learning-Based Audio-Visual Speech Enhancement and Separation
Michelsanti, D., Tan, Z.-H., Zhang, S.-X., Xu, Y., Yu, M., Yu, D., & Jensen, J. (2021) · IEEE/ACM Transactions on Audio, Speech, and Language Processing · doi:10.1109/taslp.2021.3066303 · Cited by 316
A New Framework for CNN-Based Speech Enhancement in the Time Domain
Pandey, A., & Wang, D. (2019) · IEEE/ACM Transactions on Audio, Speech, and Language Processing · doi:10.1109/taslp.2019.2913512 · Cited by 299
Wavesplit: End-to-End Speech Separation by Speaker Clustering
Zeghidour, N., & Grangier, D. (2021) · IEEE/ACM Transactions on Audio, Speech, and Language Processing · doi:10.1109/taslp.2021.3099291 · Cited by 257
AudioGPT: Understanding and Generating Speech, Music, Sound, and Talking Head
Huang, R., Li, M., Yang, D., Shi, J., Chang, X., Ye, Z., Wu, Y., Hong, Z., Huang, J., Liu, J., et al. (2023) · arXiv · doi:10.48550/arxiv.2304.12995 · Cited by 131
Kinoshita, K., Ochiai, T., Delcroix, M., & Nakatani, T. (2020) · ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp40776.2020.9053266 · Cited by 104
Noise Estimation for Generative Diffusion Models
San-Roman, R., Nachmani, E., & Wolf, L. (2021) · arXiv · doi:10.48550/arxiv.2104.02600 · Cited by 44
Sense and Learn: Self-supervision for omnipresent sensors
Saeed, A., Ungureanu, V., & Gfeller, B. (2021) · Machine Learning with Applications · doi:10.1016/j.mlwa.2021.100152 · Cited by 34
The Right to Talk: An Audio-Visual Transformer Approach
Truong, T.-D., Duong, C. N., De Vu, T., Pham, H. A., Raj, B., Le, N., & Luu, K. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00114 · Cited by 38
Wang, N. Y.-H., Wang, H.-L. S., Wang, T.-W., Fu, S.-W., Lu, X., Wang, H.-M., & Tsao, Y. (2021) · IEEE Transactions on Neural Systems and Rehabilitation Engineering · doi:10.1109/tnsre.2020.3042655 · Cited by 24
Automating medical simulations
Gershov, S., Braunold, D., Spektor, R., Ioscovich, A., Raz, A., & Laufer, S. (2023) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2023.104446 · Cited by 8
Metadata
- Authors
- Luo, Y., & Mesgarani, N.
- Venue
- IEEE/ACM Transactions on Audio, Speech, and Language Processing
- Published
- 2019-08
- Volume
- 27
- Issue
- 8
- Pages
- 1256-1266
- ISSN
- 2329-9290 (ISSN-L)
- 2329-9304
- OpenAlex
- W2952218014
- Cited by
- 2,042
- References
- 61
- FWCI
- 137.85
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.