Joint Speech Recognition and Speaker Diarization via Sequence Transduction
Open accessAbstract
Speech applications dealing with conversations require not only recognizing the spoken words, but also determining who spoke when. The task of assigning words to speakers is typically addressed by merging the outputs of two separate systems, namely, an automatic speech recognition (ASR) system and a speaker diarization (SD) system. The two systems are trained independently with different objective functions. Often the SD systems operate directly on the acoustics and are not constrained to respect word boundaries and this deficiency is overcome in an ad hoc manner. Motivated by recent advances in sequence to sequence learning, we propose a novel approach to tackle the two tasks by a joint ASR and SD system using a recurrent neural network transducer. Our approach utilizes both linguistic and acoustic cues to infer speaker roles, as opposed to typical SD systems, which only use acoustic cues. We evaluated the performance of our approach on a large corpus of medical conversations between physicians and patients. Compared to a competitive conventional baseline, our approach improves word-level diarization error rate from 15.8% to 2.2%.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The position of the recording device has a strong impact on the captured audio.
“The position of the recording device also has a strong impact on the captured audio.18,20”
From Challenges of developing a digital scribe to reduce clinical documentation burden (2019) · cites this paper as [20]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · see citing works
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · see citing works
Speech recognition with deep recurrent neural networks
Graves, A., Mohamed, A.-R., & Hinton, G. (2013) · 2013 IEEE International Conference on Acoustics, Speech and Signal Processing · doi:10.1109/icassp.2013.6638947 · Cited by 8,893 · see citing works
Graves, A., Fernández, S., Gomez, F., & Schmidhuber, J. (2006) · Proceedings of the 23rd international conference on Machine learning - ICML '06 · doi:10.1145/1143844.1143891 · Cited by 5,370 · see citing works
In-Datacenter Performance Analysis of a Tensor Processing Unit
Jouppi, N. P., Young, C., Patil, N., Patterson, D., Agrawal, G., Bajwa, R., Bates, S., Bhatia, S., Boden, N., Borchers, A., et al. (2017) · Proceedings of the 44th Annual International Symposium on Computer Architecture · doi:10.1145/3079856.3080246 · Cited by 4,537 · see citing works
X-Vectors: Robust DNN Embeddings for Speaker Recognition
Snyder, D., Garcia-Romero, D., Sell, G., Povey, D., & Khudanpur, S. P. (2018) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2018.8461375 · Cited by 2,698 · see citing works
VoxCeleb2: Deep Speaker Recognition
Chung, J. S., Nagrani, A., & Zisserman, A. (2018) · Interspeech 2018 · doi:10.21437/interspeech.2018-1929 · Cited by 2,256 · see citing works
Sequence Transduction with Recurrent Neural NetworksPreprint
Graves, A. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1211.3711 · Cited by 1,320 · see citing works
Speaker Diarization: A Review of Recent Research
Anguera, X., Bozonnet, S., Evans, N., Fredouille, C., Friedland, G. H., & Vinyals, O. (2012) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/tasl.2011.2125954 · Cited by 671 · see citing works
Streaming End-to-end Speech Recognition for Mobile Devices
He, Y., Sainath, T. N., Prabhavalkar, R., McGraw, I., Alvarez, R., Zhao, D., Rybach, D., Kannan, A., Wu, Y., Pang, R., et al. (2019) · ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2019.8682336 · Cited by 601 · see citing works
An overview of automatic speaker diarization systems
Tranter, S. E., & Reynolds, D. A. (2006) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/tasl.2006.878256 · Cited by 597 · see citing works
End-to-end text-dependent speaker verification
Heigold, G., Moreno, I. L., Samy Bengio, A., & Shazeer, N. (2016) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2016.7472652 · Cited by 531 · see citing works
Wang, Q., Downey, C., Wan, L., Mansfield, P. A., & Moreno, I. L. (2018) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2018.8462628 · Cited by 324 · see citing works
Neural Speech Recognizer: Acoustic-to-Word LSTM Model for Large Vocabulary Speech Recognition
Soltau, H., Liao, H., & Sak, H. (2017) · Interspeech 2017 · doi:10.21437/interspeech.2017-1566 · Cited by 304 · see citing works
Self-citationPhoneme Recognition Using Time-Delay Neural Networks
Waibel, A. H., Hanazawa, T., Hinton, G. E., Shikano, K., & LANG, K. J. (1990) · Elsevier eBooks · doi:10.1016/b978-0-08-051584-7.50037-1 · Cited by 301 · see citing works
Speaker diarization using deep neural network embeddings
Garcia-Romero, D., Snyder, D., Sell, G., Povey, D., & McCree, A. V. (2017) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2017.7953094 · Cited by 244 · see citing works
A robust speaker clustering algorithm
Ajmera, J., & Wooters, C. (2004) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru.2003.1318476 · Cited by 243 · see citing works
Fully Supervised Speaker Diarization
Zhang, A., Wang, Q., Zhu, Z., Paisley, J. W., & Wang, C. (2019) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2019.8683892 · Cited by 235 · see citing works
Speaker diarization with plda i-vector scoring and unsupervised calibration
Sell, G., & Garcia-Romero, D. (2014) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt.2014.7078610 · Cited by 213 · see citing works
Sell, G., Snyder, D., McCree, A. V., Garcia-Romero, D., Villalba, J., Maciejewski, M., Manohar, V., Dehak, N., Povey, D., Watanabe, S., et al. (2018) · Interspeech (USB) · doi:10.21437/interspeech.2018-1893 · Cited by 211 · see citing works
Multistage speaker diarization of broadcast news
Barras, C., Zhu, X., Meignier, S., & Gauvain, J.-L. (2006) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/tasl.2006.878261 · Cited by 180 · see citing works
TristouNet: Triplet loss for speaker turn embedding
Bredin, H. (2017) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2017.7953194 · Cited by 173 · see citing works
Morfessor 2.0: Python Implementation and Extensions for Morfessor Baseline
Virpioja, S., Smit, P., Grönroos, S.-A., & Kurimo, M. (2013) · Aaltodoc (Aalto University) · Cited by 156 · see citing works
The Use of Recurrent Neural Networks in Continuous Speech Recognition
Robinson, T., Hochberg, M. M., & Renals, S. J. (1996) · Kluwer international series in engineering and computer science · doi:10.1007/978-1-4613-1367-0_10 · Cited by 109 · see citing works
Feature Learning with Raw-Waveform CLDNNs for Voice Activity Detection
Zazo, R., Sainath, T. N., Simko, G., & Parada, C. (2016) · Interspeech (USB) · doi:10.21437/interspeech.2016-268 · Cited by 108 · see citing works
Speaker diarization from speech transcripts
Lamel, L. F., Gauvain, J., & Canseco-Rodriguez, L. (2004) · Interspeech (USB) · doi:10.21437/interspeech.2004-250 · Cited by 46 · see citing works
Park, T., & Georgiou, P. (2018) · Interspeech (USB) · doi:10.21437/interspeech.2018-1364 · Cited by 35 · see citing works
Efficient Implementation of Recurrent Neural Network Transducer in Tensorflow
Bagby, T., Rao, K., & Sim, K. C. (2018) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt.2018.8639690 · Cited by 29 · see citing works
Streaming End-to-end Speech Recognition For Mobile DevicesPreprint
He, Y., Sainath, T. N., Prabhavalkar, R., et al. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.06621 · Cited by 23 · see citing works
Reducing the computational complexity for whole word models
Soltau, H., Liao, H., & Sak, H. (2017) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru.2017.8268917 · Cited by 14 · see citing works
Self-citationImproving the efficiency of forward-backward algorithm using batched computation in TensorFlow
Sim, K. C., Narayanan, A., Bagby, T., Sainath, T. N., & Bacchiani, M. (2017) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru.2017.8268944 · Cited by 9 · see citing works
Fully Supervised Speaker DiarizationPreprint
Zhang, A., Wang, Q., Zhu, Z., et al. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.04719 · Cited by 8 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Recent Advances in End-to-End Automatic Speech Recognition
Li, J. (2022) · APSIPA Transactions on Signal and Information Processing · doi:10.1561/116.00000050 · Cited by 411 · see citing works
Speaker recognition based on deep learning: An overview
Bai, Z., & Zhang, X.-L. (2021) · Neural Networks · doi:10.1016/j.neunet.2021.03.004 · Cited by 386 · see citing works
A review of speaker diarization: Recent advances with deep learning
Park, T., Kanda, N., Dimitriadis, D., Han, K. J., Watanabe, S., & Narayanan, S. S. (2021) · Computer Speech & Language · doi:10.1016/j.csl.2021.101317 · Cited by 363 · see citing works
Challenges of developing a digital scribe to reduce clinical documentation burden
Quiroz, J. C., Laranjo, L., Kocaballi, A. B., Berkovsky, S., Rezazadegan, D., & Coiera, E. (2019) · npj Digital Medicine · doi:10.1038/s41746-019-0190-1 · Cited by 162 · see citing works
The digital scribe in clinical practice: a scoping review and research agenda
van Buchem, M. M., Boosman, H., Bauer, M. P., Kant, I. M. J., Cammel, S. A., & Steyerberg, E. W. (2021) · npj Digital Medicine · doi:10.1038/s41746-021-00432-5 · Cited by 152 · see citing works
Kanda, N., Gaur, Y., Wang, X., Meng, Z., Chen, Z., Zhou, T., & Yoshioka, T. (2020) · Interspeech (USB) · doi:10.21437/interspeech.2020-1085 · Cited by 70 · see citing works
Serialized Output Training for End-to-End Overlapped Speech Recognition
Kanda, N., Gaur, Y., Wang, X., Meng, Z., & Yoshioka, T. (2020) · Interspeech (USB) · doi:10.21437/interspeech.2020-999 · Cited by 67 · see citing works
Xia, W., Han, L., Wang, Q., Tripathi, A., Huang, Y., Moreno, I. L., & Sak, H. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9746531 · Cited by 52 · see citing works
Speech Recognition and Multi-Speaker Diarization of Long Conversations
Mao, H. H., Li, S. X., McAuley, J., & Cottrell, G. W. (2020) · Interspeech (USB) · doi:10.21437/interspeech.2020-3039 · Cited by 52 · see citing works
Serialized Output Training for End-to-End Overlapped Speech Recognition
Kanda, N., Gaur, Y., Wang, X., Meng, Z., & Yoshioka, T. (2020) · Interspeech (USB) · doi:10.21437/interspeech.2020-0999 · Cited by 51 · see citing works
Alignment-Length Synchronous Decoding for RNN Transducer
Saon, G., Tüske, Z., & Audhkhasi, K. (2020) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp40776.2020.9053040 · Cited by 49 · see citing works
Investigation of End-to-End Speaker-Attributed ASR for Continuous Multi-Talker Recordings
Kanda, N., Chang, X., Gaur, Y., Wang, X., Meng, Z., Chen, Z., & Yoshioka, T. (2021) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt48900.2021.9383600 · Cited by 41 · see citing works
A dataset of simulated patient-physician medical interviews with a focus on respiratory cases
Fareez, F., Parikh, T., Wavell, C., Shahab, S., Chevalier, M., Good, S., De Blasi, I., Rhouma, R., McMahon, C., Lam, J.-P., et al. (2022) · Scientific Data · doi:10.1038/s41597-022-01423-1 · Cited by 39 · see citing works
Towards Neural Diarization for Unlimited Numbers of Speakers Using Global and Local Attractors
Horiguchi, S., Watanabe, S., Garcia, L. P., Xue, Y., Takashima, Y., & Kawaguchi, Y. (2021) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru51503.2021.9687875 · Cited by 36 · see citing works
Multi-Task Learning for Speaker Verification and Voice Trigger Detection
Sigtia, S., Marchi, E., Kajarekar, S. S., Naik, D. K., & Bridle, J. S. (2020) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp40776.2020.9054760 · Cited by 30 · see citing works
End-to-End Speaker-Attributed ASR with Transformer
Kanda, N., Ye, G., Gaur, Y., Wang, X., Meng, Z., Chen, Z., & Yoshioka, T. (2021) · Interspeech (USB) · doi:10.21437/interspeech.2021-101 · Cited by 31 · see citing works
Streaming Multi-Speaker ASR with RNN-T
Sklyar, I., Piunova, A., & Liu, Y. (2021) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp39728.2021.9413471 · Cited by 27 · see citing works
Lyu, K.-M., Lyu, R.-Y., & Chang, H.-T. (2024) · PeerJ Computer Science · doi:10.7717/peerj-cs.1973 · Cited by 26 · see citing works
End-to-End Speech Summarization Using Restricted Self-Attention
Sharma, R., Palaskar, S., Black, A. W., & Metze, F. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9747320 · Cited by 25 · see citing works
Integrating Text Inputs for Training and Adapting RNN Transducer ASR Models
Thomas, S., Kingsbury, B. E. D., Saon, G., & Kuo, H.-K. J. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9747862 · Cited by 25 · see citing works
Maiti, S., Erdoğan, H., Wilson, K., Wisdom, S., Watanabe, S., & Hershey, J. R. (2021) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp39728.2021.9414841 · Cited by 21 · see citing works
Pretext Tasks Selection for Multitask Self-Supervised Audio Representation Learning
Zaiem, S., Parcollet, T., Essid, S., & Heba, A. (2022) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2022.3195430 · Cited by 22 · see citing works
Serafini, L., Cornell, S., Morrone, G., Zovato, E., Brutti, A., & Squartini, S. (2023) · Computer Speech & Language · doi:10.1016/j.csl.2023.101534 · Cited by 23 · see citing works
Rnn-transducer With Language Bias For End-to-end Mandarin-English Code-switching Speech Recognition
Zhang, S., Yi, J., Tian, Z., Tao, J., & Bai, Y. (2021) · International Symposium on Chinese Spoken Language Processing (ISCSLP) · doi:10.1109/iscslp49672.2021.9362075 · Cited by 21 · see citing works
Streaming Speaker-Attributed ASR with Token-Level Speaker Embeddings
Kanda, N., Wu, J., Wu, Y., Xiao, X., Meng, Z., Wang, X., Gaur, Y., Chen, Z., Li, J., & Yoshioka, T. (2022) · Interspeech (USB) · doi:10.21437/interspeech.2022-253 · Cited by 19 · see citing works
Identifying relevant information in medical conversations to summarize a clinician-patient encounter
Quiroz, J. C., Laranjo, L., Kocaballi, A. B., Briatore, A., Berkovsky, S., Rezazadegan, D., & Coiera, E. W. (2020) · Health Informatics Journal · doi:10.1177/1460458220951719 · Cited by 19 · see citing works
Zuluaga-Gómez, J., Sarfjoo, S. S., Prasad, A., Nigmatulina, I., Motlíček, P., Ondrej, K., Ohneiser, O., & Helmke, H. (2023) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt54892.2023.10022718 · Cited by 18 · see citing works
RNN Transducer Models for Spoken Language Understanding
Thomas, S., Kuo, H.-K. J., Saon, G., Tüske, Z., Kingsbury, B. E. D., Kurata, G., Kons, Z., & Hoory, R. (2021) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp39728.2021.9414029 · Cited by 16 · see citing works
A Comprehensive Evaluation of Incremental Speech Recognition and Diarization for Conversational AI
Addlesee, A., Yu, Y., & Eshghi, A. (2020) · Proceedings of the 28th International Conference on Computational Linguistics · doi:10.18653/v1/2020.coling-main.312 · Cited by 15 · see citing works
Towards Reducing the Need for Speech Training Data to Build Spoken Language Understanding Systems
Thomas, S., Kuo, H.-K. J., Kingsbury, B. E. D., & Saon, G. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9747555 · Cited by 14 · see citing works
Minimum Bayes Risk Training for End-to-End Speaker-Attributed ASR
Kanda, N., Meng, Z., Lu, L., Gaur, Y., Wang, X., Chen, Z., & Yoshioka, T. (2021) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp39728.2021.9415062 · Cited by 11 · see citing works
USED: Universal Speaker Extraction and Diarization
Ao, J., Yldrm, M. S., Tao, R., Ge, M., Wang, S., Qian, Y., & Li, H. (2024) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2024.3511268 · Cited by 12 · see citing works
Hybrid Speech and Text Analysis Methods for Speaker Change Detection
Anidjar, O. H., Lapidot, I., Hajaj, C., Dvir, A. Z., & Gilad, I. (2021) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2021.3093817 · Cited by 11 · see citing works
Yang, C.-H. H., Park, T., Gong, Y., Li, Y., Chen, Z., Lin, Y., Chen, C., Hu, Y., Dhawan, K., Żelasko, P., et al. (2024) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt61566.2024.10832176 · Cited by 13 · see citing works
DiCoW: Diarization-conditioned Whisper for target speaker automatic speech recognition
Polok, A., Klement, D., Kocour, M., Han, J., Landini, F., Yusuf, B., Wiesner, M., Khudanpur, S. P., Černocký, J., & Burget, L. (2025) · Computer Speech & Language · doi:10.1016/j.csl.2025.101841 · Cited by 11 · see citing works
Kanda, N., Xiao, X., Wu, J., Zhou, T., Gaur, Y., Wang, X., Meng, Z., Chen, Z., & Yoshioka, T. (2021) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru51503.2021.9687974 · Cited by 12 · see citing works
Cheng, G., Chen, Y., Yang, R., Li, Q., Yang, Z., Ye, L., Zhang, P., Zhang, Q., Xie, L., Qian, Y., et al. (2022) · International Symposium on Chinese Spoken Language Processing (ISCSLP) · doi:10.1109/iscslp57327.2022.10038258 · Cited by 11 · see citing works
Speaker Diarization: A Review of Objectives and Methods
O’Shaughnessy, D. D. (2025) · Applied Sciences · doi:10.3390/app15042002 · Cited by 10 · see citing works
Enhancing Speaker Diarization with Large Language Models: A Contextual Beam Search Approach
Park, T., Dhawan, K., Koluguri, N. R., & Balam, J. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10446204 · Cited by 10 · see citing works
Auxiliary Networks for Joint Speaker Adaptation and Speaker Change Detection
Sarı, L., Hasegawa‐Johnson, M., & Thomas, S. (2020) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2020.3040626 · Cited by 10 · see citing works
Hypothesis Stitcher for End-to-End Speaker-Attributed ASR on Long-Form Multi-Talker Recordings
Chang, X., Kanda, N., Gaur, Y., Wang, X., Meng, Z., & Yoshioka, T. (2021) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp39728.2021.9414432 · Cited by 10 · see citing works
Augmenting Transformer-Transducer Based Speaker Change Detection with Token-Level Training Loss
Zhao, G., Wang, Q., Han, L., Huang, Y., & Moreno, I. L. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10094955 · Cited by 9 · see citing works
Integrating Dialog History into End-to-End Spoken Language Understanding Systems
Ganhotra, J., Thomas, S., Kuo, H.-K. J., Joshi, S., Saon, G., Tüske, Z., & Kingsbury, B. E. D. (2021) · Interspeech (USB) · doi:10.21437/interspeech.2021-1460 · Cited by 7 · see citing works
Speaker Diarization based on Deep Learning Techniques: A Review
Al-Hadithy, T. M., Frikha, M., & Maseer, Z. K. (2022) · International Symposium on Multidisciplinary Studies and Innovative Technologies (ISMSIT) · doi:10.1109/ismsit56059.2022.9932710 · Cited by 7 · see citing works
Unsupervised Speaker Diarization that is Agnostic to Language, Overlap-Aware, and Tuning Free
Tanveer, M. I., Casabuena, D., Jussi Karlgren, C., & Jones, R. (2022) · Interspeech (USB) · doi:10.21437/interspeech.2022-10605 · Cited by 7 · see citing works
Li, Y., Parsan, A., Wang, B., Dong, P., Yao, S., & Qin, R. (2022) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2022.105597 · Cited by 7 · see citing works
Language modelling for speaker diarization in telephonic interviews
India, M., Hernando, J., & Fonollosa, J. A. R. (2022) · Computer Speech & Language · doi:10.1016/j.csl.2022.101441 · Cited by 7 · see citing works
VoxMM: Rich Transcription of Conversations in the Wild
Kwak, D., Jung, J., Nam, K.-H., Jang, Y., Jung, J.-W., Watanabe, S., & Chung, J. S. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10446300 · Cited by 6 · see citing works
Masumura, R., Okamura, D., Makishima, N., Ihori, M., Takashima, A., Tanaka, T., & Orihashi, S. (2021) · Interspeech (USB) · doi:10.21437/interspeech.2021-2043 · Cited by 6 · see citing works
Synthetic Conversations Improve Multi-Talker ASR
Nguyen, T. B., & Waibel, A. H. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10446589 · Cited by 6 · see citing works
Joint speaker diarization and speech recognition based on region proposal networks
Huang, Z., Delcroix, M., Garcia, L. P., Watanabe, S., Raj, D., & Khudanpur, S. P. (2021) · Computer Speech & Language · doi:10.1016/j.csl.2021.101316 · Cited by 6 · see citing works
Keshishian, M., Mischler, G., Thomas, S., Kingsbury, B. E. D., Bickel, S., Mehta, A. D., & Mesgarani, N. (2026) · Nature Machine Intelligence · doi:10.1038/s42256-026-01185-0 · Cited by 5 · see citing works
Masuyama, Y., Chang, X., Zhang, W., Cornell, S., Wang, Z.-Q., Ono, N., Qian, Y., & Watanabe, S. (2025) · Computer Speech & Language · doi:10.1016/j.csl.2025.101813 · Cited by 5 · see citing works
Extending RNN-T-based speech recognition systems with emotion and language classification
Kons, Z., Aronowitz, H., Morais, E. D. S., Damasceno, M., Kuo, H.-K. J., Thomas, S., & Saon, G. (2022) · Interspeech (USB) · doi:10.21437/interspeech.2022-10480 · Cited by 5 · see citing works
End-to-end speech recognition modeling from de-identified data
Flechl, M., Yin, S.-C., Park, J., & Skala, P. (2022) · Interspeech (USB) · doi:10.21437/interspeech.2022-10484 · Cited by 5 · see citing works
End-to-end ASR to jointly predict transcriptions and linguistic annotations
Omachi, M., Fujita, Y., Watanabe, S., & Wiesner, M. (2021) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2021.naacl-main.149 · Cited by 5 · see citing works
Understanding Medical Conversations: Rich Transcription, Confidence Scores & Information ExtractionPreprint
Soltau, H., Wang, M., Izhak Shafran, J., & Shafey, L. E. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.02219 · Cited by 5 · see citing works
Self-citationPicknet: Real-Time Channel Selection for Ad Hoc Microphone Arrays
Yoshioka, T., Wang, X., & Wang, D. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9746879 · Cited by 4 · see citing works
Speaker Change Detection For Transformer Transducer ASR
Wu, J., Chen, Z., Hu, M., Xiao, X., & Li, J. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10096361 · Cited by 4 · see citing works
Unsupervised Speaker Diarization in Distributed IoT Networks Using Federated Learning
Bhuyan, A. K., Dutta, H., & Biswas, S. K. (2024) · IEEE Transactions on Emerging Topics in Computational Intelligence · doi:10.1109/tetci.2024.3482855 · Cited by 5 · see citing works
Improving Morpheme Segmentation Using BERT Embeddings
Sorokin, A. A. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-16500-9_13 · Cited by 4 · see citing works
Chollet, G., Sansen, H., Tevissen, Y., Boudy, J., Hariz, M., Lohr, C., & Yassa, F. (2024) · AHFE international · doi:10.54941/ahfe1004577 · Cited by 4 · see citing works
Liang, D., Shao, N., & Li, X. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10446568 · Cited by 3 · see citing works
Multimodal Classroom Diarization with GPT Re-scoring: Teacher or Student?
Perez, M., Coker, B., Kocabagli, K. B., Vitale, J., & Camp, A. V. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-98462-4_29 · Cited by 3 · see citing works
Considerations on creating conversational agents for multiple environments and users
Cebrián, J., Martínez, R., Rodríguez, N., & D’Haro, L. F. (2021) · AI Magazine · doi:10.1609/aimag.v42i2.7484 · Cited by 3 · see citing works
Speaker conditioned acoustic modeling for multi-speaker conversational ASR
Chetupalli, S. R., & Ganapathy, S. (2022) · Interspeech (USB) · doi:10.21437/interspeech.2022-11267 · Cited by 3 · see citing works
Speaker Embeddings Incorporating Acoustic Conditions for Diarization
Higuchi, Y., Suzuki, M., & Kurata, G. (2020) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp40776.2020.9054273 · Cited by 3 · see citing works
Investigation of End-To-End Speaker-Attributed ASR for Continuous Multi-Talker RecordingsPreprint
Kanda, N., Chang, X., Gaur, Y., Wang, X., Meng, Z., Chen, Z., & Yoshioka, T. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2008.04546 · Cited by 3 · see citing works
Streaming Multi-speaker ASR with RNN-TPreprint
Sklyar, I., Piunova, A., & Liu, Y. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2011.11671 · Cited by 3 · see citing works
Multi-Speaker Data Augmentation for Improved end-to-end Automatic Speech Recognition
Thomas, S., Kuo, H.-K. J., Saon, G., & Kingsbury, B. E. D. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10095622 · Cited by 2 · see citing works
Multitask Speech Recognition and Speaker Change Detection for Unknown Number of Speakers
Kumar, S., Madikeri, S. R., Nigmatulina, I., Villatoro-Tello, E., Motlíček, P., Pandia, K., Dubagunta, S. P., & Ganapathiraju, A. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10446130 · Cited by 2 · see citing works
Resource-Efficient Adaptation of Speech Foundation Models for Multi-Speaker ASR
Wang, W., Dhawan, K., Park, T., Puvvada, K. C., Medennikov, I., Majumdar, S., Huang, H., Balam, J., & Ginsburg, B. (2024) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt61566.2024.10832215 · Cited by 2 · see citing works
Speaker Diarization For Vietnamese Conversations Using Deep Neural Network Embeddings
Nguyen, T. L., Thang, T. B., Hai, V., Tran, T. A. X., & Le, N. M. (2022) · International Conference on Communications and Electronics (ICCE) · doi:10.1109/icce55644.2022.9852042 · Cited by 2 · see citing works
Minimum Bayes Risk Training for End-to-End Speaker-Attributed ASRPreprint
Kanda, N., Meng, Z., Lu, L., Gaur, Y., Wang, X., Chen, Z., & Yoshioka, T. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2011.02921 · Cited by 2 · see citing works
Speech Summarization using Restricted Self-AttentionPreprint
Sharma, R., Palaskar, S., Black, A. W., & Metze, F. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.06263 · Cited by 2 · see citing works
SCDiar: a streaming diarization system based on speaker change detection and speech recognition
Zheng, N., Wan, X., Liu, K., & Zhou, H. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10888692 · Cited by 1 · see citing works
Leveraging broadcast media subtitle transcripts for automatic speech recognition and subtitling
Poncelet, J., & hamme, H. V. (2026) · EURASIP Journal on Audio Speech and Music Processing · doi:10.1186/s13636-026-00450-9 · Cited by 1 · see citing works
Semantic Tree Driven Thyroid Ultrasound Report Generation by Voice Input
Liu, L., Wang, M., Dong, Y., Zhao, W., Yang, J. G., & Su, J. (2021) · Transactions on computational science and computational intelligence · doi:10.1007/978-3-030-71051-4_32 · Cited by 1 · see citing works
Are You Dictating to Me? Detecting Embedded Dictations in Doctor-Patient Conversations
Schaaf, T., Zhang, L., Bayestehtashk, A., Fuhs, M. C., Durrani, S., Burger, S., Woszczyna, M., & Polzin, T. (2021) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru51503.2021.9688118 · Cited by 1 · see citing works
Speaker Diarization with Deep Learning Techniques
Sarmah, K. (2020) · Türk bilgisayar ve matematik eğitimi dergisi · doi:10.61841/turcomat.v11i3.14309 · Cited by 1 · see citing works
Analysis of Time Domain Features of Dysarthria Speech
Singh, A., Kittur, A., Sonawane, K., Singh, A., & Upadhya, S. S. (2020) · International Conference on Computing Methodologies and Communication (ICCMC) · doi:10.1109/iccmc48092.2020.iccmc-00025 · Cited by 1 · see citing works
Language Aided Speaker Diarization Using Speaker Role InformationPreprint
Flemotomos, N., Georgiou, P., & Narayanan, S. S. (2019) · arXiv (Cornell University) · Cited by 1 · see citing works
The HUAWEI Speaker Diarisation System for the VoxCeleb Speaker Diarisation ChallengePreprint
Wang, R., Tong, R., Yeung, Y. T., & Chen, X. D. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.11657 · Cited by 1 · see citing works
Hypothesis Stitcher for End-to-End Speaker-attributed ASR on Long-form Multi-talker RecordingsPreprint
Chang, X., Kanda, N., Gaur, Y., Wang, X., Meng, Z., & Yoshioka, T. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2101.01853 · Cited by 1 · see citing works
End-to-End Speaker-Attributed ASR with TransformerPreprint
Kanda, N., Ye, G., Gaur, Y., Wang, X., Meng, Z., Chen, Z., & Yoshioka, T. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.02128 · Cited by 1 · see citing works
Bhuyan, A. K., Dutta, H., & Biswas, S. K. (2024) · doi:10.36227/techrxiv.170492258.87092583/v1 · Cited by 1 · see citing works
Dicow: Diarization-Conditioned Whisper for Target Speaker Automatic Speech RecognitionPreprint
Polok, A., Klement, D., Kocour, M., Han, J., Landini, F., Yusuf, B., Wiesner, M., Khudanpur, S. P., Černocký, J., & Burget, L. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5123534 · Cited by 1 · see citing works
Informed Speech Self-supervised Representation Learning
Zaiem, S. (2024) · theses.fr (ABES) · doi:10.70675/dc53ad72z177fz4bb4z91fbzd00576ba3f2e · Cited by 1 · see citing works
Streaming Multi-talker Speech Recognition with Joint Speaker IdentificationPreprint
Lu, L., Kanda, N., Li, J., & Gong, Y. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.02109
Speaker diarization assisted ASR for multi-speaker conversationsPreprint
Chetupalli, S. R., & Ganapathy, S. (2021) · arXiv (Cornell University)
Masumura, R., Okamura, D., Makishima, N., Ihori, M., Takashima, A., Tanaka, T., & Orihashi, S. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2107.01549
Improving End-to-end Models for Set Prediction in Spoken Language Understanding
Kuo, H.-K. J., Tüske, Z., Thomas, S., Kingsbury, B. E. D., & Saon, G. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9746086
Towards lifelong human assisted speaker diarization
Shamsi, M., Larcher, A., Barrault, L., Meignier, S., Prokopalo, Y., Tahon, M., Mehrish, A., Petitrenaud, S., Galibert, O., Gaist, S., et al. (2022) · Computer Speech & Language · doi:10.1016/j.csl.2022.101437
Speaker Diarization for Unlimited Number of Speakers Using Dynamic Linear
Wang, S., & Zhang, C. (2024) · International Symposium on Chinese Spoken Language Processing (ISCSLP) · doi:10.1109/iscslp63861.2024.10800134
SEAL: Speaker Error Correction using Acoustic-conditioned Large Language Models
Kumar, A., Paturi, R., Afshan, A., & Srinivasan, S. (2025) · IEEE International Conference on Acoustics, Speech, and Signal Processing · doi:10.1109/icassp49660.2025.10890148 · Cited by 2 · see citing works
On the Limitations of Speaker Diarization
Amorim, J., Pimentel, J., & Rudzicz, F. (2026) · Expert Systems · doi:10.1111/exsy.70221
TokenVerse++: Towards Flexible Multitask Learning with Dynamic Task Activation
Kumar, S., Madikeri, S. R., Villatoro-Tello, E., Burdisso, S., Rangappa, P., Carofilis, A., Motlíček, P., Pandia, K., Venkatesan, S. M., Hacıoğlu, K., et al. (2025) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru65441.2025.11434791
Cheng, T., Xi, C., Pan, J., Wang, R., Chen, H., Han, J., Burget, L., Gao, J., & Du, J. (2026) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp55912.2026.11461587
Whisper with Benefits: A Unified Approach to Speech and Speaker Attribute Recognition
Duong, R. N. Q. K., Torre, I. G., Nguyen, H., Nancy Bertin, B., Lebarbenchon, R., Camberlein, E., Souviraà-Labastie, N., & Han, K. J. (2026) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp55912.2026.11462763
Speaker Attributed Automatic Speech Recognition Using Speech Aware Llms
Aronowitz, H., Kons, Z., Dekel, A., Saon, G., & Hoory, R. (2026) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp55912.2026.11464609
- Authors
- Shafey, L. E., Soltau, H., & Shafran, I.
- Affiliations
- Google (United States) (US)
- Venue
- Interspeech 2019
- Publisher
- ISCA
- Published
- 2019-09-15
- Pages
- 396-400
- ISSN
- 2308-457X (ISSN-L)
- OpenAlex
- W2957946919
- Cited by
- 104
- References
- 30
- Citations by year
- 2026: 9 · 2025: 8 · 2024: 14 · 2023: 5 · 2022: 21 · 2021: 29 · 2020: 16 · 2019: 2
- FWCI
- 6.50
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- Interspeech (USB) (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
- Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
- Physical Sciences › Computer Science › Signal Processing › Speech and Audio Processing
- Keywords
- speaker diarization, speech recognition, RNN-T, medical conversation, physician-patient interaction
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
