WhisperX: Time-Accurate Speech Transcription of Long-Form Audio
Open accessAbstract
Large-scale, weakly-supervised speech recognition models, such as Whisper, have demonstrated impressive results on speech recognition across domains and languages. However, their application to long audio transcription via buffered or sliding window approaches is prone to drifting, hallucination & repetition; and prohibits batched transcription due to their sequential nature. Further, timestamps corresponding each utterance are prone to inaccuracies and word-level timestamps are not available out-of-the-box. To overcome these challenges, we present WhisperX, a time-accurate speech recognition system with word-level timestamps utilising voice activity detection and forced phoneme alignment. In doing so, we demonstrate state-of-the-art performance on long-form transcription and word segmentation benchmarks. Additionally, we show that pre-segmenting audio with our proposed VAD Cut & Merge strategy improves transcription quality and enables a twelve-fold transcription speedup via batched inference.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
E2E models tend to be inaccurate in predicting word-level timestamps.
“They also tend to be inaccurate in predicting word-level timestamps [Bain et al. 2023].”
From Measuring the Accuracy of Automatic Speech Recognition Solutions (2023) · cites this paper as Bain et al. 2023
Section 3.4.1 reports that extracting word-level timestamps directly from Whisper significantly underperforms in word-segmentation precision and recall, and Section 2.7 says that methods using Whisper’s cross-attention scores are prone to timestamp inaccuracies. These findings support the point for Whisper, but the cited paper does not establish that E2E models generally tend to be inaccurate. (LLM, conf=0.84)
Experimental — accuracy not yet measuredA related study reported a speed increase of up to twelve times without sacrificing transcription accuracy.
“A related study reported a speed increase of up to twelve times without sacriicing transcription accuracy [Bain et al. 2023].”
From Measuring the Accuracy of Automatic Speech Recognition Solutions (2023) · cites this paper as Bain et al. 2023
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · see citing works
“Librispeech: an ASR corpus based on public domain audio books,”
Panayotov, V., Chen, G., Povey, D., & Khudanpur, S. (2015) · 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2015.7178964 · Cited by 6,072 · see citing works
HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units
Hsu, W.-N., Bolte, B., Tsai, Y.-H. H., Lakhotia, K., Salakhutdinov, R., & Mohamed, A. (2021) · IEEE/ACM Transactions on Audio, Speech, and Language Processing · doi:10.1109/taslp.2021.3122291 · Cited by 2,983 · see citing works
wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations
Baevski, A., Zhou, H., Mohamed, A., & Auli, M. (2020) · arXiv · doi:10.48550/arxiv.2006.11477 · Cited by 2,369 · see citing works
Robust Speech Recognition via Large-Scale Weak Supervision
Radford, A., Kim, J. W., Xu, T., Brockman, G., McLeavey, C., & Sutskever, I. (2022) · arXiv · doi:10.48550/arxiv.2212.04356 · Cited by 1,953 · see citing works
WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing
Chen, S., Wang, C., Chen, Z., Wu, Y., Liu, S., Chen, Z., Li, J., Kanda, N., Yoshioka, T., Xiao, X., et al. (2022) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2022.3188113 · Cited by 1,926 · see citing works
Montreal Forced Aligner: Trainable Text-Speech Alignment Using Kaldi
McAuliffe, M., Socolof, M., Mihuc, S., Wagner, M., & Sonderegger, M. (2017) · Interspeech (USB) · doi:10.21437/interspeech.2017-1386 · Cited by 1,169 · see citing works
HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clips
Miech, A., Zhukov, D., Alayrac, J.-B., Tapaswi, M., Laptev, I., & Sivic, J. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00272 · Cited by 986 · see citing works
The AMI Meeting Corpus: A Pre-announcement
Carletta, J., Ashby, S., Bourban, S., Flynn, M., Guillemot, M., Hain, T., Kadlec, J., Vasilis Karaiskos, B., Kraaij, W., Kronenthal, M., et al. (2006) · Lecture notes in computer science · doi:10.1007/11677482_3 · Cited by 939 · see citing works
Unsupervised Cross-Lingual Representation Learning for Speech Recognition
Conneau, A., Baevski, A., Collobert, R., Mohamed, A., & Auli, M. (2021) · Interspeech 2021 · doi:10.21437/interspeech.2021-329 · Cited by 637 · see citing works
Wang, C., Rivière, M., Lee, A., Wu, A., Talnikar, C., Haziza, D., Williamson, M., Pino, J., & Emmanuel Dupoux, A. (2021) · Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2021.acl-long.80 · Cited by 339 · see citing works
Wang, Q., Downey, C., Wan, L., Mansfield, P. A., & Moreno, I. L. (2018) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2018.8462628 · Cited by 324 · see citing works
Ssast: Self-supervised audio spectrogram transformer
Gong, Y., Lai, C.-I., Chung, Y.-A., & Glass, J. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i10.21315 · Cited by 294 · see citing works
TED-LIUM 3: Twice as Much Data and Corpus Repartition for Experiments on Speaker Adaptation
Hernandez, F., Nguyen, V., Ghannay, S., Tomashenko, N., & Estève, Y. (2018) · Lecture notes in computer science · doi:10.1007/978-3-319-99579-3_21 · Cited by 280 · see citing works
Fully Supervised Speaker Diarization
Zhang, A., Wang, Q., Zhu, Z., Paisley, J. W., & Wang, C. (2019) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2019.8683892 · Cited by 235 · see citing works
Prosodylab-aligner: A tool for forced alignment of laboratory speech
Gorman, K., Howell, J., & Wagner, M. (2011) · Canadian acoustics · Cited by 223 · see citing works
Automatic segmentation and labeling of speech based on Hidden Markov Models
Brugnara, F., Falavigna, D., & Omologo, M. (1993) · Speech Communication · doi:10.1016/0167-6393(93)90083-w · Cited by 167 · see citing works
Leveraging Weakly Supervised Data to Improve End-to-end Speech-to-text Translation
Ye, J., Johnson, M., Macherey, W., Weiss, R. J., Cao, Y. J., Chiu, C., Ari, N., Laurenzo, S., & Wu, Y. (2019) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2019.8683343 · Cited by 153 · see citing works
Unsupervised Sound Separation Using Mixture Invariant TrainingPreprint
Wisdom, S., Tzinis, E., Erdoğan, H., Weiss, R. J., Wilson, K., & Hershey, J. R. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.12701 · Cited by 127 · see citing works
Sub-word Level Lip Reading With Visual Attention
Prajwal, K. R., Afouras, T., & Zisserman, A. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00510 · Cited by 115 · see citing works
Self-citationA Comparison of End-to-End Models for Long-Form Speech Recognition
Chiu, C., Han, W., Zhang, Y., Pang, R., Kishchenko, S., Nguyen, P., Narayanan, A., Liao, H., Zhang, S., Kannan, A., et al. (2019) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru46091.2019.9003854 · Cited by 83 · see citing works
CTC-Segmentation of Large Corpora for German End-to-End Speech Recognition
Kürzinger, L., Winkelbauer, D., Li, L., Watzel, T., & Rigoll, G. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-60276-5_27 · Cited by 80 · see citing works
Automatic segmentation combining an HMM-based approach and spectral boundary correction
Kim, Y.-J., & Conkie, A. D. (2002) · International Conference on Spoken Language Processing (ICSLP) · doi:10.21437/icslp.2002-88 · Cited by 61 · see citing works
Automatic phonetic segmentation using boundary models
Yuan, J., Ryant, N., Liberman, M., Stolcke, A., Mitra, V., & Wang, W. (2013) · Interspeech (USB) · doi:10.21437/interspeech.2013-540 · Cited by 59 · see citing works
Highly accurate phonetic segmentation using boundary correction models and system fusion
Stolcke, A., Ryant, N., Mitra, V., Yuan, J., Wang, W., & Liberman, M. (2014) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2014.6854665 · Cited by 42 · see citing works
Neufa: Neural Network Based End-to-End Forced Alignment with Bidirectional Attention Mechanism
Li, J., Meng, Y., Wu, Z., Meng, H. M. L., Tian, Q., Wang, Y., & Wang, Y. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9747085 · Cited by 21 · see citing works
Augmentation adversarial training for self-supervised speaker recognitionPreprint
Huh, J., Heo, H. S., Kang, J., Watanabe, S., & Chung, J. S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.12085 · Cited by 19 · see citing works
Self-citationAugmentation Adversarial Training for Self-Supervised Speaker Representation Learning
Kang, J., Huh, J., Heo, H. S., & Chung, J. S. (2022) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2022.3200915 · Cited by 16 · see citing works
Self-citationVisual Keyword Spotting with AttentionPreprint
Prajwal, K. R., Momeni, L., Afouras, T., & Zisserman, A. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.15957 · Cited by 8 · see citing works
Self-citation
Flagged first, then by citations
Research works printed but not linked: 4
- H. Chen, H. Zhang, L. Wang, K. A. Lee, M. Liu, and J. Dang, "Self-supervised audio-visual speaker representation with co- meta learning," in ICASSP, 2023.
- G. Gelly and J.-L. Gauvain, "Optimization of rnn-based speech activity detection," IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 26, no. 3, pp. 646-656, 2018.
- H. Bredin, R. Yin, J. M. Coria, G. Gelly, P. Korshunov, M. Lavechin, D. Fustes, H. Titeux, W. Bouaziz, and M.-P. Gill, "pyannote.audio: neural building blocks for speaker diarization," in ICASSP, 2020.
- Y.-Y. Yang, M. Hira, Z. Ni, A. Astafurov, C. Chen, C. Puhrsch, D. Pollack, D. Genzel, D. Greenberg, E. Z. Yang et al., "Torchau- dio: Building blocks for audio and speech processing," in ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2022, pp. 6982-6986.
Cited by
Sorted by citations
Automatic Speech Recognition: A survey of deep learning techniques and approaches
Ahlawat, H., Aggarwal, N., & Gupta, D. (2025) · International Journal of Cognitive Computing in Engineering · doi:10.1016/j.ijcce.2024.12.007 · Cited by 70 · see citing works
Emilia: An Extensive, Multilingual, and Diverse Speech Dataset For Large-Scale Speech Generation
He, H., Shang, Z., Wang, C., Li, X., Gu, Y., Hua, H., Liu, L., Yang, C., Li, J., Shi, P., et al. (2024) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt61566.2024.10832365 · Cited by 51 · see citing works
Measuring the Accuracy of Automatic Speech Recognition Solutions
Kuhn, K., Kersken, V., Reuter, B., Egger, N., & Zimmermann, G. (2023) · ACM Transactions on Accessible Computing · doi:10.1145/3636513 · Cited by 44 · see citing works
Take the aTrain. Introducing an interface for the Accessible Transcription of Interviews
Haberl, A., Fleiß, J., Kowald, D., & Thalmann, S. (2024) · Journal of Behavioral and Experimental Finance · doi:10.1016/j.jbef.2024.100891 · Cited by 44 · see citing works
AutoAD: Movie Description in Context
Han, T., Bain, M., Nagrani, A., Varol, G., Xie, W., & Zisserman, A. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.01815 · Cited by 43 · see citing works
Self-citationScaling Up Video Summarization Pretraining with Large Language Models
Argaw, D. M., Yoon, S., Heilbron, F. C., Deilamsalehy, H., Bui, T., Wang, Z., Dernoncourt, F., & Chung, J. S. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00796 · Cited by 35 · see citing works
Natural language processing models reveal neural dynamics of human conversation
Cai, J., Hadjinicolaou, A. E., Paulk, A. C., Soper, D. J., Xia, T., Wang, A. F., Rolston, J. D., Richardson, R. M., Williams, Z. M., & Cash, S. S. (2025) · Nature Communications · doi:10.1038/s41467-025-58620-w · Cited by 43 · see citing works
AutoAD II: The Sequel – Who, When, and What in Movie Audio Description
Han, T., Bain, M., Nagrani, A., Varol, G., Xie, W., & Zisserman, A. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01255 · Cited by 33 · see citing works
Self-citationLyu, K.-M., Lyu, R.-Y., & Chang, H.-T. (2024) · PeerJ Computer Science · doi:10.7717/peerj-cs.1973 · Cited by 26 · see citing works
LLMREI: Automating Requirements Elicitation Interviews with LLMs
Korn, A., Gorsch, S., & Vogelsang, A. (2025) · IEEE International Requirements Engineering Conference (RE) · doi:10.1109/re63999.2025.00013 · Cited by 25 · see citing works
Retrieval-Augmented Egocentric Video Captioning
Xu, J., Huang, Y., Hou, J., Chen, G., Zhang, Y., Feng, R., & Xie, W. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01284 · Cited by 24 · see citing works
Automatic recognition of second language speech-in-noise
Kim, S.-E., Chernyak, B. R., Seleznova, O., Keshet, J., Goldrick, M., & Bradlow, A. R. (2024) · JASA Express Letters · doi:10.1121/10.0024877 · Cited by 24 · see citing works
MM-Narrator: Narrating Long-form Videos with Multimodal In-Context Learning
Zhang, C., Lin, K., Yang, Z., Wang, J., Li, L., Lin, C.-C., Liu, Z., & Wang, L. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.01295 · Cited by 24 · see citing works
AutoAD III: The Prequel - Back to the Pixels
Han, T., Bain, M., Nagrani, A., Varol, G., Xie, W., & Zisserman, A. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01720 · Cited by 24 · see citing works
Self-citationFavaro, A., Butala, A., Thebaud, T., Villalba, J., Dehak, N., & Moro-Velázquez, L. (2024) · npj Parkinson s Disease · doi:10.1038/s41531-024-00817-9 · Cited by 22 · see citing works
SpoofCeleb: Speech Deepfake Detection and SASV in the Wild
Jung, J.-W., Wu, Y., Wang, X., Kim, J.-H., Maiti, S., Matsunaga, Y., Shim, H.-J., Tian, J., Evans, N., Chung, J. S., et al. (2025) · IEEE Open Journal of Signal Processing · doi:10.1109/ojsp.2025.3529377 · Cited by 21 · see citing works
Advancing Naturalistic Affective Science with Deep Learning
Lin, C., Bulls, L. S., Tepfer, L. J., Vyas, A. D., & Thornton, M. A. (2023) · Affective Science · doi:10.1007/s42761-023-00215-z · Cited by 21 · see citing works
Ferraro, A., Galli, A., Gatta, V. L., & Postiglione, M. (2023) · Frontiers in Big Data · doi:10.3389/fdata.2023.1210559 · Cited by 21 · see citing works
Towards decoding individual words from non-invasive brain recordings
d’Ascoli, S., Bel, C., Rapin, J., Banville, H. J., Benchetrit, Y., Pallier, C., & King, J.-R. (2025) · Nature Communications · doi:10.1038/s41467-025-65499-0 · Cited by 20 · see citing works
Eberhardt, S. T., Vehlen, A., Schaffrath, J., Schwartz, B., Baur, T., Schiller, D., Hallmen, T., André, E., & Lutz, W. (2025) · Scientific Reports · doi:10.1038/s41598-025-14923-y · Cited by 18 · see citing works
Large Language Models for Psychological Assessment: A Comprehensive Overview
Brickman, J., Gupta, M., & Oltmanns, J. R. (2025) · Advances in Methods and Practices in Psychological Science · doi:10.1177/25152459251343582 · Cited by 21 · see citing works
AI-driven mock interview assessment: leveraging generative language models for automated evaluation
Uppalapati, P. J., Dabbiru, M., & Kasukurthi, V. R. (2025) · International Journal of Machine Learning and Cybernetics · doi:10.1007/s13042-025-02529-9 · Cited by 17 · see citing works
Kalda, J., Pagès, C., Marxer, R., Alumäe, T., & Bredin, H. (2024) · The Speaker and Language Recognition Workshop (Odyssey) · doi:10.21437/odyssey.2024-17 · Cited by 17 · see citing works
Relax: Composable Abstractions for End-to-End Dynamic Machine Learning
Lai, R., Shao, J., Feng, S. M., Lyubomirsky, S., Hou, B., Lin, W., Ye, Z., Jin, H., Jin, Y., Liu, J., et al. (2025) · International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS) · doi:10.1145/3676641.3716249 · Cited by 17 · see citing works
What automatic speech recognition can and cannot do for conversational speech transcription
Russell, S. O., Gessinger, I., Krason, A., Vigliocco, G., & Harte, N. (2024) · Research Methods in Applied Linguistics · doi:10.1016/j.rmal.2024.100163 · Cited by 16 · see citing works
Funk, M., Okada, S., & André, E. (2024) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/3678957.3685757 · Cited by 16 · see citing works
Bauer, J. F., Gerczuk, M., Schindler‐Gmelch, L., Amiriparian, S., Ebert, D. D., Krajewski, J., Schuller, B. W., & Berking, M. (2024) · Depression and Anxiety · doi:10.1155/2024/9667377 · Cited by 16 · see citing works
Learning visual language models for video understanding
Yang, A. (2023) · theses.fr (ABES) · doi:10.70675/802872a7z8092z4556za1fdz2c7af45f0e72 · Cited by 15 · see citing works
Hirschi, K., Kang, O., Mu, Y., Hansen, J. H. L., & Beloin, K. (2025) · Language Learning · doi:10.1111/lang.12719 · Cited by 14 · see citing works
Dong, Z., Zhang, Z., Xu, W., Han, J., Ou, J., & Schuller, B. W. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10446795 · Cited by 14 · see citing works
AutoAD-Zero: A Training-Free Framework for Zero-Shot Audio Description
Xie, J., Han, T., Bain, M., Nagrani, A., Varol, G., Xie, W., & Zisserman, A. (2024) · Lecture notes in computer science · doi:10.1007/978-981-96-0908-6_5 · Cited by 16 · see citing works
Self-citationGómez-Zaragozá, L., Wills, S., Tejedor-Garcı́a, C., Marín‐Morales, J., Alcañíz, M., & Strik, H. (2023) · Interspeech (USB) · doi:10.21437/interspeech.2023-1734 · Cited by 13 · see citing works
Zero-Shot Domain-Sensitive Speech Recognition with Prompt-Conditioning Fine-Tuning
Liao, F.-T., Chan, Y.-C., Chen, Y., Hsu, C.-J., & Shiu, D.-S. (2023) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru57964.2023.10389617 · Cited by 13 · see citing works
PITCH: AI-assisted Tagging of Deepfake Audio Calls using Challenge-Response
Mittal, G., Jakobsson, A., Marshall, K. A., Hegde, C., & Memon, N. D. (2025) · ACM Asia Conference on Computer and Communications Security (ASIA CCS) · doi:10.1145/3708821.3733861 · Cited by 12 · see citing works
Less Peaky and More Accurate CTC Forced Alignment by Label Priors
Huang, R., Zhang, X., Ni, Z., Sun, L., Hira, M., Hwang, J., Manohar, V., Pratap, V., Wiesner, M., Watanabe, S., et al. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10446111 · Cited by 11 · see citing works
LLM-based speaker diarization correction: A generalizable approach
Efstathiadis, G., Yadav, V., & Abbas, A. (2025) · Speech Communication · doi:10.1016/j.specom.2025.103224 · Cited by 12 · see citing works
Listen Closely: Measuring Vocal Tone in Corporate Disclosures
Ewertz, J., Knickrehm, C., Nienhaus, M., & Reichmann, D. (2025) · Journal of Accounting Research · doi:10.1111/1475-679x.70015 · Cited by 15 · see citing works
Fonteles, J. H., Cohn, C., Ayalon, E., Zhou, M., Ashwin, T. S., Davalos, E., Li, Z., Rayala, S., Mereddy, D., Coursey, A., et al. (2026) · Learning and Instruction · doi:10.1016/j.learninstruc.2025.102274 · Cited by 10 · see citing works
Non-Invasive Suicide Risk Prediction Through Speech Analysis
Amiriparian, S., Gerczuk, M., Lutz, J., Strube, W., Papazova, I., Hasan, A., Kathan, A., & Schuller, B. W. (2024) · E-Health and Bioengineering Conference (EHB) · doi:10.1109/ehb64556.2024.10805581 · Cited by 10 · see citing works
Assessing reading fluency in elementary grades: A machine learning approach
Silva, G., Rodrigues, R. L., Amorim, A. N., Jeon, L., Albuquerque, E. X. S., Silva, V. C., da Silva, V. F., Pinheiro, A. L. A., Nunes, J. P. J. R., Souza, S., et al. (2025) · Computers and Education Artificial Intelligence · doi:10.1016/j.caeai.2025.100411 · Cited by 10 · see citing works
Schmid, B. (2024) · Urban Studies · doi:10.1177/00420980241305322 · Cited by 9 · see citing works
High-Quality Classroom Dialogue Automatic Analysis System
Jia, L., Sun, H., Jiang, J., & Yang, X. (2025) · Applied Sciences · doi:10.3390/app15031613 · Cited by 9 · see citing works
Towards Multimodal Large-Language Models for Parent-Child Interaction: A Focus on Joint Attention
Shi, W., Le, H. V., & Choo, K. T. W. (2025) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3706599.3720215 · Cited by 9 · see citing works
Live: Learning Video LLM with Streaming Speech Transcription at Scale
Chen, J., Zeng, Z., Lin, Y., Li, W., Ma, Z., & Shou, M. Z. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.02708 · Cited by 9 · see citing works
Unconstrained Dysfluency Modeling for Dysfluent Speech Transcription and Detection
Lian, J., Feng, C., Farooqi, N., Li, S., Kashyap, A., Cho, C. J., Wu, P., Netzorg, R., Li, T., & Anumanchipalli, G. K. (2023) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru57964.2023.10389771 · Cited by 9 · see citing works
FunnyNet-W: Multimodal Learning of Funny Moments in Videos in the Wild
Liu, Z.-S., Courant, R., & Kalogeiton, V. S. (2024) · International Journal of Computer Vision · doi:10.1007/s11263-024-02000-2 · Cited by 9 · see citing works
Emilia: A Large-Scale, Extensive, Multilingual, and Diverse Dataset for Speech Generation
He, H., Shang, Z., Wang, C., Li, X., Gu, Y., Hua, H., Liu, L., Yang, C., Li, J., Shi, P., et al. (2025) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/taslpro.2025.3612835 · Cited by 8 · see citing works
Zada, Z., Nastase, S. A., Speer, S. P. H., Mwilambwe-Tshilobo, L., Tsoi, L., Burns, S. M., Falk, E. B., Hasson, U., & Tamir, D. (2025) · Neuron · doi:10.1016/j.neuron.2025.11.004 · Cited by 8 · see citing works
Transcribing and Aligning Conversational Speech: A Hybrid Pipeline Applied to French Conversations
Yamasaki, H., Louradour, J., Hunter, J., & Prévot, L. (2023) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru57964.2023.10389691 · Cited by 7 · see citing works
Coulange, S., Kato, T., Rossato, S., & Masperi, M. (2024) · Languages · doi:10.3390/languages9030078 · Cited by 7 · see citing works
Look, Listen and Recognise: Character-Aware Audio-Visual Subtitling
Korbar, B., Huh, J., & Zisserman, A. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10446480 · Cited by 7 · see citing works
Self-citationEnhancing Whisper’s Accuracy and Speed for Indian Languages through Prompt-Tuning and Tokenization
Tripathi, K., Gothi, R., & Wasnik, P. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10887595 · Cited by 7 · see citing works
SpeakEasy: Enhancing Text-to-Speech Interactions for Expressive Content Creation
Brade, S., Anderson, S., Kumar, R., Jin, Z., & Truong, A. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3714263 · Cited by 7 · see citing works
Streaming ASR Encoder for Whisper-to-Speech Online Voice Conversion
Avdeeva, A., Gusev, A., Andzhukaev, T., & Ivanov, A. (2023) · IEEE Open Journal of Signal Processing · doi:10.1109/ojsp.2023.3343342 · Cited by 7 · see citing works
Diarist: Streaming Speech Translation with Speaker Diarization
Yang, M., Kanda, N., Wang, X., Chen, J., Wang, P., Xue, J., Li, J., & Yoshioka, T. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10446050 · Cited by 7 · see citing works
MMSummary: Multimodal Summary Generation for Fetal Ultrasound Video
Guo, X., Men, Q., & Noble, J. A. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72083-3_63 · Cited by 7 · see citing works
VideoMix: Aggregating How-To Videos for Task-Oriented Learning
Yang, S., Truong, A., Kim, J., & Li, D. (2025) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3708359.3712144 · Cited by 7 · see citing works
Visualizing speech styles in captions for deaf and hard-of-hearing viewers
Ahn, S., Kim, J., Shin, C., & Hong, J.-H. (2024) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2024.103386 · Cited by 6 · see citing works
Pu, Y., & Zhang, W.-Q. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10888563 · Cited by 6 · see citing works
Xu, L., Chen, K., Mueller, K. D., Liss, J., & Berisha, V. (2024) · Journal of Alzheimer s Disease · doi:10.1177/13872877241300149 · Cited by 8 · see citing works
Student perceptions of the usefulness of core concepts when reasoning in physiology
Cole, E. J., & Doherty, J. H. (2024) · AJP Advances in Physiology Education · doi:10.1152/advan.00198.2024 · Cited by 6 · see citing works
Quantitative analysis of communication dynamics in agile software teams through multimodal analytics
Miranda, D., Noël, R., Godoy, J., Escobedo, C., Cechinel, C., & Muñoz, R. (2025) · Scientific Reports · doi:10.1038/s41598-025-91328-x · Cited by 6 · see citing works
Parola, A., Jessen, E. T., Rybner, A. S., Mortensen, M. D., Larsen, S. N., Simonsen, A., Lin, J. M., Zhou, Y., Wang, H., Koelkebeck, K., et al. (2025) · Schizophrenia Bulletin · doi:10.1093/schbul/sbaf124 · Cited by 6 · see citing works
Scaling Multilingual Visual Speech Recognition
Prajwal, K. R., Hegde, S. B., & Zisserman, A. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10890395 · Cited by 5 · see citing works
Self-citationUncertainty modeling in multimodal speech analysis across the psychosis spectrum
Rohanian, M., Hüppi, R. M., Nooralahzadeh, F., Dannecker, N., Pauli, Y., Surbeck, W., Sommer, I. E., Hinzen, W., Langer, N., Krauthammer, M., et al. (2026) · npj Digital Medicine · doi:10.1038/s41746-025-02309-3 · Cited by 10 · see citing works
Guo, M., & Han, L. (2024) · Interpreting International Journal of Research and Practice in Interpreting · doi:10.1075/intp.00100.guo · Cited by 5 · see citing works
Barchet, A. V., Bruera, A., Wend, J., Rimmele, J. M., Obleser, J., & Hartwigsen, G. (2025) · Journal of Neuroscience · doi:10.1523/jneurosci.0657-25.2025 · Cited by 5 · see citing works
Geometry of neural dynamics along the cortical attractor landscape reflects changes in attention
Song, H., Chen, R.-Q., Botch, T. L., Braver, T. S., Rosenberg, M. D., Zacks, J. M., & Ching, S.-N. (2025) · bioRxiv · doi:10.1038/s41467-026-69041-8 · Cited by 4 · see citing works
Narrative “twist” shifts within-individual neural representations of dissociable story features
Sava‐Segal, C., Grall, C., & Finn, E. S. (2026) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2512071123 · Cited by 6 · see citing works
Christie, I., McGill, M., & Brewster, S. A. (2024) · ACM International Conference on Interactive Media Experiences · doi:10.1145/3639701.3656321 · Cited by 5 · see citing works
Synthetic AI Data Pipeline for Domain-Specific Speech-to-Text Solutions
Karl, A. L., Fernandes, G. S., Pires, L. A., Serpa, Y. R., & Caminha, C. (2024) · Simpósio Brasileiro de Tecnologia da Informação e da Linguagem Humana (STIL) · doi:10.5753/stil.2024.245336 · Cited by 5 · see citing works
Automatic Detection of Articulatory-Based Disfluencies in Primary Progressive Aphasia
Lian, J., Zhou, X., Guo, C., Ye, Z., Ezzes, Z., Vonk, J. M. J., Morin, B. T., Baquirin, D. P., Miller, Z., Tempini, M. L. G., et al. (2025) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2025.3579972 · Cited by 5 · see citing works
Hippocampus serves as a repository for spoken and heard word meanings during conversationsPreprint
Chavez, A. G., Yan, X., Franch, M. C., Mickiewicz, E. A., Baltazar, W., Belanger, J. L., Devara, D., Etta, M., Hamre, T., Ismail, T., et al. (2025) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2025.09.20.677504 · Cited by 5 · see citing works
Hallmen, T., Schiller, D., Vehlen, A., Eberhardt, S. T., Baur, T., Don, D. S. W., Lutz, W., & André, E. (2025) · Frontiers in Digital Health · doi:10.3389/fdgth.2025.1638539 · Cited by 10 · see citing works
van der Klis, A., Adriaans, F., Han, M., & Kager, R. (2023) · Multimodal Technologies and Interaction · doi:10.3390/mti7070068 · Cited by 4 · see citing works
Multi-sentence Grounding for Long-Term Instructional Video
Li, Z., Chen, Q., Han, T., Zhang, Y., Wang, Y., & Xie, W. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72992-8_12 · Cited by 4 · see citing works
Self-citationGirmaji, R., Beri, B., Subramanian, R., & Gandhi, V. (2025) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3708359.3712113 · Cited by 4 · see citing works
WhisBERT: Multimodal Text-Audio Language Modeling on 100M Words
Wolf, L., Kotar, K., Tuckute, G., Hosseini, E. A., Regev, T. I., Wilcox, E., & Warstadt, A. S. (2023) · BabyLM Challenge · doi:10.18653/v1/2023.conll-babylm.21 · Cited by 3 · see citing works
AiModerator: A Co-Pilot for Hyper-Contextualization in Political Debate Video
Andrews, P., Borch, N. T., & Fjeld, M. (2025) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3708359.3712148 · Cited by 3 · see citing works
PlantPal: Leveraging Precision Agriculture Robots to Facilitate Remote Engagement in Urban Gardening
Zeqiri, A., Britten, J., Schramm, C., Jansen, P., Rietzler, M., & Rukzio, E. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713180 · Cited by 4 · see citing works
Identifying storytelling in job interviews using deep learning
Germanier, E., He, M., Rufai, A. M., Garner, P. N., Bangerter, A., Renier, L. A., Mast, M. S., & Orji, K. (2025) · Computers in Human Behavior Reports · doi:10.1016/j.chbr.2025.100688 · Cited by 4 · see citing works
Predicting accentedness and comprehensibility through ASR scores and acoustic features
Dong, W., Catia Cucchiarini, A., van Hout, R. W. N. M., & Strik, H. (2025) · Computer Speech & Language · doi:10.1016/j.csl.2025.101858 · Cited by 3 · see citing works
Chapter-Llama: Efficient Chaptering in Hour-Long Videos with LLMs
Ventura, L., Yang, A., Schmid, C., & Varol, G. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.01765 · Cited by 3 · see citing works
Exploiting Music Source Separation for Automatic Lyrics Transcription with Whisper
Syed, J., Higgs, I. M., Cífka, O., & Sandler, M. B. (2025) · IEEE International Conference on Multimedia and Expo workshops · doi:10.1109/icmew68306.2025.11152264 · Cited by 3 · see citing works
Rodolitz, J., Hamilton, V., Tabassum, M., Lerner, A., & Hofmann, M. K. (2025) · International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) · doi:10.1145/3663547.3746375 · Cited by 3 · see citing works
PIA: Deepfake Detection Using Phoneme-Temporal and Identity-Dynamic Analysis
Datta, S. K., Ranga, T., Sun, C., & Lyu, S. (2025) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw69036.2025.00170 · Cited by 4 · see citing works
Yuan, Z., Stefan Benus, A., & D’Ausilio, A. (2024) · Speech prosody · doi:10.21437/speechprosody.2024-255 · Cited by 3 · see citing works
Nascimento, P., Ferreira, J. C., & Batista, F. (2024) · International Journal of Speech Technology · doi:10.1007/s10772-024-10126-4 · Cited by 3 · see citing works
Ochôa, X., Huang, X., & Charlton, A. (2024) · Journal of Learning Analytics · doi:10.18608/jla.2024.8355 · Cited by 3 · see citing works
Jemetz, M., Dolezal, D., & Motschnig, R. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73474-8_1 · Cited by 5 · see citing works
Lima, R. D. S., Leal, S. E., Cândido, A., & Aluísio, S. M. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-79029-4_3 · Cited by 3 · see citing works
ChunkFormer: Masked Chunking Conformer For Long-Form Speech Transcription
Le, K., Ho, T. V., Tran, D. K., & Chau, D. T. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10888640 · Cited by 3 · see citing works
Automating sedation state assessments using natural language processing
Conway, A., Li, J., Rad, M. G., Warren, B. E., Mafeld, S. C., & Taati, B. (2024) · Journal of Nursing Scholarship · doi:10.1111/jnu.12968 · Cited by 3 · see citing works
Computer-Assisted English Presentation Learning System Based on Word Stress in Pronunciation
Tagami, Y., Kojima, T., & Yamaguchi, S. (2024) · IEEE Annual International Computer Software and Applications Conference (COMPSAC) · doi:10.1109/compsac61105.2024.00271 · Cited by 3 · see citing works
Average Token Delay: A Duration-aware Latency Metric for Simultaneous Translation
Kano, Y., Sudoh, K., & Nakamura, S. (2024) · Journal of Natural Language Processing · doi:10.5715/jnlp.31.1049 · Cited by 3 · see citing works
Peng, W.-H., Chen, S., & Chen, B. (2024) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt61566.2024.10832269 · Cited by 3 · see citing works
Low-Resource Text-to-Speech Synthesis Using Noise-Augmented Training of ForwardTacotron
Lakshminarayana, K. K., Zalkow, F., Dittmar, C., Pia, N., & Habets, E. A. P. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10890686 · Cited by 3 · see citing works
Rangappa, P., Zuluaga-Gómez, J., Madikeri, S. R., Carofilis, A., Prakash, J., Burdisso, S., Kumar, S., Villatoro-Tello, E., Nigmatulina, I., Motlíček, P., et al. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10888138 · Cited by 3 · see citing works
LiveGuard: Voice Liveness Detection via Wavelet Scattering Transform and Mel Spectrogram Scaling
Shan, L., Zhang, X., Hossen, M. I., & Hei, X. (2025) · International Conference on Dependable Systems & Networks/Proceedings · doi:10.1109/dsn64029.2025.00041 · Cited by 3 · see citing works
Yeo, E., Liss, J., Berisha, V., & Mortensen, D. R. (2025) · Perspectives of the ASHA Special Interest Groups · doi:10.1044/2025_persp-25-00030 · Cited by 3 · see citing works
- Authors
- Bain, M., Huh, J., Han, T., & Zisserman, A.
- Affiliations
- University of Oxford (GB)
- Venue
- INTERSPEECH 2023
- Publisher
- ISCA
- Published
- 2023-08-20
- Pages
- 4489-4493
- ISSN
- 2308-457X (ISSN-L)
- OpenAlex
- W4385822293
- Cited by
- 337
- References
- 29
- Citations by year
- 2026: 134 · 2025: 123 · 2024: 65 · 2023: 15
- FWCI
- 59.71
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- cc-by
- Locations
- Interspeech (USB) (publishedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Funders
- Engineering and Physical Sciences Research Council (EP/T028572/1)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
- Physical Sciences › Computer Science › Signal Processing › Speech and Audio Processing
- Physical Sciences › Computer Science › Signal Processing › Music and Audio Processing
- Keywords
- speech transcription, voice activity detection, forced alignment, phone recognition, timestamping
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
