The DOI matches its Crossref record; 29 of the 29 references it cites were found in a registry or an index.
WhisperX: Time-Accurate Speech Transcription of Long-Form Audio
Open accessAbstract
Large-scale, weakly-supervised speech recognition models, such as Whisper, have demonstrated impressive results on speech recognition across domains and languages. However, their application to long audio transcription via buffered or sliding window approaches is prone to drifting, hallucination & repetition; and prohibits batched transcription due to their sequential nature. Further, timestamps corresponding each utterance are prone to inaccuracies and word-level timestamps are not available out-of-the-box. To overcome these challenges, we present WhisperX, a time-accurate speech recognition system with word-level timestamps utilising voice activity detection and forced phoneme alignment. In doing so, we demonstrate state-of-the-art performance on long-form transcription and word segmentation benchmarks. Additionally, we show that pre-segmenting audio with our proposed VAD Cut & Merge strategy improves transcription quality and enables a twelve-fold transcription speedup via batched inference.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“They also tend to be inaccurate in predicting word-level timestamps [Bain et al. 2023].”
From Measuring the Accuracy of Automatic Speech Recognition Solutions (2023) · cites this paper as Bain et al. 2023
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“A related study reported a speed increase of up to twelve times without sacriicing transcription accuracy [Bain et al. 2023].”
From Measuring the Accuracy of Automatic Speech Recognition Solutions (2023) · cites this paper as Bain et al. 2023
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
“Librispeech: an ASR corpus based on public domain audio books,”
Panayotov, V., Chen, G., Povey, D., & Khudanpur, S. (2015) · 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2015.7178964 · Cited by 6,072
HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units
Hsu, W.-N., Bolte, B., Tsai, Y.-H. H., Lakhotia, K., Salakhutdinov, R., & Mohamed, A. (2021) · IEEE/ACM Transactions on Audio, Speech, and Language Processing · doi:10.1109/taslp.2021.3122291 · Cited by 2,983
wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations
Baevski, A., Zhou, H., Mohamed, A., & Auli, M. (2020) · arXiv · doi:10.48550/arxiv.2006.11477 · Cited by 2,369
Robust Speech Recognition via Large-Scale Weak Supervision
Radford, A., Kim, J. W., Xu, T., Brockman, G., McLeavey, C., & Sutskever, I. (2022) · arXiv · doi:10.48550/arxiv.2212.04356 · Cited by 1,953
WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing
Chen, S., Wang, C., Chen, Z., Wu, Y., Liu, S., Chen, Z., Li, J., Kanda, N., Yoshioka, T., Xiao, X., et al. (2022) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2022.3188113 · Cited by 1,926
Montreal Forced Aligner: Trainable Text-Speech Alignment Using Kaldi
McAuliffe, M., Socolof, M., Mihuc, S., Wagner, M., & Sonderegger, M. (2017) · Interspeech (USB) · doi:10.21437/interspeech.2017-1386 · Cited by 1,169
HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clips
Miech, A., Zhukov, D., Alayrac, J.-B., Tapaswi, M., Laptev, I., & Sivic, J. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00272 · Cited by 986
The AMI Meeting Corpus: A Pre-announcement
Carletta, J., Ashby, S., Bourban, S., Flynn, M., Guillemot, M., Hain, T., Kadlec, J., Vasilis Karaiskos, B., Kraaij, W., Kronenthal, M., et al. (2006) · Lecture notes in computer science · doi:10.1007/11677482_3 · Cited by 939
Unsupervised Cross-Lingual Representation Learning for Speech Recognition
Conneau, A., Baevski, A., Collobert, R., Mohamed, A., & Auli, M. (2021) · Interspeech 2021 · doi:10.21437/interspeech.2021-329 · Cited by 637
Flagged first, then by citations
Research works printed but not linked: 5
- H. Chen, H. Zhang, L. Wang, K. A. Lee, M. Liu, and J. Dang, "Self-supervised audio-visual speaker representation with co- meta learning," in ICASSP, 2023.
- S. Wisdom, E. Tzinis, H. Erdogan, R. Weiss, K. Wilson, and J. Hershey, "Unsupervised sound separation using mixture invari- ant training," NeurIPS, vol. 33, pp. 3846-3857, 2020.
- G. Gelly and J.-L. Gauvain, "Optimization of rnn-based speech activity detection," IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 26, no. 3, pp. 646-656, 2018.
- H. Bredin, R. Yin, J. M. Coria, G. Gelly, P. Korshunov, M. Lavechin, D. Fustes, H. Titeux, W. Bouaziz, and M.-P. Gill, "pyannote.audio: neural building blocks for speaker diarization," in ICASSP, 2020.
- Y.-Y. Yang, M. Hira, Z. Ni, A. Astafurov, C. Chen, C. Puhrsch, D. Pollack, D. Genzel, D. Greenberg, E. Z. Yang et al., "Torchau- dio: Building blocks for audio and speech processing," in ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2022, pp. 6982-6986.
Cited by
Sorted by citations
Items 301–338 of 338
Sečujski, M., Vujinović, A., Kovačević, A., Simić, N., Stanojev, V., Suzić, S., Nosek, T., & Delić, V. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37867-5_12
Vanhée, L., Boström, E., Lithner, J., Andersson, C., Palmberg, B., Winberg, M., Svahn, O., & Palm, T. (2026) · IEEE Access · doi:10.1109/access.2026.3733324
Talking Faces in Serbian: From Speech and Text to Blendshape Animation
Stojicic, N., Bosnjak, M., Petrovic, T., Jovanovic, I., Suzić, S., & Nosek, T. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37870-5_14
Jaballi, S., Zrigui, S., Nicolas, H., & Zrigui, M. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-36868-3_25
Augmentation-Enhanced Whisper for Vietnamese-English Code-Switching Speech Recognition
My, N. T. T., Do, H., & Chau, D. (2026) · International Symposium on Communications and Information Technologies · doi:10.1109/iscit70453.2026.11695357
FOCUS: Towards Fetal Obstetric Corrective UltraSound Guidance
Lamdouar, H., Wang, A. F., Guo, X., Men, Q., Lander, J., Papageorghiou, A. T., & Noble, J. A. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-38236-8_19
Uteuliev, N., Kudaybergenov, J., Kudaybergenov, T., Rakhimovich, M. A., & Khudaybergenov, K. K. (2026) · Information · doi:10.3390/info17101048
Badea, D. G., Ghita, A., Rughinis, R. V., Zaim-Oprea, F., & Țurcanu, D. (2026) · Computers · doi:10.3390/computers15100654
Analytical validation of digital measure of intelligibility in progressive supranuclear palsy
Potter, İ. Y., Mishra, R. K., Pawar, T., Wang, J., Mefferd, A. S., de Riesthal, M. R., Kuruvilla-Dugdale, M., Wills, A.-M., Pantelyat, A., & Vaziri, A. (2026) · Frontiers in Neuroscience · doi:10.3389/fnins.2026.1961535
MULTIDATA: An Interactive Web Platform for Speech-Gesture Data Extraction and Visualization
Raúl, S. S., Carrión, D. A., Martínez, I. B., Garibyan, A., Iabdounane, Y., Turner, M. B., Uhrig, P., & Cánovas, C. P. (2026) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/3776574.3832530
Vollebregt, R., Bornemann, L. R., Bruijniks, S. J. E., & Salah, A. A. (2026) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/3776574.3831165
Altalhi, S., Guha, T., & Vinciarelli, A. (2026) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/3776574.3831162
Breathing Signal Prediction from Speech: Toward Reproducible and Comparable Models
Pessanha, F., Wildt, G., MacIntyre, A. D., Kaya, H., & Salah, A. A. A. (2026) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/3776574.3831208
Continuous wrist kinematics carry lexical cues for temporal deixis
Raúl, S. S., & Cánovas, C. P. (2026) · Companion Publication of the International Conference on Multimodal Interaction (ICMI Companion) · doi:10.1145/3776591.3832506
Hao, C., Hrkalović, T. M., Dudzik, B., & Hung, H. (2026) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/3776574.3831154
Enabling AI to Find Multimodal Signals of Storytelling Cross-Culturally: A Multi-Step Approach
Bonifazi, A., Al-qaoud, J. K., Carrión, D. A., Crosilla, G., Debreslioska, S., Castillo, M. D. R. I., Eide, Ø., Frings, M., Garibyan, A., Gipper, S., et al. (2026) · Companion Publication of the International Conference on Multimodal Interaction (ICMI Companion) · doi:10.1145/3776591.3836646
Multimodal detection and classification of humor in human-human and human-machine interactions
Callejas, S. (2026) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/3776574.3828962
Shaffi, S. A., Seikavandi, M. J., Obara, A., Modica, A., & Narcizo, F. B. (2026) · Companion Publication of the International Conference on Multimodal Interaction (ICMI Companion) · doi:10.1145/3776591.3837049
Decoding Humor in Stand-up Comedy: A Multilingual and Multimodal Sequence Labeling Approach
Callejas, S., Pélachaud, C., Ravenet, B., & Barrière, V. (2026) · Companion Publication of the International Conference on Multimodal Interaction (ICMI Companion) · doi:10.1145/3776591.3838174
de Brito, R. X., Rocha, F. A. D. A., & Nascimento, F. G. B. D. (2026) · Revista de Ciência da Computação · doi:10.22481/recic.v8i1.19821
Castillo-Segura, P., Fernández-Panadero, C., Alario‐Hoyos, C., & Kloos, C. D. (2026) · Applied Sciences · doi:10.3390/app16199806
Speak in Any Language: An End-to-End Framework for Faithful Multilingual Dubbing
Raju, K. S. P., Reddy, K. R. S., Goyal, N., & Choudhary, N. (2026) · International Conference on Emerging Techniques in Computational Intelligence (ICETCI) · doi:10.1109/icetci71772.2026.11710457
EgoTL: Egocentric Think-Aloud Chains for Long-Horizon Tasks
Liu, L., Li, D., Liang, Y., Jiang, S., Vijay, H., Hu, H., Xu, X., Liu, Z., Shakkottai, S., Li, M., et al. (2026) · doi:10.1109/cvprf71909.2026.00198
Evaluating Textual and Acoustic De-Identification for Code-Switched Indonesian Speech
Uliniansyah, M. T., Pebiana, S., Latief, A. D., Wabula, Y., Widyawati, N., Ocktaviana, S., Santoso, W. M., & Lan, T. J. (2026) · International Conference on Computer, Control, Informatics and its Applications · doi:10.1109/ic3ina72033.2026.11714599
Xu, Z., Wu, Z., Zhang, C., Wang, Z., Xiao, C., & Liang, C. (2026) · doi:10.1109/cvprf71909.2026.00792
Vocal Loudness and Stability as Behavioral Indicators of Surgeons’ Non-Technical Skills
Wang, J., Anton, N. E., Liu, P., Yan, Y., Stefanidis, D., & Yu, D. (2026) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/10711813261493597
Liu, D., Yang, J., Miller, C., Hadley, P. A., Redmond, S. M., & Xiong, J. (2026) · Language Speech and Hearing Services in Schools · doi:10.1044/2026_lshss-25-00268
Dobre, A. L., Țapu, R., SULTANA, A.-E., & Mocanu, B. (2026) · Neurocomputing · doi:10.1016/j.neucom.2026.135349
Park, S. M. (2026) · Journal of Broadcast Engineering · doi:10.5909/jbe.2026.31.5.1028
Moreno-Sandoval, L. G., Castillo-Laverde, J. D., Gómez-Arenas, J. J., Lee-Rodríguez, L. F., & Mejía-Estrada, N. (2026) · Communications in computer and information science · doi:10.1007/978-3-032-28444-0_5
A sociolinguistic approach to pitch range and speech rate in an oral Spanish corpus from Valencia
Nebot, A. C. (2026) · Journal of Experimental Phonetics · doi:10.1344/efe-2026-35-235-256
Shi, X., Zhao, H., Cun, X., Wu, Q., & Jin, X. (2026) · Computers & Graphics · doi:10.1016/j.cag.2026.104802
Berta, an Open-Source Modular AI Scribe for Clinical Documentation: Implementation Report (Preprint)
Vaid, S., Weldon, M., Dunn, J., Davis, S., Lonergan, K., Li, H., Franc, J. M., Abdalla, M., Baumgart, D. C., Hayward, J., et al. (2026) · JMIR Medical Informatics · doi:10.2196/105385
Gershov, S., Bentov, I., Mahameed, F., Raz, A., & Laufer, S. (2026) · JMIR Medical Education · doi:10.2196/96628
Obara, A., Seikavandi, M. J., Haddad, K., Overholt, D., Modica, A., Shaffi, S. A., & Narcizo, F. B. (2026) · International Workshop on Human-centric Multimedia Analysis (HCMA) · doi:10.1145/3841192.3841747
Lau, W. S.-Y., Swords, C. M., Thairu, M. W., Julia, N. H., Tatar, R., Rahrig, H., Wilson‐Mendenhall, C. D., Grupe, D. W., Goldman, R. I., Vack, N. J., et al. (2026) · JMIR Mental Health · doi:10.2196/92589
Fine-tuning ASR for corpus linguistics: Singapore English
Coats, S., Basile, A., Morin, C., & Fuchs, R. (2026) · Digital Scholarship in the Humanities · doi:10.1093/llc/fqag142
Pressmann, P., Zerrer, P., & Puschmann, C. (2026) · Springer eBooks · doi:10.1007/978-3-658-51008-4_12
Metadata
- Authors
- Bain, M., Huh, J., Han, T., & Zisserman, A.
- Affiliations
- University of Oxford (GB)
- Venue
- INTERSPEECH 2023
- Publisher
- ISCA
- Published
- 2023-08-20
- Pages
- 4489-4493
- ISSN
- 2308-457X (ISSN-L)
- OpenAlex
- W4385822293
- Cited by
- 337
- References
- 29
- Citations by year
- 2026: 134 · 2025: 123 · 2024: 65 · 2023: 15
- FWCI
- 59.71
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- cc-by
- Locations
- Interspeech (USB) (publishedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Funders
- Engineering and Physical Sciences Research Council (EP/T028572/1)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
- Physical Sciences › Computer Science › Signal Processing › Speech and Audio Processing
- Physical Sciences › Computer Science › Signal Processing › Music and Audio Processing
- Keywords
- speech transcription, voice activity detection, forced alignment, phone recognition, timestamping
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
