CoChat
Sign Up
PreprintCited by 116Speech Recognition and SynthesisSpeech and Audio ProcessingSpeech and dialogue systems

The DOI matches its DataCite record; 1 of the 1 references it cites were found in a registry or an index.

Scaling Speech Technology to 1, 000+ Languages

Open access
Pratap, Vineel, Tjandra, Andros, Shi, Bowen, Tomasello, Paden, Babu, Arun, Kundu, Sayani, Elkahky, Ali, Ni, Zhaoheng, Vyas, Apoorv, Fazel-Zarandi, Maryam, Baevski, Alexei, Adi, Yossi, Zhang, Xiaohui, Hsu, Wei-Ning, Conneau, Alexis, Auli, Michael·2023·arXiv
DOI 10.48550/arxiv.2305.13516arXiv 2305.13516
Open this paper in CoChat — free accountOpen PDF

Abstract

Expanding the language coverage of speech technology has the potential to improve access to information for many more people. However, current speech technology is restricted to about one hundred languages which is a small fraction of the over 7,000 languages spoken around the world. The Massively Multilingual Speech (MMS) project increases the number of supported languages by 10-40x, depending on the task. The main ingredients are a new dataset based on readings of publicly available religious texts and effectively leveraging self-supervised learning. We built pre-trained wav2vec 2.0 models covering 1,406 languages, a single multilingual automatic speech recognition model for 1,107 languages, speech synthesis models for the same number of languages, as well as a language identification model for 4,017 languages. Experiments show that our multilingual speech recognition model more than halves the word error rate of Whisper on 54 languages of the FLEURS benchmark while being trained on a small fraction of the labeled data.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2305.13516

Cites

1 of 1 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Language identification is cross-validated using fastText on recognized text and MMS-LID on raw waveform to make language annotations robust against ASR errors, short utterances, and mixed-language cases.

    “Furthermore, language identification (LID) is cross-validated using fastText [29, 30] on the recognized text and MMS-LID [31] on the raw waveform, making the final language annotations robust against ASR errors, short utterances, and mixed-language cases.”

    From MOSS-Audio Technical Report (2026) · cites this paper as [31]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Flagged first, then by citations

Research works printed but not linked: 91

  1. R. Ardila, M. Branson, K. Davis, M. Henretty, M. Kohler, J. Meyer, R. Morais, L. Saunders, F. M. Tyers, and G. Weber. Common voice: A massively-multilingual speech corpus. Proc. of LREC, 2020.
  2. J. L. Ba, J. R. Kiros, and G. E. Hinton. Layer normalization. arXiv, 2016.
  3. A. Baevski, S. Schneider, and M. Auli. vq-wav2vec: Self-supervised learning of discrete speech representations. In Proc. of ICLR, 2020a.
  4. A. Baevski, Y. Zhou, A. Mohamed, and M. Auli. wav2vec 2.0: A framework for self-supervised learning of speech representations. In Proc. of NeurIPS, 2020b.
  5. A. Baevski, W.-N. Hsu, A. Conneau, and M. Auli. Unsupervised speech recognition. In A. Beygelz- imer, Y. Dauphin, P. Liang, and J. W. Vaughan, editors, Advances in Neural Information Processing Systems, 2021.
  6. A. Bapna, I. Caswell, J. Kreutzer, O. Firat, D. van Esch, A. Siddhant, M. Niu, P. Baljekar, X. Garcia, W. Macherey, T. Breiner, V. Axelrod, J. Riesa, Y. Cao, M. X. Chen, K. Macherey, M. Krikun, P. Wang, A. Gutkin, A. Shah, Y. Huang, Z. Chen, Y. Wu, and M. Hughes. Building machine translation systems for the next thousand languages. arXiv, abs/2205.03983, 2022a. · arXiv
  7. A. Bapna, C. Cherry, Y. Zhang, Y. Jia, M. Johnson, Y. Cheng, S. Khanuja, J. Riesa, and A. Conneau. mslam: Massively multilingual joint pre-training for speech and text. ArXiv, abs/2202.01374, 2022b. · arXiv
  8. A. W. Black. Cmu wilderness multilingual speech dataset. In Proc. of ICASSP, 2019.
  9. L. Burget, P. Schwarz, et al. Multilingual acoustic modeling for speech recognition based on subspace gaussian mixture models. In Proc. of ICASSP, 2010.
  10. X. Cai, J. Yuan, Y. Bian, G. Xun, J. Huang, and K. Church. W-CTC: a connectionist temporal classification loss with wild cards. In International Conference on Learning Representations, 2022. URL https://openreview.net/forum?id=0RqDp8FCW5Z.
  11. E. Casanova, J. Weber, C. D. Shulby, A. C. Junior, E. Gölge, and M. A. Ponti. Yourtts: Towards zero- shot multi-speaker tts and zero-shot voice conversion for everyone. In International Conference on Machine Learning, pages 2709-2720. PMLR, 2022.
  12. W. Chan, N. Jaitly, Q. V. Le, and O. Vinyals. Listen, attend and spell. arXiv, 2015.
  13. W. Chan, D. Park, C. Lee, Y. Zhang, Q. Le, and M. Norouzi. Speechstew: Simply mix all available speech recognition data to train one large neural network. arXiv, abs/2104.02133, 2021. · arXiv
  14. T. Chen, B. Xu, C. Zhang, and C. Guestrin. Training deep nets with sublinear memory cost. arXiv, abs/1604.06174, 2016. · arXiv
  15. W. Chen, B. Yan, J. Shi, Y. Peng, S. Maiti, and S. Watanabe. Improving massively multilingual asr with auxiliary ctc objectives. In Proc. of ICASSP, 2023.
  16. Z. Chen, Y. Zhang, A. Rosenberg, B. Ramabhadran, P. Moreno, A. Bapna, and H. Zen. Maestro: Matched speech text representations through modality matching. arXiv, abs/2204.03409, 2022. · arXiv
  17. J. Cho, M. K. Baskar, et al. Multilingual sequence-to-sequence speech recognition: architecture, transfer learning, and language modeling. In Proc. of IEEE SLT, 2018.
  18. C. Christodouloupoulos and M. Steedman. A massively parallel corpus: the bible in 100 languages. In Proc. of LREC, 2015.
  19. R. Collobert, C. Puhrsch, and G. Synnaeve. Wav2letter: an end-to-end convnet-based speech recognition system. arXiv, abs/1609.03193, 2016. · arXiv
  20. A. Conneau and G. Lample. Cross-lingual language model pretraining. Proc. of NeurIPS, 2019.
  21. A. Conneau, A. Baevski, R. Collobert, A. Mohamed, and M. Auli. Unsupervised cross-lingual representation learning for speech recognition. arXiv, abs/2006.13979, 2020a. · arXiv
  22. A. Conneau, K. Khandelwal, N. Goyal, V. Chaudhary, G. Wenzek, F. Guzmán, E. Grave, M. Ott, L. Zettlemoyer, and V. Stoyanov. Unsupervised cross-lingual representation learning at scale. In Proc. of ACL, 2020b.
  23. A. Conneau, M. Ma, S. Khanuja, Y. Zhang, V. Axelrod, S. Dalmia, J. Riesa, C. Rivera, and A. Bapna. Fleurs: Few-shot learning evaluation of universal representations of speech. arXiv, 2022.
  24. A. Defossez, G. Synnaeve, and Y. Adi. Real time speech enhancement in the waveform domain. In Interspeech, 2020.
  25. J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. Proc. of NAACL, 2019.
  26. S. Dieleman, A. van den Oord, and K. Simonyan. The challenge of realistic music generation: modelling raw audio at scale. Proc of NIPS, 2018.
  27. L. Dinh, J. Sohl-Dickstein, and S. Bengio. Density estimation using real NVP. In ICLR. OpenRe- view.net, 2017.
  28. D. Doukhan, E. Lechapt, M. Evrard, and J. Carrive. Ina's mirex 2018 music and speech detection system. In Proc. of MIREX, 2018.
  29. Z. Fan, M. Li, S. Zhou, and B. Xu. Exploring wav2vec 2.0 on speaker verification and language identification. arXiv, 2021.
  30. A. Graves, S. Fernández, and F. Gomez. Connectionist temporal classification: Labelling unsegmented sequence data with recurrent neural networks. In Proc. of ICML, 2006.
  31. M. He, J. Yang, L. He, and F. K. Soong. Multilingual byte2speech models for scalable low-resource speech synthesis. arXiv, abs/arXiv:2103.03541, 2021.
  32. K. Heafield. KenLM: Faster and smaller language model queries. In Proceedings of the Sixth Workshop on Statistical Machine Translation, pages 187-197, Edinburgh, Scotland, July 2011. Association for Computational Linguistics.
  33. G. Heigold, V. Vanhoucke, A. Senior, P. Nguyen, M. Ranzato, M. Devin, and J. Dean. Multilingual acoustic models using distributed deep neural networks. In Proc. of ICASSP, 2013.
  34. U. Hermjakob, J. May, and K. Knight. Out-of-the-box universal Romanization tool uroman. In Proceedings of ACL 2018, System Demonstrations, pages 13-18, Melbourne, Australia, July 2018. Association for Computational Linguistics. doi: 10.18653/v1/P18-4003. URL https: //aclanthology.org/P18-4003. · DOI
  35. N. Houlsby, A. Giurgiu, S. Jastrzebski, B. Morrone, Q. De Laroussilhe, A. Gesmundo, M. Attariyan, and S. Gelly. Parameter-efficient transfer learning for nlp. In International Conference on Machine Learning, pages 2790-2799. PMLR, 2019.
  36. W.-N. Hsu, A. Sriram, A. Baevski, T. Likhomanenko, Q. Xu, V. Pratap, J. Kahn, A. Lee, R. Collobert, G. Synnaeve, et al. Robust wav2vec 2.0: Analyzing domain shift in self-supervised pre-training. arXiv preprint arXiv:2104.01027, 2021. · arXiv
  37. E. Jang, S. Gu, and B. Poole. Categorical reparameterization with gumbel-softmax. Proc. of ICLR, 2016.
  38. T. Javed, S. Doddapaneni, A. Raman, K. S. Bhogale, G. Ramesh, A. Kunchukuttan, P. Kumar, and M. M. Khapra. Towards building asr systems for the next billion users. In Proc. of AAAI CAI, 2022.
  39. H. Jegou, M. Douze, and C. Schmid. Product quantization for nearest neighbor search. IEEE Trans. Pattern Anal. Mach. Intell., 33(1):117-128, Jan. 2011.
  40. F. Jia, N. R. Koluguri, J. Balam, and B. Ginsburg. Ambernet: A compact end-to-end model for spoken language identification. arXiv, abs/2210.15781, 2022. · arXiv
  41. J. D. Kahn, V. Pratap, T. Likhomanenko, Q. Xu, A. Hannun, J. Cai, P. Tomasello, A. Lee, E. Grave, G. Avidov, et al. Flashlight: Enabling innovation in tools for machine learning. In International Conference on Machine Learning, pages 10557-10574. PMLR, 2022.
  42. A. Kannan, A. Datta, et al. Large-scale multilingual speech recognition with a streaming end-to-end model. In Proc. of Interspeech, 2019.
  43. J. Kim, J. Kong, and J. Son. Conditional variational autoencoder with adversarial learning for end-to-end text-to-speech. In Proc. of ICML, 2021.
  44. D. P. Kingma and J. Ba. Adam: A Method for Stochastic Optimization. In Proc. of ICLR, 2015.
  45. D. P. Kingma and M. Welling. Auto-encoding variational bayes. CoRR, abs/1312.6114, 2013. · arXiv
  46. A. Koenecke, A. Nam, E. Lake, J. Nudell, M. Quartey, Z. Mengesha, C. Toups, J. R. Rickford, D. Jurafsky, and S. Goel. Racial disparities in automated speech recognition. Proceedings of the National Academy of Sciences, 2020.
  47. J. Kong, J. Kim, and J. Bae. Hifi-gan: Generative adversarial networks for efficient and high fidelity speech synthesis. In NeurIPS, 2020.
  48. L. Kürzinger, D. Winkelbauer, L. Li, T. Watzel, and G. Rigoll. Ctc-segmentation of large corpora for german end-to-end speech recognition. In A. Karpov and R. Potapova, editors, Speech and Computer, pages 267-278, Cham, 2020. Springer International Publishing.
  49. C. Leong, J. Nemecek, J. Mansdorfer, A. Filighera, A. Owodunni, and D. Whitenack. Bloom library: Multimodal datasets in 300+ languages for a variety of downstream tasks. In Proc. of EMNLP, 2022.
  50. M. P. Lewis, G. F. Simon, and C. D. Fennig. Ethnologue: Languages of the world, nineteenth edition. Online version: http://www.ethnologue.com, 2016.
  51. B. Li, Y. Zhang, T. Sainath, Y. Wu, and W. Chan. Bytes are all you need: End-to-end multilingual speech recognition and synthesis with bytes. In Proc. of ICASSP, 2019.
  52. B. Li, R. Pang, T. N. Sainath, A. Gulati, Y. Zhang, J. Qin, P. Haghani, W. R. Huang, M. Ma, and J. Bai. Scaling end-to-end models for large-scale multilingual asr. In Proc. of ASRU, 2021.
  53. X. Li, F. Metze, D. R. Mortensen, A. W. Black, and S. Watanabe. Asr2k: Speech recognition for around 2000 languages without audio. arXiv preprint arXiv:2209.02842, 2022. · arXiv
  54. T. Likhomanenko, G. Synnaeve, and R. Collobert. Who needs words? lexicon-free speech recognition. In Proc. of Interspeech, 2019.
  55. T. Likhomanenko, Q. Xu, V. Pratap, P. Tomasello, J. Kahn, G. Avidov, R. Collobert, and G. Synnaeve. Rethinking evaluation in ASR: are our models robust enough? In Proc. of Interspeech, 2020.
  56. H. Lin, L. Deng, D. Yu, Y.-f. Gong, A. Acero, and C.-H. Lee. A study on multilingual acoustic modeling for large vocabulary asr. In Proc. of ICASSP, 2009.
  57. A. H. Liu, W.-N. Hsu, M. Auli, and A. Baevski. Towards end-to-end unsupervised speech recognition. arXiv, 2022.
  58. Y. Liu, J. Gu, N. Goyal, X. Li, S. Edunov, M. Ghazvininejad, M. Lewis, and L. Zettlemoyer. Multilingual denoising pre-training for neural machine translation. arXiv, abs/2001.08210, 2020. · arXiv
  59. L. Lugosch, T. Likhomanenko, G. Synnaeve, and R. Collobert. Pseudo-labeling for massively multilingual speech recognition. In Proc. of ICASSP, 2022.
  60. M. Mauch and S. Dixon. Pyin: A fundamental frequency estimator using probabilistic threshold distributions. 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 659-663, 2014.
  61. A. McCarthy, R. Wicks, D. Lewis, A. Mueller, W. Wu, O. Adams, G. Nicolai, M. Post, and D. Yarowsky. The johns hopkins university bible corpus: 1600+ tongues for typological exploration. In Proc. of LREC, 2020.
  62. J. Meyer, D. I. Adelani, E. Casanova, A. Öktem, D. W. J. Weber, S. Kabongo, E. Salesky, I. Orife, C. Leong, P. Ogayo, C. Emezue, J. Mukiibi, S. Osei, A. Agbolo, V. Akinode, B. Opoku, S. Olan- rewaju, J. Alabi, and S. Muhammad. Bibletts: a large, high-fidelity, multilingual, and uniquely african speech corpus. arXiv, abs/2207.03546, 2022. · arXiv
  63. T. Nekvinda and O. Dušek. One model, many languages: Meta-learning for multilingual text-to- speech. In Interspeech, 2020.
  64. M. Ott, S. Edunov, A. Baevski, A. Fan, S. Gross, N. Ng, D. Grangier, and M. Auli. fairseq: A fast, extensible toolkit for sequence modeling. In Proc. of NAACL System Demonstrations, 2019.
  65. D. Povey, A. Ghoshal, G. Boulianne, L. Burget, O. Glembek, N. Goel, M. Hannemann, P. Motlicek, Y. Qian, P. Schwarz, J. Silovsky, G. Stemmer, and K. Vesely. The kaldi speech recognition toolkit. In Proc. of ASRU, 2011.
  66. V. Pratap, A. Hannun, Q. Xu, J. Cai, J. Kahn, G. Synnaeve, V. Liptchinsky, and R. Collobert. Wav2letter++: A fast open-source speech recognition system. In Proc. of ICASSP, 2019.
  67. V. Pratap, A. Sriram, P. Tomasello, A. Hannun, V. Liptchinsky, G. Synnaeve, and R. Collobert. Massively Multilingual ASR: 50 Languages, 1 Model, 1 Billion Parameters. In Proc. of Interspeech 2020, 2020a.
  68. V. Pratap, A. Sriram, et al. Massively multilingual asr: 50 languages, 1 model, 1 billion parameters. arXiv, abs/2007.03001, 2020b. · arXiv
  69. V. Pratap, Q. Xu, A. Sriram, G. Synnaeve, and R. Collobert. Mls: A large-scale multilingual dataset for speech research. In Proc. of Interspeech, 2020c.
  70. V. Pratap, A. Hannun, G. Synnaeve, and R. Collobert. Star temporal classification: Sequence modeling with partially labeled data. In A. H. Oh, A. Agarwal, D. Belgrave, and K. Cho, editors, Advances in Neural Information Processing Systems, 2022. URL https://openreview.net/ forum?id=ldRyJb_cjXa.
  71. T. Qian, K. Hollingshead, S.-y. Yoon, K.-y. Kim, and R. Sproat. A python toolkit for universal transliteration. In Proc. of LREC, 2010.
  72. A. Radford, J. W. Kim, T. Xu, G. Brockman, C. McLeavey, and I. Sutskever. Robust speech recognition via large-scale weak supervision. arXiv, 2022.
  73. S. Rajbhandari, O. Ruwase, J. Rasley, S. Smith, and Y. He. Zero-infinity: Breaking the GPU memory wall for extreme scale deep learning. arXiv, abs/2104.07857, 2021. · arXiv
  74. M. Ravanelli, T. Parcollet, P. Plantinga, A. Rouhe, S. Cornell, L. Lugosch, C. Subakan, N. Dawalatabad, A. Heba, J. Zhong, J.-C. Chou, S.-L. Yeh, S.-W. Fu, C.-F. Liao, E. Rastorgueva, F. Grondin, W. Aris, H. Na, Y. Gao, R. D. Mori, and Y. Bengio. SpeechBrain: A general-purpose speech toolkit, 2021. arXiv:2106.04624. · arXiv
  75. F. Ribeiro, D. Florêncio, C. Zhang, and M. Seltzer. Crowdmos: An approach for crowdsourcing mean opinion score studies. In 2011 IEEE international conference on acoustics, speech and signal processing (ICASSP), pages 2416-2419. IEEE, 2011.
  76. T. Saeki, S. Maiti, X. Li, S. Watanabe, S. Takamichi, and H. Saruwatari. Learning to speak from text: Zero-shot multilingual text-to-speech with unsupervised text pretraining. arXiv, abs/2301.12596, 2023a. · arXiv
  77. T. Saeki, H. Zen, Z. Chen, N. Morioka, G. Wang, Y. Zhang, A. Bapna, A. Rosenberg, and B. Ram- abhadran. Virtuoso: Massive multilingual speech-text joint semi-supervised learning for text-to- speech. In Proc. of ICASSP, 2023b.
  78. S. Schneider, A. Baevski, R. Collobert, and M. Auli. wav2vec: Unsupervised pre-training for speech recognition. In Proc. of Interspeech, 2019.
  79. M. Staib, T. H. Teh, A. Torresquintero, D. S. R. Mohan, L. Foglianti, R. Lenain, and J. Gao. Phonological features for 0-shot multilingual speech synthesis. In Interspeech, 2020.
  80. I. Sutskever, O. Vinyals, and Q. V. Le. Sequence to sequence learning with neural networks. Proc. of NIPS, 2014.
  81. X. Tan, T. Qin, F. Soong, and T.-Y. Liu. A survey on neural speech synthesis. arXiv preprint arXiv:2106.15561, 2021. · arXiv
  82. A. Tjandra, D. G. Choudhury, F. Zhang, K. Singh, A. Conneau, A. Baevski, A. Sela, Y. Saraf, and M. Auli. Improved language identification through cross-lingual self-supervised learning. In ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 6877-6881. IEEE, 2022a.
  83. A. Tjandra, N. Singhal, D. Zhang, O. Kalinli, A. Mohamed, D. Le, and M. L. Seltzer. Massively multilingual asr on 70 languages: Tokenization, architecture, and generalization capabilities. arXiv, abs/2211.05756, 2022b. · arXiv
  84. S. Toshniwal, T. N. Sainath, R. J. Weiss, B. Li, P. Moreno, E. Weinstein, and K. Rao. Multilingual speech recognition with a single end-to-end model. In Proc. of ICASSP, 2018.
  85. J. Valk and T. Alumäe. Voxlingua107: a dataset for spoken language recognition. In Proc. of SLT, 2020.
  86. A. van den Oord, Y. Li, and O. Vinyals. Representation learning with contrastive predictive coding. Proc. of NIPS, 2018.
  87. A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin. Attention is all you need. In Proc. of NIPS, 2017.
  88. C. Wang, M. Riviere, A. Lee, A. Wu, C. Talnikar, D. Haziza, M. Williamson, J. Pino, and E. Dupoux. VoxPopuli: A large-scale multilingual speech corpus for representation learning, semi-supervised learning and interpretation. In Proc. of ACL, 2021.
  89. S. Watanabe, T. Hori, S. Karita, T. Hayashi, J. Nishitoba, Y. Unno, N. Enrique Yalta Soplin, J. Hey- mann, M. Wiesner, N. Chen, A. Renduchintala, and T. Ochiai. ESPnet: End-to-end speech processing toolkit. In Proceedings of Interspeech, pages 2207-2211, 2018. doi: 10.21437/ Interspeech.2018-1456. URL http://dx.doi.org/10.21437/Interspeech.2018-1456. · DOI
  90. Y.-Y. Yang, M. Hira, Z. Ni, A. Chourdia, A. Astafurov, C. Chen, C.-F. Yeh, C. Puhrsch, D. Pollack, D. Genzel, D. Greenberg, E. Z. Yang, J. Lian, J. Mahadeokar, J. Hwang, J. Chen, P. Goldsborough, P. Roy, S. Narenthiran, S. Watanabe, S. Chintala, V. Quenneville-Bélair, and Y. Shi. Torchaudio: Building blocks for audio and speech processing. arXiv preprint arXiv:2110.15018, 2021. · arXiv
  91. S.-Y. Yoon, K.-Y. Kim, and R. Sproat. Multilingual transliteration using feature based phonetic method. In Proc. of ACL, 2007. Amharic 229.3 140.3 48.7 30.7 52.4 32.5 52.9 30.1 53.3 31.1 Arabic 20.4 16.0 34.9 19.6 35.8 19.9 44.0 23.4 41.3 21.0 Assamese 102.3 106.2 29.5 18.8 28.4 18.6 37.6 21.2 30.5 19.2 Azerbaijani 33.1 23.4 40.7 21.3 38.3 19.8 45.0 21.2 40.1 19.1 Bengali 100.6 104.1 19.7 11.6 20.0 12.1 25.0 12.5 23.5 12.1 Bulgarian 21.4 14.6 23.4 13.1 23.9 13.3 27.9 12.9 25.5 13.5 Burmese 123.0 115.7 22.2 14.2 22.3 14.5 29.2 20.2 24.5 16.0 Catalan 9.6 7.3 18.1 11.0 18.1 11.0 25.9 11.5 20.1 10.8 Dutch 9.9 6.7 26.9 13.7 26.4 14.3 38.1 14.9 27.6 14.5 English 4.4 4.2 23.8 10.7 24.8 11.8 38.8 12.2 27.8 12.3 Filipino 19.1 13.8 19.3 11.9 19.4 12.2 26.2 13.5 20.2 12.4 Finnish 13.9 9.7 26.4 22.5 26.9 23.1 32.3 22.2 28.8 23.1 French 8.7 8.3 24.3 13.7 24.5 14.1 35.8 15.4 29.3 15.0 German 6.5 4.5 22.5 13.2 22.3 13.7 38.4 13.1 22.5 13.3 Greek 19.0 12.5 40.8 14.0 40.5 13.6 57.5 13.0 40.1 13.6 Gujarati 104.8 102.7 23.0 13.0 22.7 12.8 73.9 56.4 24.0 12.8 Hausa 106.6 88.9 35.9 26.7 36.3 27.3 40.4 26.7 38.3 26.4 Hebrew 33.1 27.1 68.5 44.8 66.6 41.5 78.7 50.9 67.1 40.0 Hindi 26.8 21.5 65.0 44.4 28.8 16.0 70.7 45.7 21.2 10.6 Hungarian 24.3 17.0 31.2 18.1 30.7 18.4 40.3 18.3 30.7 18.0 Icelandic 49.9 38.2 42.9 18.3 42.3 19.9 53.6 20.5 45.3 18.6 Indonesian 10.2 7.1 25.5 11.7 23.8 12.1 31.9 11.6 23.4 11.8 Javanese 67.9 68.5 32.8 19.6 32.8 20.0 58.8 27.2 34.2 19.5 Kannada 77.7 37.0 18.8 14.4 15.8 12.9 41.3 25.2 17.7 13.3 Kazakh 48.8 37.7 30.2 17.4 30.2 17.7 63.8 19.5 31.6 17.4 Khmer 103.8 128.9 26.0 19.9 25.7 19.8 70.7 52.4 26.7 19.7 Korean 16.4 14.3 58.7 37.5 59.9 37.3 82.1 58.2 68.3 40.1 Lao 101.4 101.5 48.9 45.4 24.2 22.8 62.1 56.6 22.6 16.9 Latvian 32.0 23.1 20.8 12.0 20.9 12.1 24.5 11.9 21.8 12.1 Malay 12.2 8.7 25.3 12.3 25.9 13.2 32.4 12.1 26.1 12.5 Malayalam 101.1 100.7 23.7 19.1 19.5 16.6 39.1 25.6 20.4 15.3 Marathi 63.2 38.3 32.5 19.0 19.2 13.5 28.0 14.9 20.9 13.4 Mongolian 103.7 110.5 55.7 29.3 54.9 32.9 67.7 28.7 55.3 32.3 Persian 41.0 32.9 39.7 22.9 39.9 22.5 44.4 21.3 42.9 22.0 Polish 8.0 5.4 21.5 11.4 20.8 11.6 33.0 11.0 25.1 11.3 Portuguese 5.0 4.3 16.1 10.8 16.3 10.8 19.3 10.2 17.7 10.5 Punjabi 102.0 102.4 41.4 29.9 30.4 20.7 99.0 91.0 31.0 19.8 Romanian 20.0 14.4 27.9 18.8 28.4 19.1 31.3 17.8 27.4 18.3 Russian 7.2 5.6 30.3 14.6 30.4 14.3 38.8 14.7 35.0 15.0 Shona 143.9 121.0 38.1 30.4 37.7 30.1 43.0 29.9 37.8 29.6 Somali 104.0 102.9 51.8 42.8 52.5 43.0 54.5 42.9 53.8 42.8 Spanish 3.6 3.0 12.2 7.8 12.4 8.2 14.0 7.8 14.0 8.7 Swahili 52.8 39.3 22.9 16.0 23.3 15.6 29.6 16.8 23.7 16.0 Swedish 11.2 8.5 29.9 17.4 30.5 17.5 38.2 17.2 33.5 17.4 Tajik 74.0 85.8 59.8 46.6 33.9 19.2 59.0 39.5 25.7 15.7 Tamil 23.1 17.5 24.2 18.3 21.9 16.3 25.3 17.3 23.9 16.3 Telugu 82.8 99.0 19.4 13.7 19.6 13.7 24.5 15.8 22.1 13.6 Thai 15.4 11.5 18.2 13.6 18.1 13.6 27.6 18.8 20.7 14.3 Turkish 10.4 8.4 28.6 17.3 28.7 17.5 31.2 16.1 30.9 16.9 Ukrainian 11.6 8.6 31.1 13.6 31.7 13.5 39.2 13.3 33.3 13.6 Urdu 28.2 22.6 42.3 22.7 33.1 20.1 46.4 25.1 36.9 20.5 Vietnamese 12.7 10.3 44.5 18.6 47.5 20.7 56.6 21.0 52.9 19.8 Welsh 40.8 33.0 48.9 20.8 49.0 20.8 54.9 21.4 51.4 20.9 Yoruba 105.1 94.8 61.2 49.7 61.9 49.4 62.7 50.2 64.2 49.4 50.1 44.3 33.3 20.7 31.0 19.1 44.2 24.8 32.5

Cited by

Sorted by citations

Metadata

Authors
Pratap, V., Tjandra, A., Shi, B., Tomasello, P., Babu, A., Kundu, S., Elkahky, A., Ni, Z., Vyas, A., Fazel-Zarandi, M., Baevski, A., Adi, Y., Zhang, X., Hsu, W.-N., Conneau, A., & Auli, M.
ORCID
Venue
arXiv
Publisher
arXiv
Published
2023
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2305.13516
arXiv
2305.13516
OpenAlex
W4378105483
Cited by
116
References
1
Citations by year
2026: 5 · 2025: 18 · 2024: 60 · 2023: 33
Type
posted content
Language
en
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
  • Physical Sciences › Computer Science › Signal Processing › Speech and Audio Processing
  • Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
Keywords
self-supervised learning, wav2vec 2.0, multilingual speech recognition, speech synthesis, language identification, low-resource languages

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review