Japanese and Korean voice search
The DOI matches its Crossref record; 7 of the 8 references it cites were found in a registry or an index.
Abstract
This paper describes challenges and solutions for building a successful voice search system as applied to Japanese and Korean at Google. We describe the techniques used to deal with an infinite vocabulary, how modeling completely in the written domain for language model and dictionary can avoid some system complexity, and how we built dictionaries, language and acoustic models in this framework. We show how to deal with the difficulty of scoring results for multiple script languages because of ambiguities. The development of voice search for these languages led to a significant simplification of the original process to build a system for any new language which in in parts became our default process for internationalization of voice search.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“LogRobust [8], which encodes log templates into semantic vectors using the FastText pre-trained model [31], cannot work well on 2 out of 4 datasets.”
From Log-based Anomaly Detection Without Log Parsing (2021) · cites this paper as [8]
In the citing paper
“Their empirical study on a Microsoft online service system shows that up to 30.3% logs are changed in the latest version.”
From Log-based Anomaly Detection Without Log Parsing (2021) · cites this paper as [8]
In the citing paper
“Zhang et al. [8] indicated that log data is unstable, meaning that new log events often appear due to software evolution at its lifetime.”
From Log-based Anomaly Detection Without Log Parsing (2021) · cites this paper as [8]
In the citing paper
“Another option is to use domain knowledge to reduce the input space of a model — such strategies include patch embeddings [Dosovitskiy et al. 2020] as well as fixed or learned tokenization [Schuster and Nakajima 2012; Sennrich et al. 2016; Kudo and Richardson 2018; van den Oord et al. 2017; Ramesh et al. 2021]. These methods may alleviate some challenges facing generative approaches, but have the”
From On the Opportunities and Risks of Foundation Models (2021) · cites this paper as Schuster and Nakajima 2012
Citations
Cites
Semi-tied covariance matrices for hidden Markov models
Gales, M. J. F. (1999) · IEEE Transactions on Speech and Audio Processing · doi:10.1109/89.759034 · Cited by 537
Boosted MMI for model and feature-space discriminative training
Povey, D., Kanevsky, D., Kingsbury, B., Ramabhadran, B., Saon, G., & Visweswariah, K. (2008) · 2008 IEEE International Conference on Acoustics, Speech and Signal Processing · doi:10.1109/icassp.2008.4518545 · Cited by 341
“Your Word is my Command”: Google Search by Voice: A Case Study
Schalkwyk, J., Beeferman, D., Beaufays, F., Byrne, B., Chelba, C., Cohen, M., Kamvar, M., & Strope, B. (2010) · Advances in Speech Recognition · doi:10.1007/978-1-4419-5951-5_4 · Cited by 232
Building transcribed speech corpora quickly and cheaply for many languages
Hughes, T., Nakajima, K., Ha, L., Vasu, A., Moreno, P. J., & LeBeau, M. (2010) · Interspeech (USB) · doi:10.21437/interspeech.2010-551 · Cited by 75
Self-citationGoogle Search by Voice: A Case Study
Schalkwyk, J., Beeferman, D., Beaufays, F., Byrne, B., Chelba, C. I., Cohen, M., Garrett, M., & Strope, B. P. (2010) · Cited by 63
Query language modeling for voice search
Chelba, C. I., Schalkwyk, J., Brants, T., Ha, V., Harb, B., Neveitt, W., Parada, C., & Xu, P. (2010) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt.2010.5700834 · Cited by 28
Search by voice in Mandarin Chinese
Shan, J., Wu, G., Hu, Z., Tang, X., Jansche, M., & Moreno, P. J. (2010) · Interspeech (USB) · doi:10.21437/interspeech.2010-129 · Cited by 18
Research works printed but not linked: 1
- sung, 0, Deploying Search by Voice in Cantonese, Proc Interspeech 2011
Cited by
Items 301–400 of 933
Cross-language Information RetrievalPreprint
Galuščáková, P., Oard, D. W., & Nair, S. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.05988 · Cited by 24
“Byte-Pair Encoding (BPE) [178] and Wordpiece [177] are now widely used in MT.”
BackgroundLog Sequence Anomaly Detection Based on Template and Parameter Parsing via BERT
Chai, X., Zhang, H., Zhang, J., Sun, Y., & Das, S. K. (2024) · IEEE Transactions on Dependable and Secure Computing · doi:10.1109/tdsc.2024.3428538 · Cited by 26
miMamba: EEG-Based Emotion Recognition With Multi-Scale Inverted Mamba Models
Zhou, X., Huang, D., Peng, X., & Yin, L. (2025) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2025.3587443 · Cited by 25
“Innaturallanguageprocessing,itisalsomoreeffec-tivetosymbolizewordsinasentenceratherthaneachletter[27], [28].”
The impact of tokenizer selection in genomic language models
Lindsey, L. M., Pershing, N. L., Habib, A. K. M. R. R., Dufault‐Thompson, K., Stephens, W. Z., Blaschke, A. J., Jiang, X., & Sundar, H. (2025) · Bioinformatics · doi:10.1093/bioinformatics/btaf456 · Cited by 25
“…studying tokenization in biological language models to date is Dotan et al. (2024), which tested five different tokenizers of various sizes: BPE (Sennrich et al. 2016), Unigram (Kudo 2018), WordPiece (Schuster and Nakajima 2012), characters, and pairs, against eight different biological datasets.”
A Compression-Based Multiple Subword Segmentation for Neural Machine Translation
Nonaka, K., Yamanouchi, K., Tomohiro, I., Okita, T., Shimada, K., & Sakamoto, H. (2022) · Electronics · doi:10.3390/electronics11071014 · Cited by 21
Ludwig, F., Dolos, K., Zesch, T., & Hobley, E. (2022) · Workshop on Online Abuse and Harms (WOAH) · doi:10.18653/v1/2022.woah-1.4 · Cited by 20
Pashto offensive language detection: a benchmark dataset and monolingual Pashto BERT
Haq, I., Qiu, W., Guo, J., & Tang, P. (2023) · PeerJ Computer Science · doi:10.7717/peerj-cs.1617 · Cited by 28
“…from scratch, we utilized the BERT Base ( Devlin et al., 2018 ) Tokenization: To tokenize the input sentences (tweets), we used the BERT WordPiece tokenizer ( Schuster & Nakajima, 2012 ), which is the recommended tokenizer for the BERT Base model, and the vocabulary size was fixed at 30K words.”
MethodPhoneme-Based Contextualization for Cross-Lingual Speech Recognition in End-to-End Models
Hu, K., Bruguier, A., Sainath, T. N., Prabhavalkar, R., & Pundak, G. (2019) · Interspeech (USB) · doi:10.21437/interspeech.2019-1868 · Cited by 20
“Further improvements such as biasing before beam pruning, and wordpiece-based biasing have been proposed to achieve state-of-the-art biasing results [12, 6, 13].”
BackgroundEmiru, E. D., Xiong, S., Li, Y., Fesseha, A., & Diallo, M. (2021) · Information · doi:10.3390/info12020062 · Cited by 23
You should evaluate your language model on marginal likelihood over tokenisations
Cao, K., & Rimell, L. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.161 · Cited by 43
“Alternative approaches, such as WordPiece (Schuster and Nakajima, 2012) and SentencePiece (Kudo, 2018), explicitly use a language modelling objective to induce a token lexicon.”
MethodMultilingual Question Answering from Formatted Text applied to Conversational AgentsPreprint
Siblini, W., Pasqual, C., Lavielle, A., Challal, M., & Cauchois, C. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.04659 · Cited by 23
“Tokens generally consist in single characters or the most frequent series of characters of the language, generated with Wordpiece Tokenization [38]. unlabeled text data to learn contextual token representations.”
BackgroundYu, Z., Yang, X., Sweeting, G. L., Ma, Y., Stolte, S. E., Fang, R., & Wu, Y. (2022) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-022-01996-2 · Cited by 19
“wordpiece[32] in BERT and Byte-Pair Encoding [33] in RoBERTa) and adopted the default parameters optimized in our clinical transformer package and clinical relation extraction with transformer package [22, 26].”
MethodAccelerating RNN-T Training and Inference Using CTC Guidance
Wang, Y., Chen, Z., Zheng, C., Zhang, Y., Han, W., & Haghani, P. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10096065 · Cited by 19
Zhang, R., Zhang, J., Chen, Q., Wang, B., Liu, Y., Qian, Q., Pan, D., Xia, J., Wang, Y., & Han, Y. (2023) · Computational Materials Science · doi:10.1016/j.commatsci.2023.112441 · Cited by 24
Sboev, A. G., Sboeva, S., Александрович, М. И., Gryaznov, A. V., Борисович, Р. Р., Naumov, A., Selivanov, A., Rylkov, G., & Ilyin, V. (2022) · Applied Sciences · doi:10.3390/app12010491 · Cited by 24
“Thirdly, it used a different tokenization algorithm: while BERT used WordPiece [54], XLM-RoBERTa used SentencePiece [55].”
MethodEmerging trends: Subwords, seriously?
Church, K. (2020) · Natural Language Engineering · doi:10.1017/s1351324920000145 · Cited by 18
“Word pieces are being used for a variety of applications: speech (Schuster and Nakajima 2012), translation (Wu et al.”
BackgroundImproving The Latency And Quality Of Cascaded Encoders
Sainath, T. N., He, Y., Narayanan, A., Botros, R., Wang, W., Qiu, D., Chiu, C., Prabhavalkar, R., Gruenstein, A., Gulati, A., et al. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9747879 · Cited by 25
Panoutsopoulos, H., Espejo-García, B., Raaijmakers, S. A., Wang, X., Fountas, S., & Brewster, C. (2024) · Computers and Electronics in Agriculture · doi:10.1016/j.compag.2024.109268 · Cited by 18
Krishna, A., Santra, B., Bandaru, S. P., Sahu, G., Sharma, V. D., Satuluri, P., & Goyal, P. (2018) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d18-1276 · Cited by 24
“cal tagging is not straightforward, as they learn a new sub-word vocabulary using the sentencepiece model (Schuster and Nakajima, 2012).”
BackgroundMøller, A. G., van der Goot, R., & Plank, B. (2020) · Workshop on Noisy User-generated Text (W-NUT) · doi:10.18653/v1/2020.wnut-1.44 · Cited by 20
“Besides the standard word and character n-grams, we also evaluate wordpiece n-grams (Schuster and Nakajima, 2012).”
MethodMulti-Task Language Modeling for Improving Speech Recognition of Rare Words
Yang, C.-H. H., Liu, L. L., Gandhe, A., Gu, Y., Raju, A., Filimonov, D., & Bulyko, I. (2021) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru51503.2021.9688282 · Cited by 18
Leveraging Synthetic Data and PU Learning For Phishing Email Detection
Qachfar, F. Z., Verma, R. M., & Mukherjee, A. (2022) · ACM Conference on Data and Application Security and Privacy (CODASPY) · doi:10.1145/3508398.3511524 · Cited by 18
“For our approach, we start by preprocessing the email content using the uncased base bert tokenizer with reference to theWordPiece algorithm [34] that was initially intended to solve the Japanese and Korean voice search problem.”
MethodFujii, T., Shibata, K., Yamaguchi, A., Morishita, T., & Sogawa, Y. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-srw.5 · Cited by 35
Incorporating Context into Subword Vocabularies
Yehezkel, S., & Pinter, Y. (2023) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2023.eacl-main.45 · Cited by 47
Boundless Byte Pair Encoding: Breaking the Pre-tokenization BarrierPreprint
Schmidt, C. W., Reddy, V., Tanner, C., & Pinter, Y. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2504.00178 · Cited by 42
“Using the widely adopted BPE, WordPiece (Wu et al., 2016; Schuster & Nakajima, 2012) and UnigramLM (Kudo, 2018) tokenizers, 3 with vocabulary sizes of 40,960, 65,536, 98,304, and 131,072, we calculated the proportion of pretokens that were ultimately represented by a single token.”
Text Counterfactuals via Latent Optimization and Shapley-Guided Search
Fern, X. Z., & Pope, Q. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.452 · Cited by 27
“HotFlip implemented in TextAttack only works for word or characterlevel classifiers, not WordPiece (Schuster and Nakajima, 2012) classifiers like RoBERTa and BERT.”
MethodCaines, A., Yannakoudakis, H., Allen, H., Pérez-Paredes, P., Byrne, B., & Buttery, P. J. (2022) · Linköping electronic conference proceedings · doi:10.3384/ecp190003 · Cited by 38
“3Note that tokens in the context of transformer language models are ‘subword tokens’ automatically derived from training corpora via byte-pair encoding or an algorithm such as WordPiece (Gage, 1994; Schuster and Nakajima, 2012).”
MethodDLInfer: Deep Learning with Static Slicing for Python Type Inference
Yan, Y., Feng, Y., Fan, H., & Xu, B. (2023) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse48619.2023.00170 · Cited by 21
“To alleviate the OOV, researchers have proposed several techniques to segment rare words into sub-word units [68]– [71].”
MethodModeling essay grading with pre-trained BERT features
Sharma, A., & Jayagopi, D. B. (2024) · Applied Intelligence · doi:10.1007/s10489-024-05410-4 · Cited by 17
Building a Word Segmenter for Sanskrit Overnight
Reddy, V., Krishna, A., Sharma, V., Gupta, P., Vineeth, M. R., & Goyal, P. (2018) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/4xp7pwb4iff6 · Cited by 17
Summarizing Legal Rulings: Comparative Experiments
Feijó, D. D. V., & Moreira, V. P. (2019) · International conference Recent advances in natural language processing · doi:10.26615/978-954-452-056-4_036 · Cited by 22
“The alternative to mitigate the disadvantages of these two approaches is to use sub-word units as the token representation (Schuster and Nakajima, 2012; Chitnis and DeNero, 2015; Sennrich et al., 2015; Kudo, 2018).”
MethodZero‐anaphora resolution in Korean based on deep language representation model: BERT
Kim, Y., Ra, D.-Y., & Lim, S. (2020) · ETRI Journal · doi:10.4218/etrij.2019-0441 · Cited by 17
“The word-piece model is used for tokenization in Google BERT [31,32], while byte-pair encoding is used in the BERT specialized for Korean [33].”
MethodDoutre, T., Han, W., Ma, M., Lu, Z., Chiu, C., Pang, R., Narayanan, A., Misra, A., Zhang, Y., & Cao, L. (2021) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp39728.2021.9413692 · Cited by 17
AnomalyAdapters: Parameter-Efficient Multi-Anomaly Task Detection
Ünal, U., & Dağ, H. (2022) · IEEE Access · doi:10.1109/access.2022.3141161 · Cited by 21
“In contrast to related studies, we use Byte-Pair Encoding [13] instead of WordPiece [14] in tokenization.”
ResultAlleviating ASR Long-Tailed Problem by Decoupling the Learning of Representation and Classification
Deng, K., Cheng, G., Yang, R., & Yan, Y. (2021) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2021.3138707 · Cited by 17
Xu, H., Chen, Y., Liu, Z., Wen, Y., & Yuan, X. (2022) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2022/615 · Cited by 17
“…slot [X] with the name of input concept c : Then, TaxoPrompt feeds the prompt into the LM for word tokenization using algorithm like WordPiece [ Schuster and Nakajima, 2012] and gets a prompt sentence s with n tokens: where t 1: i is the left-to-right context for masked positions and t j : n…”
MethodCompositional Evaluation on Japanese Textual Entailment and Similarity
Yanaka, H., & Mineshima, K. (2022) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00518 · Cited by 41
“co/cl-tohoku/bert-large-japanese (SUBWORD) setting, the model processes input texts with word-level tokenization by the MeCab morphological parser with a standard Japanese dictionary IPAdic (Asahara and Matsumoto, 2003), followed by WordPiece subword tokenization (Schuster and Nakajima, 2012).”
MethodModular Hybrid Autoregressive Transducer
Meng, Z., Chen, T., Prabhavalkar, R., Zhang, Y., Wang, G., Audhkhasi, K., Emond, J., Strohman, T., Ramabhadran, B., Huang, W., et al. (2023) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt54892.2023.10023194 · Cited by 17
“The label decoder output is projected to 4095-dim output layer predicting word pieces [51].”
Weighted combination of BERT and N-GRAM features for Nuanced Arabic Dialect Identification
Mekki, A. E., Alami, A., Alami, H., Khoumsi, A., & Berrada, I. (2020) · Cited by 17
Residual Energy-Based Models for End-to-End Speech Recognition
Li, Q., Zhang, Y., Li, B., Cao, L., & Woodland, P. C. (2021) · Interspeech (USB) · doi:10.21437/interspeech.2021-690 · Cited by 17
Tokenization for Molecular Foundation Models
Wadell, A., Bhutani, A., & Viswanathan, V. (2026) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.5c01856 · Cited by 17
DTT: An Example-Driven Tabular Transformer for Joinability by Leveraging Large Language Models
Nobari, A. D., & Rafiei, D. (2024) · Proceedings of the ACM on Management of Data · doi:10.1145/3639279 · Cited by 16
“Inan ideal setting where there is no noise or inconsistency in the data and the model performs with no error, all of the predicted values for a specific source 𝑠 𝑖 should be the same, i.e. 𝑜 𝑖 1 = 𝑜 𝑖 2 = . . . = 𝑜 𝑖𝑛 .”
BackgroundImproving Neural Machine Translation by Incorporating Hierarchical Subword Features
Morishita, M., Suzuki, J., & Nagata, M. (2018) · International Conference on Computational Linguistics · Cited by 24
Better Neural Machine Translation by Extracting Linguistic Information from BERT
Shavarani, H. S., & Sarkar, A. (2021) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2021.eacl-main.241 · Cited by 18
Zhao, S., & Birbilis, N. (2023) · npj Materials Degradation · doi:10.1038/s41529-022-00319-0 · Cited by 16
Adaptive language model training for molecular design
Blanchard, A. E., Bhowmik, D., Fox, Z., Gounley, J. P., Gläser, J., Akpa, B. S., & Irle, S. (2023) · Journal of Cheminformatics · doi:10.1186/s13321-023-00719-7 · Cited by 19
“Starting with a text-based representation for molecules, Simplified Molecular Input Line Entry System (SMILES) [28], the process of tokenization is used to determine commonly occurring subsequences [29, 30].”
MethodConstructing sub-word units for spoken term detection
van Heerden, C., Karakos, D., Narasimhan, K., Davel, M. H., & Schwartz, R. M. (2017) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2017.7953264 · Cited by 16
“As has been documented in a number of papers [1, 2, 3, 4, 5, 6, 7], the best techniques for OOV detection involve decoding with a variety of units, such as syllables, morphemes and 1-n phone units.”
MethodRETRACTED: Consumer sentiment analysis with aspect fusion and GAN‐BERT aided adversarial learning
Jain, P. K., Quamer, W., & Pamula, R. (2023) · Expert Systems · doi:10.1111/exsy.13247 · Cited by 15
“The BERT components are the transformer (Vaswani et al., 2017) and an attention-based approach that determines the word's contextual relationships, sub-words in a sentence (Schuster & Nakajima, 2012).”
BackgroundRetractedsustain.AI: a Recommender System to analyze Sustainability Reports
Hillebrand, L., Pielka, M., Leonhard, D., Deußer, T., Dilmaghani, T., Kliem, B., Loitz, R., Morad, M., Temath, C., Bell, T., et al. (2023) · International Conference on Artificial Intelligence and Law · doi:10.1145/3594536.3595131 · Cited by 15
“Formally, we first apply WordPiece [15] tokenization to transform an examplary input segment s into a sequence of sub-word tokens t = ([CLS], t1, .”
MethodLarge language models and unsupervised feature learning: implications for log analysis
Karlsen, E., Luo, X., Zincir-Heywood, N., & Heywood, M. I. (2024) · Annals of Telecommunications · doi:10.1007/s12243-024-01028-2 · Cited by 18
Transformers for molecular property prediction: domain adaptation efficiently improves performance
Sultan, A., Rausch-Dupont, M., Khan, S. R., Kalinina, O., Klakow, D., & Volkamer, A. (2026) · Journal of Cheminformatics · doi:10.1186/s13321-026-01252-z · Cited by 15
“We also used absolute positional embed-dings and the same tokenizer (i.e., WordPiece [48, 49]) as BERT.”
Parallel Rescoring with Transformer for Streaming On-Device Speech Recognition
Li, W., Qin, J., Chiu, C., Pang, R., & He, Y. (2020) · Interspeech (USB) · doi:10.21437/interspeech.2020-2875 · Cited by 16
Real-Time Social Media Analytics with Deep Transformer Language Models: A Big Data Approach
Ahmet, A., & Abdullah, T. (2020) · International Conference on Big Data Science and Engineering · doi:10.1109/bigdatase50710.2020.00014 · Cited by 15
“BERT introduced Wordpiece [7] which processes words out of vocabulary using a segmentation algorithm and eliminates the need for traditional data pre-processing.”
MethodOn the hidden treasure of dialog in video question answering
Engin, D., Schnitzler, F., Duong, N. Q. K., & Avrithis, Y. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00207 · Cited by 15
“Tokens are represented by WordPiece embeddings [28, 35], concatenated with position embeddings representing their position in the input sequence and segment embeddings, where segments correspond to sentences and are defined according to occurrences of the separator token [SEP].”
BackgroundStatic Embeddings as Efficient Knowledge Bases?
Dufter, P., Kassner, N., & Schütze, H. (2021) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2021.naacl-main.186 · Cited by 25
Making Punctuation Restoration Robust and Fast with Multi-Task Learning and Knowledge Distillation
Hentschel, M., Tsunoo, E., & Okuda, T. (2021) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp39728.2021.9414518 · Cited by 17
“Each model was trained using sequences of 512 wordpiece to-kens [23] of their pre-trained tokeniser.”
MethodUnsupervised Data Selection via Discrete Speech Representation for ASR
Lu, Z., Wang, Y., Zhang, Y., Han, W., Chen, Z., & Haghani, P. (2022) · Interspeech (USB) · doi:10.21437/interspeech.2022-399 · Cited by 15
“For transcript tokenization, we use a 1024-token WordPiece model [43] for all English experiments, and 4096-token WordPiece models for French, German, and Spanish experiments.”
MethodNLPashto: NLP Toolkit for Low-resource Pashto Language
Haq, I., Qiu, W., Jie, G., & Tang, P. (2023) · International Journal of Advanced Computer Science and Applications · doi:10.14569/ijacsa.2023.01406142 · Cited by 15
“To tokenize the input sequences, we used WordPiece tokenizer [26], which is the recommended tokenizer for BERTBase.”
MethodWord, Subword or Character? An Empirical Study of Granularity in Chinese-English NMT
Wang, Y., Zhou, L., Zhang, J., & Zong, C. (2017) · Communications in computer and information science · doi:10.1007/978-981-10-7134-8_4 · Cited by 15
Shin, D., Kam, H. J., Jeon, M.-S., & Kim, H. Y. (2021) · JMIR Medical Informatics · doi:10.2196/30223 · Cited by 15
“WordPiece [31-33], which was originally introduced for Japanese/Korean segmentation, was employed for the transformer [34] encoder–based models such as BERT and ELECTRA for various purposes.”
MethodSpeech Corpus of Ainu Folklore and End-to-end Speech Recognition for Ainu LanguagePreprint
Matsuura, K., Ueno, S., Mimura, M., Sakai, S., & Kawahara, T. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.06675 · Cited by 18
“In addition to these both models, a word piece unit, which is defined by automatically dividing a word into frequent parts, has been proposed (Schuster and Nakajima, 2012; Lüscher et al., 2019), and its vocabulary size can be determined almost freely.”
MethodZeineldeen, M., Zeyer, A., Zhou, W. G., Ng, T., Schlüter, R., & Ney, H. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2005.09336 · Cited by 16
“In all cases, these models usually operate on • graphemes (characters), • subword units ( byte-pair encoding (BPE) [6, 20, 21], WordPieces / word piece model (WPM) [7,22,23], uni-gram language model (LM) based segmentation [24,25], or pronunciation-based segmentation [26]) • or whole words [27–32].”
BackgroundLanguage model fusion for streaming end to end speech recognitionPreprint
Cabrera, R., Liu, X., Ghodsi, M., Matteson, Z., Weinstein, E., & Kannan, A. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.04487 · Cited by 15
“It directly predicts a sequence of words, subwords [19] or graphemes without using an external pronunciation model a r X i v : 2104 .”
BackgroundHow Much is Enough? The Diminishing Returns of Tokenization Training DataPreprint
Reddy, V., Schmidt, C. W., Pinter, Y., & Tanner, C. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2502.20273 · Cited by 19
“Copyright 2025 by the author(s). fault to standard tokenization algorithms such as Byte-Pair Encoding (BPE; Sennrich et al., 2016), UnigramLM (Kudo, 2018) or WordPiece (Schuster & Nakajima, 2012; Devlin et al., 2019), sourced directly from libraries such as Hugging Face.”
InfluentialReal-Time Anomaly Detection Using Distributed Tracing in Microservice Cloud Applications
Raeiszadeh, M., Ebrahimzadeh, A., Saleem, A., Glitho, R., Eker, J., & Mini, R. A. F. (2023) · IEEE International Conference on Cloud Networking · doi:10.1109/cloudnet59005.2023.10490038 · Cited by 20
“To handle the dynamic nature of service and operation names in microservices systems, we employ the WordPiece algorithm [14] for tokenization, which splits words into tokens based on character combinations.”
Training Experimental Language Models with Low Resources, for the Hungarian Language
Yang, Z. G., & Váradi, T. (2023) · Acta Polytechnica Hungarica · doi:10.12700/aph.20.5.2023.5.11 · Cited by 18
Petrillo, L., Martinelli, F., Santone, A., & Mercaldo, F. (2024) · Electronics · doi:10.3390/electronics13204057 · Cited by 16
Tokenization and deep learning architectures in genomics: A comprehensive review
Testagrose, C., & Boucher, C. A. (2025) · Computational and Structural Biotechnology Journal · doi:10.1016/j.csbj.2025.07.038 · Cited by 15
Reducing the computational complexity for whole word models
Soltau, H., Liao, H., & Sak, H. (2017) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru.2017.8268917 · Cited by 14
“While we use an OOV symbol to cover non modeled words in training, word piece models [16] might be a good compromise between long units and good coverage.”
BackgroundMethodAlleviating the Burden of Labeling: Sentence Generation by Attention Branch Encoder–Decoder Network
Ogura, T., Magassouba, A., Sugiura, K., Hirakawa, T., Yamashita, T., Fujiyoshi, H., & Kawai, H. (2020) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2020.3010735 · Cited by 15
Low Latency Speech Recognition Using End-to-End Prefetching
Chang, S.-Y., Li, B., Rybach, D., He, Y., Li, W., Sainath, T. N., & Strohman, T. (2020) · Interspeech (USB) · doi:10.21437/interspeech.2020-1898 · Cited by 14
“Both decoders are trained to predict 4,096 word pieces [22].”
BackgroundA Statistical Extension of Byte-Pair Encoding
Vilar, D., & Federico, M. (2021) · International Conference on Spoken Language Translation (IWSLT) · doi:10.18653/v1/2021.iwslt-1.31 · Cited by 26
“An alternative model to BPE which is also widely used was presented by Kudo (2018), which can be considered as an extension of (Schuster and Nakajima, 2012).”
MethodLamproudis, A., Henriksson, A., & Dalianis, H. (2021) · International conference Recent advances in natural language processing · doi:10.26615/978-954-452-072-4_090 · Cited by 14
“This tokenizer is a word piece tokenizer, as described by Schuster and Nakajima (2012), and is responsible for constructing the vocabulary – gradually building it from the character level and upwards – by maximizing the likelihood of the training data with respect to the vocabulary.”
MethodTowards automating Numerical Consistency Checks in Financial Reports
Hillebrand, L., Deußer, T., Dilmaghani, T., Kliem, B., Loitz, R., Bauckhage, C., & Sifa, R. (2022) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata55660.2022.10020308 · Cited by 14
“1) Sentence Encoder: A pre-trained BERT [7] model encodes a WordPiece [28] tokenized sentence to obtain a contextualized sub-word token embedding sequence.”
MethodHuang, J., Wu, Z., Mahowald, K., & Potts, C. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.770 · Cited by 24
A Contextual Dependency-Aware Graph Convolutional Network for extracting entity relations
Liao, J., Du, Y., Hu, J., Li, H., Li, X., & Chen, X. (2022) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.122366 · Cited by 15
Deep learning workflow for the inverse design of molecules with specific optoelectronic properties
Yoo, P., Bhowmik, D., Mehta, K. R., Zhang, P., Liu, F., Pasini, M. L., & Irle, S. (2023) · Scientific Reports · doi:10.1038/s41598-023-45385-9 · Cited by 17
Tezcan, A., Bulté, B., & Vanroy, B. (2021) · Informatics · doi:10.3390/informatics8010007 · Cited by 19
“WordPiece uses the same approach for vocabulary reduction as BPE, with the difference that the merge choice is based on likelihood maximisation rather than frequency [69].”
MethodAutoKG - An Automotive Domain Knowledge Graph for Software Testing: A position paper
Kesri, V., Nayak, A., & Ponnalagu, K. (2021) · IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW) · doi:10.1109/icstw52544.2021.00047 · Cited by 14
“Dynamic embedding models such as BERT provide contextualized representations for a word and can even handle OOV words as it uses the WordPiece tok-enization algorithm [11] to break down even previously unseen words.”
MethodTowards End-to-End Speech Recognition
Palaz, D. (2016) · Infoscience (Ecole Polytechnique Fédérale de Lausanne) · doi:10.5075/epfl-thesis-7054 · Cited by 15
Finding Generalizable Evidence by Learning to Convince Q&A ModelsPreprint
Perez, E., Karamcheti, S., Fergus, R., Weston, J., Kiela, D., & Cho, K. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.05863 · Cited by 15
Grid Search Hyperparameter Benchmarking of BERT, ALBERT, and LongFormer on DuoRCPreprint
Quijano, A. J., Nguyen, S., & Ordoñez, J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2101.06326 · Cited by 14
“The entire tokenization process is done using WordPiece tokenization [15].”
MethodExploring Tokenization Strategies and Vocabulary Sizes for Enhanced Arabic Language ModelsPreprint
Alrefaie, M. T., Morsy, N. E., & Samir, N. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2403.11130 · Cited by 18
“In the realm of Arabic Natural Language Processing (NLP), subword tokenization stands as a prevalent choice, evident in the adoption of WordPiece by HULMonA and SentencePiece by AraBERT.”
BackgroundInfluentialHasan, M. J., Rahman, F., & Mohammed, N. (2024) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2024.105764 · Cited by 16
Large language models – the future of fundamental physics?
Heneka, C. S., Nieser, F., Ore, A., Plehn, T., & Schiller, D. (2026) · SciPost Physics · doi:10.21468/scipostphys.20.3.070 · Cited by 14
“Word-Piece tokenization [ 55, 56 ] also extends a base vocabulary, but instead of merging tokens by frequency, it merges them based on high mutual information between them.”
Analyzing the Quality and Stability of a Streaming End-to-End On-Device Speech Recognizer
Shangguan, Y., Knister, K., He, Y., Mcgraw, I. C., & Beaufays, F. (2020) · Interspeech (USB) · doi:10.21437/interspeech.2020-1194 · Cited by 13
A Vocabulary-Free Multilingual Neural Tokenizer for End-to-End Task Learning
Islam, M. M., Aguilar, G., Ponnusamy, P., Mathialagan, C. S., Ma, C., & Guo, C. (2022) · Workshop on Representation Learning for NLP (RepL4NLP) · doi:10.18653/v1/2022.repl4nlp-1.10 · Cited by 16
SPRoBERTa: protein embedding learning with local fragment modeling
Wu, L., Yin, C., Zhu, J., Wu, Z., He, L., Xia, Y., Xie, S., Qin, T., & Liu, T. (2022) · Briefings in Bioinformatics · doi:10.1093/bib/bbac401 · Cited by 13
“Widely used subword encoding algorithms include Byte-Pair Encoding (BPE) [20], SentencePiece [5, 21] and WordPiece [22], and they all split sentences into subword units by statistics of consequent characters.”
MethodWang, X., Macdonald, C., Tonellotto, N., & Ounis, I. (2023) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3539618.3591916 · Cited by 13
“Prevalent tokenisation techniques used by large pretrained language models include WordPiece [31], Byte-Pair En-coding (BPE) [32] and SentencePiece [14] tokenisation techniques.”
MethodInfluentialAnalyzing Cognitive Plausibility of Subword Tokenization
Beinborn, L., & Pinter, Y. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.272 · Cited by 50
Character-Aware Attention-Based End-to-End Speech Recognition
Meng, Z., Gaur, Y., Li, J., & Gong, Y. (2019) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru46091.2019.9004018 · Cited by 13
“One popular type of WSUs is the word pieces model generated by iteratively combining two units out of the current inventory that increase the likelihood the most on the training data [13, 16].”
MethodModel Unit Exploration for Sequence-to-Sequence Speech Recognition.Preprint
Irie, K., Prabhavalkar, R., Kannan, A., Bruguier, A., Rybach, D., & Nguyen, P. (2019) · arXiv (Cornell University) · Cited by 13
AdaptiVocab: Enhancing LLM Efficiency in Focused Domains through Lightweight Vocabulary AdaptationPreprint
Nakash, I., Calderon, N., David, E. B., Hoffer, E., & Reichart, R. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2503.19693 · Cited by 16
“The vocabulary, which consists of this set of tokens, is Table 1: Tokenization before and after vocabulary adaptation to the History of Physics domain. typically selected using widely adopted methods such as Byte Pair Encoding (BPE) (Sen-nrich et al., 2016; Schuster & Nakajima, 2012; Kudo, 2018).”
Huawei’s NMT Systems for the WMT 2019 Biomedical Translation Task
Peng, W., Liu, J., Li, L., & Liu, Q. (2019) · Conference on Machine Translation (WMT) · doi:10.18653/v1/w19-5420 · Cited by 12
“It is generated by the WordPiece (Schuster and Nakajima, 2012) model trained on Wikipedia dump.”
MethodTokenization as Finite-State Transduction
Cognetta, M., & Okazaki, N. (2025) · Computational Linguistics · doi:10.1162/coli.a.23 · Cited by 15
Acoustic Data-Driven Subword Modeling for End-to-End Speech Recognition
Zhou, W. G., Zeineldeen, M., Zheng, Z., Schlüter, R., & Ney, H. (2021) · Interspeech (USB) · doi:10.21437/interspeech.2021-1623 · Cited by 17
Drozdov, I. A., Szubert, B., Reda, E., Makary, P., Forbes, D., Chang, S. L., Ezhil, A., Puttagunta, S. A., Hall, M., Carlin, C. M., et al. (2021) · Scientific Reports · doi:10.1038/s41598-021-99986-3 · Cited by 12
Both, M., Müller, J. M., & Diedrich, C. (2021) · at - Automatisierungstechnik · doi:10.1515/auto-2021-0050 · Cited by 12
Optimal transformers based image captioning using beam search
Shetty, A., Kale, Y., Patil, Y., Patil, R., & Sharma, S. (2023) · Multimedia Tools and Applications · doi:10.1007/s11042-023-17359-6 · Cited by 13
Automating petition classification in Brazil’s legal system: a two-step deep learning approach
Costa, Y. D. R., Oliveira, H., Nogueira, V., Massa, L., Yang, X., Barbosa, A., Oliveira, K., & Vieira, T. M. D. A. (2023) · Artificial Intelligence and Law · doi:10.1007/s10506-023-09385-4 · Cited by 12
“…removing non alpha-numeric symbols from the text (including punctuation) and then tokenizing the text into a sequence of integers by using word tokenization or WordPiece tokenization, a subword segmentation algorithm that decomposes rare words into meaningful subwords (Schuster and Nakajima 2012).”
Metadata
- Authors
- Schuster, M., & Nakajima, K.
- Affiliations
- Google (United States) (US)
- Venue
- 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
- Publisher
- IEEE
- Published
- 2012-03
- Pages
- 5149-5152
- ISSN
- 1520-6149 (ISSN-L)
- OpenAlex
- W2121879602
- Cited by
- 862
- References
- 8
- Citations by year
- 2026: 36 · 2025: 66 · 2024: 90 · 2023: 124 · 2022: 137 · 2021: 155 · 2020: 144 · 2019: 59 · 2018: 36 · 2017: 10 · 2016: 3 · 2013: 2
- FWCI
- 0.49
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- IEEE International Conference on Acoustics Speech and Signal Processing (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
- Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Keywords
- voice search, Japanese language, Korean language, acoustic modeling
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.