Japanese and Korean voice search
The DOI matches its Crossref record; 7 of the 8 references it cites were found in a registry or an index.
Abstract
This paper describes challenges and solutions for building a successful voice search system as applied to Japanese and Korean at Google. We describe the techniques used to deal with an infinite vocabulary, how modeling completely in the written domain for language model and dictionary can avoid some system complexity, and how we built dictionaries, language and acoustic models in this framework. We show how to deal with the difficulty of scoring results for multiple script languages because of ambiguities. The development of voice search for these languages led to a significant simplification of the original process to build a system for any new language which in in parts became our default process for internationalization of voice search.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“LogRobust [8], which encodes log templates into semantic vectors using the FastText pre-trained model [31], cannot work well on 2 out of 4 datasets.”
From Log-based Anomaly Detection Without Log Parsing (2021) · cites this paper as [8]
In the citing paper
“Their empirical study on a Microsoft online service system shows that up to 30.3% logs are changed in the latest version.”
From Log-based Anomaly Detection Without Log Parsing (2021) · cites this paper as [8]
In the citing paper
“Zhang et al. [8] indicated that log data is unstable, meaning that new log events often appear due to software evolution at its lifetime.”
From Log-based Anomaly Detection Without Log Parsing (2021) · cites this paper as [8]
In the citing paper
“Another option is to use domain knowledge to reduce the input space of a model — such strategies include patch embeddings [Dosovitskiy et al. 2020] as well as fixed or learned tokenization [Schuster and Nakajima 2012; Sennrich et al. 2016; Kudo and Richardson 2018; van den Oord et al. 2017; Ramesh et al. 2021]. These methods may alleviate some challenges facing generative approaches, but have the”
From On the Opportunities and Risks of Foundation Models (2021) · cites this paper as Schuster and Nakajima 2012
Citations
Cites
Semi-tied covariance matrices for hidden Markov models
Gales, M. J. F. (1999) · IEEE Transactions on Speech and Audio Processing · doi:10.1109/89.759034 · Cited by 537
Boosted MMI for model and feature-space discriminative training
Povey, D., Kanevsky, D., Kingsbury, B., Ramabhadran, B., Saon, G., & Visweswariah, K. (2008) · 2008 IEEE International Conference on Acoustics, Speech and Signal Processing · doi:10.1109/icassp.2008.4518545 · Cited by 341
“Your Word is my Command”: Google Search by Voice: A Case Study
Schalkwyk, J., Beeferman, D., Beaufays, F., Byrne, B., Chelba, C., Cohen, M., Kamvar, M., & Strope, B. (2010) · Advances in Speech Recognition · doi:10.1007/978-1-4419-5951-5_4 · Cited by 232
Building transcribed speech corpora quickly and cheaply for many languages
Hughes, T., Nakajima, K., Ha, L., Vasu, A., Moreno, P. J., & LeBeau, M. (2010) · Interspeech (USB) · doi:10.21437/interspeech.2010-551 · Cited by 75
Self-citationGoogle Search by Voice: A Case Study
Schalkwyk, J., Beeferman, D., Beaufays, F., Byrne, B., Chelba, C. I., Cohen, M., Garrett, M., & Strope, B. P. (2010) · Cited by 63
Query language modeling for voice search
Chelba, C. I., Schalkwyk, J., Brants, T., Ha, V., Harb, B., Neveitt, W., Parada, C., & Xu, P. (2010) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt.2010.5700834 · Cited by 28
Search by voice in Mandarin Chinese
Shan, J., Wu, G., Hu, Z., Tang, X., Jansche, M., & Moreno, P. J. (2010) · Interspeech (USB) · doi:10.21437/interspeech.2010-129 · Cited by 18
Research works printed but not linked: 1
- sung, 0, Deploying Search by Voice in Cantonese, Proc Interspeech 2011
Cited by
Items 501–600 of 933
Should you marginalize over possible tokenizations?
Chirkova, N., Kruszewski, G., Rozen, J., & Dymetman, M. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-short.1 · Cited by 21
nanoT5: Fast & Simple Pre-training and Fine-tuning of T5 Models with Limited Resources
Nawrot, P. (2023) · Workshop for NLP Open Source Software (NLP-OSS) · doi:10.18653/v1/2023.nlposs-1.11 · Cited by 15
Assessing Grammatical Correctness in Language Learning
Katinskaia, A., & Yangarber, R. (2021) · Työväentutkimus Vuosikirja · Cited by 7
Alshamrani, S. S., Abusnaina, A., Abuhamad, M., Lee, A., Nyang, D., & Mohaisen, A. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-66046-8_7 · Cited by 7
“This technique has been proven efficient as opposed to other embedding mechanisms that map all of the out-of-vocabulary words to one token such as ‘UNK’ [15, 21].”
MethodClinical de-identification using sub-document analysis and ELECTRA
Catelli, R., Gargiulo, F., Damiano, E., Esposito, M., & Pietro, G. D. (2021) · IEEE International Conference on Digital Health (ICDH) · doi:10.1109/icdh52753.2021.00050 · Cited by 8
Representation Learning: Recommendation With Knowledge Graph via Triple-Autoencoder
Geng, Y., Xiao, X., Sun, X., & Zhu, Y. Q. (2022) · Frontiers in Genetics · doi:10.3389/fgene.2022.891265 · Cited by 12
“Secondly, we then tokenize the text using the WordPiece tokenizer (Schuster and Nakajima, 2012).”
MethodDiscovering Interdisciplinary Research Based on Neural Networks
He, T., Fu, W., Xu, J., Zhang, Z., Zhou, J., Yin, Y., & Xie, Z. (2022) · Frontiers in Bioengineering and Biotechnology · doi:10.3389/fbioe.2022.908733 · Cited by 7
“Before inputting the author keyword into IRD-BERT, they need to be expressed in the form of wordpiece (Schuster et al., 2012).”
MethodEnd-to-End Speech Recognition Models
Chan, W. (2016) · KiltHub Repository · doi:10.1184/r1/6716237 · Cited by 8
Situating Sentence Embedders with Nearest Neighbor OverlapPreprint
Lin, L. H., & Smith, N. A. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.10724 · Cited by 9
“Subword information is incorporated using the WordPiece model (Schuster and Nakajima, 2012).”
Understanding character-level models for representing morphology
Vania, C. (2020) · ERA · doi:10.7488/era/49 · Cited by 13
“A similar word segmentation algorithm (Schuster and Nakajima, 2012) is used by Google’s neural machine translation system (Wu et al., 2016) and BERT pretrained language model (Devlin et al., 2019) to segment word into subword units called “wordpieces”.”
MethodValueNet: A Neural Text-to-SQL Architecture Incorporating ValuesPreprint
Brunner, U., & Stockinger, K. (2020) · arXiv (Cornell University) · Cited by 10
Chen, T., Vahab, N., Tyagi, N., Cummins, E., Peleg, A. Y., & Tyagi, S. (2023) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2023.05.31.542682 · Cited by 8
Action abstractions for amortized samplingPreprint
Boussif, O., Ezzine, L. N., Viviano, J. D., Koziarski, M., Jain, M., Whitammer, E. S., Bengio, E., Assouel, R., & Bengio, Y. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2410.15184 · Cited by 7
“WordPiece (Schuster & Nakajima, 2012) works similarly to BPE where a pair is only merged if the ratio of the likelihood of its joint to that the product of its marginals is the highest.”
Chen, D., Chen, H., Yang, Y., Lin, A., & Yu, Z. (2021) · arXiv · doi:10.48550/arxiv.2104.00783 · Cited by 6
“More specifically, given the dialogue history as a series of utterances, we first join the utterances together with a [SEP] token and then tokenize the entire input using WordPiece (Schuster and Nakajima, 2012).”
MethodAn Efficient Algorithm for Streaming BPE Tokenization
Mamouras, K., Li, A. W., & Yang, Y. (2026) · Proceedings of the ACM on Programming Languages · doi:10.1145/3808330 · Cited by 8
Cheng, L., Jia, W., & Yang, W. (2022) · ACM Transactions on Information Systems · doi:10.1145/3545800 · Cited by 6
“Considering that SNIPS contains much more Out-of-Vocabulary (OOV) tokens, we attribute the improvement of BERT to alleviating the OOV problem by the WordPiece encoding [41].”
MethodEnhancing Semantic Relation Classification With Shortest Dependency Path Reasoning
Li, J., Shuang, K., Guo, J., Shi, Z., & Wang, H. (2023) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2023.3265205 · Cited by 15
“In detail, a tokenizer with word segmentation algorithms such as BPE [35] and WordPiece [36] will convert a sentence to a sequence S , we feed it into the encoder to get the word embedding E word ∈ R N × e : Where e is the hidden dimension of the encoder.”
MethodPrabhavalkar, R., Meng, Z., Wang, W., Stooke, A., Cai, X., He, Y., Narayanan, A., Hwang, D., Sainath, T. N., & Moreno, P. J. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10446985 · Cited by 10
“All models tokenize the transcripts using 4096 word-pieces [35].”
MethodWataya, T., Miura, A., Sakisuka, T., Fujiwara, M., Tanaka, H., Hiraoka, Y., Sato, J., Tomiyama, M., Nishigaki, D., Kita, K., et al. (2024) · Japanese Journal of Radiology · doi:10.1007/s11604-024-01549-9 · Cited by 6
“Morphological analysis is followed by Word-piece [15] tokenization, in which some words, especially those rare in pretraining corpus, are separated into subwords, so that the number of token types becomes the designated number (a hyperparameter in pretrain).”
MethodThe Impact of Word Splitting on the Semantic Content of Contextualized Word Representations
Soler, A. G., Labeau, M., & Clavel, C. (2024) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00647 · Cited by 10
“Subword tokenization algorithms were first proposed by Schuster and Nakajima (2012) and became widespread after the adaptation of Byte Pair Encoding to word segmentation (Gage, 1994; Sen-nrich et al., 2016).”
MethodCroce, D., Smirnov, A., Tiburzi, L., Travaglini, S., Costa, R. M. D. S., Calabrese, A., Basili, R., Ghiron, N. L., & Melino, G. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.125126 · Cited by 7
Wang, J., Kim, J., Kim, S., & Lee, Y. (2020) · Interspeech (USB) · doi:10.21437/interspeech.2020-2440 · Cited by 7
An Investigation of Using Hybrid Modeling Units for Improving End-to-End Speech Recognition System
Chen, S., Hu, X., Li, S., & Xu, X. (2021) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp39728.2021.9414598 · Cited by 6
“Among the most two popular such algorithms WordPiece [19] and Byte Pair Encoding (BPE), the BPE is more robust and commonly used for creating subwords in ASR and machine translation.”
MethodAn Experience Report on Technical Debt in Pull Requests: Challenges and Lessons Learned
Karmakar, S., Codabux, Z., & Vidoni, M. C. (2022) · Proceedings of the ... ACM-IEEE International Symposium on Empirical Software Engineering and Measurement/International Symposium on Empirical Software Engineering and Measurement · doi:10.1145/3544902.3546637 · Cited by 9
“Additionally, we used WordPiece [34] and Byte-Pair Encoding (BPE) [12] with the state-of-the-art transformer models (DistilBERT [33], AlBERT [21], RoBERTa [40]).”
MethodContent-based Multiclass Classification on Indonesian SMS Messages
Nafiisah, H. R., & Ruskanda, F. Z. (2022) · International Symposium on Electronics and Smart Devices (ISESD) · doi:10.1109/isesd56103.2022.9980769 · Cited by 6
“We implemented WordPieceModel [17] for feature extraction in Java language and exported vocabularies from pretrained IndoBERT-base phase 2 tokenizer.”
MethodWingfield, C., Zhang, C., Devereux, B. J., Fonteneau, E., Thwaites, A., Liu, X., Woodland, P. C., Marslen‐Wilson, W. D., & Su, L. (2022) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2022.1057439 · Cited by 7
“, 2013), recent studies mostly rely on writtenform-based word piece units (Schuster and Nakajima, 2012; Wu et al., 2016) that are not directly associated with phonetic units.”
BackgroundCASBERT: BERT-based retrieval for compositely annotated biosimulation model entities
Munarko, Y., Rampadarath, A., & Nickerson, D. P. (2023) · Frontiers in Bioinformatics · doi:10.3389/fbinf.2023.1107467 · Cited by 6
“The tokens used are WordPieces (Schuster and Nakajima, 2012) rather than words, so they can more precisely represent unique words in the biosimulation modelling domain and are more resistant to typos.”
BackgroundImproving Numeracy by Input Reframing and Quantitative Pre-Finetuning Task
Chen, C.-C., Takamura, H., Kobayashi, I., & Miyao, Y. (2023) · Findings of the Association for Computational Linguistics: EACL · doi:10.18653/v1/2023.findings-eacl.4 · Cited by 6
“Original signifies that we did not perform any preprocessing on the input data, and the results are tokenized based on WordPiece (Schuster and Nakajima, 2012; Wu et al., 2016) and Byte-Pair Encoding (BPE) (Sennrich et al.”
MethodNepaliBERT: Pre-training of Masked Language Model in Nepali Corpus
Pudasaini, S., Shakya, S., Tamang, A., Adhikari, S., Thapa, S., & Lamichhane, S. (2023) · International Conference on I-SMAC (IoT in Social, Mobile, Analytics and Cloud) (I-SMAC) · doi:10.1109/i-smac58438.2023.10290690 · Cited by 14
“Among the various tokenization methods available, we used the WordPiece Tokenizer which was introduced by [12].”
MethodCytonMT: an Efficient Neural Machine Translation Open-source Toolkit Implemented in C++
Wang, X., Utiyama, M., & Sumita, E. (2018) · Conference on Empirical Methods in Natural Language Processing: System Demonstrations (EMNLP) · doi:10.18653/v1/d18-2023 · Cited by 7
“Both datasets are processed and converted using byte-pair encoding(Gage, 1994; Schuster and Nakajima, 2012) with a shared source-target vocabulary of about 37000 tokens.”
MethodStress Test Evaluation of Biomedical Word Embeddings
Araujo, V., Carvallo, A., Aspillaga, C., Thorne, C., & Parra, D. (2021) · Workshop on Biomedical Language Processing (BioNLP) · doi:10.18653/v1/2021.bionlp-1.13 · Cited by 9
“, 2017) and WordPiece tokenizer (Schuster and Nakajima, 2012) that allow it to learn better representations.”
BackgroundMachine Understandable Contracts with Deep Learning
Dolga, R., Treleaven, P. C., & Denny, M. T. (2020) · International Conference on Computational Science and Computational Intelligence (CSCI) · doi:10.1109/csci51800.2020.00099 · Cited by 6
“Furthermore, even if specific legal language vocabulary is not used, it can still be captured in different subwords by the WordPiece tokenizer [20].”
BackgroundPan, C., Sun, M., & Deng, K. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.13 · Cited by 6
“TopWORDS is closely related to methods widely used in neural machine translation for constructing sub-word dictionary, and can be viewed as an advanced version of WordPiece (Schuster and Nakajima, 2012), Byte Pair Encoding (Sennrich et al.”
MethodImproving Contextual Biasing with Text Injection
Sainath, T. N., Prabhavalkar, R., Caseiro, D., Rondon, P., & Allauzen, C. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10096287 · Cited by 6
“All models are trained with the Hybrid Autoregressive Transducer (HAT) factorization [28] to predict 4,096 word pieces [7].”
BackgroundMethodCross-lingual Transferring of Pre-trained Contextualized Language ModelsPreprint
Li, Z., Parnow, K., Zhao, H., Zhang, Z., Wang, R., Utiyama, M., & Sumita, E. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2107.12627 · Cited by 6
“For all models migrated in the same direction, regardless of their original vocabulary, we used the same single vocabulary that we trained on the joint language data using the WordPiece Subword scheme [38].”
MethodLarge Language Models in Psycholinguistic Studies
Günther, F., & Cassani, G. (2025) · Elsevier eBooks · doi:10.1016/b978-0-323-95504-1.00749-3 · Cited by 6
Morphology Without Borders: Clause-Level Morphology
Goldman, O., & Tsarfaty, R. (2022) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00528 · Cited by 13
“However, the lack of a word/token delimiting symbol in some languages prompted the development of more sophisticated tokenization methods, supervised (Xue, 2003; Nakagawa, 2004) or statistical (Schuster and Nakajima, 2012), mostly for east Asian languages.”
MethodSmishing Detection From a Messaging Platform View
Schwarz, S. F., Fonseca, P. D. T., & Rocha, A. D. R. (2025) · IEEE Access · doi:10.1109/access.2025.3597903 · Cited by 5
“Instead, this algorithm was primarily developed to handle out-of-vocabulary words by segmenting an input text via the longest-prefix match strategy [29].”
The cost of language: tokenization as a metric of labor
Caffoni, P. (2026) · Ai & Society · doi:10.1007/s00146-026-02990-2 · Cited by 3
“While not a compression algorithm technically speaking, WordPiece reduces the complexity of language modeling by encoding text in a more efficient and generalizable form. measurable units that can be operationalized within work-flows of machine learning.”
InfluentialInvestigation into Phone-Based Subword Units for Multilingual End-to-End Speech Recognition
Yusuyin, S., Huang, H., Liu, J., & Liu, C. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10096812 · Cited by 5
“BPE variants include using bytes instead of characters as input or estimating unigram probabilities for the BPE subwords [2].”
MethodMorpheme-Based Neural Machine Translation Models for Low-Resource Fusion Languages
Gezmu, A. M., & Nürnberger, A. (2023) · ACM Transactions on Asian and Low-Resource Language Information Processing · doi:10.1145/3610773 · Cited by 6
“However, the most conventional word segmentation methods commonly used in NMT [55] such as Byte Pair Encoding (BPE) [68], Word-Piece [67, 78], and Sentence Piece with Unigram Language Modeling (SPULM) [48] conform to agglutinative morphology.”
MethodImproving Speech Recognition for African American English with Audio Classification
Garg, S., Huo, Z., Sim, K. C., Schwartz, S., Chua, M., Aksënova, A., Munkhdalai, T., King, L., Wright, D., Mengesha, Z., et al. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10447116 · Cited by 5
A Method for AI-generated sentence detection through Large Language Models
Martinelli, F., Mercaldo, F., Petrillo, L., & Santone, A. (2024) · Procedia Computer Science · doi:10.1016/j.procs.2024.09.351 · Cited by 5
Verberne, S. (2024) · ACM eBooks · doi:10.1145/3674127.3674141 · Cited by 5
Trippas, J. R. (2024) · ACM eBooks · doi:10.1145/3674127.3674135 · Cited by 6
Sub-Subword N-Gram Features for Subword-Level Neural Machine Translation
Martinez, A., Sudoh, K., & Matsumoto, Y. (2021) · Journal of Natural Language Processing · doi:10.5715/jnlp.28.82 · Cited by 6
“One such approach is the WordPieceModel (Schuster and Nakajima 2012).”
A Deliberation-Based Joint Acoustic and Text Decoder
Mavandadi, S., Sainath, T. N., Hu, K., & Wu, Z. (2021) · Interspeech (USB) · doi:10.21437/interspeech.2021-165 · Cited by 5
“The second-pass attention decoder has a 4,096-dimensional softmax layer to predict the same mixedcase wordpieces [31] as the RNN-T.”
MethodTransducer-Based Streaming Deliberation for Cascaded Encoders
Hu, K., Sainath, T. N., Narayanan, A., Pang, R., & Strohman, T. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9746406 · Cited by 5
Hybrid sub-word segmentation for handling long tail in morphologically rich low resource languages
Manghat, S., Manghat, S., & Schultz, T. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9746652 · Cited by 6
Neural text summarization for Hungarian
Yang, Z. G. (2022) · Acta Linguistica Academica · doi:10.1556/2062.2022.00577 · Cited by 8
Sun, L., Florian Luisier, A., Batmanghelich, K. N., Florencio, D. A. F., & Zhang, C. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.200 · Cited by 18
Shin, J., Jo, E., Yoon, Y., & Jung, J. (2023) · Applied Sciences · doi:10.3390/app132011545 · Cited by 5
To token or not to token: A Comparative Study of Text Representations for Cross-Lingual Transfer
Rahman, M. M., Sakib, F. A., Faisal, F., & Anastasopoulos, A. (2023) · Workshop on Multilingual Representation Learning (MRL) · doi:10.18653/v1/2023.mrl-1.6 · Cited by 8
Now It Sounds Like You: Learning Personalized Vocabulary On Device
Shenoy, A., Wang, S., Chuang, P., & Nguyen, J. (2024) · Proceedings of the AAAI Symposium Series · doi:10.1609/aaaiss.v3i1.31224 · Cited by 5
Out-of-vocabulary handling and topic quality control strategies in streaming topic models
Nguyễn, T. L., Pham, T. T., Van, L. N., Ban, H.-B., & Than, K. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.128757 · Cited by 13
Speaker-Independent Automatic Speech Recognition System for Mobile Phone Applications in Punjabi
Mittal, P., & Singh, N. (2017) · Advances in intelligent systems and computing · doi:10.1007/978-3-319-67934-1_33 · Cited by 5
Parsimonious Morpheme Segmentation with an Application to Enriching Word Embeddings
El-Kishky, A., Xu, F. F., Zhang, A., & Han, J. (2019) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata47090.2019.9005957 · Cited by 10
“Baselines As a baseline for segmentation, we utilize a unigram language model segmentation of “word-pieces" and byte-pair encoding segmentation as described in the related work [39], [44].”
MethodAdversarial Subword Regularization for Robust Neural Machine Translation
Park, J., Sung, M., Lee, J., & Kang, J. (2020) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2020.findings-emnlp.175 · Cited by 9
Koehn, P. (2020) · Cambridge University Press eBooks · doi:10.1017/9781108608480.003 · Cited by 5
Emitting Word Timings with HMM-Free End-to-End System in Automatic Speech Recognition
Chen, X., Ni, H., He, Y., Wang, K., Ma, Z., & Xie, Z. (2021) · Interspeech (USB) · doi:10.21437/interspeech.2021-894 · Cited by 5
“W LAS and W CTC are the sets of word-piece [14] units, where a word-piece starting with a meta symbol (U+2581) indicates the beginning of a word.”
BackgroundIntent-based Product Collections for E-commerce using Pretrained Language Models
Kim, H., Jeong, J., Kim, K.-M., Lee, D., Lee, H. D., Seo, D., Han, J., Park, D. W., Heo, J. A., & Kim, R. Y. (2021) · IEEE ... International Conference on Data Mining workshops · doi:10.1109/icdmw53433.2021.00036 · Cited by 5
Shim, H., Lowet, D., Luca, S., & Vanrumste, B. (2021) · WNUT · doi:10.18653/v1/2021.wnut-1.29 · Cited by 6
“During pre-processing, we lowercased and tokenised data sets by using a WordPiece algorithm (Schuster and Nakajima, 2012).”
MethodThe boundaries of meaning: a case study in neural machine translation
Balashov, Y. V. (2022) · Inquiry · doi:10.1080/0020174x.2022.2113429 · Cited by 5
Semantic VL-BERT: Visual Grounding via Attribute Learning
Wanigasekara, P., Qin, K., Barut, E., Yang, F., Ruan, W., & Su, C. (2022) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn55064.2022.9892420 · Cited by 5
“Each input either represents a word from the query (e.g., via a WordPiece embedding [15]), or a region-of-interest from the input image that has been identified with an object detector, such as a Faster R-CNN.”
MethodExploring Natural Language Processing Methods for Interactive Behaviour Modelling
Zhang, G., Bortoletto, M., Hu, Z., Shi, L., Bâce, M., & Bulling, A. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-42286-7_1 · Cited by 9
Power Law Graph Transformer for Machine Translation and Representation LearningPreprint
Gokden, B. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2107.02039 · Cited by 6
“The datasets were used to create a subword vocabulary of maximum 15k tokens using the wordpiece approach used in BERT implementation [20, 15].”
MethodCombining dictionary- and rule-based approximate entity linking with tuned BioBERTPreprint
Mobasher, G., Mertová, L., Ghosh, S., Krebs, O., Heinlein, B., & Müller, W. (2021) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2021.11.09.467905 · Cited by 5
Tokens with Meaning: A Hybrid Tokenization Approach for NLPPreprint
Bayram, M. A., Fincan, A. A., Gümüş, A. S., Karakaş, S., Diri, B., Yıldırım, S., & Çelik, D. (2025) · Research Square · doi:10.21203/rs.3.rs-6513777/v1 · Cited by 5
Incremental BPE TokenizationPreprint
Jiang, S., & Gong, R.-H. (2026) · arXiv.org · doi:10.48550/arxiv.2605.30813 · Cited by 3
“Despite alternatives like WordPiece (Schuster & Nakajima, 2012) used in BERT (Devlin et al., 2019) and Unigram (Kudo, 2018) adopted by T5 (Raffel et al., 2020), BPE remains the predominant choice for current models.”
Ding, Y., & Chen, W. (2025) · Journal of King Saud University - Computer and Information Sciences · doi:10.1007/s44443-025-00233-w · Cited by 4
SAGE: Semantic-Aware Global Explanations for Named Entity Recognition
Zugarini, A., & Rigutini, L. (2023) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn54540.2023.10191364 · Cited by 4
“[16] in BERT, which requires particular attention in designing the text perturbation step.”
Optimizing the Size of Subword Vocabularies in Dialect Classification
Kanjirangat, V., Samardżić, T., Dolamic, L., & Rinaldi, F. (2023) · Workshop on NLP for Similar Languages, Varieties and Dialects (VarDial) · doi:10.18653/v1/2023.vardial-1.2 · Cited by 9
“Quite a few solutions have been proposed for unsupervised subword segmentation (Creutz and Lagus, 2005; Schuster and Nakajima, 2012; Poon et al., 2009; Narasimhan et al., 2015; Sennrich et al., 2016; Bergmanis and Goldwater, 2017; Grönroos et al., 2020; Kudo, 2018).”
BackgroundTokenizers for African Languages
Ndomba, G. E., Mswahili, M. E., & Jeong, Y.-S. (2024) · IEEE Access · doi:10.1109/access.2024.3522285 · Cited by 4
TextDefense: Adversarial Text Detection Based on Word Importance Score Dispersion
Shen, L., Pu, Y., Zhang, X., Ge, C., Yang, X., Peng, H., Wang, W., & Ji, S. (2025) · IEEE Transactions on Dependable and Secure Computing · doi:10.1109/tdsc.2025.3551162 · Cited by 4
“…reason for using the WIR algorithm is that current transformer-based models (e.g., BERT, XLNet, GPT, etc.) utilize the WordPiece tokenization method [32] to construct the vocabulary of the model, unlike RNN-based models such as vanilla RNN, GRU, and LSTM, which use the entire English vocabulary.”
Multi-task learning for multi-dialect Arabic sentiment classification and sarcasm detection
Barmati, M. E., Said, B., & Dahou, A. (2025) · Language Resources and Evaluation · doi:10.1007/s10579-025-09823-6 · Cited by 4
Timmapathini, H., Nayak, A., Mandadi, S., Sangada, S., Kesri, V., Ponnalagu, K., & Venkoparao, V. G. (2021) · National Conference on Artificial Intelligence · Cited by 9
Sharma, M., Kandasamy, I., & Vasantha, W. B. (2021) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/2021.semeval-1.130 · Cited by 4
“, 2016), WordPiece (Schuster and Nakajima, 2012), and SentencePiece (Kudo and Richardson, 2018).”
Assessing the use of attention weights to interpret BERT-based stance classification
Sáenz, C. A. C., & Becker, K. E. (2021) · IEEE/WIC/ACM International Conference on Web Intelligence · doi:10.1145/3486622.3493966 · Cited by 6
“If not, the word is divided into several tokens using the WordPiece algorithm [13].”
MethodStreaming Intended Query Detection using E2E Modeling for Continued Conversation
Chang, S.-Y., Prakash, G., Wu, Z., Sainath, T. N., Li, B., Liang, Q., Stambler, A., Upadhyay, S., Faruqui, M., & Strohman, T. (2022) · Interspeech (USB) · doi:10.21437/interspeech.2022-569 · Cited by 4
Hasan, A., Wu, J., Nguyen, Q., Andres, S., Guellil, I., Zhang, H., Casey, A., Alex, B., Guthrie, B., & Wu, H. (2025) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2025.110747 · Cited by 4
Lennox, M., Robertson, N. M., & Devereux, B. J. (2020) · Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) · doi:10.1109/embc44109.2020.9176380 · Cited by 4
“During the data processing stage, this encoding method breaks the original sequence into a set of subsequences of varying length [27, 4, 28, 34].”
MethodUPB at GermEval-2020 Task 3: Assessing Summaries for German Texts using BERTScore and Sentence-BERT.
Paraschiv, A., & Cercel, D.-C. (2020) · Cited by 4
SubICap: Towards Subword-informed Image Captioning
Sharif, N., Bennamoun, M., Liu, W., & Shah, S. A. A. (2021) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv48630.2021.00358 · Cited by 4
“While both BPE and WordPiece require data to learn word segmentations, WordPiece forms the new subwords based on likelihood instead of pairing frequency.”
MethodEfficient Estimation of Influence of a Training Instance
Kobayashi, S., Yokoi, S., Suzuki, J., & Inui, K. (2021) · Journal of Natural Language Processing · doi:10.5715/jnlp.28.573 · Cited by 4
Koehn, P. (2020) · Cambridge University Press eBooks · doi:10.1017/9781108608480.009 · Cited by 7
Palakodety, S., KhudaBukhsh, A. R., & Jayachandran, G. (2021) · SpringerBriefs in computer science · doi:10.1007/978-981-16-5625-5_4 · Cited by 4
Löwenmark, K., Taal, C., Nivre, J., Liwicki, M., & Sandin, F. (2022) · PHM Society European Conference · doi:10.36001/phme.2022.v7i1.3356 · Cited by 4
“However, the WordPiece splitting does not always produce intelligible tokens, such as for remanufacture, which is tokenised as [re, ##man, ##uf, ##act, ##re], despite [manufacture] also existing as a token in the tokenizer vocabulary.”
MethodInfluentialGale, R. C., Salem, A. C., Fergadiotis, G., & Bedrick, S. D. (2023) · Workshop on Representation Learning for NLP (RepL4NLP) · doi:10.18653/v1/2023.repl4nlp-1.18 · Cited by 11
“Today’s LLMs typically rely on sub-word representations such as WordPieces (Schuster and Nakajima, 2012) or Byte-Pair Encoding (BPE) (Gage, 1994; Sen-nrich et al., 2016), which strike a balance between vocabulary size and semantic precision while allowing un-restricted input and avoiding the limitations arising from a…”
MethodFinDeBERTaV2: Word-Segmentation-Free Pre-trained Language Model for Finance
Suzuki, M., Sakaji, H., Hirano, M., & Izumi, K. (2024) · Transactions of the Japanese Society for Artificial Intelligence · doi:10.1527/tjsai.39-4_fin23-g · Cited by 4
Recent Progresses in Deep Learning based Acoustic Models (Updated)Preprint
Yu, D., & Li, J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1804.09298 · Cited by 6
“However, recently Google significantly advanced the research of attention-based model by • modeling with word piece units [86] which are more stable and helpful to LM modeling; • including scheduled sampling [87] which feeds the previous predicted label instead of the ground truth during training so…”
BackgroundWiki to Automotive: Understanding the Distribution Shift and its impact on Named Entity RecognitionPreprint
Nayak, A., & Timmapathini, H. P. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2112.00283 · Cited by 4
“Entity and Context characteristics: As BERT variants rely on the WordPiece tokenization al-gorithm (Schuster and Nakajima, 2012), understanding the distribution of entities that are In-Vocabulary (IV) vs Out-Of-Vocabulary (OOV) is important as the meaning of a word will then be determined by how…”
Re(Visiting) Large Language Models in FinancePreprint
Rahimikia, E., & Drinkall, F. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4963618 · Cited by 17
Overcoming Vocabulary Constraints with Pixel-level FallbackPreprint
Lotz, J. F., Setiawan, H., Peitz, S., & Kementchedjhieva, Y. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2504.02122 · Cited by 5
“Subword tokenization is an intrinsic part of the modern language modeling pipeline (Schuster & Nakajima, 2012; Sennrich et al., 2016; Kudo, 2018).”
FinBERT-QA: Financial Question Answering with pre-trained BERT Language ModelsPreprint
Yuan, B. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2505.00725 · Cited by 4
BERTDom: Protein Domain Boundary Prediction Using BERT
Haseeb, A., Bashir, M., & Wali, A. (2023) · Computing and Informatics · doi:10.31577/cai_2023_3_667 · Cited by 3
“WordPiece algorithm [31] is a tokenizer that splits sentences into words and then words into sub-words.”
BackgroundNguyễn, V. M., Nur, N., Stern, W., Mercer, T., Sen, C., Bhattacharyya, S., Tumbiolo, V., & Goh, S. J. (2023) · International Conference on Machine Learning and Applications · doi:10.1109/icmla58977.2023.00316 · Cited by 4
“The tokenization method for the BERT and BERT based models is WordPiece [3], a tokenization method where words are divided into a limited set of common sub-words ("wordpieces") based on frequency of occurences and decide its merge strategy by maximizing the likelihood of the sub-words pair[29, 30].”
MethodFerro, N., & Maistro, M. (2024) · ACM eBooks · doi:10.1145/3674127.3674132 · Cited by 5
Fine-Tuning Transformer Models for Structuring Spanish Psychiatric Clinical Notes
Rubio-Martín, S., Crespo-Álvaro, A., García-Ordás, M. T., García, A. S., Pato, C. M. F., & Benítez‐Andrades, J. A. (2025) · Proceedings - IEEE Symposium on Computer-Based Medical Systems · doi:10.1109/cbms65348.2025.00028 · Cited by 5
Dynamic Diffusion Programming and Classification
Chien, J., & Yeh, C.-K. (2026) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/taslpro.2026.3671483 · Cited by 3
“The tokens refer to those in the smallest units within a text sequence, which are typically the characters, words, or sub-words such as those generated by the byte-pair encoding [23] or the word-piece models [24].”
Metadata
- Authors
- Schuster, M., & Nakajima, K.
- Affiliations
- Google (United States) (US)
- Venue
- 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
- Publisher
- IEEE
- Published
- 2012-03
- Pages
- 5149-5152
- ISSN
- 1520-6149 (ISSN-L)
- OpenAlex
- W2121879602
- Cited by
- 862
- References
- 8
- Citations by year
- 2026: 36 · 2025: 66 · 2024: 90 · 2023: 124 · 2022: 137 · 2021: 155 · 2020: 144 · 2019: 59 · 2018: 36 · 2017: 10 · 2016: 3 · 2013: 2
- FWCI
- 0.49
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- IEEE International Conference on Acoustics Speech and Signal Processing (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
- Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Keywords
- voice search, Japanese language, Korean language, acoustic modeling
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.