Japanese and Korean voice search
The DOI matches its Crossref record; 7 of the 8 references it cites were found in a registry or an index.
Abstract
This paper describes challenges and solutions for building a successful voice search system as applied to Japanese and Korean at Google. We describe the techniques used to deal with an infinite vocabulary, how modeling completely in the written domain for language model and dictionary can avoid some system complexity, and how we built dictionaries, language and acoustic models in this framework. We show how to deal with the difficulty of scoring results for multiple script languages because of ambiguities. The development of voice search for these languages led to a significant simplification of the original process to build a system for any new language which in in parts became our default process for internationalization of voice search.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“LogRobust [8], which encodes log templates into semantic vectors using the FastText pre-trained model [31], cannot work well on 2 out of 4 datasets.”
From Log-based Anomaly Detection Without Log Parsing (2021) · cites this paper as [8]
In the citing paper
“Their empirical study on a Microsoft online service system shows that up to 30.3% logs are changed in the latest version.”
From Log-based Anomaly Detection Without Log Parsing (2021) · cites this paper as [8]
In the citing paper
“Zhang et al. [8] indicated that log data is unstable, meaning that new log events often appear due to software evolution at its lifetime.”
From Log-based Anomaly Detection Without Log Parsing (2021) · cites this paper as [8]
In the citing paper
“Another option is to use domain knowledge to reduce the input space of a model — such strategies include patch embeddings [Dosovitskiy et al. 2020] as well as fixed or learned tokenization [Schuster and Nakajima 2012; Sennrich et al. 2016; Kudo and Richardson 2018; van den Oord et al. 2017; Ramesh et al. 2021]. These methods may alleviate some challenges facing generative approaches, but have the”
From On the Opportunities and Risks of Foundation Models (2021) · cites this paper as Schuster and Nakajima 2012
Citations
Cites
Semi-tied covariance matrices for hidden Markov models
Gales, M. J. F. (1999) · IEEE Transactions on Speech and Audio Processing · doi:10.1109/89.759034 · Cited by 537
Boosted MMI for model and feature-space discriminative training
Povey, D., Kanevsky, D., Kingsbury, B., Ramabhadran, B., Saon, G., & Visweswariah, K. (2008) · 2008 IEEE International Conference on Acoustics, Speech and Signal Processing · doi:10.1109/icassp.2008.4518545 · Cited by 341
“Your Word is my Command”: Google Search by Voice: A Case Study
Schalkwyk, J., Beeferman, D., Beaufays, F., Byrne, B., Chelba, C., Cohen, M., Kamvar, M., & Strope, B. (2010) · Advances in Speech Recognition · doi:10.1007/978-1-4419-5951-5_4 · Cited by 232
Building transcribed speech corpora quickly and cheaply for many languages
Hughes, T., Nakajima, K., Ha, L., Vasu, A., Moreno, P. J., & LeBeau, M. (2010) · Interspeech (USB) · doi:10.21437/interspeech.2010-551 · Cited by 75
Self-citationGoogle Search by Voice: A Case Study
Schalkwyk, J., Beeferman, D., Beaufays, F., Byrne, B., Chelba, C. I., Cohen, M., Garrett, M., & Strope, B. P. (2010) · Cited by 63
Query language modeling for voice search
Chelba, C. I., Schalkwyk, J., Brants, T., Ha, V., Harb, B., Neveitt, W., Parada, C., & Xu, P. (2010) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt.2010.5700834 · Cited by 28
Search by voice in Mandarin Chinese
Shan, J., Wu, G., Hu, Z., Tang, X., Jansche, M., & Moreno, P. J. (2010) · Interspeech (USB) · doi:10.21437/interspeech.2010-129 · Cited by 18
Research works printed but not linked: 1
- sung, 0, Deploying Search by Voice in Cantonese, Proc Interspeech 2011
Cited by
Items 401–500 of 933
Multi-Granularity Prediction with Learnable Fusion for Scene Text Recognition
Da, C., Wang, P., & Yao, C. (2026) · International Journal of Computer Vision · doi:10.1007/s11263-025-02653-7 · Cited by 13
“In particular, subword representations (BPE and WordPiece) are utilized to induce linguistic information in an implicit manner through multi-task learning and optimization.”
InfluentialOntology-based semantic data interestingness using BERT models
Abhilash, C. B., Mahesh, K., & Sanda, N. (2023) · Connection Science · doi:10.1080/09540091.2023.2190499 · Cited by 12
Krishna, A., Majumder, B. P., Bhat, R. S., & Goyal, P. (2018) · Conference on Computational Natural Language Learning · doi:10.18653/v1/k18-1034 · Cited by 12
“Methods such as wordpiece (Schuster and Nakajima, 2012) as well as Sennrich et al.”
MethodWhich transformer architecture fits my data? A vocabulary bottleneck in self-attentionPreprint
Wies, N., Levine, Y., Jannai, D., & Shashua, A. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.03928 · Cited by 28
“…word-level vocabularies, the number of unique words in web based sources can surpass 1M, and therefore sub-word based encodings such as WordPiece (Schuster & Nakajima, 2012), Sentence- Piece (Kudo & Richardson, 2018) or BPE (Sennrich et al., 2016) are used to reduce the number of vocabulary…”
BackgroundAn Overview on Large Language Models
Dakhel, A. M., Nikanjam, A., Khomh, F., Desmarais, M. C., & Washizaki, H. (2024) · Springer eBooks · doi:10.1007/978-3-031-55642-5_1 · Cited by 15
Xu, X., Wen, F., Chu, B., Fu, Z., Lin, Q., Liu, J., Fei, B., Li, Y., Zhou, L., & Yang, Z. (2025) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3711896.3737219 · Cited by 11
“Using the WordPiece algorithm [32], we extract domain-specific vocabulary from our 32B-token financial corpus and the “Colossal, Cleaned, Common Crawl (C4)” dataset 4 , the latter being part of the original pre-training data for Chinese-RoBERTa.”
Consistency Training with Virtual Adversarial Discrete Perturbation
Park, J., Kim, G., & Kang, J. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-main.414 · Cited by 18
LMSOC: An Approach for Socially Sensitive Pretraining
Kulkarni, V., Mishra, S., & Haghighi, A. (2021) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2021.findings-emnlp.254 · Cited by 19
Improving Deliberation by Text-Only and Semi-Supervised Training
Hu, K., Sainath, T. N., He, Y., Prabhavalkar, R., Strohman, T., Mavandadi, S., & Wang, W. (2022) · Interspeech (USB) · doi:10.21437/interspeech.2022-243 · Cited by 11
Classification of open source software bug report based on transfer learning
Liao, Z., Wang, K., Zeng, Q., Liu, S., Yan, Z., & He, J. (2022) · Expert Systems · doi:10.1111/exsy.13184 · Cited by 11
“BERT performs subword division based on Word Piece (Schuster & Nakajima, 2012).”
MethodChoi, W., Yoon, H.-M., Hyun, M.-H., Lee, H., Seol, J.-W., Lee, K. D., Yoon, Y. J., & Kong, H. (2023) · PLoS ONE · doi:10.1371/journal.pone.0280637 · Cited by 16
MergeDistill: Merging Language Models using Pre-trained Distillation
Khanuja, S., Johnson, M., & Talukdar, P. P. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.254 · Cited by 21
HintedBT: Augmenting Back-Translation with Quality and Transliteration Hints
Ramnath, S., Johnson, M., Gupta, A., & Raghuveer, A. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.129 · Cited by 15
“and target using WordPiece tokenization (Schuster and Nakajima, 2012; Wu et al., 2016).”
Multi-Task Learning for End-to-End ASR Word and Utterance Confidence with Deletion Prediction
Qiu, D., He, Y., Li, Q., Zhang, Y., Cao, L., & Mcgraw, I. C. (2021) · Interspeech (USB) · doi:10.21437/interspeech.2021-1207 · Cited by 11
Improving the Fusion of Acoustic and Text Representations in RNN-T
Zhang, C., Li, B., Lu, Z., Sainath, T. N., & Chang, S.-Y. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9747418 · Cited by 14
Technical Language Supervision for Intelligent Fault Diagnosis in Process Industry
Löwenmark, K., Taal, C., Schnabel, S., Liwicki, M., & Sandin, F. (2022) · International Journal of Prognostics and Health Management · doi:10.36001/ijphm.2022.v13i2.3137 · Cited by 14
Rationalizing predictions by adversarial information calibration
Sha, L., Camburu, O.-M., & Lukasiewicz, T. (2022) · Artificial Intelligence · doi:10.1016/j.artint.2022.103828 · Cited by 14
“We do not compare the perplexity with Transformer-based models like GPT [95, 96, 12], because these models usually use subword vocabularies (like Byte Pair Encoding (BPE) [96] and WordPiece [106, 28] ) which makes the perplexities not comparable with our work.”
Kobayashi, S., Okazaki, N., & Inui, K. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1709.01679 · Cited by 11
“An approach to addressing the unknown word problem used in recent studies (Kim et al., 2016; Sennrich et al., 2016; Luong and Manning, 2016; Schuster and Nakajima, 2012) comprises the embeddings of unknown words from character embeddings or subword embeddings.”
MethodSentence-Level Fluency Evaluation: References Help, But Can Be Spared!Preprint
Kann, K., Rothe, S., & Filippova, K. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1809.08731 · Cited by 15
“WordPiece models are estimated using a data-driven approach which maximizes the LM likelihood of the training corpus as described in Wu et al. (2016) and Schuster and Nakajima (2012).”
MethodMulti-Image Summarization: Textual Summary from a Set of Cohesive ImagesPreprint
Trieu, N., Goodman, S., Narayana, P., Sone, K., & Soricut, R. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.08686 · Cited by 12
“For the purpose of training our models over this dataset, the groundtruth labels are tokenized by a Wordpiece model (Schuster and Nakajima, 2012), using the same algorithm as the one in (Devlin et al., 2018).”
MethodInfluentialAbstractive text summarization for Hungarian
Yang, Z. G., Agócs, Á., Kusper, G., & Váradi, T. (2021) · Az Eszterházy Károly Tanárképző Főiskola tudományos közleményei. Tanulmányok a matematikai tudományok köréből/Az Eszterházy Károly Főiskola tudományos közleményei. Tanulmányok a matematikai tudományok köréből/Annales mathematicae et informaticae · doi:10.33039/ami.2021.04.002 · Cited by 21
“Yang, Á. Agócs, G. Kusper, T. Váradi WordPiece tokenizer [18] was used.”
MergeDistill: Merging Pre-trained Language Models using DistillationPreprint
Khanuja, S., Johnson, M., & Talukdar, P. P. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.02834 · Cited by 12
“tokenization (Schuster and Nakajima, 2012; Wu et al., 2016) in all our experiments.”
Lassébie, J., Marcolin, L., Vandeweyer, M., & Vignal, B. (2021) · OECD social employment and migration working papers · doi:10.1787/adb03746-en · Cited by 17
Timor, N., Mamou, J., Korat, D., Berchansky, M., Jain, G., Pereg, O., Wasserblat, M., & Harel, D. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2502.05202 · Cited by 20
“8: With data parallelism (batching), compute via one target forward pass the m + 1 logits of the target model conditioned on the prompt and all the draft continuations, (Sennrich et al., 2016), WordPiece (Schuster & Nakajima, 2012), Unigram (Kudo, 2018), and SentencePiece (Kudo & Richardson, 2018) are heuristic and gr…”
Bridging Language Gaps: Advances in Cross-Lingual Information Retrieval with Multilingual LLMsPreprint
Goworek, R., Macmillan-Scott, O., & Bilici, E. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2510.00908 · Cited by 17
Castells, P., & Jannach, D. (2024) · ACM eBooks · doi:10.1145/3674127.3674131 · Cited by 11
Efficient Estimation of Influence of a Training Instance
Kobayashi, S., Yokoi, S., Suzuki, J., & Inui, K. (2020) · Workshop on Simple and Efficient Natural Language Processing (SustaiNLP) · doi:10.18653/v1/2020.sustainlp-1.6 · Cited by 10
“Rare words of named entities were divided into many subwords (Schuster and Nakajima, 2012; Sennrich et al., 2016; Wu et al., 2016) and requiring more complex processing.”
BackgroundBERT-Based Vulnerability Type Identification with Effective Program Representation
Zhu, C., Du, G., Wu, T., Cui, N., Chen, L., & Shi, G. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19208-1_23 · Cited by 10
Knafou, J., Haas, Q., Borissov, N., Counotte, M. J., Low, N. N., Imeri, H., Ipekci, A. M., Buitrago‐García, D., Heron, L., Amini, P., et al. (2023) · Systematic Reviews · doi:10.1186/s13643-023-02247-9 · Cited by 15
Visually Grounded Language Learning: A Review of Language Games, Datasets, Tasks, and Models
Suglia, A., Konstas, I., & Lemon, O. (2024) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.15185 · Cited by 21
TransformCode: A Contrastive Learning Framework for Code Embedding via Subtree Transformation
Xian, Z., Huang, R., Towey, D., Fang, C., & Chen, Z. (2024) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2024.3393419 · Cited by 17
“Some common subword-based tokenization algorithms are: Byte-Pair Encoding (BPE) [50]; Word-Piece [24], [51]; and Unigram [52].”
MethodLarge Language Models for Software Engineering: A Systematic Mapping Study
Görmez, M. K., Yılmaz, M., & Clarke, P. M. (2024) · Communications in computer and information science · doi:10.1007/978-3-031-71139-8_5 · Cited by 18
West: Word Encoded Sequence Transducers
Variani, E., Suresh, A. T., & Weintraub, M. (2019) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2019.8683694 · Cited by 10
Causal and Masked Language Modeling of Javanese Language using Transformer-based Architectures
Wongso, W., Setiawan, D. S., & Suhartono, D. (2021) · International Conference on Advanced Computer Science and Information Systems · doi:10.1109/icacsis53237.2021.9631331 · Cited by 10
“On the other hand, our BERT-based and DistilBERT-based model used Hugging Face’s implementation of BERT’s Word-Piece Tokenizer [31].”
MethodNamed Entity Recognition in Long Documents: An End-to-end Case Study in the Legal Domain
Keshavarz, H., Vagena, Z., Kouki, P., Fountalis, I., Mabrouki, M., Belaweid, A., & Vasiloglou, N. (2022) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata55660.2022.10020873 · Cited by 10
“For the SEC-SE model, we use the WordPiece tokenizer outlined in [14].”
MethodSubsumption Prediction for E-Commerce Taxonomies
Shi, J., Chen, J., Dong, H., Khan, I., Liang, L., Zhou, Q., Wu, Z., & Horrocks, I. R. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-33455-9_15 · Cited by 10
On the Discrepancy between Density Estimation and Sequence Generation
Lee, J., Tran, D., Fırat, O., & Cho, K. (2020) · Workshop on Structured Prediction for NLP (SPNLP) · doi:10.18653/v1/2020.spnlp-1.10 · Cited by 11
E-Stitchup: Data Augmentation for Pre-Trained EmbeddingsPreprint
Wolfe, C. R., & Lundgaard, K. T. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.00772 · Cited by 11
“To create these textual embeddings, input phrases are first tokenized using a WordPiece tokenizer [14].”
MethodKrzyzanowski, A., Pickett, S. D., & Pogány, P. (2025) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.5c00359 · Cited by 12
Phonetically Induced Subwords for End-to-End Speech Recognition
Papadourakis, V., Müller, M., Liu, J., Mouchtaris, A., & Omologo, M. (2021) · Interspeech (USB) · doi:10.21437/interspeech.2021-1787 · Cited by 10
“BPE variants include using bytes instead of characters as input [10] or estimating unigram probabilities for the BPE subwords [11].”
MethodWine is not v i n. On the Compatibility of Tokenizations across Languages
Maronikolakis, A., Dufter, P., & Schütze, H. (2021) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2021.findings-emnlp.205 · Cited by 18
Comparison of preprocessing approaches for text data in digital shop floor management systems
Müller, M., Longard, L., & Metternich, J. (2022) · Procedia CIRP · doi:10.1016/j.procir.2022.04.030 · Cited by 9
Identifying Corporate Credit Risk Sentiments from Financial News
Ahbali, N., Liu, X., Nanda, A., Stark, J., Talukder, A., & Khandpur, R. P. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-industry.40 · Cited by 17
“The raw documents are first split into sentences using syntok 3 and then on each sentence a pre-trained WordPiece tokenizer (Schuster and Nakajima, 2012) is applied.”
MethodJEIT: Joint End-to-End Model and Internal Language Model Training for Speech Recognition
Meng, Z., Wang, W., Prabhavalkar, R., Sainath, T. N., Chen, T., Variani, E., Zhang, Y., Li, B., Rosenberg, A., & Ramabhadran, B. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10095249 · Cited by 9
“In HAT, the label decoder output passes through a 640-dim feedforward joint network before projected to 4096 output units representing word pieces [40].”
BackgroundTADA: Efficient Task-Agnostic Domain Adaptation for Transformers
Hung, C.-C., Lange, L., & Strötgen, J. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.31 · Cited by 9
Multi-word Tokenization for Sequence Compression
Gee, L., Rigutini, L., Ernandes, M., & Zugarini, A. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-industry.58 · Cited by 17
“To do this, a vocabulary is first constructed by statistically learning the most frequent tokens from a large general-purpose corpus (Sennrich et al., 2016; Schuster and Nakajima, 2012; Kudo and Richardson, 2018).”
BackgroundContext-aware Transliteration of Romanized South Asian Languages
Kirov, C., Johny, C., Katanova, A., Gutkin, A., & Roark, B. (2023) · Computational Linguistics · doi:10.1162/coli_a_00510 · Cited by 22
MAGIC: Multi-granularity domain adaptation for text recognition
Zhang, J., Liu, X., Xue, Z.-Y., Luo, X., & Xu, X.-S. (2024) · Pattern Recognition · doi:10.1016/j.patcog.2024.111229 · Cited by 10
Zheng, J., Xie, L., & Xu, H. (2025) · Neurocomputing · doi:10.1016/j.neucom.2025.130132 · Cited by 10
Kim, M., Jung, J.-W., Rha, H., Maiti, S., Arora, S., Chang, X., Watanabe, S., & Ro, Y. M. (2026) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2026.3659297 · Cited by 10
“For the text input, we employ the BERT tokenizer [50], [68] to derive text tokens Advantages of tokenizing tri-modalities: Tokenizing all three modalities brings two main advantages.”
Exploring Model Units and Training Strategies for End-to-End Speech Recognition
Huang, M., Lu, Y., Wang, L., Qian, Y., & Yu, K. (2019) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru46091.2019.9003834 · Cited by 9
“We train a statistical wordpiece model [21] with word counts obtained from training set text data for segmenting each word individually into subwords.”
BackgroundMethodIPR: The Semantic Textual Similarity and Recognizing Textual Entailment Systems.
Rodrigues, R., Couto, P., & Rodrigues, I. P. (2019) · Cited by 9
An Attention-Based Joint Acoustic and Text on-Device End-To-End Model
Sainath, T. N., Pang, R., Weiss, R. J., He, Y., Chiu, C., & Strohman, T. (2020) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp40776.2020.9053510 · Cited by 9
“Both decoders are trained to predict 4,096 word pieces [23], which are derived from a large corpus of text transcripts.”
BackgroundAltuncu, M. T., Yaliraki, S. N., & Barahona, M. (2021) · Studies in computational intelligence · doi:10.1007/978-3-030-65351-4_13 · Cited by 9
A Semantic Annotation Pipeline towards the Generation of Knowledge Graphs in Tribology
Kügler, P., Marian, M., Dorsch, R., Schleich, B., & Wartzack, S. J. (2022) · Lubricants · doi:10.3390/lubricants10020018 · Cited by 9
“Tokenizers like WordPiece [51] represent words as subword vectors [49], e.”
BackgroundTrojBits: A Hardware Aware Inference-Time Attack on Transformer-Based Language Models
Ghanim, M. A., Santriaji, M. H., Lou, Q., & Solihin, Y. (2023) · Frontiers in artificial intelligence and applications · doi:10.3233/faia230254 · Cited by 19
Japanese Abstractive Text Summarization using BERT
Iwasaki, Y., Yamashita, A., Konno, Y., & Matsubayashi, K. (2020) · Advances in Science Technology and Engineering Systems Journal · doi:10.25046/aj0506199 · Cited by 9
Alami, H., En-Nahnahi, N., Mahdaouy, A. E., & Alaoui, S. O. E. (2022) · Journal of King Saud University - Computer and Information Sciences · doi:10.1016/j.jksuci.2022.03.015 · Cited by 10
“Wordpiece model segments questions basing on sub-words units approach (Schuster et al., 2012; Wu et al., 2016).”
NorBERT: NetwOrk Representations Through BERT for Network Analysis & Management
Le, F., Wertheimer, D., Calo, S. B., & Nahum, E. (2022) · Proceedings - International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems · doi:10.1109/mascots56607.2022.00012 · Cited by 12
“To achieve this, a tokenizer [23], [24] splits the text into words or subwords (e.”
BackgroundImproving Tokenisation by Alternative Treatment of Spaces
Gow-Smith, E., Madabushi, H. T., Scarton, C., & Villavicencio, A. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.786 · Cited by 24
Bollegala, D., Kiryo, R., Tsujino, K., & Yukawa, H. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.11004 · Cited by 10
“LIT has gained popularity as an alternative to LST (Sennrich et al., 2016; Zhu et al., 2019; Kudo and Richardson, 2018; Kudo, 2018; Schuster and Nakajima, 2012) because, unlike LST, LIT methods do not require predefined”
MethodInteractive Teaching for Conversational AIPreprint
Ping, Q., Niu, F., Thattai, G., Chengottusseriyil, J., Gao, Q., Reganti, A., Rajagopal, P., Tür, G., Hakkani-Tür, D. Z., & Natarajan, P. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2012.00958 · Cited by 13
“The input sentence is first tokenized with WordPiece [19].”
BackgroundSelf-supervised and Supervised Joint Training for Resource-rich Machine TranslationPreprint
Cheng, Y., Wang., W., Jiang, L., & Macherey, W. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.04060 · Cited by 14
ConvFiT: Conversational Fine-Tuning of Pretrained Language ModelsPreprint
Vulić, I., Su, P.-H., Coope, S., Gerz, D., Budzianowski, P., Casanueva, I., Mrkšić, N., & Wen, T.-H. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2109.10126 · Cited by 10
“The text t is tokenized into subwords (Schuster and Nakajima, 2012) relying on each en-coder’s dedicated tokeniser.”
MethodData-Centric AI Governance: Addressing the Limitations of Model-Focused PoliciesPreprint
Gupta, R., Walker, L., Corona, R., Fu, S., Petryk, S., Napolitano, J., Darrell, T., & Reddie, A. W. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2409.17216 · Cited by 13
“Further, it remains unclear what exactly constitutes a “data point,” especially with modern methods like transformers, which rely on tokens, the amount of which varies with different tokenization methods (Sennrich et al., 2016; Schuster & Nakajima, 2012).”
BackgroundLone Pine at SemEval-2021 Task 5: Fine-Grained Detection of Hate Speech Using BERToxic
Khan, Y., Ma, W., & Vosoughi, S. (2021) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/2021.semeval-1.132 · Cited by 11
“This sub-word tokenization algorithm (Schuster and Nakajima, 2012) tokenizes a word like "moron" into ["mo","##ron"] and we Figure 1: The BERToxic model architecture.”
MethodCEQE to SQET: A study of contextualized embeddings for query expansion
Naseri, S., Dalton, J., Yates, A., & Allan, J. (2022) · Information Retrieval · doi:10.1007/s10791-022-09405-y · Cited by 10
“We compute the average embedding vector of word w by ��⃗ w ≜ 1 � w � ∑ p i ∈ w ��⃗ p i , where p i is a WordPiece of word w and | w | is the number of WordPieces in the word w .”
MethodInfluentialGaleev, D. T., & Славиевич, П. В. (2022) · Informatics and Automation · doi:10.15622/ia.21.3.3 · Cited by 8
Context-Aware end-to-end ASR Using Self-Attentive Embedding and Tensor Fusion
Chang, S.-Y., Zhang, C., Sainath, T. N., Li, B., & Strohman, T. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10095204 · Cited by 14
“E2E models are trained to predict 4,096 word pieces [33].”
MethodSentiment analysis in Portuguese tweets: an evaluation of diverse word representation models
Vianna, D., Carneiro, F., Carvalho, J., Plastino, A., & Paes, A. (2023) · Language Resources and Evaluation · doi:10.1007/s10579-023-09661-4 · Cited by 10
“Precisely, fastText tokenizer splits words into sub-words based on character N-grams, mBERT and BERTimbau follow the Word Piece tokenization (Schuster and Nakajima, 2012), and BERTweet employs the Byte-Pair encoding (Gage, 1994; Heinzerling & Strube, 2018).”
Unsupervised Term Extraction for Highly Technical Domains
Fusco, F., Staar, P., & Antognini, D. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-industry.1 · Cited by 8
Automatic Morpheme Segmentation for Russian: Can an Algorithm Replace Experts?
Morozov, D. A., Garipov, T., Lyashevskaya, O., Savchuk, S. O., Iomdin, B. L., & Glazkova, A. V. (2024) · Journal of language and Education · doi:10.17323/jle.2024.22237 · Cited by 11
Performance evaluation of tokenizers in large language models for the Assamese language
Tamang, S., & Bora, D. J. (2025) · International Journal of Information Technology · doi:10.1007/s41870-025-02454-8 · Cited by 9
“We selected Assamese language for our study due to it being a low-resource language, which presents us with a unique challenge and opportunities for evaluating tokenizers.”
Lakomkin, E., Heymann, J., Sklyar, I., & Wiesler, S. (2020) · Interspeech (USB) · doi:10.21437/interspeech.2020-1569 · Cited by 8
Subword Mapping and Anchoring across Languages
Vernikos, G., & Popescu-Belis, A. (2021) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2021.findings-emnlp.224 · Cited by 18
SecTKG: A Knowledge Graph for Open‐Source Security Tools
Sun, S., Huang, C., Tie-jun, W., & Shen, Y. (2023) · International Journal of Intelligent Systems · doi:10.1155/2023/4464974 · Cited by 12
Multistate Encoding with End-To-End Speech RNN Transducer Network
Wu, Z., Li, B., Zhang, Y., Aleksic, P. S., & Sainath, T. N. (2020) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp40776.2020.9054287 · Cited by 8
“The target transcript is represented as a sequence of word-piece tokens [23] with a vocabulary size of 4,096; these are embedded into 128 dimensions before being fed into the prediction network.”
BackgroundLiterature Retrieval for Precision Medicine with Neural Matching and Faceted Summarization
Noh, J., & Kavuluru, R. (2020) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2020.findings-emnlp.304 · Cited by 9
Improving Confidence Estimation on Out-of-Domain Data for End-to-End Speech Recognition
Li, Q., Zhang, Y., Qiu, D., He, Y., Cao, L., & Woodland, P. C. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9746979 · Cited by 19
Yanaka, H., & Mineshima, K. (2021) · BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP (BlackboxNLP) · doi:10.18653/v1/2021.blackboxnlp-1.26 · Cited by 24
“…the model processes input texts with word-level tokenization based on a standard Japanese dictionary (ipadic) (Asahara and Matsumoto, 2003), followed by the WordPiece subword tokenization (Schuster and Nakajima, 2012), trained with whole-word masking enabled for the masked language model objective.”
MethodUpcycle Your OCR: Reusing OCRs for Post-OCR Text Correction in Romanised SanskritPreprint
Krishna, A., Majumder, B. P., Bhat, R. S., & Goyal, P. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1809.02147 · Cited by 11
“Methods such as wordpiece (Schuster and Nakajima, 2012) as well as Sennrich et al.”
MethodElangovan, A., Davis, M. J., & Verspoor, K. M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2008.08727 · Cited by 8
“s in substantially increased F-score of 54.0 (PPI-BioBERTx10, Table 2), where basing the confidence score on the average score of Hsieh, 2017 #71Zhang, 2019 #72Hsieh, 2017 #71Lee, 2019 #82Wu, 2016 #89Schuster, 2012 #90Lee, 2019 #82Hsieh, 2017 #71Zhang, 2019 #72Yadav, 2018 #25Wei, 2015 #18models can improve uncertainty…”
BackgroundInfluentialWang, X., Liu, Y., Zhao, S., & Li, J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2108.07493 · Cited by 8
“The contextual LM, min(det(S ◦G)), is obtained by compiling the list of biasing phrases into an n-gram weighted finite state transducer (WFST) [22] G, Where G is left composed with a “speller” FST S to transduce wordpieces [23] into the corresponding word.”
MethodSpeeding up Deep Model Training by Sharing Weights and Then UnsharingPreprint
Yang, S., Hou, L., Song, X., Liu, Q., & Zhou, D. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.03848 · Cited by 10
“We preprocess all datasets with WordPiece tokenization (Schuster & Nakajima, 2012).”
Siamese BERT-Based Model for Web Search Relevance Ranking Evaluated on a New Czech Dataset
Kocián, M., Náplava, J., Stancl, D., & Kadlec, V. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i11.21502 · Cited by 7
An Information-Theoretic Perspective on LLM Tokenizers
Erdogan, M., Gorle, A. R., Chandak, S., Pilancı, M., & Weissman, T. (2026) · IEEE International Symposium on Information Theory (ISIT) · doi:10.1109/isit62367.2026.11653797 · Cited by 7
Entropy-Based Subword Mining with an Application to Word Embeddings
El-Kishky, A., Xu, F. F., Zhang, A., Macke, S., & Han, J. (2018) · Proceedings of the Second Workshop on Subword/Character LEvel Models · doi:10.18653/v1/w18-1202 · Cited by 7
“Other methods apply a maximum likelihood approach to identifying subwords (also called wordpieces) and has been successfully applied to a variety of NLP tasks (Wu et al., 2016; Schuster and Nakajima, 2012).”
MethodText-Text Neural Machine Translation: A Survey
Gemechu, E. A., & Kanagachidambaresan, G. R. (2023) · Optical Memory and Neural Networks · doi:10.3103/s1060992x23020042 · Cited by 7
Karita, S., Sproat, R., & Ishikawa, H. (2023) · doi:10.18653/v1/2023.cawl-1.8 · Cited by 18
A Systematic Review of Computational Approaches to Deciphering Bronze Age Aegean and Cypriot Scripts
Braović, M., Krstinić, D., Štula, M., & Ivanda, A. (2024) · Computational Linguistics · doi:10.1162/coli_a_00514 · Cited by 9
Knowledge complementation based dual-table filling approach for aspect sentiment triplet extraction
Wang, B., Jiang, B., Wang, G., Shi, L., & Liu, P. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.128625 · Cited by 7
Research on morphological knowledge-guided low-resource agglutinative languages-Chinese translation
Abudouwaili, G., Ruzmamat, S., Abiderexiti, K., Yibulayin, T., Yi, N., & Wumaier, A. (2025) · Complex & Intelligent Systems · doi:10.1007/s40747-025-01780-5 · Cited by 8
Cohen, D., Rosenberger, I., Butman, M., & Bar, K. (2023) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2023.1091443 · Cited by 9
“Following standard practices, we apply WordPiece (Schuster and Nakajima, 2012) tokenization on the input transcriptions, place a [CLS] token in front of the sequence of tokens, and close it with a final [SEP] token.”
MethodUML: A Universal Monolingual Output Layer For Multilingual Asr
Zhang, C., Li, B., Sainath, T. N., Strohman, T., & Chang, S.-Y. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10096228 · Cited by 9
“By incrementally combining graphemes into frequent pieces of words, the vocabulary size can be controlled using WPMs [12].”
BackgroundMethodEvaluating and improving transformers pre-trained on ASTs for Code Completion
Ochs, M., Narasimhan, K., & Mezini, M. (2023) · IEEE International Conference on Software Analysis, Evolution, and Reengineering · doi:10.1109/saner56733.2023.00096 · Cited by 8
“We chose the subword tokenizer that splits less frequent words into frequent subwords[14].”
MethodDe-identification techniques to preserve privacy in medical records
Catelli, R., & Esposito, M. (2023) · Elsevier eBooks · doi:10.1016/b978-0-323-90531-2.00007-2 · Cited by 9
SelfSeg: A Self-supervised Sub-word Segmentation Method for Neural Machine Translation
Song, H., Dabre, R., Chu, C., Kurohashi, S., & Sumita, E. (2023) · ACM Transactions on Asian and Low-Resource Language Information Processing · doi:10.1145/3610611 · Cited by 9
“WordPiece (WPM) [49] is similar to BPE where it chooses the adjacent fragment pair that maximizes the likelihood of the training data rather than based on word frequency.”
BackgroundMethodBiomedical Language Models are Robust to Sub-optimal Tokenization
Gutiérrez, B. J., Sun, H., & Su, Y. (2023) · Workshop on Biomedical Language Processing (BioNLP) · doi:10.18653/v1/2023.bionlp-1.32 · Cited by 7
Metadata
- Authors
- Schuster, M., & Nakajima, K.
- Affiliations
- Google (United States) (US)
- Venue
- 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
- Publisher
- IEEE
- Published
- 2012-03
- Pages
- 5149-5152
- ISSN
- 1520-6149 (ISSN-L)
- OpenAlex
- W2121879602
- Cited by
- 862
- References
- 8
- Citations by year
- 2026: 36 · 2025: 66 · 2024: 90 · 2023: 124 · 2022: 137 · 2021: 155 · 2020: 144 · 2019: 59 · 2018: 36 · 2017: 10 · 2016: 3 · 2013: 2
- FWCI
- 0.49
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- IEEE International Conference on Acoustics Speech and Signal Processing (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
- Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Keywords
- voice search, Japanese language, Korean language, acoustic modeling
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.