Japanese and Korean voice search
The DOI matches its Crossref record; 7 of the 8 references it cites were found in a registry or an index.
Abstract
This paper describes challenges and solutions for building a successful voice search system as applied to Japanese and Korean at Google. We describe the techniques used to deal with an infinite vocabulary, how modeling completely in the written domain for language model and dictionary can avoid some system complexity, and how we built dictionaries, language and acoustic models in this framework. We show how to deal with the difficulty of scoring results for multiple script languages because of ambiguities. The development of voice search for these languages led to a significant simplification of the original process to build a system for any new language which in in parts became our default process for internationalization of voice search.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“LogRobust [8], which encodes log templates into semantic vectors using the FastText pre-trained model [31], cannot work well on 2 out of 4 datasets.”
From Log-based Anomaly Detection Without Log Parsing (2021) · cites this paper as [8]
In the citing paper
“Their empirical study on a Microsoft online service system shows that up to 30.3% logs are changed in the latest version.”
From Log-based Anomaly Detection Without Log Parsing (2021) · cites this paper as [8]
In the citing paper
“Zhang et al. [8] indicated that log data is unstable, meaning that new log events often appear due to software evolution at its lifetime.”
From Log-based Anomaly Detection Without Log Parsing (2021) · cites this paper as [8]
In the citing paper
“Another option is to use domain knowledge to reduce the input space of a model — such strategies include patch embeddings [Dosovitskiy et al. 2020] as well as fixed or learned tokenization [Schuster and Nakajima 2012; Sennrich et al. 2016; Kudo and Richardson 2018; van den Oord et al. 2017; Ramesh et al. 2021]. These methods may alleviate some challenges facing generative approaches, but have the”
From On the Opportunities and Risks of Foundation Models (2021) · cites this paper as Schuster and Nakajima 2012
Citations
Cites
Semi-tied covariance matrices for hidden Markov models
Gales, M. J. F. (1999) · IEEE Transactions on Speech and Audio Processing · doi:10.1109/89.759034 · Cited by 537
Boosted MMI for model and feature-space discriminative training
Povey, D., Kanevsky, D., Kingsbury, B., Ramabhadran, B., Saon, G., & Visweswariah, K. (2008) · 2008 IEEE International Conference on Acoustics, Speech and Signal Processing · doi:10.1109/icassp.2008.4518545 · Cited by 341
“Your Word is my Command”: Google Search by Voice: A Case Study
Schalkwyk, J., Beeferman, D., Beaufays, F., Byrne, B., Chelba, C., Cohen, M., Kamvar, M., & Strope, B. (2010) · Advances in Speech Recognition · doi:10.1007/978-1-4419-5951-5_4 · Cited by 232
Building transcribed speech corpora quickly and cheaply for many languages
Hughes, T., Nakajima, K., Ha, L., Vasu, A., Moreno, P. J., & LeBeau, M. (2010) · Interspeech (USB) · doi:10.21437/interspeech.2010-551 · Cited by 75
Self-citationGoogle Search by Voice: A Case Study
Schalkwyk, J., Beeferman, D., Beaufays, F., Byrne, B., Chelba, C. I., Cohen, M., Garrett, M., & Strope, B. P. (2010) · Cited by 63
Query language modeling for voice search
Chelba, C. I., Schalkwyk, J., Brants, T., Ha, V., Harb, B., Neveitt, W., Parada, C., & Xu, P. (2010) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt.2010.5700834 · Cited by 28
Search by voice in Mandarin Chinese
Shan, J., Wu, G., Hu, Z., Tang, X., Jansche, M., & Moreno, P. J. (2010) · Interspeech (USB) · doi:10.21437/interspeech.2010-129 · Cited by 18
Research works printed but not linked: 1
- sung, 0, Deploying Search by Voice in Cantonese, Proc Interspeech 2011
Cited by
Items 601–700 of 933
To Case or Not to Case: An Empirical Study in Learned Sparse Retrieval
Lionis, E. G., Ju, J.-H., Nalmpantis, A., Thuis, C., MacAvaney, S., & Yates, A. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-21289-4_33 · Cited by 3
Towards a unified framework for imperceptible textual attacks
Shi, J., Li, L., & Zeng, D. D. (2024) · Applied Intelligence · doi:10.1007/s10489-024-05292-6 · Cited by 3
“However, BERT utilizes the WordPiece tokenization [31], which breaks down words into smaller subwords.”
Tonellotto, N. (2024) · ACM eBooks · doi:10.1145/3674127.3674130 · Cited by 5
Classifying Genetic Essentialist Biases using Large Language Models
Reimann, R., Lynch, K. E., Gawronski, S. A., Chan, J., & Griffiths, P. E. (2025) · Review of Philosophy and Psychology · doi:10.1007/s13164-025-00770-3 · Cited by 3
Word and Text Similarity Using Classical Word Embeddings in Quantum NLP Systems
Ćavar, D., & Parukola, K. R. (2025) · IEEE International Conference on Acoustics, Speech, and Signal Processing Workshops (ICASSPW) · doi:10.1109/icasspw65056.2025.11010999 · Cited by 3
“The WordPiece algorithm was first laid out in [24] to cope with out-of-vocabulary words and improve the general performance of machine learning and NLP tasks.”
Koehn, P. (2020) · Cambridge University Press eBooks · doi:10.1017/9781108608480.018 · Cited by 3
Improving Entity Recall in Automatic Speech Recognition with Neural Embeddings
Li, C., Rondon, P., Caseiro, D., Velikovich, L., Velez, X., & Aleksic, P. S. (2021) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp39728.2021.9413756 · Cited by 3
“For ASR models that output wordpieces [21] during decoding, such as the RNN-T described in Section 3.2, we convert the wordpiece lattice to a word lattice.”
MethodInfluentialMulti-Granularity Sequence Alignment Mapping for Encoder-Decoder Based End-to-End ASR
Tang, J., Zhang, J., Song, Y., McLoughlin, I. V., & Dai, L.-R. (2021) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2021.3101921 · Cited by 3
Analysis of Bias in NLP Models With Regression and Effect Sizes
Dayanık, E., Vu, N. T., & Padó, S. (2022) · Northern European Journal of Language Technology · doi:10.3384/nejlt.2000-1533.2022.3505 · Cited by 4
“…the BERT-based one were trained on the English portion of the 2012 CoNLL Shared Task dataset (Pradhan et al., 2012 2020) was pretrained on BooksCorpus (Zhu et al., 2015a) and English Wikipedia using cased Wordpieces tokens (Schuster and Nakajima, 2012) and fine-tuned on the 2012 CoNLL ST dataset.”
MethodASCL: Adversarial supervised contrastive learning for defense against word substitution attacks
Shi, J., Li, L., & Zeng, D. D. (2022) · Neurocomputing · doi:10.1016/j.neucom.2022.09.032 · Cited by 3
Multilingual BERT-based Word Alignment By Incorporating Common Chinese Characters
Li, Z., Sun, X. N., Ren, F., Ma, J., Huang, D., & Shi, P. (2023) · ACM Transactions on Asian and Low-Resource Language Information Processing · doi:10.1145/3594634 · Cited by 3
“Before being input into mBERT, the sentences are tokenized based on a shared WordPiece vocabulary [23].”
MethodExploring the Impact of Vocabulary Techniques on Code Completion: A Comparative Approach
Hussain, Y., Huang, Z., Zhou, Y., & Khan, I. A. (2023) · International Journal of Software Engineering and Knowledge Engineering · doi:10.1142/s0218194023500687 · Cited by 3
MQDD: Pre-training of Multimodal Question Duplicity Detection for Software Engineering Domain
Pašek, J., Sido, J., Konopík, M., & Pražák, O. (2023) · International conference Recent advances in natural language processing · doi:10.26615/978-954-452-092-2_089 · Cited by 5
Encoder models at the European Patent Office: Pre-training and use cases
Hähnke, V. D., Wéry, A., Wirth, M., & Klenner-Bajaja, A. (2025) · World Patent Information · doi:10.1016/j.wpi.2025.102360 · Cited by 3
Mixed Case Contextual ASR Using Capitalization Masks
Caseiro, D., Rondon, P., The, Q.-N. L., & Aleksic, P. S. (2020) · Interspeech (USB) · doi:10.21437/interspeech.2020-2367 · Cited by 3
“We use these data with the algorithm described in [10] to generate wordpiece models containing 4,096 pieces.”
MethodControlled Randomness Improves the Performance of Transformer Models
Deußer, T., Zhao, C., Krämer, W., Leonhard, D., Bauckhage, C., & Sifa, R. (2023) · International Conference on Machine Learning and Applications · doi:10.1109/icmla58977.2023.00274 · Cited by 3
“Given this sentence from [20], “In 2021 and 2020 the total net revenue To give more detail, given an input sentence tokenized using WordPiece [68], we utilize the pre-trained BERT model to obtain the encoded token embeddings.”
MethodData Augmentation for Deep Transfer Learning.Preprint
Wolfe, C., & Lundgaard, K. T. (2019) · arXiv (Cornell University) · Cited by 3
Mischief: A Simple Black-Box Attack Against Transformer ArchitecturesPreprint
de Wynter, A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.08542 · Cited by 3
“We conjecture that, as long as their tokenizer operates in a similar fashion to the WordPiece tokenizer from Schuster and Nakajima (2012), and their parameter size is large enough, the effects from this study extend to them.”
BackgroundAkbari, H., Palangi, H., Yang, J., Rao, S., Çelikyılmaz, A., Fernandez, R., Smolensky, P., Gao, J., & Chang, S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2011.09530 · Cited by 3
“T5-tokenizer is based on the Huggingface implementation of T5 [74] and performs WordPiece tokenization [56] for a given string.”
MethodNeural Transfer Learning with Transformers for Social Science Text Analysis.Preprint
Wankmüller, S. (2021) · arXiv (Cornell University) · Cited by 3
SHUOWEN-JIEZI: Linguistically Informed Tokenizers For Chinese Language Model PretrainingPreprint
Si, C., Zhang, Z., Chen, Y., Qi, F., Wang, X., Liu, Z., & Sun, M. (2021) · arXiv (Cornell University) · Cited by 3
Blanchard, A., Zhang, P., Bhowmik, D., Mehta, K. R., Gounley, J. P., Reeve, S. T., Irle, S., & Pasini, M. L. (2022) · ChemRxiv · doi:10.26434/chemrxiv-2022-gw1n3 · Cited by 3
Lossless Vocabulary Reduction for Auto-Regressive Language ModelsPreprint
Chijiwa, D., Hasegawa, T., Nishida, K., Yamaguchi, S., Ohba, T., Sakao, T., & Takeuchi, S. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2510.08102 · Cited by 3
“Tokenization—the process of decomposing a given text into a sequence of subwords called tokens — plays an important role in modern language models (Schuster & Nakajima, 2012; Sennrich et al., 2016; Kudo, 2018), where tokens are the minimum unit for their input and output.”
ByteSpan: Information-Driven Subword TokenisationPreprint
Goriely, Z., Salhan, S., Lesci, P., Cheng, J., & Buttery, P. J. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2506.18639 · Cited by 4
Decoding the Obfuscated: Advanced NER Techniques on Commercial Sex Advertisement Data
Rodríguez, A., Rivas, P., Turek, J. S., Sooksatra, K., Quevedo, E., Bichler, G., Cerny, T., Giddens, L., & Petter, S. (2023) · International Conference on Computational Science and Computational Intelligence (CSCI) · doi:10.1109/csci62032.2023.00029 · Cited by 3
Probing a pretrained RoBERTa on Khasi language for POS tagging
Mitri, A. M., Lyngdoh, E. L., Warjri, S., Saha, G. K., Lyngdoh, S. A., & Maji, A. K. (2024) · Natural language processing. · doi:10.1017/nlp.2024.24 · Cited by 2
Temporal Extraction and Retrieval
Alonso, O., & Spaniol, M. (2024) · ACM eBooks · doi:10.1145/3674127.3674137 · Cited by 2
Not All Exceptions Are Created Equal: Triaging Error Logs in Real-World Enterprises
Liu, J., Yao, M., Li, S., Yang, D., Wu, Z., Qu, X., Zhang, Z., Li, D., Guo, Y., & Chen, X. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3721126 · Cited by 4
“However, this task is hard and time-consuming.”
NICTokenizer: Tokenizer offload to a SmartNIC
Galvankar, S., Mehta, S., & Choi, S. (2025) · ACM SIGCOMM Conference Posters and Demos · doi:10.1145/3744969.3748428 · Cited by 2
“AI pipelines run a wide range of tokenizers [8],[13],[12] on the Host Server’s CPU.”
Language-Specific Tokenization for Assamese: Efficiency and Downstream Integration with LLMs
Nath, B., & Tamang, S. (2025) · Impact in Computics · doi:10.65500/computics-2025-004 · Cited by 2
Maghrebi dialects – Arabic bidirectional translation: an improved transformer with transfer learning
R’baiti, J., Hmamouche, Y., & Seghrouchni, A. E. F. (2026) · Natural language processing. · doi:10.1017/nlp.2026.10018 · Cited by 2
“Experiment 2 : As a second experiment, we used the WordPiece tokeniser (Schuster et al. 2012; Wu et al. 2016) for both input and output data within the transformer system.”
How to Segment Turkish Words for Neural Text Classification?
Nahas, A. A., Kulunk, A., Gözütok, B., Kalkan, S. C., & Erdinc, H. Y. (2020) · International Conference on Innovations in Intelligent Systems and Applications (INISTA) · doi:10.1109/inista49547.2020.9194661 · Cited by 2
“WordPiece is another word segmentation algorithm originally introduced to segment Korean and Japanese, which are non-space separated languages [6].”
BackgroundAccurate polyglot semantic parsing with DAG grammars
Fancellu, F., Kádár, Á., Zhang, R., & Fazly, A. (2020) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2020.findings-emnlp.320 · Cited by 2
“4 The final word-level representations are obtained through mean-pooling the sub-word representations of mBERT computed using the Wordpiece algorithm (Schuster and Nakajima, 2012).”
MethodA novel approach towards voice-based video content search
Bhowmik, T., Rai, A., Pandey, S., Boddu, P., Patel, N., Veni, S., & Manchala, V. (2021) · IEEE International Conference for Convergence in Technology (I2CT) · doi:10.1109/i2ct51068.2021.9417955 · Cited by 2
“Schuster et. al show how the development of voice search in Japanese and Korean language led to build a simplified version of the original system for internationalization of voice search [3].”
BackgroundData Science, Human-Centered Computing, and Intelligent Technologies
Jerewan, C. (. 2., Hajian, A., Baloian, N., Inoue, T., Luther, W. J., & Berlin, L. V. (2022) · Logos Verlag Berlin eBooks · doi:10.30819/5520 · Cited by 2
Transfer Learning using BERT & Comparative Analysis of ML Algorithms for Opinion Mining
Gandhi, B. S., Kumar, S., Victor, A., & Selvanambi, R. (2022) · IEEE Global Conference for Advancement in Technology (GCAT) · doi:10.1109/gcat55367.2022.9971839 · Cited by 2
Analysis of Subword based Word Representations Case Study: Fasttext Malayalam
Vivek, M., & Chandran, P. (2022) · IEEE India Conference · doi:10.1109/indicon56171.2022.10040147 · Cited by 2
“WordPiece [24] is very similar to BPE, except in the way of choosing the symbol pairs.”
BackgroundLanguage Models for Automatic Distribution of Review Notes in Movie Production
Garcés, D., Santos, M., & Llorca, D. F. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-48232-8_23 · Cited by 2
Log Analysis For Network Attack Detection Using Deep Learning Models
Pham, H. P., Nguyen, V. H., Tran, N. A., & Nguyen, M. H. (2023) · International Symposium on Information and Communication Technology · doi:10.1145/3628797.3628943 · Cited by 2
“Our work adopts the WordPiece tokenization method [10], [11], a widely utilized approach in recent language modeling studies [12], [13], [14].”
MethodLearning to Abstract with Nonparametric Variational Information Bottleneck
Behjati, M., Fehr, F., & Henderson, J. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.106 · Cited by 2
Hybrid Tokenization Strategy for Turkish Abstractive Text Summarization
Kayalı, N. Z., & Omurca, S. İ. (2024) · International Artificial Intelligence and Data Processing Symposium (IDAP) · doi:10.1109/idap64064.2024.10711036 · Cited by 2
“The investigation considered different ways in which texts can be split into words such as widely used subword techniques like WordPiece [23] or Byte-Pair Encoding (BPE) [24] together with those that are based on knowledge specific to languages such as morphemes and syllables in Turkish.”
MethodGaluščáková, P., Oard, D. W., & Nair, S. (2024) · ACM eBooks · doi:10.1145/3674127.3674136 · Cited by 3
Gómez, E., Cuesta, H., Gkiokas, A., Gómez-Cañón, J. S., Porcaro, L., & Yesiler, F. (2024) · ACM eBooks · doi:10.1145/3674127.3702959 · Cited by 5
Privacy in Information Retrieval
Soboroff, I. M. (2024) · ACM eBooks · doi:10.1145/3674127.3674139 · Cited by 2
Detecting Text-Based Cybercrimes Using BERT
Manna, A., Al-Fayoumi, M. A., & Al-Fawa’reh, M. (2024) · International Jordanian Cybersecurity Conference (IJCC) · doi:10.1109/ijcc64742.2024.10847273 · Cited by 2
Skansi, S. (2018) · Undergraduate topics in computer science · doi:10.1007/978-3-319-73004-2_9 · Cited by 2
Self-improvement of voice interface with user-input spoken query at early stage of commercialization
Kim, K. H., Lee, D., Cho, N., Jeon, H., & Kim, J. (2013) · IEEE Transactions on Consumer Electronics · doi:10.1109/tce.2013.6689699 · Cited by 3
“[4] expressed reliability in recognition result for speech queries with confidence measure.”
BackgroundA Study of State-of-the-Art Neural Machine Translation Approaches
Kumar, S., & Asger, M. (2018) · International Journal of Scientific Research in Computer Science Engineering and Information Technology · doi:10.32628/cseit411822 · Cited by 2
Moriya, T., Wang, J., Tanaka, T., Masumura, R., Shinohara, Y., Yamaguchi, Y., & Aono, Y. (2019) · Interspeech (USB) · doi:10.21437/interspeech.2019-1558 · Cited by 2
“As one solution, English studies [9–12] use characters, word-piece and subword units [13,14].”
MethodTreat the Word As a Whole or Look Inside? Subword Embeddings Model Language Change and Typology
Xu, Y., Zhang, J., & Reitter, D. (2019) · Workshop on Computational Approaches to Historical Language Change (LChange) · doi:10.18653/v1/w19-4717 · Cited by 4
“html (2)Another approach is to tokenize words into subwords while optimizing a language model acquired over these word pieces (Schuster and Nakajima, 2012; Sennrich et al., 2015).”
BackgroundLanguage Model CNN-driven Similarity Matching and Classification for HTML-embedded Product Data.
Borst, J., Körner, E., Opasjumruskit, K., & Niekler, A. (2020) · elib (German Aerospace Center) · Cited by 2
BertLoc: duplicate location record detection in a large-scale location dataset
Park, S., Lee, S., & Woo, S. S. (2021) · ACM Symposium on Applied Computing · doi:10.1145/3412841.3441969 · Cited by 2
“In fact, BERT based models use word piece tokenizing [24] based on byte pair encoding (BPE) algorithms [26] to breakdown sentences into meaningful units.”
MethodA Hybrid Model for Drug-Drug Interaction Extraction from Structured Product Labeling Documents.
Mahajan, D., Poddar, A., & Lin, Y.-T. L. Y.-T. (2019) · Theory and applications of categories · Cited by 2
Koehn, P. (2020) · Cambridge University Press eBooks · doi:10.1017/9781108608480.002 · Cited by 2
Koehn, P. (2020) · Cambridge University Press eBooks · doi:10.1017/9781108608480.005 · Cited by 2
Koehn, P. (2020) · Cambridge University Press eBooks · doi:10.1017/9781108608480.008 · Cited by 2
Wang, Z., Zhang, G., & Lashkarashvili, N. (2022) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/2022.semeval-1.2 · Cited by 2
Blanchard, A. E., Zhang, P., Bhowmik, D., Mehta, K. R., Gounley, J. P., Reeve, S. T., Irle, S., & Pasini, M. L. (2022) · Communications in computer and information science · doi:10.1007/978-3-031-23606-8_1 · Cited by 3
“In tokenization, commonly occurring sequences are used to generate a vocabulary for the model [54, 55].”
BackgroundMethodInfluentialPaaß, G., & Giesselbach, S. (2023) · Artificial intelligence: foundations, theory, and algorithms/Artificial intelligence: Foundations, theory, and algorithms · doi:10.1007/978-3-031-23190-2_1 · Cited by 2
LFPS: Learned Formal Proof Strengthening for Efficient Hardware Verification
Kang, M., Nova, A., Singh, E., Bathini, G. S., & Viktorov, Y. (2023) · doi:10.1109/iccad57390.2023.10323681 · Cited by 3
“In other words, a binary label is assigned for an unexplored helper x ∈ X \ { x i } as: We can then express the conditional probability as:”
BackgroundA Machine Learning-Based Pipeline for the Extraction of Insights from Customer Reviews
Lakatos, R., Bogacsovics, G., Harangi, B., Lakatos, I., Tiba, A., Tóth, J. A., Szabó, M., & Hajdú, A. (2024) · Big Data and Cognitive Computing · doi:10.3390/bdcc8030020 · Cited by 2
“Of course, the regression values we generated also allow for a finer resolution.”
BackgroundAn In-depth Walkthrough on Evolution of Neural Machine TranslationPreprint
Jagtap, R., & Dhage, S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.04902 · Cited by 2
“…with parallelism, GNMT also introduces noble segmentation approaches to address out of vocabulary (OOV) words including wordpiece model which was originally developed o solve a Japanese/Korean segmentation problem for the Google speech recognition system[18]; and the mixed word/character model.”
MethodKoehn, P. (2020) · Cambridge University Press eBooks · doi:10.1017/9781108608480.017 · Cited by 2
Rodríguez, A., & Rivas, P. (2023) · arXiv (Cornell University) · doi:10.6084/m9.figshare.24602823.v1 · Cited by 6
Self-Organizing LanguagePreprint
Eugenio, P. M., & Beavers, A. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2506.23293 · Cited by 2
SOMBRERO: Measuring and Steering Boundary Placement in End-to-End Hierarchical Sequence ModelsPreprint
Neitemeier, P., Serra, A., Li, J., Wirges, S., Balles, L., & Metzen, J. H. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2601.22805 · Cited by 2
LiteToken: Removing Intermediate Merge Residues From BPE TokenizersPreprint
Sun, Y., Yang, H., Lin, Z., & Zhang, M. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2602.04706 · Cited by 2
“Classic merge-based methods include BPE (Gage, 1994), which repeatedly merges the most frequent adjacent symbol pairs, and WordPiece (Schuster & Nakajima, 2012), widely used in BERT-like models (Devlin et al., 2019) with a likelihood-inspired objective and longest-match decoding.”
A comprehensive benchmark and empirical study of trace anomaly detection
Sun, Y., Shao, M., Nie, X., Yang, K., Li, X., Hao, B., Zhang, S., Pei, C., He, D., Li, Y., et al. (2025) · IEEE Transactions on Services Computing · doi:10.1109/tsc.2025.3622122 · Cited by 1
“Notably, when generating graph representations for traces,PUTraceADusesWordPiece[27]andapre-trainedBERT model[28]togenerate768-dimensionalembeddingsforservice andoperationnameswhilecreatinggraphrepresentationsfor traces.”
Singh, G., Pan, Y., Andrés-Ferrer, J., del-Agua, M. A., Diehl, F., Pinto, J., & Vozila, P. (2023) · Clinical Natural Language Processing Workshop (ClinicalNLP) · doi:10.18653/v1/2023.clinicalnlp-1.18 · Cited by 7
“Most language generation systems use subword modeling to create an open vocabulary system (Sennrich et al., 2016; Schuster and Nakajima, 2012; Kudo and Richardson, 2018).”
BackgroundAra--CANINE: Character-Based Pre-Trained Language Model for Arabic Language Understanding
Alkesaiberi, A., Alkhathlan, A., & Abdelalí, A. (2024) · International Journal on Cybernetics & Informatics · doi:10.5121/ijci.2024.130204 · Cited by 1
“Later, authors in [14] proposed two models based on WordPiece subword tokenization [15]: ARBERT which was pre-trained on 6 different data sources, and MARBERT which was pre-trained on 1 Billion tweets.”
BackgroundMethodMultilingual Transliteration for Pan-Indic Keyboard Input
Bellegarda, J. R. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10446873 · Cited by 1
eCommerce Search and Discovery
Kallumadi, S. T., & King, T. H. (2024) · ACM eBooks · doi:10.1145/3674127.3674140 · Cited by 1
Beyond Dataset Watermarking: Model-Level Copyright Protection for Code Summarization Models
Zhang, J., Li, H., Wu, D., Sun, X., Lu, Q., & Long, G. (2025) · International World Wide Web Conference (WWW) · doi:10.1145/3696410.3714641 · Cited by 1
“To address this gap, we investigate methods to protect the digital rights of CSMs through digital watermarking.”
Study of Kyrgyz-Kazakh Neural Machine Translation
Abduali, B., Tukeyev, U., Zhumanov, Z., & Israilova, N. (2025) · Communications in computer and information science · doi:10.1007/978-981-96-5881-7_21 · Cited by 1
Various large language model types
Nayyar, A., Vairamani, A. D., & Kaswan, K. S. (2025) · Elsevier eBooks · doi:10.1016/b978-0-443-33904-2.00004-5 · Cited by 1
Layer Cake: On Language Representation and Compute Characteristics in Text Classification
Worth, P. J., & Cardei, I. (2025) · International Journal of Intelligence Science · doi:10.4236/ijis.2025.154010 · Cited by 1
X-HEART: eXplainable heterogeneous log anomaly detection using robust transformers
Mvula, P. K., Branco, P., Jourdan, G.-V., & Viktor, H. L. (2025) · Knowledge and Information Systems · doi:10.1007/s10115-025-02604-1 · Cited by 1
Yang, D., Saito, Y., Saeki, T., Koriyama, T., Nakata, W., Xin, D., & Saruwatari, H. (2025) · Speech Communication · doi:10.1016/j.specom.2025.103331 · Cited by 1
Bonfiglio, A. (2025) · The Journal of Supercomputing · doi:10.1007/s11227-025-08100-1 · Cited by 1
“Unlike LSTM, the BERT model considers the entire context to generate word representations, employing subword embeddings [56] and attention mechanisms.”
Efficient detection of AI-generated scientific abstracts with a lightweight transformer
Zhang, C., & Zhou, W. (2026) · Scientific Reports · doi:10.1038/s41598-026-35203-3 · Cited by 3
Numerical Constraint-Aware Dense Retrieval with Two-Phase Contrastive Learning
Wang, M., Wang, Y., & Wu, F. (2026) · Big Data Mining and Analytics · doi:10.26599/bdma.2025.9020047 · Cited by 1
Emoji-aware multimodal representations for multi-label emotion analysis in Indonesian social media
Amalia, A., Lydia, M. S., Rangkuti, R. P., Marulitua, F. P., Hanif, F., Fitra, M., & Gunawan, D. (2026) · PeerJ Computer Science · doi:10.7717/peerj-cs.3927 · Cited by 1
Geng, Z., & Yang, Y. (2026) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3805712.3809724 · Cited by 1
miLLi: Model Integrating Local Linguistic Insights for Morphologically Robust Tokenization
Rahimov, E. (2026) · Bilişim Teknolojileri Dergisi · doi:10.17671/gazibtd.1856668 · Cited by 1
Lukáč, M., Duchoň, F., Ivan, J., Dekan, M., Zelenay, E., Kocúr, M., Trepáčová, I., Jurov, T., & Pšenka, R. (2026) · Applied Sciences · doi:10.3390/app16157849 · Cited by 1
“Because languages build words from morphemes, an unfamiliar word may still decompose into familiar components: under the WordPiece strategy [10,11], unhappiness segments as un + happi + ness , each likely present in the vocabulary, allowing the model to infer meaning without prior exposure to the…”
Natural language embeddings of synthesis and testing conditions enhance glass dissolution prediction
Mannan, S., Nambudiripad, K. S., Mandal, I., Gosvami, N. N., & Krishnan, N. M. A. (2026) · Digital Discovery · doi:10.1039/d6dd00191b · Cited by 1
“The collected data was tokenized using MatSciBERT, which uses a WordPiece tokenizer 33,56,57 .”
Qizi, M. S. T., Djamaldinova, S., & Umirzakova, S. (2026) · Electronics · doi:10.3390/electronics15184236 · Cited by 1
Deep Bidirectional Transformers for Italian Question Answering.
Croce, D., Brandi, G., & Basili, R. (2019) · Cited by 1
R-grams: Unsupervised Learning of Semantic Units in Natural Language
Gyllensten, A. C., Ekgren, A., & Sahlgren, M. (2019) · Proceedings of the 13th International Conference on Computational Semantics - Student Papers · doi:10.18653/v1/w19-0607 · Cited by 3
“A close relative to the re-pair algorithm is Byte Pair Encoding (BPE) (Gage, 1994), first used in the context of segmentation by Schuster and Nakajima (2012) and recently popularized in within deep learning by Sennrich et al. (2016).”
Learning Embeddings from Scientific Corpora using Lexical, Grammatical and Semantic Information
García-Silva, A., Denaux, R., & Gómez-Pérez, J. M. (2019) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.4059300 · Cited by 1
Error Analysis for Visual Question Answering
Podtikhov, A., Shaban, M., Kovalev, A. K., & Panov, A. I. (2020) · Studies in computational intelligence · doi:10.1007/978-3-030-60577-3_34 · Cited by 1
Deep Learning-based Syllable Recognition Framework for Korean Children
Yang, S., Lee, M.-H., & Kim, H. (2021) · International Conference on Information Networking · doi:10.1109/icoin50884.2021.9333866 · Cited by 1
Learning to Lemmatize in the Word Representation Space.
Lagus, J., & Arto Klami, A. (2021) · DSpace repository (University of Tartu) · Cited by 1
Regularizing Word Segmentation by Creating Misspellings
Xu, H., Audhkhasi, K., Huang, Y., Emond, J., & Ramabhadran, B. (2021) · Interspeech (USB) · doi:10.21437/interspeech.2021-648 · Cited by 1
“We propose a new word tokenization algorithm, which is highly extensible and achieves better performance compared to the wordpiece model [9].”
ResultImproving Neural Language Processing with Named Entities
Matsushita, K., Makino, T., & Iwakura, T. (2021) · International conference Recent advances in natural language processing · doi:10.26615/978-954-452-072-4_107 · Cited by 1
“Then, cl-tohoku/bert-base-japanese-whole-word-masking’s tokenizer based on the WordPiece (Schuster and Nakajima, 2012) was applied to tokenized text with MeCab to de-compose each word into subwords.”
MethodHow to pretrain an efficient cross-disciplinary language model: The ScilitBERT use case
de la Broise, J.-B., Bernard, N., Dubuc, J.-P., Perlato, A., & Latard, B. (2021) · International Conference on Information Technology Research (ICITR) · doi:10.1109/icitr54349.2021.9657164 · Cited by 1
Contextualizer: Connecting the Dots of Context with Second-Order Attention
Maupomé, D., & Meurs, M. (2022) · Information · doi:10.3390/info13060290 · Cited by 1
“Following the Transformer approaches [7,10,27], the documents were tokenized into word-piece tokens [28,29].”
MethodBrian Roark, A., & Gutkin, A. (2022) · Workshop on Speech and Language Processing for Assistive Technologies · doi:10.18653/v1/2022.slpat-1.1 · Cited by 1
“For example, byte-pair encoding (Sennrich et al., 2016) or word-pieces (Schuster and Nakajima, 2012) are learned tokenizations that group together frequent character units, resulting in a configurable balance between the size of the vocabulary and the lengths of dependencies being effectively…”
Background
Metadata
- Authors
- Schuster, M., & Nakajima, K.
- Affiliations
- Google (United States) (US)
- Venue
- 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
- Publisher
- IEEE
- Published
- 2012-03
- Pages
- 5149-5152
- ISSN
- 1520-6149 (ISSN-L)
- OpenAlex
- W2121879602
- Cited by
- 862
- References
- 8
- Citations by year
- 2026: 36 · 2025: 66 · 2024: 90 · 2023: 124 · 2022: 137 · 2021: 155 · 2020: 144 · 2019: 59 · 2018: 36 · 2017: 10 · 2016: 3 · 2013: 2
- FWCI
- 0.49
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- IEEE International Conference on Acoustics Speech and Signal Processing (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
- Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Keywords
- voice search, Japanese language, Korean language, acoustic modeling
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.