Mimicking Word Embeddings using Subword RNNs
Open accessThe DOI matches its Crossref record; 21 of the 21 references it cites were found in a registry or an index.
Abstract
Word embeddings improve generalization over lexical features by placing each word in a lower-dimensional space, using distributional information obtained from unlabeled data.However, the effectiveness of word embeddings for downstream NLP tasks is limited by out-of-vocabulary (OOV) words, for which embeddings do not exist.In this paper, we present MIM-ICK, an approach to generating OOV word embeddings compositionally, by learning a function from spellings to distributional embeddings.Unlike prior work, MIMICK does not require re-training on the original word embedding corpus; instead, learning is performed at the type level.Intrinsic and extrinsic evaluations demonstrate the power of this simple approach.On 23 languages, MIMICK improves performance over a word-based baseline for tagging part-of-speech and morphosyntactic attributes.It is competitive with (and complementary to) a supervised characterbased model in low-resource settings.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“However, it is hard to extract contextual information from characters to form meaningful words [50], [51], [52].”
From Log-based Anomaly Detection Without Log Parsing (2021) · cites this paper as [50]
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Therefore, Log2Vec could not effectively handle some domain-specific words, such as technical terms or entity names [50], [51], [52].”
From Log-based Anomaly Detection Without Log Parsing (2021) · cites this paper as [50]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · cited 1×
“(2015) propose a multi-level long shortterm memory (LSTM; Hochreiter and Schmidhuber, 1997), in which word embeddings are built compositionally from an LSTM over characters, and then tagging is performed by an LSTM over”
BackgroundEnriching Word Vectors with Subword Information
Bojanowski, P., Grave, E., Joulin, A., & Mikolov, T. (2017) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00051 · Cited by 9,645 · cited 5×
“compare to FastText (Bojanowski et al., 2016), a high-vocabulary, high-dimensionality embedding benchmark.”
MethodResultInfluentialCharacter-Aware Neural Language Models
Kim, Y., Jernite, Y., Sontag, D., & Rush, A. M. (2015) · arXiv · doi:10.48550/arxiv.1508.06615 · Cited by 1,530 · cited 1×
“Character-based approaches avoid these problems: for example, Kim et al. (2016) train a recurrent neural network over words, whose embeddings are constructed by convolution over character embeddings; Wiet-ing et al. (2016) learn embeddings of character n-grams, and then sum them into word…”
BackgroundCharacter-Aware Neural Language Models
Kim, Y., Jernite, Y., Sontag, D., & Rush, A. (2016) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v30i1.10362 · Cited by 1,491 · cited 1×
“Character-based approaches avoid these problems: for example, Kim et al. (2016) train a recurrent neural network over words, whose embeddings are constructed by convolution over character embeddings; Wiet-ing et al. (2016) learn embeddings of character n-grams, and then sum them into word…”
BackgroundBetter Word Representations with Recursive Neural Networks for Morphology
Luong, M.-T., Socher, R., & Manning, C. D. (2013) · Conference on Computational Natural Language Learning · Cited by 1,020
A Universal Part-of-Speech Tagset
Petrov, S., Das, D., & McDonald, R. (2012) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/3pjyez8kmkhz · Cited by 722
Learning Character-level Representations for Part-of-Speech Tagging
dos Santos, C., & Zadrozny, B. (2014) · Cited by 519
Universal Stanford dependencies: A cross-linguistic typology
de Marneffe, M.-C., Dozat, T., Silveira, N., Haverinen, K., Ginter, F., Nivre, J., & Manning, C. D. (2014) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/4ok3zdjx8t96 · Cited by 456 · cited 5×
“phosyntactic annotations through the Universal Dependencies (UD) corpus (De Marneffe et al., 2014). For example, Faruqui et al. (2016) propose a graph-based technique for propagating typelevel morphological information across a lexicon, improving token-level morphosyntactic tagging in 11 languages, using an SVM tagger.”
BackgroundMethodInfluentialEnriching Word Vectors with Subword Information
Bojanowski, P., Grave, É., Joulin, A., & Mikolov, T. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1607.04606 · Cited by 384 · cited 5×
“compare to FastText (Bojanowski et al., 2016), a high-vocabulary, high-dimensionality embedding benchmark.”
MethodResultInfluentialDyNet: The Dynamic Neural Network Toolkit
Neubig, G., Dyer, C., Goldberg, Y., Matthews, A., Ammar, W., Anastasopoulos, A., Ballesteros, M., Chiang, D., Clothiaux, D., Cohn, T., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1701.03980 · Cited by 302 · cited 2×
“We initialize character embeddings randomly, and use DyNet to implement the model (Neubig et al., 2017).”
Method
Research works printed but not linked: 8
- Željko Agić, Nikola Ljubešić, and Danijela Merkler. 2013. Lemmatization and morphosyntactic tagging of Croatian and Serbian. In 4th Biennial Interna- tional Workshop on Balto-Slavic Natural Language Processing (BSNLP 2013).
- Marta R Costa-jussà, David Aldón, and José AR Fonol- losa. 2017. Chinese-spanish neural machine trans- lation enhanced with character and word bitmap fonts. Machine Translation, pages 1-13.
- Marie-Catherine De Marneffe, Timothy Dozat, Na- talia Silveira, Katri Haverinen, Filip Ginter, Joakim Nivre, and Christopher D Manning. 2014. Univer- sal stanford dependencies: A cross-linguistic typol- ogy. In Proceedings of the Language Resources and Evaluation Conference (LREC), pages 4585-4592.
- Manaal Faruqui, Jesse Dodge, Sujay K Jauhar, Chris Dyer, Eduard Hovy, and Noah A Smith. 2015. Retrofitting word vectors to semantic lexicons. In Proceedings of the North American Chapter of the Association for Computational Linguistics (NAACL), Denver, CO.
- Jan Hajič and Barbora Hladká. 1998. Tagging inflec- tive languages: Prediction of morphological cate- gories for a rich, structured tagset. In Proceed- ings of the Association for Computational Linguis- tics (ACL), pages 483-490.
- Wang Ling, Tiago Luís, Luís Marujo, Ramón Fernan- dez Astudillo, Silvio Amir, Chris Dyer, Alan W Black, and Isabel Trancoso. 2015. Finding function in form: Compositional character models for open vocabulary word representation. In Proceedings of Empirical Methods for Natural Language Process- ing (EMNLP).
- Thomas Müller, Helmut Schmid, and Hinrich Schütze. 2013. Efficient higher-order CRFs for morpholog- ical tagging. In Proceedings of Empirical Methods for Natural Language Processing (EMNLP), pages 322-332.
- Barbara Plank, Anders Søgaard, and Yoav Goldberg. 2016. Multilingual part-of-speech tagging with bidirectional long short-term memory models and auxiliary loss. In Proceedings of the Association for Computational Linguistics (ACL), Berlin.
Cited by
Items 101–174 of 174
Hide and Seek: Revisiting DNS-based User Tracking
Chang, D., Chen, J. Q., Li, Z., & Li, X. (2022) · IEEE European Symposium on Security and Privacy · doi:10.1109/eurosp53844.2022.00020 · Cited by 11
“The NLP community also faces this “out-of-vocabulary” problem, and methods in this area [75], [76] can be leveraged to tackle this issue.”
BackgroundNeural Semi-Markov Conditional Random Fields for \nRobust Character-Based Part-of-Speech Tagging
Kemos, A., Adel, H., & Schütze, H. (2019) · Open access LMU (Ludwid Maxmilian's Universitat Munchen) · doi:10.5282/ubm/epub.61846 · Cited by 4
Lifelong Learning of Topics and Domain-Specific Word Embeddings
Qin, X., Lu, Y., Chen, Y., & Rao, Y. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.202 · Cited by 4
“The motivation of introducing SG is to capture more information from the structures of words themselves as the complement for global and local contexts (Bojanowski et al., 2017; Pinter et al., 2017).”
BackgroundLi, X., You, S., Niu, Y., & Chen, W. (2021) · Joint Conference on Lexical and Computational Semantics · doi:10.18653/v1/2021.starsem-1.26 · Cited by 4
“Pinter et al. (2017) proposed the Mim-ick model to predict vectors for out-of-vocabulary words by learning a function from spellings to distributional embeddings.”
MethodImproving Word Embeddings through Iterative Refinement of Word- and Character-level Models
Ha, P., Zhang, S., Djuric, N., & Vučetić, S. (2020) · International Conference on Computational Linguistics · doi:10.18653/v1/2020.coling-main.104 · Cited by 4
“Recently proposed mimicking approach is a promising solution to this problem, with two representative algorithms being Mimick (Pinter et al., 2017) and GWR (Kim et al., 2018).”
BackgroundMethodResultInfluentialSubword-Based Compact Reconstruction for Open-Vocabulary Neural Word Embeddings
Sasaki, S., Suzuki, J., & Inui, K. (2021) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2021.3125133 · Cited by 5
Neural Semi-Markov Conditional Random Fields for Robust Character-Based Part-of-Speech TaggingPreprint
Kemos, A., Adel, H., & Schütze, H. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1808.04208 · Cited by 5
“Character-level models are attractive since they can effectively model morphological variants of words and build representations even for unknown words, suffering less from out-of-vocabulary problems (Pinter et al., 2017).”
BackgroundImproving Pre-Trained Multilingual Models with Vocabulary ExpansionPreprint
Wang, H., Yu, D., Sun, K., Chen, J., & Yu, D. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.12440 · Cited by 4
“, 2017) or develop hybrid word/subword-character architectures (Chung et al., 2016; Luong and Manning, 2016; Pinter et al., 2017; Bahdanau et al., 2017; Matthews et al., 2018; Li et al., 2018).”
BackgroundMethodEnhancing Out-Of-Vocabulary Word Representations with Large Language Models
Kim, M., Lee, S.-J., & Choi, J. (2024) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata62323.2024.10825494 · Cited by 3
Learning to Generate Representations for Novel Words: Mimic the OOV Situation in Training
Xing, X., Peng, M., Zhang, Q., Liu, Q., & Huang, X. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-60450-9_26 · Cited by 3
drsphelps at SemEval-2022 Task 2: Learning idiom representations using BERTRAM
Phelps, D. (2022) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/2022.semeval-1.18 · Cited by 3
Ganguly, D., Bhat, S., & Biswas, C. (2022) · Annual Meeting of the Forum for Information Retrieval Evaluation (FIRE) · doi:10.1145/3574318.3574346 · Cited by 3
“A recurrent neural network based solution is proposed in [37] to generate embedding of out-of-vocabulary words.”
BackgroundControl Stochastic Selection-Based Biomedical Text Summarization Using Sim-TLBO
Gupta, S., Sharaff, A., & Nagwani, N. K. (2023) · Arabian Journal for Science and Engineering · doi:10.1007/s13369-023-08056-5 · Cited by 3
“Sim-TLBO is a novel iterative technique with unique properties providing powerful shortening of text content [34].”
A Margin-based Loss with Synthetic Negative Samples for Continuous-output Machine Translation
Bhat, G., Kumar, S., & Tsvetkov, Y. (2019) · Workshop on Neural Generation and Translation (WNGT) · doi:10.18653/v1/d19-5621 · Cited by 3
“Closely related work on embedding prediction includes zero-shot learning for word translation (Nakashole, 2018; Conneau et al., 2018) and image labeling (Lazaridou et al., 2015), as well as rare word prediction (Pinter et al., 2018) and classification (Card et al., 2019).”
BackgroundAutomatic Adaptation of Swedish Text for Increased Inclusion
Rennes, E. (2022) · Linköping studies in science and technology. Dissertations · doi:10.3384/9789179292706 · Cited by 14
Robust Word Vectors: Context-Informed Embeddings for Noisy Texts
Malykh, V., Khakhulin, T., & Logacheva, V. (2023) · Journal of Mathematical Sciences · doi:10.1007/s10958-023-06523-w · Cited by 3
“The work [24] approximates pretrained word embeddings with a character-level model.”
MethodEfficient Contextual Representation Learning Without Softmax LayerPreprint
Li, L. H., Chen, P. H., Hsieh, C., & Chang, K.-W. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.11269 · Cited by 3
“We exploit a particular kind of word representation, the open-vocabulary word embedding, such as the FastText embedding and the MIMICK model (Pinter et al., 2017).”
MethodPredicting Semantic Relations using Global Graph PropertiesPreprint
Pinter, Y., & Eisenstein, J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1808.08644 · Cited by 7
“5 To handle these cases, we trained a character-based model called M IMICK , which learns to predict embed-dings for out-of-vocabulary items based on their spellings (Pinter et al., 2017).”
MethodSelf-citationWord Embedding Algorithms as Generalized Low Rank Models and their Canonical FormPreprint
Kenyon-Dean, K. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.02639 · Cited by 3
“These methods can involve training a characterlevel LSTM on the pre-trained word embeddings with the objective of “mimicking” them (Pinter et al., 2017), and least-squares-based methods have also proven to work quite well in producing such subword embeddings (Stratos, 2017; Zhao et al.”
MethodBridging Subword Gaps in Pretrain-Finetune Paradigm for Natural Language GenerationPreprint
Liu, X., Yang, B., Liu, D., Zhang, H., Luo, W., Zhang, M., Zhang, H., & Su, J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.06125 · Cited by 3
“In this context, researchers use embeddings of characters or subwords to predict those of unseen words (Pinter et al., 2017; Zhao et al., 2018; Sasaki et al., 2019; Fukuda et al., 2020).”
BackgroundIntegrating Approaches to Word RepresentationPreprint
Pinter, Y. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2109.04876 · Cited by 4
“Empirical analyses such as the one in Pinter et al. (2017) show that indeed, the overwhelming majority of downstream datasets contain words not present in the pre-training corpora.”
BackgroundSelf-citationAnalysis of Subword based Word Representations Case Study: Fasttext Malayalam
Vivek, M., & Chandran, P. (2022) · IEEE India Conference · doi:10.1109/indicon56171.2022.10040147 · Cited by 2
“Luong & Manning [10], Costa-Jussa & Fonollosa [11] and Pinter et al. [12] generate character representations for out-of-vocabulary (OOV) words using LSTM architecture and use existing word representations for available words.”
Enhanced anomaly detection in log-based systems using GAN-based models
Farjamfard, E., & Shameli‐Sendi, A. (2025) · Cluster Computing · doi:10.1007/s10586-025-05535-2 · Cited by 3
“To manage OOV words at runtime, the MIMICK approach [ 67] is employed.”
Is it simpler? An Evaluation of an Aligned Corpus of Standard-Simple Sentences
Rennes, E. (2020) · Cited by 2
Deep Learning Trends and Inspired Systems in Natural Language Processing
Ansari, A. A., Rautaray, S. S., & Pandey, M. (2022) · Studies in big data · doi:10.1007/978-981-19-5154-1_5 · Cited by 2
Author-Topic Modelling for Reviewer Assignment of Scientific Papers in Bahasa Indonesia
Kusumawardani, R. P., & Khairunnisa, S. O. (2018) · International Conference on Asian Language Processing · doi:10.1109/ialp.2018.8629124 · Cited by 2
“In this paper, we extend Author-Topic Modelling with the specific aim of handling morphologically agglutinative language, in this case Bahasa Indonesia [1].”
BackgroundMethodCharacter N-gram-Based Word Embeddings for Morphological Analysis of Texts
Ghukasyan, T. (2020) · Proceedings of the Institute for System Programming of RAS · doi:10.15514/ispras-2020-32(2)-1 · Cited by 2
Learning New Word Semantics with Conceptual Text
Pan, W., Liu, T., Zhang, W., & Sun, Y. (2021) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn52387.2021.9533503 · Cited by 2
“Pinter et al. [7] proposed the mimic method to represent a word by its subword embeddings and show that with these embeddings the performances on some natural language tasks are better than with value <unk> on unknown words.”
MethodComposing Word Embeddings for Compound Words Using Linguistic Knowledge
Komiya, K., Kono, S., Seito, T., & Hirabayashi, T. (2022) · ACM Transactions on Asian and Low-Resource Language Information Processing · doi:10.1145/3561299 · Cited by 4
“[21] proposed a method called MIMICK, which is used to compose word embeddings by mimicking word embeddings using subwords and bi-LSTM.”
MethodLearning to Learn to Predict Performance Regressions in Production at Meta
Beller, M., Li, H., Nair, V., Murali, V., Ahmad, I., Cito, J., Carlson, D., Aye, G. A., & Dyer, W. (2023) · doi:10.1109/ast58925.2023.00010 · Cited by 9
“This is because DiffBERT (1) uses a sub-word tokenizer, which means that there is substantial overlap between code changes in various programming languages (also to help deal with the out of vocabulary problem, [57]), (2) has been pre-trained on a wide number of different languages, apparently enabling it to work on a…”
MethodAbzianidze, L., Bylinina, L., & Paperno, D. (2024) · Cambridge University Press eBooks · doi:10.1017/9781009542340 · Cited by 1
Tiny Word Embeddings Using Globally Informed Reconstruction
Ohashi, S., Isogawa, M., Kajiwara, T., & Arase, Y. (2020) · International Conference on Computational Linguistics · doi:10.18653/v1/2020.coling-main.103 · Cited by 2
“As reconstruction networks, a recurrent neural network (RNN) (Pinter et al., 2017), convolutional neural network (CNN) (Kim et al., 2018), and self-attention mechanism (Sasaki et al., 2019) were used in the previous studies.”
BackgroundMethodInfluentialAl-Khdour, N., Younes, M. B., Abdullah, M., & AL-Smadi, M. (2020) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/2020.semeval-1.37 · Cited by 2
“• fastText : FastText Embeddings provide vectors for Out Of Vocabulary (oov) (Pinter et al., 2017) words by using the sub-word information.”
MethodA Hybrid Semantic Parsing Approach for Tabular Data AnalysisPreprint
Gao, Y., Lou, J., & Zhang, D. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.10363 · Cited by 4
FastContext: Handling Out-of-Vocabulary Words Using the Word Structure and Context
Silva, R. M., Lochter, J. V., Almeida, T. A., & Yamakami, A. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-21689-3_38 · Cited by 1
A probabilistic model for API contract specification retrieval focusing on the openAPI standard
Moon, S., Kerr, G., Silavong, F., & Moran, S. (2024) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-024-01073-4 · Cited by 1
“Although there have been efforts to fine-tune embeddings for low-resource domains, these approaches struggle with the rapidly growing vocabulary described in challenge (1) (Bahdanau et al. 2018)(Schick and Schütze 2019) (Pinter et al. 2017)(Moreo et al. 2021).”
Learning Dynamic Representations in Large Language Models for Evolving Data Streams
Srivastava, A., Bhatnagar, S., Murty, M. N., & Raman, J. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-78169-8_16 · Cited by 1
Context and Embeddings in Language Modelling - an Exploration.
Nitsche, M., & Tropmann-Frick, M. (2018) · DAMDID/RCDL · Cited by 1
Morphologically-Aware Vocabulary Reduction of Word Embeddings
Chia, C. C., Tkachenko, M., & Lauw, H. W. (2022) · IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology (WI-IAT) · doi:10.1109/wi-iat55865.2022.00018 · Cited by 1
“[26], [27] requires separate models to represent OOV words.”
BackgroundEnhancing Out-of-Vocabulary Estimation with Subword Attention
Patel, R. N., & Domeniconi, C. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.221 · Cited by 3
“Some OOV strategies use word roots of the OOV word to estimate OOV embeddings (Bojanowski et al., 2017; Pinter et al., 2017; Sasaki et al., 2019) while other methods use the OOV word’s context (Lazaridou et al., 2017; Horn, 2017; Herbelot and Baroni, 2017; Arora et al., 2017; Mu and Viswanath,…”
MethodScope and Challenges of Language Modelling - An Interrogative Survey on Context and Embeddings
Nitsche, M., & Tropmann-Frick, M. (2019) · Communications in computer and information science · doi:10.1007/978-3-030-23584-0_8 · Cited by 1
Beyond Context: A New Perspective for Word Embeddings
Zhou, Y., & Srikumar, V. (2019) · Joint Conference on Lexical and Computational Semantics · doi:10.18653/v1/s19-1003 · Cited by 1
“Some recent work (Pinter et al., 2017; Kim et al., 2018; Zhao et al., 2018; Artetxe et al., 2018) address the OOV problem using pre-trained embeddings and mimicking them by training a second model using substrings of a given word.”
BackgroundMethodLi, X., You, S., & Chen, W. (2019) · International Conference on Machine Learning and Applications · doi:10.1109/icmla.2019.00175 · Cited by 1
“[4] proposed the Mimick model to predict vectors for out-of-vocabulary (OOV) words compositionally by learning a function from spellings to distributional embeddings.”
MethodZong, C., Xia, R., & Zhang, J. (2021) · Springer eBooks · doi:10.1007/978-981-16-0100-2_3 · Cited by 1
Attending Form and Context to Generate Specialized Out-of-VocabularyWords RepresentationsPreprint
Garneau, N., Leboeuf, J.-S., Pinter, Y., & Lamontagne, L. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.06876 · Cited by 1
“A consequence of the proposed approach is that our model removes the need for a separate pre-training step as in previous work (Pinter et al., 2017; Zhao et al., 2018; Schick and Schütze, 2018), since it learns how to generate context-dependent and task-specific representations using the sequence…”
MethodInfluentialSelf-citationKnowledge Efficient Deep Learning for Natural Language ProcessingPreprint
Wang, H. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2008.12878 · Cited by 3
“OOV poses challenges for many tasks [141] such as machine translation [136, 142] and sentiment analysis [143].”
MethodTrajectory-Based Meta-Learning for Out-Of-Vocabulary Word Embedding LearningPreprint
Buck, G., & Vlachos, A. (2021) · arXiv (Cornell University) · doi:10.17863/cam.78765 · Cited by 2
Millour, A. (2020) · theses.fr (ABES) · doi:10.70675/9bad1817z09cez47d6zb8bbz333581314df3 · Cited by 1
Leveraging Context Information for Joint Entity and Relation Linking
Zhao, Y. H., Xu, Z., & Hu, W. P. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-33982-1_3
Motion Generation Using Humanoid Robot with Language Understanding
Hamazono, Y., Kobayashi, I., Asoh, H., NAKAMURA, T., Nagai, T., & Mochihashi, D. (2020) · Journal of Japan Society for Fuzzy Theory and Intelligent Informatics · doi:10.3156/jsoft.32.1_632
Representation Learning for Unseen Words by Bridging Subwords to Semantic Networks
Kim, Y., Kim, K. M., & Lee, S. K. (2020)
Yang, Z., Zhu, C., Sachidananda, V., & Darve, E. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.03753 · Cited by 2
“…to impute embed-dings, we conduct experiments on the following rare and unseen word similarity tasks. embedding imputation models, including Mimick (Pinter et al., 2017), Definition centroid (Herbe-lot and Baroni, 2017), Definition LSTM (Bah-danau et al., 2017), SemLand (Pilehvar and Col-lier,…”
MethodDeep Learning for Unstructured Data by Leveraging Domain Knowledge
Zhang, S. (2019) · TUScholarShare (Temple University)
Compact Word Embeddings Based on Global Similarity
Ohashi, S., Isogawa, M., Kajiwara, T., & Arase, Y. (2021) · Journal of Natural Language Processing · doi:10.5715/jnlp.28.235
“模倣元の単語分散表現には 300次元の学習済み fastText (crawl-300d-2M-subword) (Bojanowski et al. 2017)を用いた.このうち高頻度な 10万単語を用いて模倣モデルを訓練した.なお,単 語の出現頻度は fastText の訓練コーパスを用いて計算した.ただし,記号や数字を含む単語は 除外し,小文字ラテンアルファベットの 26種類の文字のみで構成される単語を対象にした. 大域的な損失計算のために,n = 10単語をサンプリングした.式 (3)の最適化には,Adam (Kingma and Ba 2015)を α = 0.001, β1 = 0.9, β2 = 0.999, ϵ =…”
InfluentialTowards an Embedding-Based Approach for the Geolocation of Texts and Users on Social Networks
Hasni, S. (2021) · Advances in geospatial technologies book series · doi:10.4018/978-1-7998-1954-7.ch012
FMEBA: A Fusion Multi-feature Model for Chinese Out of Vocabulary Word Embedding Generation
Huang, X., Yang, Q., Liu, S., Yuan, X., & Guo, R. (2021) · Lecture notes on data engineering and communications technologies · doi:10.1007/978-3-030-70665-4_173
SocialBully: A Social Information-Driven Cyberbullying Detector with Similarity-Based Word Embedding
Zhao, Z., Gao, M., Luo, F., & Ranzi, G. (2021) · Asian Conference on Artificial Intelligence Technology (ACAIT) · doi:10.1109/acait53529.2021.9731340
A Deep Neural Network Model for Cross-Domain Sentiment Analysis
Kumari, S., Agarwal, B., & Mittal, M. (2022) · IGI Global eBooks · doi:10.4018/978-1-6684-6303-1.ch008
Comparative Evaluation on Effect of ELMo in Combination with Machine Learning, and Ensemble Models
Yazdizadeh, T. (2022) · Carleton University's Institutional Repository (MacOdrum Library, Carleton University) · doi:10.22215/etd/2022-15230
Low-Dimensionality Information Extraction Model for Semi-structured Documents
Belhadj, D., Belaïd, A., & Belaïd, Y. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-44237-7_8 · Cited by 1
Dictionary-based Sentiment Analysis at Subword Level
Yang, J. (2024) · IEEE International Conference on Industry 4.0, Artificial Intelligence, and Communications Technology (IAICT) · doi:10.1109/iaict62357.2024.10617606
“Subword embeddings for deep learning are known to often improve NLP performance [21][22].”
BackgroundATA: Attentive Text Augmentation for Typo-Robust Language Models
Han, J.-H., & Lee, S. K. (2025) · Lecture notes in computer science · doi:10.1007/978-981-97-8702-9_24
Handling Korean Out-of-Vocabulary Words with Phoneme Representation Learning
Kim, N. D., Jeon, E., Park, J.-H., & Lee, S. (2025) · Lecture notes in computer science · doi:10.1007/978-981-96-8180-8_38
“As an alternative method, [21] has proposed a character-level recurrent neural network that mimics pre-trained word embeddings.”
InfluentialOn resolving the out of vocabulary problem in DisCoCat-based quantum natural language processing
Bhatuse, A., Khandelwal, A., Udmale, S. S., Chandra, M. G., & Kolte, S. (2025) · Evolving Systems · doi:10.1007/s12530-025-09714-9
“Few works have been proposed to address this problem in the classical NLP domain (Pinter et al. 2017; Garneau et al. 2018).”
A Multi-Aspect Evaluation of Tokenizer Adaptation Methods for Large Language Models on Russian
Andriushchenko, G. D., Godunova, M. E., Иванов, В. В., Kuzmin, D. S., Parinov, A. A., Schenikova, A. Y., & Zhemchuzhina, E. V. (2025) · Doklady Mathematics · doi:10.1134/s1064562425700395
Poltavchenko, I. (2018) · theses.fr (ABES) · doi:10.70675/ac5626fez9961z4f2fz8355z0ce6c4c2287f
Neural Transfer Learning for Domain Adaptation in Natural Language Processing
Meftah, S. (2021) · theses.fr (ABES) · doi:10.70675/9111562bzfa37z4fd8zae6bz2e8ae8d5d403 · Cited by 2
Towards efficient, general and robust entity disambiguation systems
Chen, L. (2023) · theses.fr (ABES) · doi:10.70675/abee8a0cz824dz4e34z917azffda9fe5cd83
Wang, Y., Liu, C., Zhang, R., Xu, H., Zhao, J., Sun, Y., & Wang, X. (2026) · Cybersecurity · doi:10.1186/s42400-025-00480-4
“Mimick (Pinter et al. 2017), as an early deep learning OOV embedding method, generated word vectors by learning word morphology structures but has inherent limitations–it cannot effectively capture the semantic shifts of poly-semous OOV words in different contexts.”
Metadata
- Authors
- Pinter, Y., Guthrie, R., & Eisenstein, J.
- Affiliations
- Georgia Institute of Technology (US)
- ORCID
- Pinter, Yuval
- Venue
- Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing
- Published
- 2017
- Pages
- 102-112
- OpenAlex
- W2737092125
- Cited by
- 177
- References
- 20
- Citations by year
- 2026: 1 · 2025: 5 · 2024: 6 · 2023: 14 · 2022: 17 · 2021: 35 · 2020: 29 · 2019: 40 · 2018: 27 · 2017: 2
- FWCI
- 12.22
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Conference on Empirical Methods in Natural Language Processing (EMNLP) (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Defense Threat Reduction Agency (HDTRA1)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- word embeddings, out-of-vocabulary words, distributional semantics, part-of-speech tagging, morphosyntactic tagging, low-resource languages, multilingual word representation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.

