The DOI matches its Crossref record; 47 of the 47 references it cites were found in a registry or an index.
Enriching Word Vectors with Subword Information
Open accessAbstract
Continuous word representations, trained on large unlabeled corpora are useful for many natural language processing tasks. Popular models that learn such representations ignore the morphology of words, by assigning a distinct vector to each word. This is a limitation, especially for languages with large vocabularies and many rare words. In this paper, we propose a new approach based on the skipgram model, where each word is represented as a bag of character n-grams. A vector representation is associated to each character n-gram; words being represented as the sum of these representations. Our method is fast, allowing to train models on large corpora quickly and allows us to compute word representations for words that did not appear in the training data. We evaluate our word representations on nine different languages, both on word similarity and analogy tasks. By comparing to recently proposed morphological word representations, we show that our vectors achieve state-of-the-art performance on these tasks.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Contrary to string similarity-based approaches, continuous word representations capture not only lexical but also semantic properties of tokens.”
From A Method for the Calculation of Parallel Passages for Buddhist Chinese Sources Based on Million-scale Nearest Neighbor Search (2020) · cites this paper as Bojanowski et al. (2017)
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“We screened 4 different word embeddings following a similar procedure reported in our previous study [46] and found that the Common Crawl embeddings—released by Facebook and trained using the fastText on the Common Crawl data set [47]—achieved better performance compared to other embeddings on a validation data set.”
From Extracting Family History of Patients From Clinical Narratives: Exploring an End-to-End Solution With Deep Learning Models (2020) · cites this paper as [47]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“I use Fasttext (Bojanowski et al. 2017) as a word representation model because of its capacity to collect sub-word information, which can be helpful in the case of compound words.”
From A Method for the Calculation of Parallel Passages for Buddhist Chinese Sources Based on Million-scale Nearest Neighbor Search (2020) · cites this paper as Bojanowski et al. 2017
The cited source says what the claim attributes to it.
In the citing paper
“This technique involves learning large-scale word embeddings (Bojanowski et al. 2017; Joulin et al. 2016), representing an advancement from Word2Vec.”
From Large language models (LLMs): survey, technical frameworks, and future challenges (2024) · cites this paper as Bojanowski et al. 2017
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Efficient Estimation of Word Representations in Vector Space
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1301.3781 · Cited by 26,568
Self-citationDistributed Representations of Words and Phrases and their Compositionality
Mikolov, T., Sutskever, I., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1310.4546 · Cited by 24,533
Self-citationThe proof and measurement of association between two things
Spearman, C. (1904) · The American Journal of Psychology · doi:10.2307/1412159 · Cited by 8,039
A unified architecture for natural language processing: deep neural networks with multitask learning
Collobert, R., & Weston, J. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390177 · Cited by 5,058
Harris, Z. S. (1954) · WORD · doi:10.1080/00437956.1954.11659520 · Cited by 4,807
Character-level Convolutional Networks for Text ClassificationPreprint
Zhang, X., Zhao, J., & LeCun, Y. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1509.01626 · Cited by 4,481
From Frequency to Meaning: Vector Space Models of Semantics
Turney, P. D., & Pantel, P. (2010) · Journal of Artificial Intelligence Research · doi:10.1613/jair.2934 · Cited by 2,949
Generating Sequences With Recurrent Neural Networks
Graves, A. (2013) · arXiv · doi:10.48550/arxiv.1308.0850 · Cited by 2,894
Producing high-dimensional semantic spaces from lexical co-occurrence
Lund, K., & Burgess, C. (1996) · Behavior Research Methods, Instruments, & Computers · doi:10.3758/bf03204766 · Cited by 1,875
Character-Aware Neural Language Models
Kim, Y., Jernite, Y., Sontag, D. A., & Rush, A. M. (2016) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v30i1.10362 · Cited by 1,590
Research works printed but not linked: 1
- Cui Qing, 2015, ACM Transactions on Information Systems, 34, 1
Cited by
Sorted by citations
Items 1,001–1,029 of 1,029
NODLINK: An Online System for Fine-Grained APT Attack Detection and Investigation
Li, S., Dong, F., Xiao, X., Wang, H., Shao, F., Chen, J., Guo, Y., Chen, X., & Li, D. (2024) · Proceedings 2024 Network and Distributed System Security Symposium · doi:10.14722/ndss.2024.23204 · Cited by 45
Latent Relation Language Models
Hayashi, H., Hu, Z., Xiong, C., & Neubig, G. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i05.6298 · Cited by 44
On the Evolution of Knowledge Graphs: A Survey and Perspective
Jiang, X., Xu, C., Shen, Y., Sun, X., Tang, L., Wang, S., Chen, Z., Wang, Y., & Guo, J. (2023) · arXiv · doi:10.48550/arxiv.2310.04835 · Cited by 42
Stojanov, R., Popovski, G., Cenikj, G., Koroušić Seljak, B., & Eftimov, T. (2021) · Journal of Medical Internet Research · doi:10.2196/28229 · Cited by 37
Natural Language Processing in Marketing
Hartmann, J., & Netzer, O. (2023) · Artificial Intelligence in Marketing · doi:10.1108/s1548-643520230000020011 · Cited by 38
Human-centered neural reasoning for subjective content processing: Hate speech, emotions, and humor
Kazienko, P., Bielaniewicz, J., Gruza, M., Kanclerz, K., Karanowski, K., Miłkowski, P., & Kocoń, J. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.01.010 · Cited by 38
LightSeq: A High Performance Inference Library for Transformers
Wang, X., Xiong, Y., Wei, Y., Wang, M., & Li, L. (2021) · Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies: Industry Papers · doi:10.18653/v1/2021.naacl-industry.15 · Cited by 36
ORBIT: A Real-World Few-Shot Dataset for Teachable Object Recognition
Massiceti, D., Zintgraf, L., Bronskill, J., Theodorou, L., Harris, M. T., Cutrell, E., Morrison, C., Hofmann, K., & Stumpf, S. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01064 · Cited by 33
Intrinsic bias metrics do not correlate with application bias
Goldfarb-Tarrant, S., Marchant, R., Muñoz Sánchez, R., Pandya, M., & Lopez, A. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.150 · Cited by 33
Yang, X., Zhang, H., He, X., Bian, J., & Wu, Y. (2020) · JMIR Medical Informatics · doi:10.2196/22982 · Cited by 29
Multi-Task, Multi-Channel, Multi-Input Learning for Mental Illness Detection using Social Media Text
Kirinde Gamaarachchige, P., & Inkpen, D. (2019) · Proceedings of the Tenth International Workshop on Health Text Mining and Information Analysis (LOUHI 2019) · doi:10.18653/v1/d19-6208 · Cited by 28
Yi, S., Goel, R., Khatri, C., Cervone, A., Chung, T., Hedayatnia, B., Venkatesh, A., Gabriel, R., & Hakkani-Tur, D. (2019) · Proceedings of the 12th International Conference on Natural Language Generation · doi:10.18653/v1/w19-8608 · Cited by 26
An Enhanced Neural Word Embedding Model for Transfer Learning
Kowsher, M., Sobuj, M. S. I., Shahriar, M. F., Prottasha, N. J., Arefin, M. S., Dhar, P. K., & Koshiba, T. (2022) · Applied Sciences · doi:10.3390/app12062848 · Cited by 27
Eliciting Latent Predictions from Transformers with the Tuned Lens
Belrose, N., Ostrovsky, I., McKinney, L., Furman, Z., Smith, L., Halawi, D., Biderman, S., & Steinhardt, J. (2023) · arXiv · doi:10.48550/arxiv.2303.08112 · Cited by 27
Human-in-the-loop AI in government: a case study
Benedikt, L., Joshi, C., Nolan, L., Henstra-Hill, R., Shaw, L., & Hook, S. (2020) · Proceedings of the 25th International Conference on Intelligent User Interfaces · doi:10.1145/3377325.3377489 · Cited by 24
Alsharid, M., Cai, Y., Sharma, H., Drukker, L., Papageorghiou, A. T., & Noble, J. A. (2022) · Medical Image Analysis · doi:10.1016/j.media.2022.102630 · Cited by 25
SemGloVe: Semantic Co-Occurrences for GloVe From BERT
Gan, L., Teng, Z., Zhang, Y., Zhu, L., Wu, F., & Yang, Y. (2022) · IEEE/ACM Transactions on Audio, Speech, and Language Processing · doi:10.1109/taslp.2022.3197316 · Cited by 22
(2025) · doi:10.1145/3833415 · Cited by 18
Investigating the Effects of Word Substitution Errors on Sentence Embeddings
Voleti, R., Liss, J. M., & Berisha, V. (2019) · ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2019.8683367 · Cited by 17
Xiao, F., Chen, S., Yang, J., He, H., Jiang, X., Tan, X., & Jin, D. (2025) · Computers & Security · doi:10.1016/j.cose.2024.104180 · Cited by 17
Abstractive Summarization of Reddit Posts with Multi-level Memory Networks
Kim, B., Kim, H., & Kim, G. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1260 · Cited by 96
Kariyawasam, H., Niwarthana, A., Palmer, A., Kay, J., & Withana, A. (2024) · Proceedings of the 29th International Conference on Intelligent User Interfaces · doi:10.1145/3640543.3645161 · Cited by 9
Improved Automated Classification of Sentences in Data Science Exercises
Angelone, A. M., Galassi, A., & Vittorini, P. (2022) · Lecture Notes in Networks and Systems · doi:10.1007/978-3-030-86618-1_2 · Cited by 10
van Slobbe, R., Herrmannova, D., Boeke, D. J., Lima-Walton, E. S., Abu-Hanna, A., & Vagliano, I. (2025) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2025.105815 · Cited by 7
HANSEN: Human and AI Spoken Text Benchmark for Authorship Analysis
Tripto, N., Uchendu, A., Le, T., Setzu, M., Giannotti, F., & Lee, D. (2023) · Findings of the Association for Computational Linguistics: EMNLP 2023 · doi:10.18653/v1/2023.findings-emnlp.916 · Cited by 6
Vacareanu, R., Valenzuela-Escárcega, M. A., Sharp, R., & Surdeanu, M. (2020) · Proceedings of the 28th International Conference on Computational Linguistics · doi:10.18653/v1/2020.coling-main.297 · Cited by 5
Nehrdich, S. (2020) · Journal of the Japanese Association for Digital Humanities · doi:10.17928/jjadh.5.2_132 · Cited by 3
“I use Fasttext (Bojanowski et al. 2017) as a word representation model because of its capacity to collect sub-word information, which can be helpful in the case of compound words.”
BackgroundMethodInfluentialSource Code Plagiarism Detection Using Siamese BLSTM Network and Embedding Models
Manahi, M., Sulaiman, S., & Bakar, N. S. A. A. (2022) · Lecture Notes in Electrical Engineering · doi:10.1007/978-981-16-8515-6_31 · Cited by 3
Stanza: A Python Natural Language Processing Toolkit for Many Human Languages
Qi, P., Zhang, Y., Zhang, Y., Bolton, J., & Manning, C. D. (2020) · arXiv · doi:10.48550/arxiv.2003.07082
Metadata
- Authors
- Bojanowski, P., Grave, E., Joulin, A., & Mikolov, T.
- Affiliations
- Meta (United States) (US)
- Facebook AI Research,
- Corresponding author
- Bojanowski, Piotr
- Joulin, Armand
- ORCID
- Bojanowski, Piotr
- Venue
- Transactions of the Association for Computational Linguistics
- Publisher
- MIT Press
- Published
- 2017-12
- Volume
- 5
- Pages
- 135-146
- ISSN
- 2307-387X (ISSN-L)
- OpenAlex
- W2493916176
- Cited by
- 9,645
- References
- 47
- Citations by year
- 2026: 389 · 2025: 677 · 2024: 790 · 2023: 1188 · 2022: 1253 · 2021: 1691 · 2020: 1561 · 2019: 1288 · 2018: 648 · 2017: 142
- FWCI
- 593.75
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- cc-by
- Locations
- Transactions of the Association for Computational Linguistics (publishedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Advanced Graph Neural Networks
- Keywords
- word embeddings, character n-grams, skip-gram model, out-of-vocabulary words, word similarity, word analogy, multilingual word representation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
