Improving Vector Space Word Representations Using Multilingual Correlation
Open accessThe DOI matches its Crossref record; 43 of the 43 references it cites were found in a registry or an index.
Abstract
The distributional hypothesis of Harris (1954), according to which the meaning of words is evidenced by the contexts they occur in, has motivated several effective techniques for obtaining vector space semantic representations of words using unannotated text corpora. This paper argues that lexico-semantic content should additionally be invariant across languages and proposes a simple technique based on canonical correlation analysis (CCA) for incorporating multilingual evidence into vectors generated monolingually. We evaluate the resulting word representations on standard lexical semantic evaluation tasks and show that our method produces substantially better semantic representations than monolingual techniques.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Besides, Hewitt and Manning [125], Jawahar et al. [126], Kim et al. [127] proposed several methods to extract dependency trees and constituency trees from BERT, which proved the BERT's ability to encode syntax structure.”
From Pre-trained models for natural language processing: A survey (2020) · cites this paper as [125]
Citations
Cites
(2008) · Journal of Machine Learning Research · Cited by 33,051
Efficient Estimation of Word Representations in Vector Space
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1301.3781 · Cited by 26,568 · cited 6×
“This dataset contains word pairs that are different syntactic forms of a given word and was prepared by Mikolov et al. (2013a).”
BackgroundMethodInfluentialIndexing by latent semantic analysis
Deerwester, S., Dumais, S. T., Furnas, G. W., Landauer, T. K., & Harshman, R. (1990) · Journal of the American Society for Information Science · doi:10.1002/(sici)1097-4571(199009)41:6<391::aid-asi1>3.0.co;2-9 · Cited by 12,751
Golub, G. H., & Loan, C. F. V. (1996) · Johns Hopkins University Press eBooks · Cited by 6,027
A unified architecture for natural language processing: deep neural networks with multitask learning
Collobert, R., & Weston, J. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390177 · Cited by 5,058 · cited 6×
“Our method can also be seen as an application of multi-view learning (Chang et al., 2013; Collobert and Weston, 2008), where one of the views can be used to capture cross-lingual information.”
BackgroundMethodInfluentialHarris, Z. S. (1954) · WORD · doi:10.1080/00437956.1954.11659520 · Cited by 4,807
Approximate Statistical Tests for Comparing Supervised Classification Learning Algorithms
Dietterich, T. G. (1998) · Neural Computation · doi:10.1162/089976698300017197 · Cited by 3,840 · cited 2×
“01) according to the McNemar’s test (Dietterich, 1998).”
MethodInfluentialWord association norms, mutual information, and lexicography
Church, K., & Hanks, P. (1990) · doi:10.5555/89086.89095 · Cited by 3,590
Linguistic Regularities in Continuous Space Word Representations
Mikolov, T., Yih, W.-T., & Zweig, G. (2013) · North American Chapter of the Association for Computational Linguistics · Cited by 2,912
From Frequency to Meaning: Vector Space Models of Semantics
Turney, P. D., & Pantel, P. (2010) · Journal of Artificial Intelligence Research · doi:10.1613/jair.2934 · Cited by 2,866 · cited 1×
“Using cooccurrence statistics from a large corpus of text (Deerwester et al., 1990; Turney and Pantel, 2010), 1 it is possible to construct high-quality semantic vectors — as judged by both correlations with human judgements of semantic relatedness (Turney, 2006; Agirre et al., 2009) and as…”
Background
Cited by
Items 101–200 of 617
Supersense Embeddings: A Unified Model for Supersense Interpretation, Prediction, and Utilization
Flek, L., & Gurevych, I. (2016) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p16-1191 · Cited by 56
“Moreover, the coarse semantic categorization contained in su-persenses was shown to be preserved in translation (Schneider et al., 2013), making them a perfect candidate for a multilingual adaptation of the vector space, e.g. extending Faruqui and Dyer (2014).”
BackgroundWord Embeddings for Code-Mixed Language Processing
Pratapa, A., Choudhury, M., & Sitaram, S. (2018) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d18-1344 · Cited by 55
“In this paper, we compare three popular bilingual word embedding techniques (Sec 2): Bilingual correlation based embeddings (BiCCA) (Faruqui and Dyer, 2014), Bilingual compositional model (BiCVM) (Hermann and Blunsom, 2014) and Bilingual Skip-gram (BiSkip) (Luong et al.”
MethodMultilingual Multimodal Pre-training for Zero-Shot Cross-Lingual Transfer of Vision-Language Models
Huang, P.-Y., Patrick, M., Hu, J., Neubig, G., Metze, F., & Hauptmann, A. G. (2021) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2021.naacl-main.195 · Cited by 91
NORMA: Neighborhood Sensitive Maps for Multilingual Word Embeddings
Nakashole, N. (2018) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d18-1047 · Cited by 61
“Another approach is to use canonical correlation analysis (CCA) to map two languages to a shared embedding space (Haghighi et al., 2008; Faruqui and Dyer, 2014; Lu et al., 2015; Ammar et al., 2016).”
BackgroundA Call for More Rigor in Unsupervised Cross-lingual Learning
Artetxe, M., Ruder, S., Yogatama, D., Labaka, G., & Agirre, E. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.658 · Cited by 87
“Early work required a bilingual dictionary to learn such a transformation (Mikolov et al., 2013a; Faruqui and Dyer, 2014).”
BackgroundSingle or Multiple? Combining Word Representations Independently Learned from Text and WordNet
Goikoetxea, J., Agirre, E., & Soroa, A. (2016) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v30i1.10321 · Cited by 61
“Table 2: Results as Spearman for the embeddings used in (Faruqui and Dyer 2014) (FAR, top row), and the absolute gain when using retrofitting to combine two varieties of WordNet information (+WNsh and +WNall).”
BackgroundMethodInfluentialAre Multilingual Models Effective in Code-Switching?
Winata, G. I., Cahyawijaya, S., Liu, Z., Lin, Z., Madotto, A., & Fung, P. (2021) · Workshop on Computational Approaches to Code Switching · doi:10.18653/v1/2021.calcs-1.20 · Cited by 105
Identifying Elements Essential for BERT’s Multilinguality
Dufter, P., & Schütze, H. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.358 · Cited by 139
“With the rise of static word embeddings, many multilingual embedding algorithms have been proposed (Mikolov et al., 2013; Hermann and Blunsom, 2014; Faruqui and Dyer, 2014); for a survey Figure 1: Multilinguality in our BERT model (0) is harmed by three architectural modifications: lang-pos, shift-special, no-random (…”
BackgroundMethodInfluentialOn Deep Multi-View Representation Learning: Objectives and OptimizationPreprint
Wang, W., Arora, R., Livescu, K., & Bilmes, J. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1602.01024 · Cited by 70
The representational geometry of word meanings acquired by neural machine translation models
Hill, F., Cho, K., Jean, S., & Bengio, Y. (2017) · Machine Translation · doi:10.1007/s10590-017-9194-2 · Cited by 54
“Consistent with the qualitative observation made by Faruqui and Dyer (2014), we suggested how the nature of the semantic correspondence between the words in languages enables NMT embeddings to distinguish synonyms and antonyms and, more generally, to encode the information needed to reflect human…”
BackgroundMethodResultInfluentialBilingual Dictionary Based Neural Machine Translation without Using Parallel Sentences
Duan, X., Ji, B., Jia, H., Tan, M., Zhang, M., Chen, B., Luo, W., & Zhang, Y. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.143 · Cited by 66
Tareq, M., Islam, M. F., Deb, S., Rahman, S., & Mahmud, A. A. (2023) · IEEE Access · doi:10.1109/access.2023.3277787 · Cited by 64
“To study the effect of word embeddings, Pratapa et al. [46] compared three bilingual word embedding approaches: bilingual correlation based embeddings [48], bilingual compositional model [49] and bilingual Skip-gram [50], to perform code-mixed sentiment analysis and part-of-speech tagging.”
MethodPolyglot Semantic Role Labeling
Mulcaire, P., Swayamdipta, S., & Smith, N. A. (2018) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p18-2106 · Cited by 47
“…embeddings from the monolingual embeddings using the method of Ammar et al. (2016b): for each non-English language, a small crosslingual dictionary and canonical correlation analysis was used to find a transformation of the non-English vectors into the English vector space (Faruqui and Dyer, 2014).”
MethodScalable and Flexible Multiview MAX-VAR Canonical Correlation Analysis
Fu, X., Huang, K., Hong, M., Sidiropoulos, N. D., & So, A. M. (2017) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2017.2698365 · Cited by 45
“In multi-lingual word embedding [10], [16], [23], for example, it is known that outlying features (“stop words”), may exist.”
BackgroundTwo-Level Transformer and Auxiliary Coherence Modeling for Improved Text Segmentation
Glavaš, G., & Somasundaran, S. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i05.6284 · Cited by 89
Unsupervised Bilingual Word Embedding Agreement for Unsupervised Neural Machine Translation
Sun, H., Wang, R., Chen, K., Utiyama, M., Sumita, E., & Zhao, T. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1119 · Cited by 41
“Since 2013, neural network based bilingual word embedding (BWE) has been applied to several natural language processing tasks (Mikolov et al., 2013; Faruqui and Dyer, 2014; Xing et al., 2015; Dinu et al., 2015; Lu et al., 2015; Wang et al., 2016; Artetxe et al., 2016; Smith et al., 2017; Wang et…”
MethodA Survey on Computational Metaphor Processing
Rai, S., & Chakraverty, S. (2020) · ACM Computing Surveys · doi:10.1145/3373265 · Cited by 76
“They use word embeddings provided by Faruqui and Dyer [2014].”
MethodMultilingual Reliability and "Semantic" Structure of Continuous Word Spaces
Köper, M., Scheible, C., & Walde, S. S. I. (2015) · Cited by 43
Multilingual Training of Crosslingual Word Embeddings
Duong, L., Kanayama, H., Ma, T., Bird, S., & Cohn, T. (2017) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/e17-1084 · Cited by 42
“Multilingual lexical resources such as PanLex (Kamholz et al., 2014) and Wiktionary2 cover thousands of languages, and have been used to construct high performance crosslingual word embeddings (Mikolov et al., 2013a; Xiao and Guo, 2014; Faruqui and Dyer, 2014).”
BackgroundInfluentialDomain Adapted Word Embeddings for Improved Sentiment Classification
Sarma, P. K., Liang, Y., & Sethares, B. (2018) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p18-2007 · Cited by 43
Improving Zero-Shot Cross-lingual Transfer for Multilingual Question Answering over Knowledge Graph
Zhou, Y., Geng, X., Shen, T., Zhang, W., & Jiang, D. (2021) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2021.naacl-main.465 · Cited by 108
“Early works focus on aligning multilingual word embedding (Mikolov et al., 2013; Faruqui and Dyer, 2014; Xu et al., 2018), while recent efforts are mainly made on large-scale pre-trained multilingual encoder, such as mBERT (Devlin et al., 2019), XLM (Conneau and Lam-ple, 2019), Unicoder (Huang et…”
BackgroundExplicit Alignment Objectives for Multilingual Bidirectional Encoders
Hu, J., Johnson, M., Fırat, O., Siddhant, A., & Neubig, G. (2021) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2021.naacl-main.284 · Cited by 57
A comparative study of cross-lingual sentiment analysis
Přibáň, P., Šmíd, J., Steinberger, J., & Mištera, A. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.123247 · Cited by 59
Joint Representation Learning of Cross-lingual Words and Entities via Attentive Distant Supervision
Cao, Y., Hou, L., Li, J., Liu, Z., Li, C., Chen, X., & Dong, T. (2018) · Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d18-1021 · Cited by 41
We Don't Speak the Same Language: Interpreting Polarization through Machine Translation
KhudaBukhsh, A. R., Sarkar, R., Kamlet, M. S., & Mitchell, T. M. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i17.17748 · Cited by 65
A Systematic Review of Cross-Lingual Sentiment Analysis: Tasks, Strategies, and Prospects
Zhao, C., Wu, M., Yang, X., Zhang, W., Zhang, S., Wang, S., & Li, D. (2024) · ACM Computing Surveys · doi:10.1145/3645106 · Cited by 46
“Pre-trained word embedding in a single language trained on a large unlabeled corpus are proposed by Faruqui and Dyer [45].”
MethodImproving Arabic information retrieval using word embedding similarities
Mahdaouy, A. E., Alaoui, S. O. E., & Gaussier, É. (2018) · International Journal of Speech Technology · doi:10.1007/s10772-018-9492-y · Cited by 42
“Lastly, the proposed extensions can be used also for Cross lingual IR by using cross-lingual word vector projection methods (Faruqui and Dyer 2014).”
MethodLearning Bilingual Word Representations by Marginalizing AlignmentsPreprint
Kočiský, T., Hermann, K. M., & Blunsom, P. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1405.0947 · Cited by 57
Cross-lingual Alignment vs Joint Training: A Comparative Study and A Simple Unified FrameworkPreprint
Wang, Z.-R., Xie, J., Xu, R., Yang, Y., Neubig, G., & Carbonell, J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.04708 · Cited by 92
“…cross-lingual mapping, a rich body of existing work relying on cross-lingual supervision, including bilingual dictionaries (Mikolov et al., 2013a; Faruqui & Dyer, 2014; Artetxe et al., 2016; Xing et al., 2015; Duong et al., 2016; Gouws & Søgaard, 2015; Joulin et al., 2018a), sentence-aligned…”
BackgroundMorph-fitting: Fine-Tuning Word Vector Spaces with Simple Language-Specific Rules
Vulić, I., Mrkšić, N., Reichart, R., Séaghdha, D. Ó., Young, S. J., & Korhonen, A. (2017) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p17-1006 · Cited by 46
“We also test the symmetric-pattern based vectors of Schwartz et al. (2016) ( SymPat-Emb ), count-based PMI-weighted vectors reduced by SVD (Baroni et al., 2014) ( Count-SVD ), a model which replaces the context modelling function from CBOW with bidirectional LSTMs (Mela-mud et al., 2016) ( Context2Vec ), and two sets…”
Cross-Lingual Propagation for Deep Sentiment Analysis
Dong, X., & de Melo, G. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.12071 · Cited by 39
“These are typically produced either by aligning multiple monolingual word embedding models using techniques such as linear projections (Mikolov et al. 2013b) or CCA (Faruqui and Dyer 2014), by jointly training in multiple languages via parallel corpora (Klementiev, Titov, and Bhattarai 2012; Luong, Pham, and Manning 2…”
BackgroundMethodA novel method based on symbolic regression for interpretable semantic similarity measurement
Martínez-Gil, J., & Chaves-Gonzalez, J. M. (2020) · Expert Systems with Applications · doi:10.1016/j.eswa.2020.113663 · Cited by 38
Contextual word representations: putting words into computers
Smith, N. A. (2020) · Communications of the ACM · doi:10.1145/3347145 · Cited by 57
“12 By constructing a function that reorients all the English vectors into the French space (or vice versa), researchers hoped to align all the English and French words, not just the ones in the bilingual dictionary.”
BackgroundEmbedding Word Similarity with Neural Machine TranslationPreprint
Hill, F., Cho, K., Jean, S., Devin, C., & Bengio, Y. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1412.6448 · Cited by 80
“Unlike the models described above, Faruqui & Dyer (2014) showed explicitly that projecting word embeddings from two languages (learned independently) into a common vector space can favourably influence the orientation of word embeddings when considered in their monolingual subspace; i.e relative to…”
BackgroundMethodResultInfluentialEvaluating Resource-Lean Cross-Lingual Embedding Models in Unsupervised Retrieval
Litschko, R., Glavaš, G., Vulić, I., & Dietz, L. (2019) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3331184.3331324 · Cited by 48
“Faruqui and Dyer [4] treat XS and XT as different views on the same data points and apply CCA to learn the data representations that maximize the correlation between the two views.”
BackgroundBrighter than Gold: Figurative Language in User Generated Comparisons
Niculae, V., & Danescu-Niculescu-Mizil, C. (2014) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1215 · Cited by 36
“We use the same cross-lingually optimized representation from Faruqui and Dyer (2014) and a simpler classifier, a logistic regression, which we find to perform as well as the random forests used in Tsvetkov et al. (2014a).”
BackgroundMethodCross-lingual sentiment transfer with limited resources
Rasooli, M. S., Farra, N., Radeva, A., Yu, T., & McKeown, K. R. (2017) · Machine Translation · doi:10.1007/s10590-017-9202-6 · Cited by 37
“There has been a great deal of work on inducing cross-lingual word representations (e.g. Täckström et al. 2012; Hermann and Blunsom 2013; Faruqui and Dyer 2014; Ammar et al. 2016).”
BackgroundConsistency Regularization for Cross-Lingual Fine-Tuning
Zheng, B., Dong, L., Huang, S., Wang, W., Chi, Z., Singhal, S., Che, W., Liu, T., Song, X., & Wei, F. (2021) · doi:10.18653/v1/2021.acl-long.264 · Cited by 73
“Cross-Lingual Transfer Besides learning crosslingual word embeddings (Mikolov et al., 2013; Faruqui and Dyer, 2014; Guo et al., 2015; Xu et al., 2018; Wang et al., 2019), most recent work of cross-lingual transfer is based on pre-trained cross-lingual language models (Conneau and Lample, 2019; Conneau et al.”
Bhattacharya, P., Goyal, P., & Sarkar, S. (2016) · Computación y Sistemas · doi:10.13053/cys-20-3-2462 · Cited by 35
“Word embedding based methods have been utilized in many different tasks, such as word similarity [4, 11, 21], cross lingual dependency parsing [11], finding semantic and syntactic relations [4], finding morphological tags [19], identifying POS and translation equivalence classes [5] and in analogical…”
MethodBilingual Learning of Multi-sense Embeddings with Discrete Autoencoders
Šuster, S., Titov, I., & van Noord, G. (2016) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n16-1160 · Cited by 39
“We adhere to the view of multilingual learning as a means of language grounding (Faruqui and Dyer, 2014b; Zou et al., 2013; Titov and Klementiev, 2012; Snyder and Barzilay, 2010; Naseem et al., 2009).”
BackgroundCharacterizing Departures from Linearity in Word Translation
Nakashole, N., & Flauger, R. (2018) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p18-2036 · Cited by 35
“A class of methods has emerged whose core technique is to learn linear maps between vector spaces of different languages (Mikolov et al., 2013a; Faruqui and Dyer, 2014; Vulic and Korhonen, 2016; Artetxe et al., 2016; Conneau et al., 2018).”
BackgroundNurse is Closer to Woman than Surgeon? Mitigating Gender-Biased Proximities in Word Embeddings
Kumar, V., Bhotia, T. S., Kumar, V., & Chakraborty, T. (2020) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00327 · Cited by 55
Learning Unsupervised Word Translations Without Adversaries
Mukherjee, T., Yamada, M., & Hospedales, T. M. (2018) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d18-1063 · Cited by 35
“Multilingual Correlation (Faruqui and Dyer, 2014), and Matching CCA (Haghighi et al., 2008), which alternates between matching and estimat-1 MUSE Dataset BLI Datasets Methods es-en en-es it-en en-it zh-en en-zh es-en en-es it-en en-it zh-en en-zh TM (Mikolov et al., ing a joint linear subspace.”
Bilingual Lexicon Induction by Learning to Combine Word-Level and Character-Level Representations
Heyman, G., Vulić, I., & Moens, M. (2017) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/e17-1102 · Cited by 33
“Current state-of-the-art BLI results are obtained by cross-lingual word embeddings (Mikolov et al., 2013b; Faruqui and Dyer, 2014; Gouws et al., 2015; Vuli´c and Moens, 2016; Duong et al., 2016, inter alia).”
BackgroundAugmenting English Adjective Senses with Supersenses
Tsvetkov, Y., Schneider, N., Hovy, D., Bhatia, A., Faruqui, M., & Dyer, C. (2014) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/5edop9rocohn · Cited by 43
“We use the cross-lingually enriched distributional word vectors constructed in Faruqui and Dyer (2014) using both monolingual word cooccurrence and bilingual word alignment information.”
MethodSelf-citationMohiuddin, T., & Joty, S. (2019) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n19-1386 · Cited by 42
“Instead of learning a mapping from the source to the target embedding space, Faruqui and Dyer (2014) use a technique based on Canonical Correlation Analysis (CCA) to project both source and target embeddings to a common low-dimensional space, where the correlation of the word pairs in the seed…”
MethodNeural Decipherment via Minimum-Cost Flow: From Ugaritic to Linear B
Luo, J., Cao, Y. J., & Barzilay, R. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1303 · Cited by 33
“, 2018), CCA (Haghighi et al., 2008; Faruqui and Dyer, 2014) or mutual information (Mukherjee et al.”
BackgroundCanonical Correlation Analysis (CCA) Based Multi-View Learning: An OverviewPreprint
Guo, C., & Wu, D. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1907.01693 · Cited by 59
“Faruqui and Dyer [68] proposed to learn word embeddings by considering the multilingual context using DCCA, and demonstrated that it gave better semantic representations in standard lexical semantic evaluation tasks.”
BackgroundMethodKementchedjhieva, Y., Ruder, S., Cotterell, R., & Søgaard, A. (2018) · Conference on Computational Natural Language Learning · doi:10.18653/v1/k18-1021 · Cited by 44
Translation Invariant Word Embeddings
Huang, K., Gardner, M., Papalexakis, E. E., Faloutsos, C., Sidiropoulos, N. D., Mitchell, T. M., Talukdar, P. P., & Fu, X. (2015) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d15-1127 · Cited by 32
“We used wordvectors.org (Faruqui and Dyer, 2014a) to evaluate our learned vectors on a variety of English-language word similarity tasks.”
BackgroundMethodInfluentialLinear transformations for cross-lingual semantic textual similarity
Brychcín, T. (2019) · Knowledge-Based Systems · doi:10.1016/j.knosys.2019.06.027 · Cited by 39
Word Embeddings with Limited Memory
Ling, S., Song, Y., & Roth, D. (2016) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p16-2063 · Cited by 32
“We use the datasets summarized by Faruqui and Dyer (Faruqui and Dyer, 2014): wordsim-353, wordsim-sim, wordsim-rel, MC-30, RG-65, MTurk-287, MTurk-771, MEN 3000, YP-130, Rare-Word, Verb-143, and SimLex-999.”
MethodBrainBench: A Brain-Image Test Suite for Distributional Semantic Models
Xu, H., Murphy, B., & Fyshe, A. (2016) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d16-1213 · Cited by 39
“This dataset was subsequently split into sets where the pairs denote similarity and relatedness, named WS353-SIM and WS-353-REL, respectively (Agirre et al., 2009).”
MethodA Distribution-based Model to Learn Bilingual Word Embeddings
Cao, H., Zhao, T., Zhang, S., & Meng, Y. (2016) · International Conference on Computational Linguistics · Cited by 31
Learning Bilingual Word Embeddings Using Lexical Definitions
Shi, W., Chen, M., Tian, Y., & Chang, K.-W. (2019) · Workshop on Representation Learning for NLP (RepL4NLP) · doi:10.18653/v1/w19-4316 · Cited by 30
“In particular, seed lexicon methods (Mikolov et al., 2013a; Faruqui and Dyer, 2014; Guo et al., 2015) learn transfor-”
MethodMinimally-Constrained Multilingual Embeddings via Artificial Code-Switching
Wick, M., Kanani, P. H., & Pocock, A. (2016) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v30i1.10360 · Cited by 29
“Alternatively, canonical correlation analysis (CCA) (Faruqui and Dyer 2014) and deep CCA (Lu et al. 2015) are suitable for mapping word vectors into a common bilingual space.”
BackgroundAutomated WordNet Construction Using Word Embeddings
Khodak, M., Risteski, A., Fellbaum, C., & Arora, S. (2017) · doi:10.18653/v1/w17-1902 · Cited by 41
“Further improvement to our work may come from other methods in word-embeddings, such as multi-lingual word-vectors (Faruqui and Dyer, 2014).”
MethodImproving Pre-Trained Multilingual Model with Vocabulary Expansion
Wang, H., Yu, D., Sun, K., Chen, J., & Yu, D. (2019) · Conference on Computational Natural Language Learning · doi:10.18653/v1/k19-1030 · Cited by 67
“While there is no direct precedent, previous work show that incorporating multilingual contexts can improve monolingual word embeddings (Zou et al., 2013; Andrew et al., 2013; Faruqui and Dyer, 2014; Lu et al., 2015; Ruder et al., 2017).”
BackgroundUnsupervised Neural Machine Translation With Cross-Lingual Language Representation Agreement
Sun, H., Wang, R., Chen, K., Utiyama, M., Sumita, E., & Zhao, T. (2020) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2020.2982282 · Cited by 33
Bilingual Lexicon Induction from Non-Parallel Data with Minimal Supervision
Zhang, M., Peng, H., Liu, Y., Luan, H., & Sun, M. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.10988 · Cited by 28
“…of bilingual representation, either in a similar form to our seed term (Shi et al. 2015), or as an extension to the translation matrix approach (Dinu, Lazaridou, and Baroni 2015; Lazaridou, Dinu, and Baroni 2015; Faruqui and Dyer 2014; Lu et al. 2015; Ammar et al. 2016; Vulić and Korhonen 2016).”
BackgroundDensity Matching for Bilingual Word Embedding
Zhou, C., Ma, X., Wang, D., & Neubig, G. (2019) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n19-1161 · Cited by 45
“Early work focused on supervised methods maximizes the similarity of the embeddings of words that exist in a manually-created dictionary, according to some similarity metric (Mikolov et al., 2013; Faruqui and Dyer, 2014; Jawanpuria et al., 2018; Joulin et al., 2018).”
BackgroundPaul, S., Saha, S., & Singh, J. P. (2022) · Multimedia Tools and Applications · doi:10.1007/s11042-021-11601-9 · Cited by 31
“In order to achieve a synthetic presentation for En-Hi code-switched text, we therefore concatenate both the embeddings while utilizing word-level alignments, by performing canonical correlation analysis7 (CCA) on two embeddings and project these into shared vector space where they are maximally correlated, as describ…”
MethodJointly Learning to Embed and Predict with Multiple Languages
Ferreira, D. C., Martins, A. F. T., & Almeida, M. S. C. (2016) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p16-1190 · Cited by 28
“A bilingual auto-encoder was proposed by Chandar et al. (2014), while Faruqui and Dyer (2014) applied canonical correlation analysis to parallel data to improve monolingual embeddings.”
MethodBridging Languages through Images with Deep Partial Canonical Correlation Analysis
Rotman, G., Vulić, I., & Reichart, R. (2018) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p18-1084 · Cited by 25
“These models rely on the basic CCA model (Chan-dar et al., 2016; Faruqui and Dyer, 2014), its deep variant (Lu et al., 2015), and a CCA extension which supports more than two views (Funaki and Nakayama, 2015; Rastogi et al., 2015).”
MethodConfusion2Vec: towards enriching vector space word representations with representational ambiguities
Shivakumar, P. G., & Georgiou, P. (2019) · PeerJ Computer Science · doi:10.7717/peerj-cs.195 · Cited by 27
“Studies by Mikolov et al. [2013b], Faruqui and Dyer [2014] demonstrate that vector space representations are extremely useful in extending the model from one language to another (or multi-lingual extensions) since the semantic relations between words are invariant across languages.”
Unsupervised Parallel Sentence Extraction with Parallel Segment Detection Helps Machine Translation
Hangya, V., & Fraser, A. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1118 · Cited by 36
“Following Mikolov et al. (2013), many authors map monolingual word embeddings into the same bilingual space (Faruqui and Dyer, 2014; Xing et al., 2015), others leverage parallel texts (Gouws et al., 2015) or create artificial cross-lingual corpora using seed lexicons or document alignments (Vuli´c…”
BackgroundLow-resource text classification using domain-adversarial learning
Grießhaber, D., Vu, N. T., & Maucher, J. (2019) · Computer Speech & Language · doi:10.1016/j.csl.2019.101056 · Cited by 29
“They are, however, not trivial to obtain due to the lack of parallel data (Faruqui & Dyer, 2014) or multilingual dictionaries (Ammar et al., 2016).”
Cross-Lingual Sentiment Analysis Without (Good) TranslationPreprint
Abdalla, M., & Hirst, G. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1707.01626 · Cited by 27
“(2013a), and later extended by Faruqui and Dyer (2014), this offline alignment is fast and low cost, but does not achieve a high translation accuracy.”
BackgroundCross-Lingual Bridges with Models of Lexical Borrowing
Tsvetkov, Y., & Dyer, C. (2016) · Journal of Artificial Intelligence Research · doi:10.1613/jair.4786 · Cited by 26
Self-citationKnowledge Distillation for Bilingual Dictionary Induction
Nakashole, N., & Flauger, R. (2017) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d17-1264 · Cited by 32
“They extend canonical correlation analysis (CCA) based methods (Haghighi et al., 2008; Faruqui and Dyer, 2014) to a multi-lingual setting where they treat English as the common vector space.”
BackgroundMethodNeural Cross-Lingual Coreference Resolution And Its Application To Entity Linking
Kundu, G., Sil, A., Florian, R., & Hamza, W. (2018) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p18-2063 · Cited by 30
“This paper investigates the use of multi-lingual embeddings (Faruqui and Dyer, 2014; Upadhyay et al., 2016) for building cross-lingual models for the task of coreference resolution (Ng and Cardie, 2002; Pradhan et al.”
BackgroundMethodInfluentialEfficient and Distributed Generalized Canonical Correlation Analysis for Big Multiview Data
Fu, X., Huang, K., Papalexakis, E. E., Song, H. A., Talukdar, P. P., Sidiropoulos, N. D., Faloutsos, C., & Mitchell, T. M. (2018) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2018.2875908 · Cited by 26
“In particular, we use the data that were originally introduced in [43] and can be downloaded from http://wordvectors.”
MethodEsuli, A., Moreo, A., & Sebastiani, F. (2019) · ACM Transactions on Information Systems · doi:10.1145/3326065 · Cited by 25
“rning approaches typically require the availability of external parallel corpora [18, 23], bi-lingual lexicons , Vol. 1, No. 1, Article 1. Publication date: January 2016. 1:4 Esuli, Moreo, Sebastiani [12], or machine translation tools [1]. Recently, Conneau et al. [7] proposed a method to align monolingual word embedd…”
MethodDomain Adapted Word Embeddings for Improved Sentiment Classification
Sarma, P. K., Liang, Y., & Sethares, B. (2018) · Workshop on Deep Learning Approaches for Low-Resource NLP (DeepLo) · doi:10.18653/v1/w18-3407 · Cited by 45
Mining Cross-Cultural Differences and Similarities in Social Media
Lin, B. Y., Xu, F. F., Zhu, K., & Hwang, S.-W. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p18-1066 · Cited by 25
Zhou, G., Zhao, Z., Huang, J. X., & He, T. (2016) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/2911451.2911490 · Cited by 29
Automatic design of semantic similarity controllers based on fuzzy logics
Martínez-Gil, J., & Chaves-Gonzalez, J. M. (2019) · Expert Systems with Applications · doi:10.1016/j.eswa.2019.04.046 · Cited by 25
Unsupervised Cross-lingual Adaptation for Sequence Tagging and BeyondPreprint
Li, X., Bing, L., Zhang, W., Li, Z., & Lam, W. P. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.12405 · Cited by 41
“learning cross-lingual word embeddings (Mikolov et al., 2013; Faruqui and Dyer, 2014; Smith et al., 2017) is no longer prerequisite for cross-lingual transfer.”
BackgroundMachine Translation in the Era of Large Language Models:A Survey of Historical and Emerging Problems
Ataman, D., Birch, A., Habash, N. Y., Federico, M., Koehn, P., & Cho, K. (2025) · Information · doi:10.3390/info16090723 · Cited by 71
Learning Unsupervised Multilingual Word Embeddings with Incremental Multilingual Hubs
Heyman, G., Verreet, B., Vulić, I., & Moens, M. (2019) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n19-1188 · Cited by 32
“…using cross-lingual supervision in the form of bilingual lexicons (Mikolov et al., 2013a; Artetxe et al., 2016; Smith et al., 2017), parallel corpora (Klementiev et al., 2012; Faruqui and Dyer, 2014; Gouws et al., 2015; Luong et al., 2015) or subject-aligned document pairs (Vuli´c and Moens, 2016).”
BackgroundMethodRobust Cross-Lingual Hypernymy Detection Using Dependency Context
Upadhyay, S., Vyas, Y., Carpuat, M. J., & Roth, D. (2018) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n18-1056 · Cited by 24
“Cross-lingual Distributional Semantics Cross-lingual word embeddings have been shown to encode semantics across languages in tasks such as word similarity (Faruqui and Dyer, 2014) and lexicon induction (Vuli´c and Moens, 2015).”
BackgroundA critical re-evaluation of neural methods for entity alignment
Leone, M., Huber, S., Arora, A., García-Durán, A., & West, R. (2022) · Proceedings of the VLDB Endowment · doi:10.14778/3529337.3529355 · Cited by 42
“Surprisingly, the authors did not benchmarkMTransE against any of the previous approaches that may work under the exact same setup, but their baselines consist of methods from the multilingual word embedding literature such as CCA [23], LM [45] and OT [76].”
Cross-lingual Lexical Sememe Prediction
Qi, F., Lin, Y., Sun, M., Zhu, H., Xie, R., & Liu, Z. (2018) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d18-1033 · Cited by 24
“There are also some works using a seed lexicon as the cross-lingual signal (Dinu et al., 2014; Faruqui and Dyer, 2014; Lazaridou et al., 2015; Shi et al., 2015; Lu et al., 2015; Gouws et al., 2015; Wick et al., 2016; Am-mar et al., 2016; Duong et al., 2016; Vulić and Ko-rhonen, 2016).”
BackgroundLeviant, I., & Reichart, R. (2015) · arXiv (Cornell University) · Cited by 24
Proceedings of the 4th Workshop on Representation Learning for NLP (RepL4NLP-2019)
Augenstein, I., Gella, S., Ruder, S., Kann, K., Can, B., Conneau, A., Welbl, J., Ren, X., & Rei, M. (2019) · doi:10.18653/v1/w19-43 · Cited by 27
Modeling Code-Switch Languages Using Bilingual Parallel Corpus
Lee, G., & Li, H. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.80 · Cited by 37
“Another is to exploit the isomorphic structure (Conneau et al., 2017; Artetxe et al., 2018), dictionary (Mikolov et al., 2013; Faruqui and Dyer, 2014; Huang et al., 2015; Zhang et al., 2016), shared cognate, vocab (Hauer et al., 2017; Smith et al., 2017), numeral (Artetxe et al., 2017) through…”
BackgroundUWB at SemEval-2020 Task 1: Lexical Semantic Change Detection
Pražák, O., Přibáň, P., Taylor, S., & Sido, J. (2020) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/2020.semeval-1.30 · Cited by 37
Incremental term representation learning for social network analysis
Peng, H. M., Bao, M., Li, J., Bhuiyan, M. Z. A., Liu, Y., He, Y., & Yang, E. (2017) · Future Generation Computer Systems · doi:10.1016/j.future.2017.05.020 · Cited by 22
Addressing Noise in Multidialectal Word Embeddings
Erdmann, A., Zalmout, N., & Habash, N. Y. (2018) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p18-2089 · Cited by 29
“They facilitate applications from parallel sentence extraction (Grover and Mitra, 2017) to machine translation (Zou et al., 2013; Cholakov and Kordoni, 2016; Artetxe et al., 2017b) and can be used to improve monolingual embeddings (Faruqui and Dyer, 2014).”
BackgroundA Multi-task Approach to Learning Multilingual Representations
Singla, K., Can, D., & Narayanan, S. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) · doi:10.18653/v1/p18-2035 · Cited by 22
“Research that has been taking place in the context of distributed text representations is learning multilingual text representations shared across languages (Faruqui and Dyer, 2014; Bengio and Corrado, 2015; Luong et al., 2015).”
BackgroundSemantic Drift in Multilingual Representations
Beinborn, L., & Choenni, R. (2020) · Computational Linguistics · doi:10.1162/coli_a_00382 · Cited by 48
“Canonical correlation analysis (Ammar et al. 2016; Faruqui and Dyer 2014), Kullback-Leibler divergence (Dou, Zhou, and Huang 2018) and Procrustes alignment (Conneau et al. 2017) are only a few methods to maximize the similarity between two representational spaces.”
BackgroundMethodInfluentialDeveloping bug severity prediction models using word2vec
Agrawal, R., & Goyal, R. (2021) · International Journal of Cognitive Computing in Engineering · doi:10.1016/j.ijcce.2021.08.001 · Cited by 24
Efficient and Distributed Algorithms for Large-Scale Generalized Canonical Correlations Analysis
Fu, X., Huang, K., Papalexakis, E. E., Song, H.-A., Talukdar, P. P., Sidiropoulos, N. D., Faloutsos, C., & Mitchell, T. M. (2016) · Proceedings · doi:10.1109/icdm.2016.0105 · Cited by 22
“The data were originally introduced in [18] and can be downloaded from http://wordvectors.”
MethodContextual Word Representations: A Contextual IntroductionPreprint
Smith, N. A. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.06006 · Cited by 51
“• We can use bilingual dictionaries to “align” the vectors for words in two languages into a single vector space, so that, for example, the vectors for the English word type cucumber and the French word type concombre have a small Euclidean distance (Faruqui and Dyer, 2014).”
MethodAcross-Time Comparative Summarization of News Articles
Duan, Y., & Jatowt, A. (2019) · ACM International Conference on Web Search and Data Mining (WSDM) · doi:10.1145/3289600.3291008 · Cited by 22
“Besides linear models, non-linear models such as "deep CCA" has also been introduced for the task of mapping multi-lingual word embeddings [24].”
Beyond Bilingual: Multi-sense Word Embeddings using Multilingual Context
Upadhyay, S., Chang, K.-W., Taddy, M. A., Kalai, A. T., & Zou, J. (2017) · Workshop on Representation Learning for NLP (RepL4NLP) · doi:10.18653/v1/w17-2613 · Cited by 21
“Incorporating crosslingual distributional information is a popular technique for learning word embeddings, and improves performance on several downstream tasks (Faruqui and Dyer, 2014; Guo et al., 2016; Upadhyay et al., 2016).”
BackgroundLexicon Stratification for Translating Out-of-Vocabulary Words
Tsvetkov, Y., & Dyer, C. (2015) · doi:10.3115/v1/p15-2021 · Cited by 21
“We use the word2vec tool (Mikolov et al., 2013) to train monolingual vectors, 6 and the CCA-based tool (Faruqui and Dyer, 2014) for projecting word vectors.”
MethodSelf-citationCross-lingual Text Classification via Model Translation with Limited Dictionaries
Xu, R., Yang, Y., Liu, H., & Hsi, A. (2016) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/2983323.2983732 · Cited by 20
Unsupervised Bilingual Lexicon Induction via Latent Variable Models
Dou, Z.-Y., Zhou, Z., & Huang, S. (2018) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d18-1062 · Cited by 25
“This simple yet effective approach has led researchers to investigate on improving cross-lingual word embeddings with the help of bilingual word lexicons (Faruqui and Dyer, 2014; Xing et al., 2015).”
BackgroundMethodEmbedding Learning Through Multilingual Concept Induction
Dufter, P., Zhao, M., Schmitt, M., Fraser, A., & Schütze, H. (2018) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p18-1141 · Cited by 20
“(2013), Xiao and Guo (2014) and Faruqui and Dyer (2014) use different data sources for identifying word pairs and creating the transformation”
Background
Metadata
- Authors
- Faruqui, M., & Dyer, C.
- Affiliations
- Carnegie Mellon University (US)
- Venue
- Proceedings of the 14th Conference of the European Chapter of the Association for Computational Linguistics
- Published
- 2014
- Pages
- 462-471
- OpenAlex
- W342285082
- Cited by
- 620
- References
- 43
- Citations by year
- 2026: 3 · 2025: 8 · 2024: 33 · 2023: 19 · 2022: 28 · 2021: 57 · 2020: 106 · 2019: 100 · 2018: 91 · 2017: 61 · 2016: 67 · 2015: 33 · 2014: 14
- FWCI
- 34.35
- Type
- conference paper
- Language
- en
- Open access
- green
- figshare.com
- Licence
- cc-by
- Locations
- Conference of the European Chapter of the Association for Computational Linguistics (publishedVersion)
- KiltHub Repository (submittedVersion)
- Figshare (submittedVersion, cc-by)
- KiltHub Repository
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Sentiment Analysis and Opinion Mining
- Keywords
- word embeddings, multilingual word representation, distributional semantics, canonical correlation analysis, word meaning, word sense disambiguation, word similarity
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.