Improving Vector Space Word Representations Using Multilingual Correlation
Open accessThe DOI matches its Crossref record; 43 of the 43 references it cites were found in a registry or an index.
Abstract
The distributional hypothesis of Harris (1954), according to which the meaning of words is evidenced by the contexts they occur in, has motivated several effective techniques for obtaining vector space semantic representations of words using unannotated text corpora. This paper argues that lexico-semantic content should additionally be invariant across languages and proposes a simple technique based on canonical correlation analysis (CCA) for incorporating multilingual evidence into vectors generated monolingually. We evaluate the resulting word representations on standard lexical semantic evaluation tasks and show that our method produces substantially better semantic representations than monolingual techniques.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Besides, Hewitt and Manning [125], Jawahar et al. [126], Kim et al. [127] proposed several methods to extract dependency trees and constituency trees from BERT, which proved the BERT's ability to encode syntax structure.”
From Pre-trained models for natural language processing: A survey (2020) · cites this paper as [125]
Citations
Cites
(2008) · Journal of Machine Learning Research · Cited by 33,051
Efficient Estimation of Word Representations in Vector Space
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1301.3781 · Cited by 26,568 · cited 6×
“This dataset contains word pairs that are different syntactic forms of a given word and was prepared by Mikolov et al. (2013a).”
BackgroundMethodInfluentialIndexing by latent semantic analysis
Deerwester, S., Dumais, S. T., Furnas, G. W., Landauer, T. K., & Harshman, R. (1990) · Journal of the American Society for Information Science · doi:10.1002/(sici)1097-4571(199009)41:6<391::aid-asi1>3.0.co;2-9 · Cited by 12,751
Golub, G. H., & Loan, C. F. V. (1996) · Johns Hopkins University Press eBooks · Cited by 6,027
A unified architecture for natural language processing: deep neural networks with multitask learning
Collobert, R., & Weston, J. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390177 · Cited by 5,058 · cited 6×
“Our method can also be seen as an application of multi-view learning (Chang et al., 2013; Collobert and Weston, 2008), where one of the views can be used to capture cross-lingual information.”
BackgroundMethodInfluentialHarris, Z. S. (1954) · WORD · doi:10.1080/00437956.1954.11659520 · Cited by 4,807
Approximate Statistical Tests for Comparing Supervised Classification Learning Algorithms
Dietterich, T. G. (1998) · Neural Computation · doi:10.1162/089976698300017197 · Cited by 3,840 · cited 2×
“01) according to the McNemar’s test (Dietterich, 1998).”
MethodInfluentialWord association norms, mutual information, and lexicography
Church, K., & Hanks, P. (1990) · doi:10.5555/89086.89095 · Cited by 3,590
Linguistic Regularities in Continuous Space Word Representations
Mikolov, T., Yih, W.-T., & Zweig, G. (2013) · North American Chapter of the Association for Computational Linguistics · Cited by 2,912
From Frequency to Meaning: Vector Space Models of Semantics
Turney, P. D., & Pantel, P. (2010) · Journal of Artificial Intelligence Research · doi:10.1613/jair.2934 · Cited by 2,866 · cited 1×
“Using cooccurrence statistics from a large corpus of text (Deerwester et al., 1990; Turney and Pantel, 2010), 1 it is possible to construct high-quality semantic vectors — as judged by both correlations with human judgements of semantic relatedness (Turney, 2006; Agirre et al., 2009) and as…”
Background
Cited by
Items 201–300 of 617
Cross-Lingual Text Classification with Minimal Resources by Transferring a Sparse Teacher
Karamanolakis, G., Hsu, D., & Gravano, L. (2020) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2020.findings-emnlp.323 · Cited by 23
Multi-Modal Representations for Improved Bilingual Lexicon Learning
Vulić, I., Kiela, D., Clark, S. R. L., & Moens, M. (2016) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p16-2031 · Cited by 21
“After mapping all vectors x , x ∈ V S , the target space R d T serves as a SCLVS. Type 2: G-EMB Another collection of BWE induction models optimizes two monolingual objectives jointly , with the cross-lingual objective acting as a cross-lingual regularizer during training (Gouws et al., 2015; Soyer et al., 2015).”
MethodInfluentialProjecting Embeddings for Domain Adaption: Joint Modeling of Sentiment Analysis in Diverse DomainsPreprint
Barnes, J., Klinger, R., & Walde, S. S. I. (2018) · arXiv (Cornell University) · Cited by 22
Learning Translations via Matrix Completion
Wijaya, D. T., Callahan, B., Hewitt, J., Gao, J., Ling, X., Apidianaki, M., & Callison-Burch, C. (2017) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d17-1152 · Cited by 27
“With the increasing popularity of word embeddings, many recent works approximate similarities between words in different languages by constructing a shared bilingual embedding space (Klementiev et al., 2012b; Zou et al., 2013; Vulić and Moens, 2013; Mikolov et al., 2013a; Faruqui and Dyer, 2014; Chandar A P et al., 20…”
BackgroundTwo Methods for Domain Adaptation of Bilingual Tasks: Delightfully Simple and Broadly Applicable
Hangya, V., Braune, F., Fraser, A., & Schütze, H. (2018) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p18-1075 · Cited by 19
“Following Mikolov et al. (2013b), many authors (Faruqui and Dyer, 2014; Xing et al., 2015; Lazaridou et al., 2015; Vulić and Korhonen, 2016) map monolingual word embeddings (MWEs) into the same bilingual space.”
BackgroundGraph-Based Bilingual Word Embedding for Statistical Machine Translation
Wang, R., Zhao, H., Ploux, S., Lu, B., Utiyama, M., & Sumita, E. (2018) · ACM Transactions on Asian and Low-Resource Language Information Processing · doi:10.1145/3203078 · Cited by 22
Multi-Modal Representations for Improved Bilingual Lexicon Learning
Vulić, I., Kiela, D., Clark, S. C., & Moens, M. (2016) · Lirias · doi:10.17863/cam.9719 · Cited by 21
Toward any-language zero-shot topic classification of textual documents
Song, Y., Upadhyay, S., Peng, H., Mayhew, S., & Roth, D. (2019) · Artificial Intelligence · doi:10.1016/j.artint.2019.02.002 · Cited by 24
“Motivated by the simplicity and success of neural network based word embedding [29, 30], multilingual [35] or cross-lingual [6, 36, 37, 38, 39, 40] representation learning are also investigated.”
ABSent: Cross-Lingual Sentence Representation Mapping with Bidirectional GANs
Fu, Z., Xian, Y., Geng, S., Ge, Y., Wang, Y., Dong, X., Wang, G., & de Melo, G. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i05.6279 · Cited by 21
Sentiment Analysis for Hinglish Code-mixed Tweets by means of Cross-lingual Word Embeddings
Singh, P., & Lefever, E. (2020) · Ghent University Academic Bibliography (Ghent University) · Cited by 41
Huang, J., Wang, M., Xu, X., Jie, B., & Zhang, D. (2020) · Medical Image Analysis · doi:10.1016/j.media.2020.101755 · Cited by 19
Akhtar, M. S., Sawant, P., Sen, S., Ekbal, A., & Bhattacharyya, P. (2018) · ACM Transactions on Asian and Low-Resource Language Information Processing · doi:10.1145/3273931 · Cited by 29
“For the analysis we employ three different techniques to compute the bilingual/cross-lingual embeddings (i.e., embeddings E1 (Luong et al. 2015), E2 (Faruqui and Dyer 2014), and E3 (Vulić and Moens 2015)).”
BackgroundMethodGeneralizing Procrustes Analysis for Better Bilingual Dictionary InductionPreprint
Kementchedjhieva, Y., Ruder, S., Cotterell, R., & Søgaard, A. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1809.00064 · Cited by 19
Bilingual embeddings with random walks over multilingual wordnets
Goikoetxea, J., Soroa, A., & Agirre, E. (2018) · Knowledge-Based Systems · doi:10.1016/j.knosys.2018.03.017 · Cited by 18
“While the latter authors learned a single transformation from target to source language, [34] used canonical cor-5 relation analysis to learn separate transformations, projecting the monolingual spaces into a shared space.”
BackgroundLessLex: Linking Multilingual Embeddings to SenSe Representations of LEXical Items
Colla, D., Mensa, E., & Radicioni, D. P. (2020) · Computational Linguistics · doi:10.1162/coli_a_00375 · Cited by 22
“Dictionaries have been used as seed lexicons of frequent terms to combine language models acquired separately over different languages (Mikolov, Le, and Sutskever 2013; Faruqui and Dyer 2014).”
MethodLearning Cross-Lingual Word Embeddings from Twitter via Distant Supervision
Camacho-Collados, J., Doval, Y., Martínez‐Cámara, E., Espinosa-Anke, L., Barbieri, F., & Schockaert, S. (2020) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v14i1.7280 · Cited by 26
“2016; Ruder, Vulić, and Søgaard 2019), especially since the recent introduction of models requiring almost no supervision (Mikolov, Le, and Sutskever 2013; Faruqui and Dyer 2014; Xing et al. 2015; Smith et al. 2017; Artetxe, Labaka, and Agirre 2017; Doval et al. 2018).”
Unsupervised Induction of Linguistic Categories with Records of Reading, Speaking, and Writing
Barrett, M., Gonzalez-Garduño, A. V., Frermann, L., & Søgaard, A. (2018) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n18-1184 · Cited by 24
“For these words and their POS tags we get the feature vector of the POS model yielding the highest result on both Ontonotes and PTB: CCA-projected eigenwords and Dundee features.”
BackgroundMethodInfluentialInducing Conceptual Embedding Spaces from Wikipedia
de Melo, G. (2017) · International World Wide Web Conference (WWW) · doi:10.1145/3041021.3054144 · Cited by 17
“Faruqui et al. 2015 [16] showed that lexical resources such as WordNet, PPDB, and FrameNet can be used to improve the quality of word vectors.”
Liu, L., Wan, X., & Guo, Z. (2018) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3240508.3241910 · Cited by 18
“We employ the BiCCA [7] method to obtain semantic representations of three different languages: English, contemporary Chinese and classical Chinese.”
MethodA Resource-Free Evaluation Metric for Cross-Lingual Word Embeddings Based on Graph Modularity
Fujinuma, Y., Boyd-Graber, J. L., & Paul, M. J. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1489 · Cited by 18
“MeCab (Kudo et al., 2004) 2014) maps two mono-lingual embeddings into a shared space by maximizing the correlation between translation pairs in a seed lexicon.”
BackgroundUnsupervised Word Translation with Adversarial Autoencoder
Mohiuddin, T., & Joty, S. (2020) · Computational Linguistics · doi:10.1162/coli_a_00374 · Cited by 22
“Use of mean-centering along each dimension of the word embeddings for maximum expected covariance shows that the method is closely related to the method proposed by Faruqui and Dyer (2014).”
BackgroundMethodInfluentialAn improved deep sequential model for context-aware POI recommendation
Thaipisutikul, T., & Chen, Y.-N. (2023) · Multimedia Tools and Applications · doi:10.1007/s11042-023-15540-5 · Cited by 17
“Firstly, a pre-trained POI representation matrix E L ∈ ℝ d × ∣ L ∣ is created by the Word2Vec [12] technique so each POI with similar embedding will be encoded in a closer distance in the same latent space.”
A Bilingual Graph-Based Semantic Model for Statistical Machine Translation
Wang, R., Zhao, H., Ploux, S., Lu, B., & Utiyama, M. (2016) · HAL · Cited by 16
Cross-lingual word analogies using linear transformations between semantic spaces
Brychcín, T., Taylor, S., & Svoboda, L. (2019) · Expert Systems with Applications · doi:10.1016/j.eswa.2019.06.021 · Cited by 16
Big BiRD: A Large, Fine-Grained, Bigram Relatedness Dataset for Examining Semantic Composition
Asaadi, S., Mohammad, S. M., & Kiritchenko, S. (2019) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n19-1050 · Cited by 50
“A common approach to evaluate representations of unigrams is through their ability to rank pairs of words by closeness in meaning (Pennington et al., 2014; Levy and Goldberg, 2014; Faruqui and Dyer, 2014).”
BackgroundWiktionary-based word embeddings
de Melo, G. (2015) · Cited by 16
A Simple and Effective Method To Eliminate the Self Language Bias in Multilingual Representations
Yang, Z., Yang, Y., Cer, D. M., & Darve, E. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.470 · Cited by 19
“Faruqui and Dyer (2014) propose a canonical correlation analysis (CCA) based method to add multilingual context to monolingual embeddings.”
Semi-supervised Deep Representation Learning for Multi-View Problems
Noroozi, V., Bahaadini, S., Zheng, L., Xie, S., Shao, W., & Yu, P. S. (2018) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata.2018.8622015 · Cited by 16
Freitas, A., Barzegar, S., Sales, J. E., Handschuh, S., & Davis, B. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-49004-5_14 · Cited by 15
“Faruqui & Dyer[6] use the distributional invariance across languages and propose a technique based on canonical correlation analysis (CCA) for merging multilingual evidence into vectors generated monolingually.”
MethodA Comprehensive Understanding of Code-Mixed Language Semantics Using Hierarchical Transformer
Suresh, T., Sengupta, A., Akhtar, M. S., & Chakraborty, T. (2024) · IEEE Transactions on Computational Social Systems · doi:10.1109/tcss.2024.3360378 · Cited by 26
“Extending on the parallel corpus approach, Faruqui and Dyer [10] proposed canonical correlation analysis (CCA) to project multilingual properties in the monolingual space.”
Hubless Nearest Neighbor Search for Bilingual Lexicon Induction
Huang, J., Qiu, Q., & Church, K. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1399 · Cited by 18
Poerner, N., & Schütze, H. (2019) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d19-1173 · Cited by 19
“A famous example is the cross-lingual projection of word embeddings (Mikolov et al., 2013b; Faruqui and Dyer, 2014).”
BackgroundWeakly Supervised POS Taggers Perform Poorly on Truly Low-Resource Languages
Kann, K., Lacroix, O., & Søgaard, A. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i05.6317 · Cited by 16
“They can be used for transferring information from resource-rich languages to low-resource languages, either by replacing words directly, transferring statistics, or for inducing cross-lingual word representations as done, e.g., by Faruqui and Dyer (2014).”
BackgroundA Survey of the Model Transfer Approaches to Cross-Lingual Dependency Parsing
Das, A., & Sarkar, S. (2020) · ACM Transactions on Asian and Low-Resource Language Information Processing · doi:10.1145/3383772 · Cited by 14
“[131] 2015 CCA based projection of target language embeddings to space of source language embeddings [47].”
BackgroundHollenstein, N., Barrett, M., & Beinborn, L. (2020) · Research at the University of Copenhagen (University of Copenhagen) · Cited by 32
Fashion Meets Computer Vision and NLP at e-Commerce Search
Zoghbi, S., Heyman, G., Gómez, J. C., & Moens, M. (2016) · International Journal of Computer and Electrical Engineering · doi:10.17706/ijcee.2016.8.1.31-43 · Cited by 15
Analyzing Semantic Change in Japanese Loanwords
Takamura, H., Nagata, R., & Kawasaki, Y. (2017) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/e17-1112 · Cited by 23
“There are a number of meth-ods for this purpose (Gouws et al., 2015; Zou et al., 2013; Faruqui and Dyer, 2014; Mikolov et al., 2013a).”
BackgroundAuguste, J., Rey, A., & Favre, B. (2017) · Workshop on Evaluating Vector-Space Representations for NLP · doi:10.18653/v1/w17-5304 · Cited by 15
“, 2014), Multilingual (Faruqui and Dyer, 2014), Dependencybased (Levy and Goldberg, 2014), and Fast-”
A Randomized Algorithm for CCAPreprint
Mineiro, P., & Karampatziakis, N. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1411.3409 · Cited by 18
“For the machine learning community, more familiar applications include learning with privileged information[15], semi-supervised learning[3, 14], monolingual[5] and multilingual[7] word representation learning, locality sensitive hashing[8] and clustering[2].”
BackgroundCross-Lingual Syntactically Informed Distributed Word Representations
Vulić, I. (2017) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/e17-2065 · Cited by 14
“…results in a shared cross-lingual word vector space (SCLVS), which leads to improvements on monolingual tasks (typically word similarity) (Faruqui and Dyer, 2014; Rastogi et al., 2015; Upadhyay et al., 2016), and also supports cross-lingual tasks such as bilingual lexicon induction…”
BackgroundEvaluating bilingual word embeddings on the long tail
Braune, F., Hangya, V., Eder, T., & Fraser, A. (2018) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n18-2030 · Cited by 15
“To create bilingual word embeddings, we use post-hoc mapping (PHM), a method that projects monolingual words embeddings (MWEs) into a shared space using a linear transformation trained with a small seed lexicon (Mikolov et al., 2013b; Faruqui and Dyer, 2014; Xing et al., 2015; Lazaridou et al., 2015; Vulić and Korhone…”
MethodA Distributed Representation-Based Framework for Cross-Lingual Transfer Parsing
Guo, J., Che, W., Yarowsky, D., Wang, H., & Liu, T. (2016) · Journal of Artificial Intelligence Research · doi:10.1613/jair.4955 · Cited by 14
“The second approach we consider is similar to that of Faruqui and Dyer (2014), which uses CCA to improve monolingual word embeddings with multilingual correlation. CCA is a way of measuring the linear relationship between multidimensional variables. For two multidimensional variables, CCA aims to find two projection m…”
MethodResultInfluentialLearning Bilingual Projections of Embeddings for Vocabulary Expansion in Machine Translation
Madhyastha, P., & España-Bonet, C. (2017) · Workshop on Representation Learning for NLP (RepL4NLP) · doi:10.18653/v1/w17-2617 · Cited by 14
“Since our model and BiCCA use similar supervision, we obtain similar results and differ in a similar way to those that use stronger supervision like BiCVM and BiSkip based embeddings.”
MethodInfluentialA Bilingual Adversarial Autoencoder for Unsupervised Bilingual Lexicon Induction
Bai, X., Cao, H., Chen, K., & Zhao, T. (2019) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2019.2925973 · Cited by 15
Multi-lingual Common Semantic Space Construction via Cluster-consistent Word Embedding
Huang, L., Cho, K., Zhang, B., Ji, H., & Knight, K. K. (2018) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d18-1023 · Cited by 16
“Using bilingual aligned words, previous methods project multiple monolingual embeddings into a shared semantic space using linear mappings (Mikolov et al., 2013; Rothe et al., 2016; Zhang et al., 2016; MarcoBaroni, 2015; Xing et al., 2015) or canonical correlation analysis (CCA) (Ammar et al., 2016b; Faruqui and Dyer,…”
InfluentialWhy Overfitting Isn’t Always Bad: Retrofitting Cross-Lingual Word Embeddings to Dictionaries
Zhang, M., Fujinuma, Y., Paul, M. J., & Boyd-Graber, J. L. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.201 · Cited by 14
Bhattacharya, P., Goyal, P., & Sarkar, S. (2018) · ACM Transactions on Asian and Low-Resource Language Information Processing · doi:10.1145/3208358 · Cited by 13
“…2015; Hermann and Blunsom 2013; Soyer et al. 2014; Chandar et al. 2014; Blunsom and Hermann 2014) —Word Similarity Task: Monolingual word similarity (Faruqui and Dyer 2014; Huang et al. 2015; Qiu et al. 2014) —Cross-Lingual Dependency Parsing: Train a dependency parser for one language using its…”
BackgroundMethodInfluentialComparing Unsupervised Word Translation Methods Step by Step
Hartmann, M., Kementchedjhieva, Y., & Søgaard, A. (2019) · Research at the University of Copenhagen (University of Copenhagen) · Cited by 20
A Variational Autoencoding Approach for Inducing Cross-lingual Word Embeddings
Wei, L., & Deng, Z. (2017) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2017/582 · Cited by 13
“It was extended by [Faruqui and Dyer, 2014], who simultanteously projected source and target language embeddings into a joint space using canonical correlation analysis.”
Geometry-aware domain adaptation for unsupervised alignment of word embeddings
Jawanpuria, P., Meghwanshi, M., & Mishra, B. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.276 · Cited by 24
Probing Multilingual BERT for Genetic and Typological Signals
Rama, T., Beinborn, L., & Eger, S. (2020) · International Conference on Computational Linguistics · doi:10.18653/v1/2020.coling-main.105 · Cited by 32
Spatial role labeling with convolutional neural networks
Mazalov, A., Martins, B., & de Matos, D. M. (2015) · Workshop on Geographic Information Retrieval (GIR) · doi:10.1145/2837689.2837706 · Cited by 13
“Research within the NLP community continues to explore different word embedding models and, in future experiments, it would be interesting to test recent alternatives, as a replacement to the GloVe embeddings used in our experiments [8, 6, 7].”
Wikipedia Based Short Text Classification Method
Li, J., Cai, Y., Cai, Z., Leung, H.-F., & Yang, K. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-55705-2_22 · Cited by 13
“And we can easily see from the experiments results that WBSTC method shows the best performance among VSM, ESA and BocSTC.”
MethodInfluentialPhrase embedding learning based on external and internal context with compositionality constraint
Li, M., Lu, Q., Xiong, D., & Long, Y. (2018) · Knowledge-Based Systems · doi:10.1016/j.knosys.2018.04.009 · Cited by 13
“studied, including context words under a specific syntactic dependency [19], context of words from different languages [20], context from a knowledge base [21], neighbor context in a semantic lexicon [22], substitute context [23], contrast context [24], path-based context [25], and morphological context [26, 22].”
BackgroundLearning to Represent Words in Context with Multilingual SupervisionPreprint
Kawakami, K., & Dyer, C. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1511.04623 · Cited by 13
“…the qualityof our representations as features in three downstream tasks: prediction of semantic supersenses (which assign nouns and verbs into a few dozen semantic classes), low res urce machine translation, and a lexical substitution task, and obtain state-of-the-art results on all of these.”
BackgroundMethodSelf-citationSemi Supervised Preposition-Sense Disambiguation using Multilingual DataPreprint
Gonen, H., & Goldberg, Y. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.08813 · Cited by 14
“Of much relevance to this work are also previous attempts to improve monolingual representations using bilingual data (Faruqui and Dyer, 2014).”
BackgroundChauhan, S., Saxena, S., & Daniel, P. (2021) · International Journal of Systems Assurance Engineering and Management · doi:10.1007/s13198-021-01182-z · Cited by 14
“Further extensions of (Faruqui et al. 2014) (Lu et al. 2015) orthogonal methods, which map the embeddings in one or both languages under the constraint of the transformation being orthogonal (Xing et al. 2015) (Smith et al. 2017).”
MethodAl-Duwais, M., Al-Khalifa, H. S., & Al‐Salman, A. S. (2024) · Electronics · doi:10.3390/electronics13173574 · Cited by 16
Vector Embedding of Wikipedia Concepts and Entities
Sherkat, E., & Milios, E. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-59569-6_50 · Cited by 12
Improving Cross-Lingual Word Embeddings by Meeting in the Middle
Doval, Y., Camacho-Collados, J., Espinosa-Anke, L., & Schockaert, S. (2018) · ORCA Online Research @Cardiff (Cardiff University) · doi:10.48550/arxiv.1808.08780 · Cited by 12
“…this paper, attempts to learn bilingual embeddings via a two-step process: first, word embeddings are trained on monolingual corpora and then the resulting monolingual spaces are aligned by taking advantage of bilingual dictionaries (Mikolov et al., 2013b; Faruqui and Dyer, 2014; Xing et al., 2015).”
BackgroundMethodInfluentialConception: Multilingually-Enhanced, Human-Readable Concept Vector Representations
Conia, S., & Navigli, R. (2020) · International Conference on Computational Linguistics · doi:10.18653/v1/2020.coling-main.291 · Cited by 17
Lightweight Random Indexing for Polylingual Text Classification
Fernández, A. M., Esuli, A., & Sebastiani, F. (2016) · Journal of Artificial Intelligence Research · doi:10.1613/jair.5194 · Cited by 12
“Alternatively, other approaches require a computationally expensive post-processing step to align word representations across languages (Mikolov, Le, & Sutskever, 2013a; Faruqui & Dyer, 2014).”
BackgroundCrosslingual Document Embedding as Reduced-Rank Ridge Regression
Josifoski, M., Paskov, I., Paskov, H. S., Jaggi, M., & West, R. (2019) · ACM International Conference on Web Search and Data Mining (WSDM) · doi:10.1145/3289600.3291023 · Cited by 12
“fine (cosine)” omitted by authors [6].) Query in Italian Query in English P@1 P@5 P@10 Mikolov et al. [27] 33.8 48.3 53.9 24.9 41.0 47.4 Dinu et al. [9] 38.5 56.4 63.9 24.6 45.4 54.1 Faruqui and Dyer [11] 36.1 52.7 58.1 31.0 49.9 57.0 Artetxe et al. [4] 39.7 54.7 60.5 33.8 52.4 59.1 Smith et al. [34] 43.1 60.7 66.4 38…”
InfluentialPandit, R., Sengupta, S., Naskar, S. K., Dash, N. S., & Sardar, M. M. (2019) · Informatics · doi:10.3390/informatics6020019 · Cited by 12
“Faruqui and Dyer [30] presented a multi-lingual view of word embeddings.”
BackgroundLearning to Represent Bilingual Dictionaries
Chen, M., Tian, Y., Chen, H., Chang, K.-W., Skiena, S., & Zaniolo, C. (2019) · Conference on Computational Natural Language Learning · doi:10.18653/v1/k19-1015 · Cited by 13
“Others adopt canonical correlation analysis to map separated monolingual embeddings to a shared embedding space (Faruqui and Dyer 2014; Lu et al. 2015 (Chopra, Auli, and Rush 2016).”
BackgroundWasserstein GAN based on Autoencoder with back-translation for cross-lingual embedding mappings
Zhang, Y., Li, Y., Zhu, Y. Q., & Hu, X. (2019) · Pattern Recognition Letters · doi:10.1016/j.patrec.2019.11.033 · Cited by 15
Loanword Identification in Low-Resource Languages with Minimal Supervision
Mi, C., Xie, L., & Zhang, Y. (2020) · ACM Transactions on Asian and Low-Resource Language Information Processing · doi:10.1145/3374212 · Cited by 12
“Inspired by the cross-lingual based embedding model, which can return the semantic similarity between two words belonging to different languages [8, 10], we propose a cross-lingual embedding based loanword identification model to produce some loanword candidates first.”
MethodLow-resource Cross-lingual Event Type Detection via Distant Supervision with Minimal Effort
Muis, A. O., Otani, N., Vyas, N., Xu, R., Yang, Y., Mitamura, T., & Hovy, E. H. (2018) · Cited by 11
TensSent: a tensor based sentimental word embedding method
Rahimi, Z., & Homayounpour, M. M. (2021) · Applied Intelligence · doi:10.1007/s10489-020-02163-8 · Cited by 12
“Some researchers [13– 16] have also used canonical correlation analysis (CCA) to extract word vectors, suchlike Faruqui and Dyer [16], who proposed to use CCA to extract word vectors with the help of bilingual information or [13–15] who proposed four spectral algorithms to extract word embeddings.”
Measuring associational thinking through word embeddings
Periñán-Pascual, C. (2021) · Artificial Intelligence Review · doi:10.1007/s10462-021-10056-6 · Cited by 15
“…they tried simple combination methods, e.g. averaging similarity results or concatenating vectors, and more complex methods, e.g. CCA (Faruqui and Faruqui and Dyer (2014)) and retrofitting (Faruqui et al. 2015), demonstrating that simple techniques outperform the more complex techniques in…”
BackgroundMethodInfluentialA Progressive and Cross-Domain Deep Transfer Learning Framework for Wrist Fracture Detection
Karam, C., Zini, J. E., Awad, M., Saade, C., Naffaa, L. N., & Amine, M. E. (2021) · Journal of Artificial Intelligence and Soft Computing Research · doi:10.2478/jaiscr-2022-0007 · Cited by 12
Bilingual Word Embeddings for Bilingual Terminology Extraction from Specialized Comparable Corpora
Hazem, A., & Morin, E. (2017) · HAL · Cited by 11
Towards a unified framework for bilingual terminology extraction of single-word and multi-word terms
Liu, J., Morin, E., & Saldarriaga, S. P. (2018) · HAL · Cited by 11
LINSPECTOR WEB: A Multilingual Probing Suite for Word Representations
Eichler, M., Şahin, G. G., & Gurevych, I. (2019) · Conference on Empirical Methods in Natural Language Processing: System Demonstrations (EMNLP) · doi:10.18653/v1/d19-3022 · Cited by 12
“and visualization for antonyms, synonyms, and female-male similarity; and later it was updated to support German, French, and Spanish word embeddings (Faruqui and Dyer, 2014b).”
BackgroundMethodInfluentialMultilevel Text Alignment with Cross-Document Attention
Zhou, X., Pappas, N., & Smith, N. A. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.407 · Cited by 18
Semantic similarity controllers: On the trade-off between accuracy and interpretability
Martínez-Gil, J., & Chaves-Gonzalez, J. M. (2021) · Knowledge-Based Systems · doi:10.1016/j.knosys.2021.107609 · Cited by 13
Berezutskaya, J., Freudenburg, Z. V., Ambrogioni, L., Güçlü, U., van Gerven, M. A. J., & Ramsey, N. F. (2020) · Scientific Reports · doi:10.1038/s41598-020-68853-y · Cited by 17
“3), we fitted a neural encoding model 42–45 where high frequency band (HFB) neural responses across all electrodes were predicted based on the extracted semantic components.”
MethodResultProjecting Embeddings for Domain Adaptation: Joint Modeling of Sentiment Analysis in Diverse DomainsPreprint
Barnes, J., Klinger, R., & Walde, S. S. I. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.04381 · Cited by 12
“The Most Frequent Source Words are a common source of projection lexicon in the multilingual embedding literature (Faruqui and Dyer, 2014; Lazaridou et al., 2015).”
BackgroundRobust Cross-lingual Embeddings from Parallel SentencesPreprint
Sabet, A., Gupta, P., Cordonnier, J.-B., West, R., & Jaggi, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.12481 · Cited by 21
“This observation is in accordance with the fact (Faruqui and Dyer, 2014) that bilingual contexts can be surprisingly effective for learning monolingual word representations.”
BackgroundResultCross-Lingual Word Representations via Spectral Graph Embeddings
Oshikiri, T., Fukui, K., & Shimodaira, H. (2016) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p16-2080 · Cited by 10
“…a linear transformation be-tween the two spaces is trained from a set of word pairs for translation task (Mikolov et al., 2013a), while other researchers use CCA for learning linear projections to a common vector space where translation pairs are strongly correlated (Faruqui and Dyer, 2014).”
MethodLanguage adaptation experiments via cross-lingual embeddings for related languages
Sharoff, S. (2018) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/2jv83scsdjqj · Cited by 10
“That study was followed by other studies aimed at (Faruqui and Dyer, 2014), Global Correction (Dinu et al., 2014) or TM orthogonali-sation (Artetxe et al., 2016).”
BackgroundMethodInfluentialCross-lingual hate speech detection using domain-specific word embeddings
Arango, A., Rojas, J. P., & Labra, B. P. (2024) · PLoS ONE · doi:10.1371/journal.pone.0306521 · Cited by 35
CMU: Arc-Factored, Discriminative Semantic Dependency Parsing
Thomson, S., O'Connor, B., Flanigan, J., Bamman, D., Dodge, J., Swayamdipta, S., Schneider, N., Dyer, C., & Smith, N. A. (2014) · International Workshop on Semantic Evaluation (SemEval) · doi:10.3115/v1/s14-2027 · Cited by 10
“Word vectors: Features derived from 64-dimensional vectors from (Faruqui and Dyer, 2014), including the concatenation, difference, inner product, and elementwise multiplication of the two vectors associated with a parent-child edge.”
BackgroundSelf-citationThe Emergence of Semantics in Neural Network Representations of Visual Information
Dharmaretnam, D., & Fyshe, A. (2018) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n18-2122 · Cited by 10
“We used the German-English model (512 dimensions), trained on WMT-2011 (Faruqui and Dyer, 2014).”
MethodMultilingual Short Text Classification via Convolutional Neural Network
Liu, J., Cui, R., & Zhao, Y. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-02934-0_3 · Cited by 10
Hamed, I., Zhu, M., Elmahdy, M., Abdennadher, S., & Vu, N. T. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-26061-3_17 · Cited by 10
“switching Language Modeling With Bilingual Word Embeddings 3 for cross-lingual word embeddings. In [28], Upadhyay et al. present an extensive evaluation of four popular crosslingual embedding methods [19,16,9,31] that all require parallel training data, but differ in the degree of data parallelism required. In this wo…”
BackgroundMethodInfluentialHIT - A Hierarchically Fused Deep Attention Network for Robust Code-mixed Language Representation
Sengupta, A., Bhattacharjee, S. K., Chakraborty, T., & Akhtar, S. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.407 · Cited by 14
Xu, N., Gui, T., Ma, R., Zhang, Q., Ye, J., Zhang, M., & Huang, X. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.552 · Cited by 24
Recent advances in machine translation using comparable corpora
Rapp, R., Sharoff, S., & Zweigenbaum, P. (2016) · Natural Language Engineering · doi:10.1017/s1351324916000115 · Cited by 10
“Word mappings (i.e. translation relations) are induced by an EM algorithm in Haghighi et al. (2008) whereas they are directly given by word alignment in parallel corpora in Faruqui and Dyer (2014).”
BackgroundMethodLanguage classification from bilingual word embedding graphs
Eger, S., Hoenen, A., & Mehler, A. (2016) · TUbilio (Technical University of Darmstadt) · doi:10.48550/arxiv.1607.05014 · Cited by 14
“The first is the canonical correlation analysis ( CCA ) approach suggested in Faruqui and Dyer (2014).”
BackgroundMethodInfluentialQuery Translation for Cross-Language Information Retrieval using Multilingual Word Clusters
Bhattacharya, P., Goyal, P., & Sarkar, S. (2016) · International Conference on Computational Linguistics · Cited by 10
Finding next of kin: Cross-lingual embedding spaces for related languages
Sharoff, S. (2019) · Natural Language Engineering · doi:10.1017/s1351324919000354 · Cited by 10
“That study was followed by other studies aimed at improving the process of TM production, for example, via canonical correspondence analysis (Faruqui and Dyer 2014), GC (Dinu et al .”
MethodMulti-Dimension Projection for Non-Linear Data Via Spearman Correlation Analysis (MD-SCA)
Khokhar, M. S., Cheng, K., Ayoub, M., Zakria, Z., & Eric, L. K. (2019) · International Conference on Information and Communication Technologies · doi:10.1109/icict47744.2019.9001973 · Cited by 10
“further SCA based related to informative projection’ applications are, Word Representations[4], medical application[5] and similar subspace learning algorithm [6] is a very efficient and popular algorithm for information projection or multi-view as well dimensional reduction.”
BackgroundA Multi-View Subspace Learning Approach to Internet Traffic Matrix Estimation
Kumar, A., Vidyapu, S., Saradhi, V. V., & Tamarapalli, V. (2020) · IEEE Transactions on Network and Service Management · doi:10.1109/tnsm.2020.2983329 · Cited by 10
“Application of multi-view learning methods, that consider multiple views of the same underlying data, has seen success in diverse application domains [22], [23], [24], [25].”
BackgroundParolin, E. S., Hosseini, M., Hu, Y., Khan, L. R., Brandt, P. T., Osorio, J., & D'Orazio, V. J. (2022) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3514094.3534178 · Cited by 18
“Such finding is consistent to previous studies in cross-lingual representations [2, 14, 22, 36, 73, 74, 89].”
ResultImage search using multilingual texts: a cross-modal learning approach between image and textPreprint
Portaz, M., Randrianarivo, H., Nivaggioli, A., Maudet, E., Servan, C., & Peyronnet, S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1903.11299 · Cited by 15
“One consists in training both models independently and then to learn a mapping from one representation to the other [10, 21].”
BackgroundSparse Overcomplete Word Vector RepresentationsPreprint
Faruqui, M., Tsvetkov, Y., Yogatama, D., Dyer, C., & Smith, N. A. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1506.02004 · Cited by 50
“Faruqui and Dyer (2014) learned vectors by first performing SVD on text in different languages, then applying canonical correlation analysis on pairs of vectors for words that align in parallel corpora.”
MethodSelf-citationUnsupervised Sentiment Analysis for Code-mixed DataPreprint
Yadav, S., & Chakraborty, T. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2001.11384 · Cited by 14
Metadata
- Authors
- Faruqui, M., & Dyer, C.
- Affiliations
- Carnegie Mellon University (US)
- Venue
- Proceedings of the 14th Conference of the European Chapter of the Association for Computational Linguistics
- Published
- 2014
- Pages
- 462-471
- OpenAlex
- W342285082
- Cited by
- 620
- References
- 43
- Citations by year
- 2026: 3 · 2025: 8 · 2024: 33 · 2023: 19 · 2022: 28 · 2021: 57 · 2020: 106 · 2019: 100 · 2018: 91 · 2017: 61 · 2016: 67 · 2015: 33 · 2014: 14
- FWCI
- 34.35
- Type
- conference paper
- Language
- en
- Open access
- green
- figshare.com
- Licence
- cc-by
- Locations
- Conference of the European Chapter of the Association for Computational Linguistics (publishedVersion)
- KiltHub Repository (submittedVersion)
- Figshare (submittedVersion, cc-by)
- KiltHub Repository
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Sentiment Analysis and Opinion Mining
- Keywords
- word embeddings, multilingual word representation, distributional semantics, canonical correlation analysis, word meaning, word sense disambiguation, word similarity
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.