Improving Vector Space Word Representations Using Multilingual Correlation
Open accessThe DOI matches its Crossref record; 43 of the 43 references it cites were found in a registry or an index.
Abstract
The distributional hypothesis of Harris (1954), according to which the meaning of words is evidenced by the contexts they occur in, has motivated several effective techniques for obtaining vector space semantic representations of words using unannotated text corpora. This paper argues that lexico-semantic content should additionally be invariant across languages and proposes a simple technique based on canonical correlation analysis (CCA) for incorporating multilingual evidence into vectors generated monolingually. We evaluate the resulting word representations on standard lexical semantic evaluation tasks and show that our method produces substantially better semantic representations than monolingual techniques.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Besides, Hewitt and Manning [125], Jawahar et al. [126], Kim et al. [127] proposed several methods to extract dependency trees and constituency trees from BERT, which proved the BERT's ability to encode syntax structure.”
From Pre-trained models for natural language processing: A survey (2020) · cites this paper as [125]
Citations
Cites
(2008) · Journal of Machine Learning Research · Cited by 33,051
Efficient Estimation of Word Representations in Vector Space
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1301.3781 · Cited by 26,568 · cited 6×
“This dataset contains word pairs that are different syntactic forms of a given word and was prepared by Mikolov et al. (2013a).”
BackgroundMethodInfluentialIndexing by latent semantic analysis
Deerwester, S., Dumais, S. T., Furnas, G. W., Landauer, T. K., & Harshman, R. (1990) · Journal of the American Society for Information Science · doi:10.1002/(sici)1097-4571(199009)41:6<391::aid-asi1>3.0.co;2-9 · Cited by 12,751
Golub, G. H., & Loan, C. F. V. (1996) · Johns Hopkins University Press eBooks · Cited by 6,027
A unified architecture for natural language processing: deep neural networks with multitask learning
Collobert, R., & Weston, J. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390177 · Cited by 5,058 · cited 6×
“Our method can also be seen as an application of multi-view learning (Chang et al., 2013; Collobert and Weston, 2008), where one of the views can be used to capture cross-lingual information.”
BackgroundMethodInfluentialHarris, Z. S. (1954) · WORD · doi:10.1080/00437956.1954.11659520 · Cited by 4,807
Approximate Statistical Tests for Comparing Supervised Classification Learning Algorithms
Dietterich, T. G. (1998) · Neural Computation · doi:10.1162/089976698300017197 · Cited by 3,840 · cited 2×
“01) according to the McNemar’s test (Dietterich, 1998).”
MethodInfluentialWord association norms, mutual information, and lexicography
Church, K., & Hanks, P. (1990) · doi:10.5555/89086.89095 · Cited by 3,590
Linguistic Regularities in Continuous Space Word Representations
Mikolov, T., Yih, W.-T., & Zweig, G. (2013) · North American Chapter of the Association for Computational Linguistics · Cited by 2,912
From Frequency to Meaning: Vector Space Models of Semantics
Turney, P. D., & Pantel, P. (2010) · Journal of Artificial Intelligence Research · doi:10.1613/jair.2934 · Cited by 2,866 · cited 1×
“Using cooccurrence statistics from a large corpus of text (Deerwester et al., 1990; Turney and Pantel, 2010), 1 it is possible to construct high-quality semantic vectors — as judged by both correlations with human judgements of semantic relatedness (Turney, 2006; Agirre et al., 2009) and as…”
Background
Cited by
Items 301–400 of 617
Identifying Necessary Elements for BERT’s MultilingualityPreprint
Dufter, P., & Schütze, H. (2020) · arXiv (Cornell University) · doi:10.5282/ubm/epub.72199 · Cited by 10
Explicit Alignment Objectives for Multilingual Bidirectional EncodersPreprint
Hu, J., Johnson, M., Fırat, O., Siddhant, A., & Neubig, G. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.07972 · Cited by 10
“Cross-lingual embeddings, both traditional noncontextualized word embeddings (Faruqui and Dyer, 2014) and the more recent contextualized word embeddings (Devlin et al.”
BackgroundLalrempuii, C., & Soni, B. (2023) · ACM Transactions on Asian and Low-Resource Language Information Processing · doi:10.1145/3609222 · Cited by 15
“This task initially required bilingual dictionaries to carry out the transformations [16].”
BackgroundCastells, P., & Jannach, D. (2024) · ACM eBooks · doi:10.1145/3674127.3674131 · Cited by 11
Translation of Unknown Words in Low Resource Languages
Gujral, B., Khayrallah, H., & Koehn, P. (2016) · Edinburgh Research Explorer (University of Edinburgh) · Cited by 9
Modeling Context Words as Regions: An Ordinal Regression Approach to Word Embedding
Jameel, S., & Schockaert, S. (2017) · Conference on Computational Natural Language Learning · doi:10.18653/v1/k17-1014 · Cited by 11
“…de Marn-effe, 2013; Gupta et al., 2015; Rothe and Sch¨utze, 2016), which makes it possible to use them as a source of semantic and linguistic knowledge, and to align word embeddings with visual features (Frome et al., 2013) or across different languages (Zou et al., 2013; Faruqui and Dyer, 2014).”
BackgroundEvaluation by Association: A Systematic Study of Quantitative Word Association Evaluation
Vulić, I., Kiela, D., & Korhonen, A. (2017) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/e17-1016 · Cited by 8
“…compared to all other models in our comparison. paragram-300d , (Wieting et al., 2015)) further re-fined using linguistic constraints ( paragram+cf-300d , (Mrkˇsi´c et al., 2016)); (3) Multilingual embedding models from Luong et al. (2015) ( biskip-256d ) and Faruqui and Dyer (2014) ( bicca-512d ).”
MethodResultReconstructed similarity for faster GANs-based word translation to mitigate hubness
Zhang, D., Luo, M., & He, F. (2019) · Neurocomputing · doi:10.1016/j.neucom.2019.06.082 · Cited by 9
Weakly-Supervised Concept-based Adversarial Learning for Cross-lingual Word Embeddings
Wang, H., Henderson, J., & Merlo, P. (2019) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d19-1450 · Cited by 9
“to map source and target embeddings (Faruqui and Dyer, 2014); or by forcing the mapping matrix to be orthogonal (Xing et al.”
BackgroundCross-Lingual Word Embeddings and the Structure of the Human Bilingual Lexicon
Merlo, P., & Rodriguez, M. A. (2019) · Conference on Computational Natural Language Learning · doi:10.18653/v1/k19-1011 · Cited by 11
“However, traditional methods for mapping two monolingual word embeddings require high quality aligned sentences or dictionaries (Faruqui and Dyer, 2014; Ammar et al., 2016).”
BackgroundA novel context-aware recommender system based on a deep sequential learning approach (CReS)
Thaipisutikul, T., & Shih, T. K. (2021) · Neural Computing and Applications · doi:10.1007/s00521-020-05640-w · Cited by 9
“We pre-trained R d (cid:5) I j j by using the Word2Vec [43] technique to learn the semantic embedding from all users’ sequential transactions L.”
MethodLearning Bilingual Word Embedding Mappings with Similar Words in Related Languages Using GAN
Alipour, G., Bagherzadeh, J., & Feizi‐Derakhshi, M. (2022) · Applied Artificial Intelligence · doi:10.1080/08839514.2021.2019885 · Cited by 9
“This paper proposes a new method to learn bilingual word embedding mapping that improves previous works (Artetxe, Labaka, and Agirre 2016; Faruqui and Dyer 2014; Smith et al. ; Xing et al. 2015; Zhang et al. 2016).”
MethodSubspace perspective on canonical correlation analysis: Dimension reduction and minimax rates
Ma, Z., & Li, X. (2019) · Bernoulli · doi:10.3150/19-bej1131 · Cited by 9
“In natural language processing, CCA is applied to the word co-occurrence matrix and learns vector representations of the words which capture the semantics (Dhillon et al., 2011; Faruqui and Dyer, 2014).”
MethodVector Embedding of Wikipedia Concepts and EntitiesPreprint
Sherkat, E., & Milios, E. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1702.03470 · Cited by 9
“Recently, other methods have been introduced that try to both improve the performance and quality of the word embedding [3] by using multilingual correlation.”
BackgroundComparing State-of-the-art Dependency Parsers on the Italian Stanford Dependency Treebank
Lavelli, A. (2016) · Accademia University Press eBooks · doi:10.4000/books.aaccademia.1790 · Cited by 8
Yang, F., Zhang, S., Pan, W., Yao, R., Zhang, W., Zhang, Y., Wang, G., Zhang, Q., Cheng, Y., Dong, J., et al. (2022) · Briefings in Bioinformatics · doi:10.1093/bib/bbac124 · Cited by 10
Inducing Bilingual Lexica From Non-Parallel Data With Earth Mover’s Distance Regularization
Zhang, M., Liu, Y., Luan, H., Liu, Y., & Sun, M. (2016) · International Conference on Computational Linguistics · Cited by 8
Reranking Translation Candidates Produced by Several Bilingual Word Similarity Sources
Jakubina, L., & Langlais, P. (2017) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/e17-2096 · Cited by 8
“In this paper, we investigate the reranking of n - best lists of translations produced by two embedding approaches (Mikolov et al., 2013b; Faruqui and Dyer, 2014) as well as a plain distributional approach (Rapp, 1995).”
BackgroundMethodImproving Bilingual Lexicon Induction for Low Frequency Words
Huang, J., Cai, X., & Church, K. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.100 · Cited by 10
“This type of methods have become especially popular in recent years (Mikolov et al., 2013; Faruqui and Dyer, 2014; Artetxe et al., 2016, 2018) and achieved state-of-art accuracies (Conneau et al.”
BackgroundMethodMultilingual Text Classification from Twitter during Emergencies
Piscitelli, S., Arnaudo, E., & Rossi, C. (2021) · International Conference on Consumer Electronics · doi:10.1109/icce50685.2021.9427581 · Cited by 13
“Various solutions have been proposed to solve this issue, ranging from transformations in latent space with Singular Value Decomposition (SVD) [26] or Canonical Correlation Analysis (CCA) [27], to fully-fledged trained multilingual embeddings such as ConceptNet [28] or MUSE [4].”
MethodMUSEDA: Multilingual Unsupervised and Supervised Embedding for Domain Adaption
Liang, X., Gu, Z., Xie, Y., Wang, L., & Tian, Z. (2023) · Knowledge-Based Systems · doi:10.1016/j.knosys.2023.110560 · Cited by 8
English–Welsh Cross-Lingual Embeddings
Espinosa-Anke, L., Palmer, G., Corcoran, P., Filimonov, M., Spasić, ., & Knight, D. (2021) · Applied Sciences · doi:10.3390/app11146541 · Cited by 9
“Therefore, the research in this space gravitated towards using bilingual dictionaries instead of aligning the words in respective languages [6,26].”
BackgroundCross Lingual Word Embeddings for Turkic Languages
Kuriyozov, E. R. (2020) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.3666696 · Cited by 8
Filtered Inner Product Projection for Crosslingual Embedding AlignmentPreprint
Sachidananda, V., Yang, Z., & Zhu, C. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.03652 · Cited by 10
Leveraging Pretrained Word Embeddings for Part-of-Speech Tagging of Code Switching Data
AlGhamdi, F., & Diab, M. (2019) · doi:10.18653/v1/w19-1410 · Cited by 8
Natural Language Understanding in Socially Interactive Agents
Pieraccini, R. (2021) · ACM eBooks · doi:10.1145/3477322.3477328 · Cited by 7
Multi-lingual Entity Discovery and Linking
Sil, A., Ji, H., Roth, D., & Cucerzan, S.-P. (2018) · doi:10.18653/v1/p18-5008 · Cited by 11
“Hence, before diving into the architectures we will survey multi-lingual embedding techniques (Mikolov et al., 2013c; Faruqui and Dyer, 2014; Ammar et al., 2016) and which ones work best for neural EL systems and motivate neural EL.”
UKP TU-DA at GermEval 2017: Deep Learning for Aspect Based Sentiment Detection
Lee, J.-U., Eger, S., Daxenberger, J., & Gurevych, I. (2017) · TUbilio (Technical University of Darmstadt) · Cited by 7
Learning Multilingual Word Embeddings Using Image-Text Data
Singhal, K., Raman, K., & ten Cate, B. (2019) · doi:10.18653/v1/w19-1807 · Cited by 9
“Most work on producing multilingual embeddings has relied on crosslingual human-labeled data, such as bilingual lexicons (Mikolov et al., 2013b; Ammar et al., 2016; Faruqui and Dyer, 2014b; Xing et al., 2015) or parallel/aligned corpora (Kle-”
BackgroundCross-lingual Sentiment Analysis via AAE and BiGRU
Shen, J., Liao, X., & Lei, S. (2020) · Asia-Pacific Conference on Image Processing, Electronics and Computers (IPEC) · doi:10.1109/ipec49694.2020.9115134 · Cited by 8
“Former works aim at learning a linear map, which schedules independently to generate monolingual space into a single shared space by means of a seed translation dictionary [5].”
BackgroundA Simple and Effective Approach to Robust Unsupervised Bilingual Dictionary Induction
Li, Y., Luo, Y., Ye, L., Du, Q., Wang, H., Huang, S., Xiao, T., & Zhu, J. (2020) · International Conference on Computational Linguistics · doi:10.18653/v1/2020.coling-main.526 · Cited by 13
A Novel Neurofuzzy Approach for Semantic Similarity Measurement
Martínez-Gil, J., Mokadem, R., Küng, J., & Hameurlain, A. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-86534-4_18 · Cited by 8
Code-switched inspired losses for spoken dialog representations
Colombo, P., Chapuis, É., Labeau, M., & Clavel, C. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.656 · Cited by 24
“However, they focus on the word level (Gouws et al., 2015; Mikolov et al., 2013; Faruqui and Dyer, 2014) or the utterance level (Devlin et al.”
BackgroundOneSpace: Detecting cross-language clones by learning a common embedding space
Elarnaoty, M., & Servant, F. (2023) · Journal of Systems and Software · doi:10.1016/j.jss.2023.111911 · Cited by 9
A Hybrid Distributional and Knowledge-based Model of Lexical Semantics
Αλέτρας, Ν., & Stevenson, M. (2015) · Joint Conference on Lexical and Computational Semantics · doi:10.18653/v1/s15-1003 · Cited by 7
“We apply CCA over the reduced spaces learned using LSA, ˜ H and ˜ H p to obtain two joint semantic spaces following a similar approach to Faruqui and Dyer (2014).”
BackgroundMethodDetecting actionable items in meetings by convolutional deep structured semantic models
Chen, Y.-N., Hakkani-Tür, D. Z., & He, X. (2015) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru.2015.7404819 · Cited by 7
“In addition, embedding adaptation has been studied using different languages and external knowledge [25, 26].”
BackgroundThe best answer prediction by exploiting heterogeneous data on software development Q&A forum
Zheng, W., & Li, M. (2017) · Neurocomputing · doi:10.1016/j.neucom.2016.12.097 · Cited by 7
“Based on above methods, multi-view learning has wide applicaions because there exists many views in real life, such as simultaeously recorded audio + video [19,20] , audio + articulation [21] , mages + text [22,23] , parallel text in two languages [24,25] , or atural languages + source code [26] .”
MethodA Chatbot Framework for the Children's Legal Centre
Morgan, J., Paiement, A., Seisenberger, M., Jane, W., & Wyner, A. (2018) · Frontiers in artificial intelligence and applications · doi:10.3233/978-1-61499-935-5-205 · Cited by 7
“2) to classify input messages as a speech act and legal type, words are tokenized and converted to word vectors of 200 dimensions, aiming to capture the semantic similarities between words [1].”
BackgroundDisentangling Latent Emotions of Word Embeddings on Complex Emotional Narratives
Wu, Z., & Jiang, Y. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-32236-6_53 · Cited by 7
Transferred Embeddings for Igbo Similarity, Analogy, and Diacritic Restoration Tasks
Ezeani, I., Onyenwe, I., & Hepple, M. (2018) · Lancaster EPrints (Lancaster University) · Cited by 7
An Empirical Study on Crosslingual Transfer in Probabilistic Topic Models
Hao, S., & Paul, M. J. (2020) · Computational Linguistics · doi:10.1162/coli_a_00369 · Cited by 11
“…embeddings is to use a projection matrix as supervision or sub-objective to learn a projection matrix that projects independently trained monolingual embeddings into a shared crosslingual space (Dinu and Baroni 2014; Faruqui and Dyer 2014; Tsvetkov and Dyer 2016; Vuli´c and Korhonen 2016).”
BackgroundEmpirical studies on word representations
Šuster, S. (2016) · University of Groningen research database (University of Groningen / Centre for Information Technology) · Cited by 7
Compass-aligned Distributional Embeddings for Studying Semantic Differences across CorporaPreprint
Bianchi, F., Carlo, V. D., Nicoli, P., & Palmonari, M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.06519 · Cited by 16
“Multilingual lexicons are used to stabilize the position of some words in the vector space (Faruqui & Dyer, 2014; Xing, Wang, Liu, & Lin, 2015; Smith et al., 2017): defining a set of anchors or a mapping dictionary requires domain knowledge and might be challenging in some contexts since it…”
MethodCode-switched inspired losses for generic spoken dialog representationsPreprint
Chapuis, É., Colombo, P., Labeau, M., & Clavel, C. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2108.12465 · Cited by 7
“However, they focus on the word level (Gouws et al., 2015; Mikolov et al., 2013; Faruqui and Dyer, 2014) or the utterance level (Devlin et al.”
BackgroundAn Overview of Cross-Language Information Retrieval
Zhang, L., & Zhao, X. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-57884-8_3 · Cited by 11
Encoding Prior Knowledge with Eigenword Embeddings
Osborne, D., Narayan, S., & Cohen, S. B. (2016) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00108 · Cited by 12
Unsupervised Evaluation of Human Translation Quality
Zhou, Y., & Bollegala, D. (2019) · International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management · doi:10.5220/0008064500550064 · Cited by 8
“A comparable corpus contains documents that are not exact parallel translations but convey the same information in different languages (Faruqui and Dyer, 2014; Gouws and Søgaard, 2015).”
BackgroundAutomatic learning of modality exclusivity norms with crosslingual word embeddings
Chersoni, E., Xiang, R., Lu, Q., & Huang, C. (2020) · PolyU Institutional Research Archive (Hong Kong Polytechnic University) · Cited by 6
Analogy Training Multilingual Encoders
Garneau, N., Hartmann, M., Sandholm, A., Ruder, S., Vulić, I., & Søgaard, A. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i14.17524 · Cited by 11
“…and Hall 2011; T¨ackstr¨om et al. 2013), e.g., through general-purpose multilingual representations , at the word level (Mikolov, Le, and Sutskever 2013; Faruqui and Dyer 2014; Artetxe, Labaka, and Agirre 2017) or at the sentence level and in context (Devlin et al. 2019; Lample and Conneau 2019).”
BackgroundLearning English–Chinese bilingual word representations from sentence-aligned parallel corpus
Yen, A.-Z., Huang, H., & Chen, H. (2019) · Computer Speech & Language · doi:10.1016/j.csl.2019.01.002 · Cited by 6
Cross-lingual Joint Entity and Word Embedding to Improve Entity Linking and Parallel Sentence Mining
Pan, X., Gowda, T., Ji, H., May, J., & Miller, S. (2019) · Workshop on Deep Learning Approaches for Low-Resource NLP (DeepLo) · doi:10.18653/v1/d19-6107 · Cited by 7
“After that, several methods attempt to improve the mapping (Faruqui and Dyer, 2014; Xing et al., 2015; Lazaridou et al., 2015; Ammar et al., 2016; Artetxe et al., 2017; Smith et al., 2017).”
MethodImproving Vector Space Word Representations Via Kernel Canonical Correlation Analysis
Bai, X., Cao, H., & Zhao, T. (2018) · ACM Transactions on Asian and Low-Resource Language Information Processing · doi:10.1145/3197566 · Cited by 6
“Faruqui and Dyer 2014 [7] extend this idea by using canonical correlation analysis (CCA) to map both languages to a shared vector space and produce semantic vectors with state-of-the-art performance.”
BackgroundMethodInfluentialGenerating Semantic Similarity Atlas for Natural Languages
Şenel, L. K., Utlu, İ., Yücesoy, V., Koç, A., & Çukur, T. (2018) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt.2018.8639521 · Cited by 10
“Among the studies that aim to learn cross-lingual embedding models, some try to learn transformation matrices to map monolingual word representations in one language to representations in another language using seed bilingual lexicons [9–12], while others learn cross-lingual embeddings directly from pseudo-aligned mul…”
BackgroundPoint Set Registration for Unsupervised Bilingual Lexicon Induction
Cao, H., & Zhao, T. (2018) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2018/555 · Cited by 6
“Learning word embeddings for cross-lingual natural language processing has attracted much attention [Klementiev et al. , 2012; Chandar et al. , 2014; Faruqui and Dyer, 2014; Hermann and Blunsom, 2014; Gouws et al. , 2015; Luong et al. , 2015; Shi et al. , 2015; Vuli´c and Moens, 2015; Upadhyay et…”
BackgroundExploring Metaphorical Senses and Word Representations for Identifying MetonymsPreprint
Zhang, W., & Gelernter, J. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1508.04515 · Cited by 6
“We used the 64 dimensional vector trained with a variation of Latent Semantic Analysis by Faruqui and Dyer (2014).”
MethodCross-lingual hate speech detection based on multilingual domain-specific word embeddingsPreprint
Arango, A., Pérez, J. E. P., & Poblete, B. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.14728 · Cited by 6
“To achieve this we adopted the method proposed by Faruqui and Dyer [23], which finds embedding projections that maximize the correlation between word embeddings from different feature spaces.”
InfluentialTowards a continuous modeling of natural language domains
Ruder, S., Ghaffari, P., & Breslin, J. G. (2016) · doi:10.18653/v1/w16-6012 · Cited by 5
“In computer vision, methods such as Subspace Alignment (Fernando et al., 2013) or the Geodesic Flow Kernel (Gong et al., 2012) have been used to find such transformations A and B. Similarly, in natural language processing, CCA (Faruqui and Dyer, 2014) and Procrustes analysis (Mogadala and Rettinger, 2016) have been us…”
MethodMultilingual Vector Representations of Words, Sentences, and Documents
de Melo, G. (2017) · International Joint Conference on Natural Language Processing · Cited by 5
Multilingual Semantic Relatedness Using Lightweight Machine Translation
Barzegar, S., Davis, B., Handschuh, S., & Freitas, A. (2018) · IEEE International Conference on Semantic Computing (ICSC) · doi:10.1109/icsc.2018.00024 · Cited by 5
“Faruqui & Dyer[4] use the distributional invariance across languages and propose a technique based on canonical correlation analysis (CCA) for merging multi-lingual evidence into vectors generated in monolingual fashion.”
MethodLeveraging Meta-Embeddings for Bilingual Lexicon Extraction from Specialized Comparable Corpora
Morin, E., & Hazem, A. (2018) · HAL · Cited by 5
MAAM: A Morphology-Aware Alignment Model for Unsupervised Bilingual Lexicon Induction
Yang, P., Luo, F., Chen, P., Liu, T., & Sun, X. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1308 · Cited by 7
Contextual Embeddings for Arabic-English Code-Switched Data
Sabty, C., Islam, M., & Abdennadher, S. (2020) · Cited by 5
Neurofuzzy semantic similarity measurement
Martínez-Gil, J., Mokadem, R., Küng, J., & Hameurlain, A. (2023) · Data & Knowledge Engineering · doi:10.1016/j.datak.2023.102155 · Cited by 6
Improving cross-lingual language understanding with consistency regularization-based fine-tuning
Zheng, B., & Che, W. (2023) · International Journal of Machine Learning and Cybernetics · doi:10.1007/s13042-023-01854-1 · Cited by 5
“See Tables 12, 13, 14, 15, 16 and 17.”
BackgroundMapping Entity Sets in News Archives Across Time
Duan, Y., Jatowt, A., Bhowmick, S. S., & Yoshikawa, M. (2019) · Data Science and Engineering · doi:10.1007/s41019-019-00102-3 · Cited by 5
“Faruqui et al. obtained bilingual word vectors using CCA [9].”
Revisiting the linearity in cross-lingual embedding mappings: from a perspective of word analogies
Peng, X., Lin, C., Stevenson, M., & li, C. (2020) · White Rose Research Online (University of Leeds, The University of Sheffield, University of York) · Cited by 4
Koehn, P. (2020) · Cambridge University Press eBooks · doi:10.1017/9781108608480.003 · Cited by 5
“Wikily” Supervised Neural Translation Tailored to Cross-Lingual Tasks
Rasooli, M. S., Callison-Burch, C., & Wijaya, D. T. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.124 · Cited by 14
Cross-lingual embeddings with auxiliary topic models
Zhou, D. M., Peng, X., Li, L., & Han, J. (2021) · Expert Systems with Applications · doi:10.1016/j.eswa.2021.116194 · Cited by 6
Aryal, S. K., Prioleau, H., & Washington, G. J. (2022) · doi:10.5121/csit.2022.122013 · Cited by 12
Refinement of Unsupervised Cross-Lingual Word Embeddings
Biesialska, M., & Costa‐jussà, M. R. (2020) · Frontiers in artificial intelligence and applications · doi:10.3233/faia200317 · Cited by 6
A Novel Bilingual Word Embedding Method for Lexical Translation Using Bilingual Sense CliquePreprint
Wang, R., Zhao, H., Ploux, S., Lu, B., Utiyama, M., & Sumita, E. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1607.08692 · Cited by 6
Cross-lingual Dataless Classification for Languages with Small Wikipedia PresencePreprint
Song, Y., Mayhew, S., & Roth, D. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.04122 · Cited by 5
“…(Al-Rfou’, Perozzi, & Skiena, 2013), or cross-lingual (Klementiev, Titov, & Bhattarai, 2012; Xiao & Guo, 2013; Hermann & Blunsom, 2014; Faruqui & Dyer, 2014; Lu, Wang, Bansal, Gimpel, & Livescu, 2015; Upadhyay, Faruqui, Dyer, & Roth, 2016) representation learning are also investigated.”
Kyadige, A., Rudd, E. M., & Berlin, K. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.06987 · Cited by 5
“Other common applications of multi-view learning, both in and outside of the computer vision space, include cross-spectral fusion [33], joint textual/visual content representation for image tagging and retrieval[17], Joint modeling of web page text and inbound hyperlinks [4], and multi-lingual modeling [15] to name a…”
BackgroundDo We Really Need Fully Unsupervised Cross-Lingual Embeddings?Preprint
Vulić, I., Glavaš, G., Reichart, R., & Korhonen, A. (2019) · arXiv (Cornell University) · doi:10.17863/cam.44007 · Cited by 6
Multilingual Multimodal Pre-training for Zero-Shot Cross-Lingual Transfer of Vision-Language ModelsPreprint
Huang, P.-Y., Patrick, M., Hu, J., Neubig, G., Metze, F., & Hauptmann, A. G. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.08849 · Cited by 5
“, 2015) or a bilingual dictionary to learn a transformation (Faruqui and Dyer, 2014; Mikolov et al., 2013).”
BackgroundCross-language Information RetrievalPreprint
Galuščáková, P., Oard, D. W., & Nair, S. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.05988 · Cited by 24
“In that work, they compared three ways of creating projection matrices: Canonical Correlation Analysis (CCA) [45], minimizing the Euclidean distance between the projected vector and the target vector [131], and maximizing a retrieval-oriented measure they call Cross-Domain Similarity Local Scaling…”
MethodImproving bilingual word embeddings mapping with monolingual context information
Zhu, S., Mi, C., Li, T., Zhang, F., Zhang, Z., & Sun, Y. (2021) · Machine Translation · doi:10.1007/s10590-021-09274-0 · Cited by 4
“Since then, several approaches have been proposed to optimize the bilingual word embedding mappings Faruqui and Dyer (2014); Xing et al. (2015); Ammar et al. (2016); Zhang et al. (2017b).”
BackgroundMethodA Multilingual Approach to Scene Text Visual Question Answering
Pujolràs, J. B. I., Gómez, L., & Karatzas, D. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-06555-2_5 · Cited by 4
Smart Chatbot for Guidance About Children’s Legal Rights
Kumar, A., Joshi, P., Saini, A. K., Kumari, A., Chaudhary, C., & Joshi, K. K. (2023) · Lecture notes in networks and systems · doi:10.1007/978-981-99-1909-3_35 · Cited by 5
Trippas, J. R. (2024) · ACM eBooks · doi:10.1145/3674127.3674135 · Cited by 6
Acoustic Feature Learning Using Cross-Domain Articulatory Measurements
Tang, Q., Wang, W., & Livescu, K. (2018) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2018.8461818 · Cited by 6
Language cognition and language computation — human and machine language understanding
王, 少., 丁, 鼐., 林, 楠., 张, 家., & 宗, 成. (2022) · Scientia Sinica Informationis · doi:10.1360/ssi-2021-0100 · Cited by 14
Retracted on March 14, 2023 : Cross-lingual transfer learning for statistical type inference
Li, Z., Xie, X., Li, H., Xu, Z., Li, Y., & Liu, Y. (2022) · ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA) · doi:10.1145/3533767.3534411 · Cited by 4
“We further propose a syntax enhancement strategy which is to augment the input representation with srcML meta-grammar [14, 32].”
MethodRetractedEnhancing Semantic Representations of Bilingual Word Embeddings with Syntactic Dependencies
Xu, L., Ouyang, W., Ren, X., Wang, Y., & Jiang, L. (2018) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2018/628 · Cited by 4
“, 2013b] utilize a set of meaning-equivalent pairs to learn the linear mapping; while canonical correlation analysis (CCA) is employed in [Faruqui and Dyer, 2014] to project words from two languages to a shared bilingual embedding space.”
MethodIncorporating Word Embedding into Cross-Lingual Topic Modeling
Chang, C., Hwang, S., & Xui, T.-H. (2018) · IEEE International Congress on Big Data (BigData Congress) · doi:10.1109/bigdatacongress.2018.00010 · Cited by 4
“For example, CCA [14] which projects the word space of each language to a common word space can be considered.”
BackgroundMethodInfluentialDiscriminative Sparse Generalized Canonical Correlation Analysis (DSGCCA)
Guo, C., & Wu, D. (2019) · Chinese Automation Congress · doi:10.1109/cac48633.2019.8997468 · Cited by 4
“CCA can be regarded as a dimensionality reduction method [27], [28] for multi-view data.”
Neural machine translation of Hindi and English
Bhatnagar, S., & Chatterjee, N. (2020) · Journal of Intelligent & Fuzzy Systems · doi:10.3233/jifs-179873 · Cited by 7
“Intuitively, if we have two words with similar meanings in two different languages, then if they are to be mapped to the same embedding space, the correlation of their embeddings should be maximum [11].”
Ishiwatari, S., Kaji, N., Yoshinaga, N., Toyoda, M., & Kitsuregawa, M. (2015) · Conference on Computational Natural Language Learning · doi:10.18653/v1/k15-1030 · Cited by 4
“…meaning representations shared by different languages (Klementiev et al., 2012; Lauly et al., 2014; Xiao and Guo, 2014; Hermann and Blun-som, 2014; Faruqui and Dyer, 2014; Gouws and Søgaard, 2015), rather than learning transformation between different languages (Fung, 1998; Mikolov et al., 2013b;…”
BackgroundExploring Implicit Semantic Constraints for Bilingual Word Embeddings
Su, J., Song, Z., Lu, Y., Xu, M., Wu, C., & Chen, Y. (2017) · Neural Processing Letters · doi:10.1007/s11063-017-9762-8 · Cited by 4
Second-Order Word Embeddings from Nearest Neighbor Topological FeaturesPreprint
Newman-Griffis, D. R., & Fosler‐Lussier, E. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.08488 · Cited by 11
“Additionally, multilingual data has been extensively investigated for improving language models: Upadhyay et al. [24] review several recent methods with multilingual corpora, while Faruqui and Dyer [25] and Lu et al. [26] used canonical correlation analysis to learn cross-lingual embeddings.”
BackgroundLearning Rare Word Representations using Semantic BridgingPreprint
Prokhorov, V., Pilehvar, M. T., Kartsaklis, D., Lió, P., & Collier, N. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1707.07554 · Cited by 5
“…used for the mapping of semantic spaces belonging to different languages, mainly with the purpose of learning multilingual semantic spaces: Least squares (Mikolov et al., 2013b; Dinu and Baroni, 2014, LS) and Canonical Correlation Analysis (Faruqui and Dyer, 2014; Upadhyay et al., 2016, CCA) .”
BackgroundRobust Cross-lingual Hypernymy Detection using Dependency ContextPreprint
Upadhyay, S., Vyas, Y., Carpuat, M. J., & Roth, D. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1803.11291 · Cited by 4
“Cross-lingual Distributional Semantics Cross-lingual word embeddings have been shown to encode semantics across languages in tasks such as word similarity (Faruqui and Dyer, 2014) and lexicon induction (Vuli´c and Moens, 2015).”
BackgroundLearning Cross-lingual Embeddings from Twitter via Distant SupervisionPreprint
Camacho-Collados, J., Doval, Y., Martínez‐Cámara, E., Espinosa-Anke, L., Barbieri, F., & Schockaert, S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.07358 · Cited by 4
“…are becoming increasingly popular in NLP (Upadhyay et al., 2016; Ruder et al., 2018), especially since the recent introduction of models requiring almost no super-vision (Mikolov et al., 2013b; Faruqui and Dyer, 2014; Xing et al., 2015; Smith et al., 2017; Artetxe et al., 2017; Doval et al., 2018).”
BackgroundImproving Pre-Trained Multilingual Models with Vocabulary ExpansionPreprint
Wang, H., Yu, D., Sun, K., Chen, J., & Yu, D. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.12440 · Cited by 4
“While there is no direct precedent, previous work show that incorporating multilingual contexts can improve monolingual word embeddings (Zou et al., 2013; Andrew et al., 2013; Faruqui and Dyer, 2014; Lu et al., 2015; Ruder et al., 2017).”
BackgroundMultilingual transfer learning for code-switched language and speech neural modeling
Winata, G. I. (2021) · doi:10.14711/thesis-991012936363903412 · Cited by 4
Consistency Regularization for Cross-Lingual Fine-TuningPreprint
Zheng, B., Dong, L., Huang, S., Wang, W., Chi, Z., Singhal, S., Che, W., Liu, T., Song, X., & Wei, F. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.08226 · Cited by 4
“Cross-Lingual Transfer Besides learning crosslingual word embeddings (Mikolov et al., 2013; Faruqui and Dyer, 2014; Guo et al., 2015; Xu et al., 2018; Wang et al., 2019), most recent work of cross-lingual transfer is based on pre-trained cross-lingual language models (Conneau and Lample, 2019; Conneau et al.”
Effective transfer learning for low-resource natural language understanding
Liu, Z. (2022) · doi:10.14711/thesis-991013106358603412 · Cited by 9
Metadata
- Authors
- Faruqui, M., & Dyer, C.
- Affiliations
- Carnegie Mellon University (US)
- Venue
- Proceedings of the 14th Conference of the European Chapter of the Association for Computational Linguistics
- Published
- 2014
- Pages
- 462-471
- OpenAlex
- W342285082
- Cited by
- 620
- References
- 43
- Citations by year
- 2026: 3 · 2025: 8 · 2024: 33 · 2023: 19 · 2022: 28 · 2021: 57 · 2020: 106 · 2019: 100 · 2018: 91 · 2017: 61 · 2016: 67 · 2015: 33 · 2014: 14
- FWCI
- 34.35
- Type
- conference paper
- Language
- en
- Open access
- green
- figshare.com
- Licence
- cc-by
- Locations
- Conference of the European Chapter of the Association for Computational Linguistics (publishedVersion)
- KiltHub Repository (submittedVersion)
- Figshare (submittedVersion, cc-by)
- KiltHub Repository
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Sentiment Analysis and Opinion Mining
- Keywords
- word embeddings, multilingual word representation, distributional semantics, canonical correlation analysis, word meaning, word sense disambiguation, word similarity
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.