The DOI matches its Crossref record; 47 of the 47 references it cites were found in a registry or an index.
Monolingual and Cross-Lingual Information Retrieval Models Based on (Bilingual) Word Embeddings
Open accessAbstract
We propose a new unified framework for monolingual (MoIR) and cross-lingual information retrieval (CLIR) which relies on the induction of dense real-valued word vectors known as word embeddings (WE) from comparable data. To this end, we make several important contributions: (1) We present a novel word representation learning model called Bilingual Word Embeddings Skip-Gram (BWESG) which is the first model able to learn bilingual word embeddings solely on the basis of document-aligned comparable data; (2) We demonstrate a simple yet effective approach to building document embeddings from single word embeddings by utilizing models from compositional distributional semantics. BWESG induces a shared cross-lingual embedding vector space in which both words, queries, and documents may be presented as dense real-valued vectors; (3) We build novel ad-hoc MoIR and CLIR models which rely on the induced word and document embeddings and the shared cross-lingual embedding space; (4) Experiments for English and Dutch MoIR, as well as for English-to-Dutch and Dutch-to-English CLIR using benchmarking CLEF 2001-2003 collections and queries demonstrate the utility of our WE-based MoIR and CLIR models. The best results on the CLEF collections are obtained by the combination of the WE-based approach and a unigram language model. We also report on significant improvements in ad-hoc IR tasks of our WE-based framework over the state-of-the-art framework for learning text representations from comparable data based on latent Dirichlet allocation (LDA).
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Glove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071
(2003) · Journal of Machine Learning Research · doi:10.5555/944919.944937 · Cited by 32,337
Learning representations by back-propagating errors
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986) · Nature · doi:10.1038/323533a0 · Cited by 31,892
Efficient Estimation of Word Representations in Vector Space
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1301.3781 · Cited by 26,568
Introduction to information retrieval
(2009) · Choice Reviews Online · doi:10.5860/choice.46-2715 · Cited by 12,278
A vector space model for automatic indexing
Salton, G., Wong, A., & Yang, C. S. (1975) · Communications of the ACM · doi:10.1145/361219.361220 · Cited by 7,521 · cited 1×
“Knowing the long-standing and firm relationship between vector space models, semantics modeling and IR [37, 16], one goal of this paper is to establish a new link between the recent text representation learning methodology based on word embeddings and modeling in information retrieval , with the…”
BackgroundLandauer, T. K., & Dumais, S. T. (1997) · Psychological Review · doi:10.1037/0033-295x.104.2.211 · Cited by 6,282
A unified architecture for natural language processing: deep neural networks with multitask learning
Collobert, R., & Weston, J. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390177 · Cited by 5,058
Natural Language Processing (almost) from Scratch
Collobert, R., Weston, J., Bottou, L., Karlen, M., Kavukcuoglu, K., & Kuksa, P. P. (2011) · arXiv (Cornell University) · doi:10.48550/arxiv.1103.0398 · Cited by 4,962
(2005) · Cited by 3,125
Linguistic Regularities in Continuous Space Word Representations
Mikolov, T., Yih, W.-T., & Zweig, G. (2013) · North American Chapter of the Association for Computational Linguistics · Cited by 2,912
A neural probabilistic language model
BengioYoshua, DucharmeRéjean, VincentPascal, & JanvinChristian (2003) · Journal of Machine Learning Research · doi:10.5555/944919.944966 · Cited by 2,644
A Language Modeling Approach to Information Retrieval
Ponte, J. M., & Croft, W. B. (2017) · ACM SIGIR Forum · doi:10.1145/3130348.3130368 · Cited by 2,503
(2010) · Cited by 1,906
A language modeling approach to information retrieval
Ponte, J. M., & Croft, W. B. (1998) · Proceedings of the 21st annual international ACM SIGIR conference on Research and development in information retrieval · doi:10.1145/290941.291008 · Cited by 1,821
Neural Word Embedding as Implicit Matrix Factorization
Levy, O., & Goldberg, Y. (2014) · neural information processing systems · Cited by 1,582
Relevance-Based Language Models
Lavrenko, V., & Croft, W. B. (2017) · ACM SIGIR Forum · doi:10.1145/3130348.3130376 · Cited by 1,422
Baroni, M., Dinu, G., & Kruszewski, G. (2014) · Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.3115/v1/p14-1023 · Cited by 1,364
Improving Distributional Similarity with Lessons Learned from Word Embeddings
Levy, O., Goldberg, Y., & Dagan, I. (2015) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00134 · Cited by 1,308
Semantic Compositionality through Recursive Matrix-Vector Spaces
Socher, R., Huval, B., Manning, C. D., & Ng, A. Y. (2012) · Cited by 1,259
word2vec Explained: deriving Mikolov et al.'s negative-sampling word-embedding method
Goldberg, Y., & Levy, O. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1402.3722 · Cited by 1,245
A study of smoothing methods for language models applied to information retrieval
Zhai, C., & Lafferty, J. (2004) · ACM Transactions on Information Systems · doi:10.1145/984321.984322 · Cited by 1,201
LDA-based document models for ad-hoc retrieval
Wei, X., & Croft, W. B. (2006) · Proceedings of the 29th annual international ACM SIGIR conference on Research and development in information retrieval · doi:10.1145/1148170.1148204 · Cited by 1,075
Dependency-Based Word Embeddings
Levy, O., & Goldberg, Y. (2014) · Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) · doi:10.3115/v1/p14-2050 · Cited by 1,047
Relevance based language models
Lavrenko, V., & Croft, W. B. (2001) · Proceedings of the 24th annual international ACM SIGIR conference on Research and development in information retrieval · doi:10.1145/383952.383972 · Cited by 1,037
Steyvers, M., & Griffiths, T. (2015) · Handbook of Latent Semantic Analysis · doi:10.4324/9780203936399.ch21 · Cited by 1,011
Vector-based Models of Semantic Composition
Mitchell, J., & Lapata, M. (2008) · Edinburgh Research Explorer (University of Edinburgh) · Cited by 872
Linguistic Regularities in Sparse and Explicit Word Representations
Levy, O., & Goldberg, Y. (2014) · Proceedings of the Eighteenth Conference on Computational Natural Language Learning · doi:10.3115/v1/w14-1618 · Cited by 628
Information Retrieval as Statistical Translation
Berger, A., & Lafferty, J. (2017) · ACM SIGIR Forum · doi:10.1145/3130348.3130371 · Cited by 621
Learning word embeddings efficiently with noise-contrastive estimation
Mnih, A., & Kavukcuoglu, K. (2013) · neural information processing systems · Cited by 446
Information retrieval as statistical translation
Berger, A., & Lafferty, J. (1999) · Proceedings of the 22nd annual international ACM SIGIR conference on Research and development in information retrieval · doi:10.1145/312624.312681 · Cited by 396
Inducing Crosslingual Distributed Representations of Words
Klementiev, A., Titov, I., & Bhattarai, B. (2012) · International Conference on Computational Linguistics · Cited by 389
Mimno, D., Wallach, H. M., Naradowsky, J., Smith, D. A., & McCallum, A. (2009) · Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing Volume 2 - EMNLP '09 · doi:10.3115/1699571.1699627 · Cited by 328
Natural Language Processing (Almost) from Scratch
CollobertRonan, WestonJason, BottouLéon, KarlenMichael, KavukcuogluKoray, & KuksaPavel (2011) · Journal of Machine Learning Research · doi:10.5555/1953048.2078186 · Cited by 306
Multilingual Models for Compositional Distributed Semantics
Hermann, K. M., & Blunsom, P. (2014) · Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.3115/v1/p14-1006 · Cited by 268
A Comparison of Vector-based Representations for Semantic Composition
Blacoe, W., & Lapata, M. (2012) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · Cited by 229
Learning Image Embeddings using Convolutional Neural Networks for Improved Multi-Modal Semantics
Kiela, D., & Bottou, L. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1005 · Cited by 224
Cross-lingual relevance models
Lavrenko, V., Choquette, M., & Croft, W. B. (2002) · Proceedings of the 25th annual international ACM SIGIR conference on Research and development in information retrieval · doi:10.1145/564376.564408 · Cited by 212
Improving the estimation of relevance models using large external corpora
Diaz, F., & Metzler, D. (2006) · Proceedings of the 29th annual international ACM SIGIR conference on Research and development in information retrieval · doi:10.1145/1148170.1148200 · Cited by 199
Anandarajan, M., Hill, C., & Nolan, T. (2019) · Advances in Analytics and Data Science · doi:10.1007/978-3-319-95663-3_8 · Cited by 173
A Comparative Study of Utilizing Topic Models for Information Retrieval
Yi, X., & Allan, J. (2009) · Lecture notes in computer science · doi:10.1007/978-3-642-00958-7_6 · Cited by 166
Evaluating Neural Word Representations in Tensor-Based Compositional Settings
Milajevs, D., Kartsaklis, D., Sadrzadeh, M., & Purver, M. (2014) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1079 · Cited by 125
Mining multilingual topics from wikipedia
Ni, X., Sun, J.-T., Hu, J., & Chen, Z. (2009) · International World Wide Web Conference (WWW) · doi:10.1145/1526709.1526904 · Cited by 114
Vulić, I., Smet, W. D., Tang, J., & Moens, M. (2014) · Information Processing & Management · doi:10.1016/j.ipm.2014.08.003 · Cited by 103
Self-citationExtracting Multilingual Topics from Unaligned Comparable Corpora
Jagarlamudi, J., & Daumé, H. (2010) · Lecture notes in computer science · doi:10.1007/978-3-642-12275-0_39 · Cited by 82
Vulić, I., Smet, W. D., & Moens, M. (2012) · Information Retrieval · doi:10.1007/s10791-012-9200-5 · Cited by 60
Self-citationVulić, I., & Moens, M. (2013) · Lecture notes in computer science · doi:10.1007/978-3-642-36973-5_9 · Cited by 11
Self-citation
Cited by
A Survey of Cross-lingual Word Embedding Models
Ruder, S., Vulić, I., & Søgaard, A. (2018) · Apollo (University of Cambridge) · doi:10.17863/cam.30462 · Cited by 510
Self-citationExplicit Semantic Ranking for Academic Search via Knowledge Graph Embedding
Xiong, C., Power, R., & Callan, J. (2017) · Proceedings of the 26th International Conference on World Wide Web · doi:10.1145/3038912.3052558 · Cited by 301
A Survey of Cross-lingual Word Embedding Models
Ruder, S., Vulić, I., & Søgaard, A. (2019) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.11640 · Cited by 266
Self-citationContextualizing Citations for Scientific Summarization using Word Embeddings and Domain Knowledge
Cohan, A., & Goharian, N. (2017) · Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval · doi:10.1145/3077136.3080740 · Cited by 23
An Efficient Methodology for Identifying the Similarity Between Languages with Levenshtein Distance
Janardhana Rao, P., Nageswara Rao, K., Gokuruboyina, S., & Neeraja, K. N. (2024) · Lecture Notes in Electrical Engineering · doi:10.1007/978-981-99-7137-4_15 · Cited by 13
Metadata
- Authors
- Vulić, I., & Moens, M.-F.
- Venue
- Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval
- Publisher
- ACM
- Published
- 2015-08-09
- Pages
- 363-372
- OpenAlex
- W2068297964
- Cited by
- 294
- References
- 47
- FWCI
- 31.87
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.