CoChat
Sign Up
Cited by 253Natural Language Processing TechniquesTopic ModelingSentiment Analysis and Opinion Mining

Embeddings for Word Sense Disambiguation: An Evaluation Study

Open access
Iacobacci, Ignacio, Pilehvar, Mohammad Taher, Navigli, Roberto·2016·Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
DOI 10.18653/v1/p16-1085
Open this paper in CoChat — free accountOpen PDF

Abstract

Recent years have seen a dramatic growth in the popularity of word embeddings mainly owing to their ability to capture semantic information from massive amounts of textual content.As a result, many tasks in Natural Language Processing have tried to take advantage of the potential of these distributional models.In this work, we study how word embeddings can be used in Word Sense Disambiguation, one of the oldest tasks in Natural Language Processing and Artificial Intelligence.We propose different methods through which word embeddings can be leveraged in a state-of-the-art supervised WSD system architecture, and perform a deep analysis of how different parameters affect performance.We show how a WSD system that makes use of word embeddings alone, if designed properly, can provide significant performance improvement over a state-ofthe-art WSD system that incorporates several standard WSD features.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.18653/v1/p16-1085

Cites

2 of 2 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • The biLM WSD performance is competitive with a state-of-the-art WSD-specific supervised model using hand crafted features and a task specific biLSTM trained with auxiliary coarse-grained semantic labels and POS tags.

    “This is competitive with a state-of-the-art WSD-specific supervised model using hand crafted features (Iacobacci et al., 2016) and a task specific biLSTM that is also trained with auxiliary coarse-grained semantic labels and POS tags (Raganato et al., 2017a).”

    From Deep Contextualized Word Representations (2018) · cites this paper as Iacobacci et al., 2016

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 52

  1. Eneko Agirre and Aitor Soroa. 2009. Personalizing PageRank For Word Sense Disambiguation. In Pro- ceedings of the 12th Conference of the EACL, pages 33-41, Athens, Greece.
  2. Eneko Agirre, David Martínez, Oier López de Lacalle, and Aitor Soroa. 2006. Two graph-based algorithms for state-of-the-art wsd. In Proceedings of the 2006 EMNLP, pages 585-593, Sydney, Australia.
  3. Eneko Agirre, Aitor Soroa, and Mark Stevenson. 2010. Graph-based word sense disambiguation of biomed- ical documents. Bioinformatics, 26(22):2889-2896.
  4. Eneko Agirre, Oier López de Lacalle, and Aitor Soroa. 2014. Random Walks for Knowledge-based Word Sense Disambiguation. Comp. Ling., 40(1):57-84.
  5. Collin F. Baker, Charles J. Fillmore, and John B. Lowe. 1998. The Berkeley FrameNet Project. In Proceed- ings of the 36th ACL, pages 86-90, Montreal, Que- bec, Canada.
  6. Marco Baroni, Georgiana Dinu, and Germán Kruszewski. 2014. Don't count, predict! A systematic comparison of context-counting vs. context-predicting semantic vectors. In Proceed- ings of the 52nd ACL, volume 1, pages 238-247, Baltimore, Maryland.
  7. Yoshua Bengio, Réjean Ducharme, Pascal Vincent, and Christian Janvin. 2003. A Neural Probabilistic Lan- guage Model. The Journal of Machine Learning Re- search, 3:1137-1155.
  8. David M. Blei, Andrew Y. Ng, and Michael I. Jordan. 2003. Latent Dirichlet Allocation. Journal of Ma- chine Learning Research, 3:993-1022.
  9. Samuel Brody and Mirella Lapata. 2009. Bayesian word sense induction. In Proceedings of the 12th Conference of the EACL, pages 103-111, Athens, Greece.
  10. Jun Fu Cai, Wee Sun Lee, and Yee Whye Teh. 2007. NUS-ML:Improving Word Sense Disam- biguation Using Topic Features. In Proceedings of the SemEval-2007, pages 249-252, Prague, Czech Republic.
  11. José Camacho-Collados, Mohammad Taher Pilehvar, and Roberto Navigli. 2015a. A Unified Multilin- gual Semantic Representation of Concepts. In Pro- ceedings of the 53rd ACL, volume 1, pages 741-751, Beijing, China.
  12. José Camacho-Collados, Mohammad Taher Pilehvar, and Roberto Navigli. 2015b. NASARI: a novel approach to a semantically-aware representation of items. In Proceedings of the 2015 NAACL, pages 567-577, Denver, Colorado.
  13. Xinxiong Chen, Zhiyuan Liu, and Maosong Sun. 2014. A unified model for word sense representa- tion and disambiguation. In Proceedings of the 2014 EMNLP, pages 1025-1035, Doha, Qatar.
  14. Ronan Collobert and Jason Weston. 2008. A uni- fied architecture for natural language processing: Deep neural networks with multitask learning. In Proceedings of the 25th ICML, pages 160-167, Helsinki, Finland.
  15. Antonio Di Marco and Roberto Navigli. 2013. Clus- tering and diversifying web search results with graph-based word sense induction. Comp. Ling., 39(3):709-754.
  16. Philip Edmonds and Scott Cotton. 2001. Senseval-2: Overview. In The Proceedings of the 2nd Interna- tional Workshop on Evaluating Word Sense Disam- biguation Systems, pages 1-5, Toulouse, France.
  17. Manaal Faruqui, Jesse Dodge, Sujay Kumar Jauhar, Chris Dyer, Eduard Hovy, and Noah A. Smith. 2015. Retrofitting Word Vectors to Semantic Lex- icons. In Proceedings of the 2015 NAACL, pages 1606-1615, Denver, Colorado.
  18. Juri Ganitkevitch, Benjamin Van Durme, and Chris Callison-Burch. 2013. PPDB: The Paraphrase Database. In Proceedings 2013 NAACL, pages 758- 764, Atlanta, Georgia.
  19. Weiwei Guo and Mona Diab. 2010. Combining Or- thogonal Monolingual and Multilingual Sources of Evidence for All Words WSD. In Proceedings of the 48th ACL, pages 1542-1551, Uppsala, Sweden.
  20. Taher H. Haveliwala. 2002. Topic-sensitive PageR- ank. In Proceedings of the 11th international con- ference on World Wide Web, pages 517-526, Hon- olulu, Hawai.
  21. Eric H. Huang, Richard Socher, Christopher D. Man- ning, and Andrew Y. Ng. 2012. Improving Word Representations Via Global Context And Multiple Word Prototypes. In Proceedings of 50th ACL, vol- ume 1, pages 873-882, Jeju Island, South Korea.
  22. Ignacio Iacobacci, Mohammad Taher Pilehvar, and Roberto Navigli. 2015. SensEmbed: Learning Sense Embeddings for Word and Relational Simi- larity. In Proceedings of the 53rd ACL, volume 1, pages 95-105, Beijing, China.
  23. Thomas K. Landauer and Susan T. Dutnais. 1997. A Solution to Platos Problem: The Latent Semantic Analysis Theory of Acquisition, Induction, and Rep- resentation of Knowledge. Psychological Review, 104(2):211-240.
  24. Yoong Keok Lee and Hwee Tou Ng. 2002. An Empir- ical Evaluation of Knowledge Sources and Learn- ing Algorithms for Word Sense Disambiguation. In Proceedings of the 2002 EMNLP, volume 10, pages 41-48, Philadelphia, Pennsylvania.
  25. Omer Levy, Yoav Goldberg, and Ido Dagan. 2015. Im- proving distributional similarity with lessons learned from word embeddings. TACL, 3:211-225.
  26. Suresh Manandhar, Ioannis P. Klapaftis, Dmitriy Dli- gach, and Sameer S. Pradhan. 2010. Semeval-2010 task 14: Word sense induction & disambiguation. In Proceedings of SemEval-2010, pages 63-68, Up- psala, Sweden.
  27. Grégoire Mesnil, Xiaodong He, Li Deng, and Yoshua Bengio. 2013. Investigation of Recurrent-neural- network Architectures and Learning Methods for Spoken Language Understanding. In INTER- SPEECH, pages 3771-3775, Lyon, France.
  28. Rada Mihalcea and Ehsanul Faruque. 2004. Sense- learner: Minimally Supervised Word Sense Disam- biguation for All Words in Open Text. In Proceed- ings of ACL/SIGLEX Senseval-3, volume 3, pages 155-158, Barcelona, Spain.
  29. Rada Mihalcea, Timothy Chklovski, and Adam Kilgar- riff. 2004. The Senseval-3 English Lexical Sample Task. In Proceedings of ACL/SIGLEX Senseval-3, pages 25-28, Barcelona, Spain.
  30. Tomas Mikolov, Quoc V. Le, and Ilya Sutskever. 2013. Exploiting similarities among languages for ma- chine translation. arXiv preprint arXiv:1309.4168. · arXiv
  31. George A. Miller and Walter G Charles. 1991. Con- textual correlates of semantic similarity. Language and Cognitive Processes, 6(1):1-28.
  32. George A. Miller, Martin Chodorow, Shari Landes, Claudia Leacock, and Robert G Thomas. 1994. Us- ing a Semantic Concordance for Sense Identifica- tion. In Proceedings of the Workshop on HLT, pages 240-243, Plainsboro, New Jersey.
  33. Tristan Miller, Chris Biemann, Torsten Zesch, and Iryna Gurevych. 2012. Using Distributional Sim- ilarity for Lexical Expansion in Knowledge-based Word Sense Disambiguation. In COLING, pages 1781-1796, Mumbai, India.
  34. Saif Mohammad and Graeme Hirst. 2006. Determin- ing Word Sense Dominance Using a Thesaurus. In Proceedings of the 11th Conference of EACL, pages 121-128, Trento, Italy.
  35. Andrea Moro, Alessandro Raganato, and Roberto Nav- igli. 2014. Entity Linking meets Word Sense Dis- ambiguation: a Unified Approach. Transactions of the ACL, 2:231-244.
  36. Roberto Navigli. 2009. Word sense disambiguation: a survey. ACM COMPUTING SURVEYS, 41(2):1-69.
  37. Hwee Tou Ng and Hian Beng Lee. 1996. Integrating Multiple Knowledge Sources to Disambiguate Word Sense: An Exemplar-based Approach. In Proceed- ings of the 34th Meeting on ACL, pages 40-47, Santa Cruz, California.
  38. Siddharth Patwardhan and Ted Pedersen. 2006. Us- ing WordNet-based Context Vectors to Estimate the Semantic Relatedness of Concepts. In Proceedings of the EACL 2006 Workshop Making Sense of Sense, volume 1501, pages 1-8, Trento, Italy.
  39. Jeffrey Pennington, Richard Socher, and Christo- pher D. Manning. 2014. GloVe: Global Vectors for Word Representation. In Proceedings of the 2014 EMNLP, pages 1532-1543, Doha, Qatar.
  40. Mohammad Taher Pilehvar and Roberto Navigli. 2014. A Large-scale Pseudoword-based Evaluation Framework for State-of-the-Art Word Sense Disam- biguation. Computational Linguistics, 40(4):837- 881.
  41. Simone Paolo Ponzetto and Roberto Navigli. 2010. Knowledge-rich word sense disambiguation rival- ing supervised systems. In Proceedings of the 48th ACL, pages 1522-1531, Uppsala, Sweden.
  42. Sameer S. Pradhan, Edward Loper, Dmitriy Dligach, and Martha Palmer. 2007. SemEval-2007 Task 17: English Lexical Sample, SRL and All Words. In Proceedings of the SemEval-2007, pages 87-92, Prague, Czech Republic.
  43. Joseph Reisinger and Raymond J. Mooney. 2010. Multi-Prototype Vector-Space Models of Word Meaning. In Proceedings of the 2010 Annual Con- ference of the NAACL, pages 109-117, Los Angeles, California.
  44. Sascha Rothe and Hinrich Schütze. 2015. Autoex- tend: Extending word embeddings to embeddings for synsets and lexemes. In Proceedings of the 53rd ACL, volume 1, pages 1793-1803, Beijing, China.
  45. Hui Shen, Razvan Bunescu, and Rada Mihalcea. 2013. Coarse to Fine Grained Sense Disambiguation in Wikipedia. In *SEM 2013: The Secound Joint Con- ference on Lexical and Computational Semantics, pages 22-31, Atlanta, Georgia.
  46. Benjamin Snyder and Martha Palmer. 2004. The Senseval-3 English All-Words Task. In Proceed- ings of ACL/SIGLEX Senseval-3, pages 41-43, Barcelona, Spain.
  47. Richard Socher, Alex Perelygin, Jean Y Wu, Jason Chuang, Christopher D Manning, Andrew Y Ng, and Christopher Potts. 2013. Recursive Deep Mod- els for Semantic Compositionality Over a Sentiment Treebank. In Proceedings of the 2013 EMNLP, pages 1631-1642, Seattle, USA.
  48. Jiri Stetina, Sadao Kurohashi, and Makoto Nagao. 1998. General Word Sense Disambiguation Method Based on a Full Sentential Context. In Usage of WordNet in Natural Language Processing, Proceed- ings of COLING-ACL Workshop, Montreal, Quebec, Canada.
  49. Kaveh Taghipour and Hwee Tou Ng. 2015. Semi- Supervised Word Sense Disambiguation Using Word Embeddings in General and Specific Domains. In Proceedings of the 2015 Annual Conference of the NAACL, pages 314-323, Denver, Colorado.
  50. Tim Van de Cruys and Marianna Apidianaki. 2011. Latent Semantic Word Sense Induction and Disam- biguation. In Proceedings of the 49th ACL, vol- ume 1, pages 1476-1485, Portland, Oregon.
  51. Mo Yu and Mark Dredze. 2014. Improving Lexical Embeddings with Semantic Knowledge. In Pro- ceedings of the 52nd ACL, volume 2, pages 545- 550, Baltimore, Maryland.
  52. Zhi Zhong and Hwee Tou Ng. 2010. It Makes Sense: A Wide-coverage Word Sense Disambiguation Sys- tem for Free Text. In Proceedings of the 48th ACL, pages 78-83, Uppsala, Sweden.

Cited by

Sorted by citations

Authors
Iacobacci, I., Pilehvar, M. T., & Navigli, R.
Affiliations
  • Sapienza University of Rome (IT)
  • University of Cambridge (GB)
ORCID
Venue
Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Publisher
Association for Computational Linguistics
Published
2016
Pages
897-907
ISSN
0736-587X (ISSN-L)
DOI
10.18653/v1/p16-1085
OpenAlex
W2518202280
Cited by
253
References
2
Citations by year
2026: 1 · 2025: 5 · 2024: 5 · 2023: 15 · 2022: 14 · 2021: 40 · 2020: 49 · 2019: 47 · 2018: 41 · 2017: 30 · 2016: 5 · 2013: 1
FWCI
22.27
Type
conference paper
Language
en
Open access
Licence
cc-by
Locations
  • Proceedings of the conference - Association for Computational Linguistics. Meeting (publishedVersion, cc-by)
  • IRIS Research product catalog (Sapienza University of Rome) (submittedVersion, other-oa)
Funders
FP7 Ideas: European Research Council (259234)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Physical Sciences › Computer Science › Artificial Intelligence › Sentiment Analysis and Opinion Mining
Keywords
word sense disambiguation, word embeddings, distributional semantics, parametric study

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review