The DOI matches its Crossref record; 84 of the 84 references it cites were found in a registry or an index.
Multiword Expression Processing: A Survey
Open accessAbstract
Multiword expressions (MWEs) are a class of linguistic forms spanning conventional word boundaries that are both idiosyncratic and pervasive across different languages. The structure of linguistic processing that depends on the clear distinction between words and phrases has to be re-thought to accommodate MWEs. The issue of MWE handling is crucial for NLP applications, where it raises a number of challenges. The emergence of solutions in the absence of guiding principles motivates this survey, whose aim is not only to provide a focused review of MWE processing, but also to clarify the nature of interactions between MWE processing and downstream applications. We propose a conceptual framework within which challenges and research contributions can be positioned. It offers a shared understanding of what is meant by “MWE processing,” distinguishing the subtasks of MWE discovery and identification. It also elucidates the interactions between MWE processing and two use cases: Parsing and machine translation. Many of the approaches in the literature can be differentiated according to how MWE processing is timed with respect to underlying use cases. We discuss how such orchestration choices affect the scope of MWE-aware systems. For each of the two MWE processing subtasks and for each of the two use cases, we conclude on open issues and research perspectives.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation
Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1179 · Cited by 25,545
BLEU: a method for automatic evaluation of machine translation
Papineni, K., Roukos, S., Ward, T., & Zhu, W.-J. (2001) · Proceedings of the 40th Annual Meeting on Association for Computational Linguistics - ACL '02 · doi:10.3115/1073083.1073135 · Cited by 21,344
Foundations of statistical natural language processing
Manning, C. D., & Schütze, H. (1999) · Cited by 9,816
Building a Large Annotated Corpus of English: The Penn Treebank
Marcus, M. (1993) · doi:10.21236/ada273556 · Cited by 7,405
Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
Wu, Y., Schuster, M., Chen, Z., Le, Q. V., Norouzi, M., Macherey, W., Krikun, M., Cao, Y., Gao, Q., Macherey, K., et al. (2016) · arXiv · doi:10.48550/arxiv.1609.08144 · Cited by 5,319
Head-driven phrase structure grammar
Sag, I. A., & Pollard, C. (1994) · Medical Entomology and Zoology · Cited by 3,889
Word association norms, mutual information, and lexicography
Church, K., & Hanks, P. (1990) · doi:10.5555/89086.89095 · Cited by 3,590
Statistical phrase-based translation
Koehn, P., Och, F. J., & Marcu, D. (2003) · Proceedings of the 2003 Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology - NAACL '03 · doi:10.3115/1073445.1073462 · Cited by 3,197
Regularity and Idiomaticity in Grammatical Constructions: The Case of Let Alone
Fillmore, C. J., Kay, P., & O'Connor, M. C. (1988) · Language · doi:10.2307/414531 · Cited by 2,915
Formulaic Language and the Lexicon
Wray, A. (2002) · doi:10.1017/cbo9780511519772 · Cited by 2,873
Accurate methods for the statistics of surprise and coincidence
Dunning, T. E. (1993) · doi:10.5555/972450.972454 · Cited by 2,647
A Fast and Accurate Dependency Parser using Neural Networks
Chen, D., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1082 · Cited by 1,795
Navigli, R., & Ponzetto, S. P. (2012) · Artificial Intelligence · doi:10.1016/j.artint.2012.07.001 · Cited by 1,465
Text Chunking Using Transformation-Based Learning
Ramshaw, L. A., & Marcus, M. P. (1999) · Text, Speech and Language Technology · doi:10.1007/978-94-017-2390-9_10 · Cited by 1,460
Statistical machine translation
(2010) · Choice Reviews Online · doi:10.5860/choice.47-6293 · Cited by 1,309
Recurrent Continuous Translation Models
Kalchbrenner, N., & Blunsom, P. (2013) · Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d13-1176 · Cited by 1,232
Hierarchical Phrase-Based Translation
Chiang, D. (2007) · Computational Linguistics · doi:10.1162/coli.2007.33.2.201 · Cited by 1,144
Multiword Expressions: A Pain in the Neck for NLP
Sag, I. A., Baldwin, T., Bond, F., Copestake, A., & Flickinger, D. (2002) · Lecture Notes in Computer Science · doi:10.1007/3-540-45715-1_1 · Cited by 1,016
Improved statistical alignment models
Och, F. J., & Ney, H. (2000) · Proceedings of the 38th Annual Meeting on Association for Computational Linguistics - ACL '00 · doi:10.3115/1075218.1075274 · Cited by 1,001
CoNLL-X shared task on multilingual dependency parsing
Buchholz, S., & Marsi, E. (2006) · Proceedings of the Tenth Conference on Computational Natural Language Learning - CoNLL-X '06 · doi:10.3115/1596276.1596305 · Cited by 963
Coarse-to-fine n-best parsing and MaxEnt discriminative reranking
Charniak, E., & Johnson, M. (2005) · Proceedings of the 43rd Annual Meeting on Association for Computational Linguistics - ACL '05 · doi:10.3115/1219840.1219862 · Cited by 872
Non-projective dependency parsing using spanning tree algorithms
McDonald, R., Pereira, F., Ribarov, K., & Hajič, J. (2005) · Proceedings of the conference on Human Language Technology and Empirical Methods in Natural Language Processing - HLT '05 · doi:10.3115/1220575.1220641 · Cited by 847
Technical terminology: some linguistic properties and an algorithm for identification in text
Justeson, J. S., & Katz, S. M. (1995) · Natural Language Engineering · doi:10.1017/s1351324900000048 · Cited by 820
Retrieving collocations from text: Xtract
Smadja, F. (1993) · Cited by 794
Joshi, A. K., Levy, L. S., & Takahashi, M. (1975) · Journal of Computer and System Sciences · doi:10.1016/s0022-0000(75)80019-5 · Cited by 770
The CoNLL 2007 Shared Task on Dependency Parsing
Nivre, J., Hall, J., Kübler, S., McDonald, R., Nilsson, J., Riedel, S., & Yüret, D. (2007) · Publication Server of Goethe University Frankfurt am Main (Goethe University Frankfurt) · Cited by 663
Transition-Based Dependency Parsing with Stack Long Short-Term Memory
Dyer, C., Ballesteros, M., Ling, W., Matthews, A., & Smith, N. A. (2015) · Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.3115/v1/p15-1033 · Cited by 553
Koehn, P., & Hoang, H. (2007) · Edinburgh Research Explorer (University of Edinburgh) · Cited by 509
Building a Treebank for French
Abeillé, A., Clément, L., & Toussenel, F. (2003) · Text, Speech and Language Technology · doi:10.1007/978-94-010-0201-1_10 · Cited by 433
Procedure for quantitatively comparing the syntactic coverage of English grammars
Klavans, J., Liberman, M., Marcus, M., Roukos, S., Santorini, B., Strzalkowski, T., Abney, S., Flickenger, S., Gdaniec, C., Grishman, C., et al. (1991) · Proceedings of the workshop on Speech and Natural Language - HLT '91 · doi:10.3115/112405.112467 · Cited by 432
Using the Web to Obtain Frequencies for Unseen Bigrams
Keller, F., & Lapata, M. (2003) · Computational Linguistics · doi:10.1162/089120103322711604 · Cited by 350
Lexical association measures and collocation extraction
Pecina, P. (2010) · Language Resources and Evaluation · doi:10.1007/s10579-009-9101-4 · Cited by 316
Apertium: a free/open-source platform for rule-based machine translation
Forcada, M. L., Ginestí-Rosell, M., Nordfalk, J., O’Regan, J., Ortiz-Rojas, S., Pérez-Ortiz, J. A., Sánchez-Martínez, F., Ramírez-Sánchez, G., & Tyers, F. M. (2011) · Machine Translation · doi:10.1007/s10590-011-9090-0 · Cited by 302
Riezler, S., King, T. H., Kaplan, R. M., Crouch, R., Maxwell, J. T., & Johnson, M. (2001) · Proceedings of the 40th Annual Meeting on Association for Computational Linguistics - ACL '02 · doi:10.3115/1073083.1073129 · Cited by 279
The Design, Implementation, and Use of the Ngram Statistics Package
Banerjee, S., & Pedersen, T. (2003) · Lecture Notes in Computer Science · doi:10.1007/3-540-36456-0_38 · Cited by 230
Robustness beyond shallowness: incremental deep parsing
AÏT-MOKHTAR, S., CHANOD, J.-P., & ROUX, C. (2002) · Natural Language Engineering · doi:10.1017/s1351324902002887 · Cited by 227
Review Article: Example-based Machine Translation
SOMERS, H. L. (1999) · Machine Translation · doi:10.1023/a:1008109312730 · Cited by 220
Seddah, D., Tsarfaty, R., Kübler, S., Candito, M., Choi, J. D., Farkas, R., Foster, J., Goenaga, I., Galletebeitia, K. G., Goldberg, Y., et al. (2013) · doi:10.18653/v1/w13-4917 · Cited by 216
A Dependency Parser for Tweets
Kong, L., Schneider, N., Swayamdipta, S., Bhatia, A., Dyer, C., & Smith, N. A. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1108 · Cited by 207
Memory-Based Dependency Parsing
Nivre, J., Hall, J., & Nilsson, J. (2004) · Cited by 204
A Clustering Approach for Nearly Unsupervised Recognition of Nonliteral Language
Birke, J., & Sarkar, A. (2005) · Cited by 200
Unsupervised Type and Token Identification of Idiomatic Expressions
Fazly, A., Cook, P., & Stevenson, S. (2009) · Computational Linguistics · doi:10.1162/coli.08-010-r1-07-048 · Cited by 200
Joint parsing and named entity recognition
Finkel, J. R., & Manning, C. D. (2009) · Proceedings of Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the Association for Computational Linguistics on - NAACL '09 · doi:10.3115/1620754.1620802 · Cited by 185
Fishing for ExactnessPreprint
Ted Pedersen, A. (1996) · arXiv (Cornell University) · doi:10.48550/arxiv.cmp-lg/9608010 · Cited by 141
The Formal Architecture of Lexical-Functional Grammar.
Kaplan, R. M. (1989) · Cited by 138
Silva, J., Dias, G. H., Guilloré, S., & Lopes, J. G. P. (1999) · Lecture notes in computer science · doi:10.1007/3-540-48159-1_9 · Cited by 127
Durrett, G., & Klein, D. (2015) · doi:10.3115/v1/p15-1030 · Cited by 119
Turbo Parsers: Dependency Parsing by Approximate Variational Inference
Martins, A. F. T., Smith, N. A., Xing, E. P., Aguiar, P. M. Q., & Figueiredo, M. A. T. (2018) · KiltHub Repository · doi:10.1184/r1/6476417 · Cited by 112
A Word Embedding Approach to Predicting the Compositionality of Multiword Expressions
Salehi, B., Cook, P. F., & Baldwin, T. (2015) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.3115/v1/n15-1099 · Cited by 107
Unsupervised recognition of literal and non-literal use of idiomatic expressions
Sporleder, C., & Li, L. (2009) · doi:10.3115/1609067.1609151 · Cited by 103
Probabilistic tree-adjoining grammar as a framework for statistical natural language processing
Resnik, P. (1992) · International Conference on Computational Linguistics · doi:10.3115/992133.992135 · Cited by 99
Discriminative Lexical Semantic Segmentation with Gaps: Running the MWE Gamut
Schneider, N., Danchik, E., Dyer, C., & Smith, N. A. (2014) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00176 · Cited by 99
Automatic Discovery of Non-Compositional Compounds in Parallel DataPreprint
Melamed, I. D. (1997) · arXiv (Cornell University) · doi:10.48550/arxiv.cmp-lg/9706027 · Cited by 95
Task-based Evaluation of Multiword Expressions: a Pilot Study in Statistical Machine Translation
Carpuat, M. J., & Diab, M. (2010) · Cited by 91
Urieli, A. (2013) · theses.fr (ABES) · doi:10.70675/c2c353efz44bfz485az8dedz1cd0b46094b7 · Cited by 85
Green, S., de Marneffe, M.-C., Bauer, J., Manning, C. D., & Processing, P. O. T. 2. C. O. E. M. I. N. L. (2011) · Cited by 80
Parsing Models for Identifying Multiword Expressions
Green, S., de Marneffe, M.-C., & Manning, C. D. (2012) · Computational Linguistics · doi:10.1162/coli_a_00139 · Cited by 74
Identifying bilingual Multi-Word Expressions for Statistical Machine Translation
Bouamor, D., Semmar, N., & Zweigenbaum, P. (2012) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/3jrrxjnqy96m · Cited by 67
Exploiting the WWW as a corpus to resolve PP attachment ambiguities
Volk, M. (2001) · Zurich Open Repository and Archive (University of Zurich) · doi:10.5167/uzh-20269 · Cited by 66
SemEval-2016 Task 10: Detecting Minimal Semantic Units and their Meanings (DiMSUM)
Schneider, N., Hovy, D., Johannsen, A., & Carpuat, M. J. (2016) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/s16-1084 · Cited by 59
The use of finite automata in the lexical representation of natural language
Gross, M. (1989) · Lecture notes in computer science · doi:10.1007/3-540-51465-1_3 · Cited by 58
Fritzinger, F., & Fraser, A. (2010) · Workshop on Statistical Machine Translation · Cited by 54
Identification of Multiword Expressions by Combining Multiple Linguistic Information Sources
Tsvetkov, Y., & Wintner, S. (2013) · Computational Linguistics · doi:10.1162/coli_a_00177 · Cited by 49
Multiword Expressions and Named Entities in the Wiki50 Corpus
Vincze, V., István Nagy, T., & Berend, G. (2011) · Cited by 48
Learning about the meaning of verb–particle constructions from corpora
Bannard, C. (2005) · Computer Speech & Language · doi:10.1016/j.csl.2005.02.003 · Cited by 41
Web-Scale Features for Full-Scale Parsing
Bansal, M., & Klein, D. (2011) · Cited by 41
Detecting Multi-Word Expressions Improves Word Sense Disambiguation
Finlayson, M. A., & Kulkarni, N. (2011) · Cited by 41
Learning Semantic Composition to Detect Non-compositionality of Multiword Expressions
Yazdani, M., Farahmand, M., & Henderson, J. (2015) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d15-1201 · Cited by 40
Distinguishing Degrees of Compositionality in Compound Splitting for Statistical Machine Translation
Weller, M., Cap, F., Müller, S., Walde, S. S. I., & Fraser, A. (2014) · doi:10.3115/v1/w14-5709 · Cited by 36
Discriminative Strategies to Integrate Multiword Expression Recognition and Parsing
Constant, M., Sigogne, A., & Watrin, P. (2012) · HAL · Cited by 35
Self-citationGeneration of Compound Words in Statistical Machine Translation into Compounding Languages
Stymne, S., Cancedda, N., & Ahrenberg, L. (2013) · Computational Linguistics · doi:10.1162/coli_a_00162 · Cited by 27
Inconsistency Detection in Semantic Annotation
Hollenstein, N., Schneider, N., & Webber, B. (2016) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/2tbvmsr9fnrw · Cited by 27
Manawi: Using Multi-Word Expressions and Named Entities to Improve Machine Translation
Tan, L., & Pal, S. (2014) · Conference on Machine Translation (WMT) · doi:10.3115/v1/w14-3323 · Cited by 24
Prague Dependency Treebank 2.5 -- a Revisited Version of PDT 2.0
BejÄek, E., Panevová, J., Popelka, J., et al. (2012) · International Conference on Computational Linguistics · Cited by 24
Introduction to the special issue on multiword expressions
Ramisch, C., Villavicencio, A., & Kordoni, V. (2013) · ACM Transactions on Speech and Language Processing · doi:10.1145/2483691.2483692 · Cited by 22
Self-citationThe Lexical Analysis of Natural Languages
Silberztein, M. D. (1997) · The MIT Press eBooks · doi:10.7551/mitpress/3007.003.0008 · Cited by 22
Parsing with automatically acquired, wide-coverage, robust, probabilistic LFG approximations
Cahill, A. (2004) · Dublin City University Open Access Institutional Repository (Dublin City University) · Cited by 19
Unsupervised Learning of Multiword Expressions
Korkontzelos, I. (2010) · White Rose eTheses Online (University of Leeds, The University of Sheffield, University of York) · Cited by 19
Multi-word expression identification using sentence surface features
Boukobza, R., & Rappoport, A. (2009) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/1699571.1699573 · Cited by 19
JRC-Names: Multilingual entity name variants and titles as Linked Data
Ehrmann, M., Jacquet, G., & Steinberger, R. (2016) · Semantic Web · doi:10.3233/sw-160228 · Cited by 19
How hard is it to automatically translate phrasal verbs from English to French?
Ramisch, C., Besacier, L., & Kobzar, A. (2013) · Cited by 13
Self-citationTaking on new challenges in multi-word unit processing for machine translation
Monti, J., Barreiro, A. M., Elia, A., Marano, F., & Napoli, A. (2011) · RECERCAT (Consorci de Serveis Universitaris de Catalunya) · Cited by 11
Self-citationNever-Ending Multiword Expressions Learning
Rondon, A., Caseli, H. D. M., & Ramisch, C. (2015) · Workshop on Multiword Expressions (MWE) · doi:10.3115/v1/w15-0908 · Cited by 9
Self-citationZhu, Y. (2017) · ResearchWorks at the University of Washington (University of Washington) · Cited by 2
Metadata
- Authors
- Constant, M., Eryiğit, G., Monti, J., van der Plas, L., Ramisch, C., Rosner, M., & Todirascu, A.
- Venue
- Computational Linguistics
- Publisher
- MIT Press
- Published
- 2017-12
- Volume
- 43
- Issue
- 4
- Pages
- 837-892
- ISSN
- 0891-2017 (ISSN-L)
- 1530-9312
- OpenAlex
- W2664496537
- Cited by
- 256
- References
- 84
- FWCI
- 17.55
- Type
- journal article
- Language
- en
- Open access
- diamond
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.