Finding Function in Form: Compositional Character Models for Open Vocabulary Word Representation
Open accessThe DOI matches its Crossref record; 30 of the 30 references it cites were found in a registry or an index.
Abstract
Wang Ling, Chris Dyer, Alan W Black, Isabel Trancoso, Ramón Fermandez, Silvio Amir, Luís Marujo, Tiago Luís. Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. 2015.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Similar to WSD, the biLM representations are competitive with carefully tuned, task specific biLSTMs (Ling et al., 2015;Ma and Hovy, 2016).”
From Deep Contextualized Word Representations (2018) · cites this paper as Ling et al., 2015
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · cited 1×
“To model the complex form– function relationship, we turn to long short-term memories (LSTMs), which are designed to be able to capture complex non-linear and non-local dynamics in sequences (Hochreiter and Schmidhuber, 1997).”
MethodLearning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation
Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1179 · Cited by 25,545 · cited 1×
“Our proposed model is similar to models used to compute composed representations of sentences from words (Cho et al., 2014; Li et al., 2015).”
MethodDistributed Representations of Words and Phrases and their Compositionality
Mikolov, T., Sutskever, I., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1310.4546 · Cited by 24,533 · cited 1×
“Unsupervised training methods such as the Skip-n-gram model (Mikolov et al., 2013) can be used to pretrain the word representations on unannotated corpora.”
MethodBuilding a Large Annotated Corpus of English: The Penn Treebank
Marcus, M. (1993) · doi:10.21236/ada273556 · Cited by 7,405
Framewise phoneme classification with bidirectional LSTM and other neural network architectures
Graves, A., & Schmidhuber, J. (2005) · Neural Networks · doi:10.1016/j.neunet.2005.06.042 · Cited by 5,839
Recurrent neural network based language model
Mikolov, T., Karafiát, M., Burget, L., Černocký, J., & Khudanpur, S. (2010) · Interspeech 2010 · doi:10.21437/interspeech.2010-343 · Cited by 5,390
Natural Language Processing (almost) from Scratch
Collobert, R., Weston, J., Bottou, L., Karlen, M., Kavukcuoglu, K., & Kuksa, P. P. (2011) · arXiv (Cornell University) · doi:10.48550/arxiv.1103.0398 · Cited by 4,962
LSTM neural networks for language modeling
Sundermeyer, M., Schlüter, R., & Ney, H. (2012) · Interspeech 2012 · doi:10.21437/interspeech.2012-65 · Cited by 1,963 · cited 2×
“Our language model computes log p(wi | w1, . . . , wi−1) by composing representations of words w1, . . . , wi−1 using an recurrent LSTM model (Mikolov et al., 2010; Sundermeyer et al., 2012).”
MethodA Fast and Accurate Dependency Parser using Neural Networks
Chen, D., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1082 · Cited by 1,795
Recurrent Continuous Translation Models
Kalchbrenner, N., & Blunsom, P. (2013) · Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d13-1176 · Cited by 1,232
Cited by
Deep Contextualized Word Representations
Peters, M., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1202 · Cited by 11,983
Enriching Word Vectors with Subword Information
Bojanowski, P., Grave, E., Joulin, A., & Mikolov, T. (2017) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00051 · Cited by 9,645
Hierarchical Attention Networks for Document Classification
Yang, Z., Yang, D., Dyer, C., He, X., Smola, A., & Hovy, E. (2016) · Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/n16-1174 · Cited by 4,712
Self-citationNeural Architectures for Named Entity Recognition
Lample, G., Ballesteros, M., Subramanian, S., Kawakami, K., & Dyer, C. (2016) · Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/n16-1030 · Cited by 4,401
“They have been found useful for morphologically rich languages and to handle the outof-vocabulary problem for tasks like part-of-speech tagging and language modeling (Ling et al., 2015b) or dependency parsing (Ballesteros et al., 2015).”
BackgroundMethodInfluentialSelf-citationEnd-to-end Sequence Labeling via Bi-directional LSTM-CNNs-CRF
Ma, X., & Hovy, E. (2016) · Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p16-1101 · Cited by 2,567
“A similar model has also been applied to Spanish and Portuguese NER (dos Santos et al., 2015) Ling et al. (2015) and Yang et al. (2016) also used BSLTM to compose character embeddings to word’s representation, which is similar to Lample et al. (2016).”
MethodNamed Entity Recognition with Bidirectional LSTM-CNNs
Chiu, J. P. C., & Nichols, E. (2016) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00104 · Cited by 1,909
Character-Aware Neural Language Models
Kim, Y., Jernite, Y., Sontag, D., & Rush, A. M. (2015) · arXiv · doi:10.48550/arxiv.1508.06615 · Cited by 1,530
Character-Aware Neural Language Models
Kim, Y., Jernite, Y., Sontag, D., & Rush, A. (2016) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v30i1.10362 · Cited by 1,491
Neural Machine Translation of Rare Words with Subword Units
Sennrich, R., Haddow, B., & Birch, A. (2016) · Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p16-1162 · Cited by 974
Neural Machine Translation of Rare Words with Subword Units
Sennrich, R., Haddow, B., & Birch, A. (2015) · arXiv · doi:10.48550/arxiv.1508.07909 · Cited by 950
Sequence-Level Knowledge Distillation
Kim, Y., & Rush, A. M. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1139 · Cited by 836
Exploring the Limits of Language Modeling
Józefowicz, R., Vinyals, O., Schuster, M., Shazeer, N., & Wu, Y. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1602.02410 · Cited by 823
Tokenizing, POS Tagging, Lemmatizing and Parsing UD 2.0 with UDPipe
Straka, M., & Straková, J. (2017) · Proceedings of the CoNLL 2017 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies · doi:10.18653/v1/k17-3009 · Cited by 567
A Joint Many-Task Model: Growing a Neural Network for Multiple NLP Tasks
Hashimoto, K., xiong, C., Tsuruoka, Y., & Socher, R. (2017) · Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d17-1206 · Cited by 535
“The best result to date has been achieved by Ling et al. (2015), which uses character-based LSTMs. Incorporating the character-based encoders into our JMT model would be an interesting direction, but we have shown that the simple pre-trained character n -gram embeddings lead to the promising result.”
BackgroundA Survey of Cross-lingual Word Embedding Models
Ruder, S., Vulić, I., & Søgaard, A. (2018) · Apollo (University of Cambridge) · doi:10.17863/cam.30462 · Cited by 510
Globally Normalized Transition-Based Neural Networks
Andor, D., Alberti, C., Weiss, D., Severyn, A., Presta, A., Ganchev, K., Petrov, S., & Collins, M. (2016) · Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p16-1231 · Cited by 472
A Survey on Recent Advances in Named Entity Recognition from Deep Learning models
Yadav, V., & Bethard, S. J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.11470 · Cited by 464
Enriching Word Vectors with Subword Information
Bojanowski, P., Grave, É., Joulin, A., & Mikolov, T. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1607.04606 · Cited by 384
English Conversational Telephone Speech Recognition by Humans and Machines
Saon, G., Kurata, G., Sercu, T., Audhkhasi, K., Thomas, S., Dimitriadis, D., Cui, X., Ramabhadran, B., Picheny, M., Lim, L.-L., et al. (2017) · Interspeech 2017 · doi:10.21437/interspeech.2017-405 · Cited by 349
Learning Text Similarity with Siamese Recurrent Networks
Neculoiu, P., Versteegh, M., & Rotaru, M. (2016) · Proceedings of the 1st Workshop on Representation Learning for NLP · doi:10.18653/v1/w16-1617 · Cited by 341
“Word embedding techniques such as word2vec (Mikolov et al., 2013) and Glove (Pen-nington et al., 2014) have seen much use in such models, but some go beyond the word level and represent text as a sequence of characters (Kim et al., 2015; Ling et al., 2015).”
BackgroundRecent trends in deep learning based personality detection
Mehta, Y., Majumder, N., Gelbukh, A., & Cambria, E. (2020) · Artificial Intelligence Review · doi:10.1007/s10462-019-09770-z · Cited by 329
Neural Architectures for Nested NER through Linearization
Straková, J., Straka, M., & Hajic, J. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1527 · Cited by 309
Latent Predictor Networks for Code Generation
Ling, W., Blunsom, P., Grefenstette, E., Hermann, K. M., Kočiský, T., Wang, F., & Senior, A. (2016) · Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p16-1057 · Cited by 307
“Inspired by these works, LPNs provide a richer framework by employing attention models (Bahdanau et al., 2014), pointer networks (Vinyals et al., 2015) and character-based embeddings (Ling et al., 2015).”
MethodImproved Transition-based Parsing by Modeling Characters instead of Words with LSTMs
Ballesteros, M., Dyer, C., & Smith, N. A. (2015) · Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d15-1041 · Cited by 285
Self-citationNeural Network-based Question Answering over Knowledge Graphs on Word and Character Level
Lukovnikov, D., Fischer, A., Lehmann, J., & Auer, S. (2017) · Proceedings of the 26th International Conference on World Wide Web · doi:10.1145/3038912.3052675 · Cited by 279
A Survey of Cross-lingual Word Embedding Models
Ruder, S., Vulić, I., & Søgaard, A. (2019) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.11640 · Cited by 266
Commonsense Knowledge Base Completion
Li, X., Taheri, A., Tu, L., & Gimpel, K. (2016) · Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p16-1137 · Cited by 166
“So our models are limited to tuples in which terms consist of words in the word embedding vocabulary, though future work could consider character-based architectures for open-vocabulary modeling (Huang et al., 2013; Ling et al., 2015).”
MethodReplicability Analysis for Natural Language Processing: Testing Significance with Multiple Datasets
Dror, R., Baumer, G., Bogomolov, M., & Reichart, R. (2017) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00074 · Cited by 63
“The authors compare their M IMICK model with the model of Ling et al. (2015), denoted with CHAR → TAG .”
MethodLi, J., Zhou, Y., Jiang, X., Natarajan, K., Pakhomov, S. V., Liu, H., & Xu, H. (2021) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocab112 · Cited by 53
Generalizing Word Embeddings using Bag of Subwords
Zhao, J., Mudgal, S., & Liang, Y. (2018) · Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d18-1059 · Cited by 53
Metadata
- Authors
- Ling, W., Dyer, C., Black, A. W., Trancoso, I., Fermandez, R., Amir, S., Marujo, L., & Luis, T.
- Venue
- Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing
- Published
- 2015
- Pages
- 1520-1530
- OpenAlex
- W1899794420
- Cited by
- 541
- References
- 30
- FWCI
- 50.76
- Type
- conference paper
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.