CoChat

Dynamic Entity Representations in Neural Language Models

Open access
Ji, Yangfeng, Tan, Chenhao, Martschat, Sebastian, Choi, Yejin, Smith, Noah A. · 2017 · Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing
Cited by 94

The DOI matches its Crossref record; 4 of the 4 references it cites were found in a registry or an index.

Abstract

Understanding a long document requires tracking how entities are introduced and evolve over time.We present a new type of language model, ENTITYNLM, that can explicitly model entities, dynamically update their representations, and contextually generate their mentions.Our model is generative and flexible; it can model an arbitrary number of entities in context while generating each entity mention at an arbitrary length.In addition, it can be used for several different tasks such as language modeling, coreference resolution, and entity prediction.Experimental results with all these tasks demonstrate that our model consistently outperforms strong baselines and prior work.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.18653/v1/d17-1195

Cites

4 of 4 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

  • Long Short-Term Memory

    Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318

  • Adam: A Method for Stochastic Optimization

    Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946

  • Recurrent neural network based language model

    Mikolov, T., Karafiát, M., Burget, L., Černocký, J., & Khudanpur, S. (2010) · Interspeech 2010 · doi:10.21437/interspeech.2010-343 · Cited by 5,390 · cited 1×

    “Our starting point for language modeling is a recurrent neural network (Mikolov et al., 2010), which defines where W h and b are parameters of the model (along with word embeddings x t ), LSTM is the widely used recurrent function known as “long short-term memory” (Hochreiter and Schmidhuber, 1997),…”
    Background
  • Recurrent Neural Network Grammars

    Dyer, C., Kuncoro, A., Ballesteros, M., & Smith, N. A. (2016) · Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/n16-1024 · Cited by 517 · cited 1×

    “Specifi-cally, with the samples from a proposal distribution Q ( R , E , L | X ) , the approximated marginal probability is defined as A similar idea of using importance sampling for language modeling evaluation has been used by Dyer et al. (2016).”
    MethodSelf-citation

Cited by

See all 94

Metadata

Authors
Ji, Y., Tan, C., Martschat, S., Choi, Y., & Smith, N. A.
Venue
Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing
Published
2017
Pages
1830-1839
OpenAlex
W2963925965
Cited by
94
References
4
FWCI
7.37
Type
conference paper
Language
en
Open access
hybrid

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.