CoChat
Sign Up
PreprintCited by 26

Sequence Level Training with Recurrent Neural Networks

Ranzato, Marc'Aurelio, Chopra, Sumit, Auli, Michael, Zaremba, Wojciech·2015·arXiv
DOI 10.48550/arxiv.1511.06732arXiv 1511.06732
Open this paper in CoChat — free account

Abstract

Many natural language processing applications use language models to generate text. These models are typically trained to predict the next word in a sequence, given the previous words and some context such as an image. However, at test time the model is expected to generate the entire sequence from scratch. This discrepancy makes generation brittle, as errors may accumulate along the way. We address this issue by proposing a novel sequence level training algorithm that directly optimizes the metric used at test time, such as BLEU or ROUGE. On three different tasks, our approach outperforms several strong baselines for greedy generation. The method is also competitive when these baselines employ beam search, while being several times faster.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.1511.06732

Cites

5 of 5 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • The models are trained on the ground truth, but during inference, they attend to the past predictions, which can be incorrect.

    “The models are trained on the ground truth, but during inference, they attend to the past predictions, which can be incorrect [124, 213].”

    From Survey of Hallucination in Natural Language Generation (2023) · cites this paper as [213]

  • Overexposure bias is a common problem in NMT, amplified by the teacher-forcing technique used in sequence-to-sequence models.

    “Overexposure bias is a common problem in NMT, amplified by the teacher-forcing technique used in sequence-to-sequence models.”

    From Survey of Hallucination in Natural Language Generation (2023) · cites this paper as [213]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 22

  1. Auli, M. and Gao, J. Decoder integration and expected bleu training for recurrent neural network language models. In Proc. of ACL, June 2014.
  2. Bahdanau, D., Cho, K., and Bengio, Y. Neural machine translation by jointly learning to align and translate. In ICLR, 2015.
  3. Bengio, S., Vinyals, O., Jaitly, N., and Shazeer, N. Scheduled sampling for sequence prediction with recurrent neural networks. In NIPS, 2015.
  4. Cettolo, M., Niehues, J., Stüker, S., Bentivogli, L., , and Federico, M. Report on the 11th iwslt evaluation campaign. In Proc. of IWSLT, 2014.
  5. Daume III, H., Langford, J., and Marcu, D. Search-based structured prediction as classification. Machine Learning Journal, 2009.
  6. Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., and Fei-Fei, L. Imagenet: a large-scale hierarchical image database. In IEEE Conference on Computer Vision and Pattern Recognition, 2009.
  7. Elman, Jeffrey L. Finding structure in time. Cognitive Science, 14(2):179-211, 1990.
  8. Graves, A. and Jaitly, N. Towards end-to-end speech recognition with recurrent neural networks. In ICML, 2014.
  9. He, X. and Deng, L. Maximum expected bleu training of phrase and lexicon translation models. In ACL, 2012.
  10. Kneser, Reinhard and Ney, Hermann. Improved backing-off for M-gram language modeling. In Proc. of the International Conference on Acoustics, Speech, and Signal Processing, pp. 181-184, May 1995.
  11. Liang, Percy, Bouchard-Côté, Alexandre, Taskar, Ben, and Klein, Dan. An end-to-end discrimina- tive approach to machine translation. In acl-coling2006, pp. 761-768, Jul 2006.
  12. Lin, C.Y. and Hovy, E.H. Automatic evaluation of summaries using n-gram co-occurrence statistics. In HLT-NAACL, 2003.
  13. McAllester, D., Hazan, T., and Keshet, J. Direct loss minimization for structured prediction. In NIPS, 2010.
  14. Mnih, V., Heess N., Graves, A., and Kavukcuoglu, K. Recurrent models of visual attention. In NIPS, 2014.
  15. Morin, F. and Bengio, Y. Hierarchical probabilistic neural network language model. In AISTATS, 2005.
  16. Reiter, E. and Dale, R. Building natural language generation systems. Cambridge university press, 2000.
  17. Ross, S., Gordon, G.J., and Bagnell, J.A. A reduction of imitation learning and structured prediction to no-regret online learning. In AISTATS, 2011.
  18. Rosti, Antti-Veikko I, Zhang, Bing, Matsoukas, Spyros, and Schwartz, Richard. Expected bleu training for graphs: Bbn system description for wmt11 system combination task. In Proc. of WMT, pp. 159-165. Association for Computational Linguistics, July 2011.
  19. Rush, A.M., Chopra, S., and Weston, J. A neural attention model for abstractive sentence summa- rization. In EMNLP, 2015.
  20. Sutskever, Ilya, Vinyals, Oriol, and Le, Quoc. Sequence to sequence learning with neural networks. In Proc. of NIPS, 2014.
  21. Sutton, R.S. and Barto, A.G. Reinforcement learning: An introduction. MIT Press, 1988.
  22. Venkatraman, A., Hebert, M., and Bagnell, J.A. Improving multi-step prediction of learned time series models. In AAAI, 2015.

Cited by

Sorted by citations

Authors
Ranzato, M., Chopra, S., Auli, M., & Zaremba, W.
Venue
arXiv
Publisher
arXiv
Published
2015
DOI
10.48550/arxiv.1511.06732
arXiv
1511.06732
References
5
Type
Preprint
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review