CoChat
Sign Up
PreprintCited by 1,621Topic ModelingNatural Language Processing TechniquesMultimodal Machine Learning Applications

Scheduled Sampling for Sequence Prediction with Recurrent Neural Networks

Open access
Bengio, Samy, Vinyals, Oriol, Jaitly, Navdeep, Shazeer, Noam·2015·arXiv
DOI 10.48550/arxiv.1506.03099arXiv 1506.03099
Open this paper in CoChat — free accountOpen PDF

Abstract

Recurrent Neural Networks can be trained to produce sequences of tokens given some input, as exemplified by recent results in machine translation and image captioning. The current approach to training them consists of maximizing the likelihood of each token in the sequence given the current (recurrent) state and the previous token. At inference, the unknown previous token is then replaced by a token generated by the model itself. This discrepancy between training and inference can yield errors that can accumulate quickly along the generated sequence. We propose a curriculum learning strategy to gently change the training process from a fully guided scheme using the true previous token, towards a less guided scheme which mostly uses the generated token instead. Experiments on several sequence prediction tasks show that this approach yields significant improvements. Moreover, it was used successfully in our winning entry to the MSCOCO image captioning challenge, 2015.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.1506.03099

Cites

6 of 6 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Exposure bias, the discrepancy between training and inference decoding, can contribute to hallucination in NLG regardless of decoding strategy.

    “Regardless of decoding strategy choices, the exposure bias problem [13, 213], defined as the discrepancy in decoding between training and inference time, can be another contributor to hallucination.”

    From Survey of Hallucination in Natural Language Generation (2023) · cites this paper as [13]

    The Introduction (p. 1) describes the training–inference discrepancy, says early mistakes can be amplified, and notes that beam search can mitigate the problem. The image-captioning results (Section 4.1, p. 5) link that mismatch to lower-quality captions. The paper does not specifically identify hallucination as an outcome or establish the claim across all decoding strategies. (LLM, conf=0.85)

    Experimental — accuracy not yet measured

  • Known-Unknown and SelfAware evaluated LLMs' awareness of uncertainty for questions without definitive answer.

    “Known-Unknown [5, 7] and SeflAware [299] evaluated LLMs' awareness of uncertainty for question without definitive answer”

    From Survey of Hallucination in Natural Language Generation (2023) · cites this paper as [7]

    The Introduction (p. 1) discusses a mismatch between training and inference in recurrent neural networks, while the Speech Recognition section (p. 7) evaluates scheduled sampling. Searches for Known-Unknown, SelfAware, LLM uncertainty awareness, and questions without definitive answers found no passage addressing the attributed claim. (LLM, conf=0.78)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 19

  1. [1][1] Y. Bengio, P. Simard, and P. Frasconi. Learning long term dependencies is hard. IEEE Transactions on Neural Networks , 5(2):157-166, 1994. -
  2. [3][3] I. Sutskever, O. Vinyals, and Q. Le. Sequence to sequence learning with neural networks. In Advances in Neural Information Processing Systems, NIPS , 2014. -
  3. [6][6] J. Donahue, L. A. Hendricks, S. Guadarrama, M. Rohrbach, S. Venugopalan, K. Saenko, and T. Darrell. Long-term recurrent convolutional networks for visual recognition and description. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR , 2015. -
  4. [8][8] J. D. Lafferty, A. McCallum, and F. C. N. Pereira. Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In Proceedings of the Eighteenth International Conference on Machine Learning, ICML , pages 282-289, San Francisco, CA, USA, 2001. Morgan Kaufmann Publishers Inc. -
  5. [9][9] H. Daum´ e III, J. Langford, and D. Marcu. Search-based structured prediction as classification. Machine Learning Journal , 2009. -
  6. [10][10] S. Ross, G. J. Gordon, and J. A. Bagnell. A reduction of imitation learning and structured prediction to no-regret online learning. In Proceedings of the Workshop on Artificial Intelligence and Statistics, AISTATS , 2011. -
  7. [11][11] A. Venkatraman, M. Herbert, and J. A. Bagnell. Improving multi-step prediction of learned time series models. In Twenty-Ninth AAAI Conference on Artificial Intelligence, AAAI , 2015. -
  8. [12][12] M. Collins and B. Roark. Incremental parsing with the perceptron algorithm. In Proceedings of the Association for Computational Linguistics, ACL , 2004. -
  9. [13][13] Y. Goldberg and J. Nivre. A dynamic oracle for arc-eager dependency parsing. In Proceedings of COLING , 2012. -
  10. [14][14] J. Mao, W. Xu, Y. Yang, J. Wang, Z. H. Huang, and A. Yuille. Deep captioning with multimodal recurrent neural networks (m-rnn). In International Conference on Learning Representations, ICLR , 2015. -
  11. [15][15] R. Kiros, R. Salakhutdinov, and R. Zemel. Unifying visual-semantic embeddings with multimodal neural language models. In TACL , 2015. -
  12. [17][17] H. Fang, S. Gupta, F. Iandola, R. K. Srivastava, L. Deng, P. Dollar, J. Gao, X. He, M. Mitchell, J. C. Platt, C. L. Zitnick, and G. Zweig. From captions to visual concepts and back. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR , 2015. -
  13. [18][18] R. Vedantam, C. L. Zitnick, and D. Parikh. CIDEr: Consensus-based image description evaluation. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR , 2015. -
  14. [19][19] T.-Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Doll´ ar, and C. L. Zitnick. Microsoft coco: Common objects in context. arXiv:1405.0312 , 2014. - · arXiv
  15. [22][22] D. Bahdanau, K. Cho, and Y. Bengio. Neural machine translation by jointly learning to align and translate. In International Conference on Learning Representations, ICLR , 2015. -
  16. [23][23] E. Hovy, M. Marcus, M. Palmer, L. Ramshaw, and R. Weischedel. Ontonotes: The 90% solution. In Proceedings of the Human Language Technology Conference of the NAACL, Short Papers , pages 57-60, New York City, USA, June 2006. Association for Computational Linguistics. -
  17. [24][24] D. Povey, A. Ghoshal, G. Boulianne, L. Burget, O. Glembek, N. Goel, M. Hannemann, P. Motlicek, Y. Qian, P. Schwarz, J. Silovsky, G. Stemmer, and K. Vesely. The kaldi speech recognition toolkit. In IEEE 2011 Workshop on Automatic Speech Recognition and Understanding . IEEE Signal Processing Society, December 2011. IEEE Catalog No.: CFP11SRW-USB. -
  18. [25][25] N. Jaitly. Exploring Deep Learning Methods for discovering features in speech signals. PhD thesis, University of Toronto, 2014. -
  19. [26][26] Jan Chorowski, Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. End-to-end continuous speech recognition using attention-based recurrent nn: First results. arXiv preprint arXiv:1412.1602 , 2014. · arXiv

Cited by

Sorted by citations

Authors
Bengio, S., Vinyals, O., Jaitly, N., & Shazeer, N.
Affiliations
Google (United States) (US)
ORCID
Vinyals, Oriol
Venue
arXiv
Publisher
arXiv
Published
2015
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.1506.03099
arXiv
1506.03099
OpenAlex
W648786980
Cited by
1,621
References
6
Citations by year
2026: 9 · 2025: 44 · 2024: 98 · 2023: 136 · 2022: 164 · 2021: 267 · 2020: 291 · 2019: 256 · 2018: 201 · 2017: 90 · 2016: 56 · 2015: 9
Type
posted content
Language
en
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
Keywords
scheduled sampling, recurrent neural networks, sequence prediction, curriculum learning, training-inference discrepancy, image captioning, token generation, error accumulation

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review