CoChat
Sign Up
PreprintCited by 142Natural Language Processing TechniquesTopic ModelingSpeech Recognition and Synthesis

The Curious Case of Hallucinations in Neural Machine Translation

Open access
Raunak, Vikas, Menezes, Arul, Junczys-Dowmunt, Marcin·2021·arXiv
DOI 10.48550/arxiv.2104.06683arXiv 2104.06683
Open this paper in CoChat — free accountOpen PDF

Abstract

In this work, we study hallucinations in Neural Machine Translation (NMT), which lie at an extreme end on the spectrum of NMT pathologies. Firstly, we connect the phenomenon of hallucinations under source perturbation to the Long-Tail theory of Feldman (2020), and present an empirically validated hypothesis that explains hallucinations under source perturbation. Secondly, we consider hallucinations under corpus-level noise (without any source perturbation) and demonstrate that two prominent types of natural hallucinations (detached and oscillatory outputs) could be generated and explained through specific corpus-level noise patterns. Finally, we elucidate the phenomenon of hallucination amplification in popular data-generation processes such as Backtranslation and sequence-level Knowledge Distillation.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2104.06683

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • The 'implicit representation' of knowledge in LLM has been shown to generate non-factual information despite linguistically coherent answers (i.e. 'hallucination') as a response to the input prompt.

    “Nevertheless, the “implicit representation” of knowledge in LLM has also been shown to generate non-factual information despite linguistically coherent answers (i.e. “hallucination”) as a response to the input prompt (Maynez et al. 2020, Raunak et al. 2021, Ji et al. 2023).”

    From Biomedical knowledge graph-optimized prompt generation for large language models (2024) · cites this paper as Raunak et al. 2021

    The Introduction describes NMT hallucinations as outputs decoupled from the source sequence despite being fluent in the target language, and the Conclusion reports experiments on when such hallucinations arise. This supports the claim’s general point about fluent but unsupported output. The paper studies neural machine translation, however, not LLM answers to prompts or an ‘implicit representation’ of knowledge. (LLM, conf=0.88)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

No references could be resolved yet.

Research works printed but not linked: 27

  1. Mikel Artetxe and Holger Schwenk. 2019. Massively Multilingual Sentence Embeddings for Zero-Shot Cross-Lingual Transfer and Beyond. Transactions of the Association for Computational Linguistics, 7:597-610. · DOI
  2. Satanjeev Banerjee and Alon Lavie. 2005. METEOR: An automatic metric for MT evaluation with im- proved correlation with human judgments. In Pro- ceedings of the ACL Workshop on Intrinsic and Ex- trinsic Evaluation Measures for Machine Transla- tion and/or Summarization, pages 65-72, Ann Ar- bor, Michigan. Association for Computational Lin- guistics.
  3. Alexandre Berard, Ioan Calapodescu, and Claude Roux. 2019. Naver labs Europe's systems for the WMT19 machine translation robustness task. In Proceedings of the Fourth Conference on Machine Translation (Volume 2: Shared Task Papers, Day 1), pages 526-532, Florence, Italy. Association for Computational Linguistics. · DOI
  4. Yong Cheng, Lu Jiang, and Wolfgang Macherey. 2019. Robust neural machine translation with doubly ad- versarial inputs. In Proceedings of the 57th Annual Meeting of the Association for Computational Lin- guistics, pages 4324-4333, Florence, Italy. Associa- tion for Computational Linguistics. · DOI
  5. Sergey Edunov, Myle Ott, Michael Auli, and David Grangier. 2018. Understanding back-translation at scale. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 489-500, Brussels, Belgium. Association for Computational Linguistics. · DOI
  6. Vitaly Feldman. 2020. Does learning require memo- rization? a short tale about a long tail. In Pro- ceedings of the 52nd Annual ACM SIGACT Sympo- sium on Theory of Computing, STOC 2020, page 954-959, New York, NY, USA. Association for Computing Machinery. · DOI · Paper page
  7. Marcin Junczys-Dowmunt. 2018. Dual conditional cross-entropy filtering of noisy parallel corpora. In Proceedings of the Third Conference on Machine Translation: Shared Task Papers, pages 888-895, Belgium, Brussels. Association for Computational Linguistics. · DOI · Paper page
  8. Daniel Kang and Tatsunori Hashimoto. 2020. Im- proved natural language generation via loss trunca- tion. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 718-731, Online. Association for Computa- tional Linguistics. · DOI
  9. Yoon Kim and Alexander M. Rush. 2016. Sequence- level knowledge distillation. In Proceedings of the 2016 Conference on Empirical Methods in Natu- ral Language Processing, pages 1317-1327, Austin, Texas. Association for Computational Linguistics. · DOI · Paper page
  10. Philipp Koehn and Rebecca Knowles. 2017. Six chal- lenges for neural machine translation. In Proceed- ings of the First Workshop on Neural Machine Trans- lation, pages 28-39, Vancouver. Association for Computational Linguistics. · DOI · Paper page
  11. Katherine Lee, Orhan Firat, Ashish Agarwal, Clara Fannjiang, and David Sussillo. 2018. Hallucinations in neural machine translation. In NIPS 2018 Inter- pretability and Robustness for Audio, Speech and Language Workshop.
  12. Tian Li, Ahmad Beirami, Maziar Sanjabi, and Virginia Smith. 2021. Tilted empirical risk minimization. In International Conference on Learning Representa- tions.
  13. Marianna Martindale, Marine Carpuat, Kevin Duh, and Paul McNamee. 2019. Identifying fluently inade- quate output in neural and statistical machine trans- lation. In Proceedings of Machine Translation Sum- mit XVII Volume 1: Research Track, pages 233-243, Dublin, Ireland. European Association for Machine Translation.
  14. Mathias Müller, Annette Rios, and Rico Sennrich. 2020. Domain robustness in neural machine trans- lation. In Proceedings of the 14th Conference of the Association for Machine Translation in the Americas (Volume 1: Research Track), pages 151-164, Virtual. Association for Machine Translation in the Ameri- cas.
  15. Myle Ott, Sergey Edunov, Alexei Baevski, Angela Fan, Sam Gross, Nathan Ng, David Grangier, and Michael Auli. 2019. fairseq: A fast, extensible toolkit for sequence modeling. In Proceedings of the 2019 Conference of the North American Chap- ter of the Association for Computational Linguistics (Demonstrations), pages 48-53, Minneapolis, Min- nesota. Association for Computational Linguistics. · DOI
  16. Kishore Papineni, Salim Roukos, Todd Ward, and Wei- Jing Zhu. 2002. Bleu: a method for automatic eval- uation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Com- putational Linguistics, pages 311-318, Philadelphia, Pennsylvania, USA. Association for Computational Linguistics. · DOI · Paper page
  17. Maja Popović. 2015. chrF: character n-gram F-score for automatic MT evaluation. In Proceedings of the Tenth Workshop on Statistical Machine Translation, pages 392-395, Lisbon, Portugal. Association for Computational Linguistics. · DOI
  18. Vikas Raunak, Siddharth Dalmia, Vivek Gupta, and Florian Metze. 2020. On long-tailed phenomena in neural machine translation. In Findings of the As- sociation for Computational Linguistics: EMNLP 2020, pages 3088-3095, Online. Association for Computational Linguistics. · DOI
  19. Holger Schwenk. 2018. Filtering and mining paral- lel data in a joint multilingual space. In Proceed- ings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Pa- pers), pages 228-234, Melbourne, Australia. Asso- ciation for Computational Linguistics. · DOI
  20. Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. Neural machine translation of rare words with subword units. In Proceedings of the 54th An- nual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1715- 1725, Berlin, Germany. Association for Computa- tional Linguistics. · DOI · Paper page
  21. Zhaopeng Tu, Zhengdong Lu, Yang Liu, Xiaohua Liu, and Hang Li. 2016. Modeling coverage for neural machine translation. In Proceedings of the 54th An- nual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 76- 85, Berlin, Germany. Association for Computational Linguistics. · DOI
  22. Arata Ugawa, Akihiro Tamura, Takashi Ninomiya, Hi- roya Takamura, and Manabu Okumura. 2018. Neu- ral machine translation incorporating named entity. In Proceedings of the 27th International Conference on Computational Linguistics, pages 3240-3250, Santa Fe, New Mexico, USA. Association for Com- putational Linguistics.
  23. Elena Voita, Rico Sennrich, and Ivan Titov. 2020. An- alyzing the source and target contributions to predic- tions in neural machine translation. arXiv preprint arXiv:2010.10907. · arXiv
  24. Chaojun Wang and Rico Sennrich. 2020. On exposure bias, hallucination and domain shift in neural ma- chine translation. In Proceedings of the 58th Annual Meeting of the Association for Computational Lin- guistics, pages 3544-3552, Online. Association for Computational Linguistics. · DOI · Paper page
  25. Xiaobo Xia, Tongliang Liu, Bo Han, Chen Gong, Nan- nan Wang, Zongyuan Ge, and Yi Chang. 2021. Ro- bust early-learning: Hindering the memorization of noisy labels. In International Conference on Learn- ing Representations.
  26. Boliang Zhang, Ajay Nagesh, and Kevin Knight. 2020. Parallel corpus filtering via pre-trained language models. In Proceedings of the 58th Annual Meet- ing of the Association for Computational Linguistics, pages 8545-8554, Online. Association for Computa- tional Linguistics. · DOI
  27. Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q Weinberger, and Yoav Artzi. 2019. Bertscore: Eval- uating text generation with bert. In International Conference on Learning Representations.

Cited by

Sorted by citations

Authors
Raunak, V., Menezes, A., & Junczys-Dowmunt, M.
Affiliations
Microsoft (United States) (US)
Venue
arXiv
Publisher
arXiv
Published
2021
Pages
1172-1183
DOI
10.48550/arxiv.2104.06683
arXiv
2104.06683
OpenAlex
W3153391445
Cited by
142
References
0
Citations by year
2026: 18 · 2025: 18 · 2024: 29 · 2023: 51 · 2022: 18 · 2021: 8
FWCI
12.24
Type
conference paper
Language
en
Open access
Licence
Locations
  • Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) (publishedVersion, cc-by)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University) (cc-by)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
Keywords
hallucinations, neural machine translation, factual errors

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review