CoChat

Six Challenges for Neural Machine Translation

Open access
Koehn, Philipp, Knowles, Rebecca · 2017 · Proceedings of the First Workshop on Neural Machine Translation

The DOI matches its Crossref record; 13 of the 13 references it cites were found in a registry or an index.

Abstract

We explore six challenges for neural machine translation: domain mismatch, amount of training data, rare words, long sentences, word alignment, and beam search.We show both deficiencies and improvements over the quality of phrasebased statistical machine translation.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.18653/v1/w17-3204

Cites

13 of 13 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 13 references

Research works printed but not linked: 19

  1. [3]3. Luisa Bentivogli, Arianna Bisazza, Mauro Cettolo, and Marcello Federico. 2016. Neural versus phrase- based machine translation quality: a case study. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. Associ- ation for Computational Linguistics, Austin, Texas, pages 257-267. https://aclweb.org/anthology/D16- 1025.
  2. [4]4. Ondřej Bojar, Rajen Chatterjee, Christian Feder- mann, Yvette Graham, Barry Haddow, Matthias Huck, Antonio Jimeno Yepes, Philipp Koehn, Varvara Logacheva, Christof Monz, Matteo Negri, Aurelie Neveol, Mariana Neves, Martin Popel, Matt Post, Raphael Rubino, Carolina Scarton, Lucia Specia, Marco Turchi, Karin Verspoor, and Marcos Zampieri. 2016. Findings of the 2016 conference on machine translation. In Proceedings of the First Conference on Ma- chine Translation. Association for Computational Linguistics, Berlin, Germany, pages 131-198. http://www.aclweb.org/anthology/W/W16/W16- 2301.
  3. [6]6. David Chiang. 2007. Hierarchical phrase-based translation. Computational Linguistics 33(2). http://www.aclweb.org/anthology-new/J/J07/J07- 2003.pdf.
  4. [7]7. Kyunghyun Cho, Bart van Merrienboer, Dzmitry Bah- danau, and Yoshua Bengio. 2014. On the proper- ties of neural machine translation: Encoder-decoder approaches. In Proceedings of SSST-8, Eighth Workshop on Syntax, Semantics and Structure in Statistical Translation. Association for Computa- tional Linguistics, Doha, Qatar, pages 103-111. http://www.aclweb.org/anthology/W14-4012.
  5. [10]10. Shuoyang Ding, Kevin Duh, Huda Khayrallah, Philipp Koehn, and Matt Post. 2016b. The JHU machine translation systems for WMT 2016. In Proceed- ings of the First Conference on Machine Translation (WMT).
  6. [11]11. Chris Dyer, Victor Chahuneau, and Noah A. Smith. 2013. A simple, fast, and effective reparameteriza- tion of ibm model 2. In Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Association for Computa- tional Linguistics, Atlanta, Georgia, pages 644-648. http://www.aclweb.org/anthology/N13-1073.
  7. [13]13. Michel Galley, Jonathan Graehl, Kevin Knight, Daniel Marcu, Steve DeNeefe, Wei Wang, and Igna- cio Thayer. 2006. Scalable inference and train- ing of context-rich syntactic translation models. In Proceedings of the 21st International Confer- ence on Computational Linguistics and 44th An- nual Meeting of the Association for Computa- tional Linguistics. Association for Computational Linguistics, Sydney, Australia, pages 961-968. http://www.aclweb.org/anthology/P/P06/P06-1121.
  8. [15]15. Ann Irvine and Chris Callison-Burch. 2013. Com- bining bilingual and comparable corpora for low resource machine translation. In Pro- ceedings of the Eighth Workshop on Statistical Machine Translation. Association for Computa- tional Linguistics, Sofia, Bulgaria, pages 262-270. http://www.aclweb.org/anthology/W13-2233.
  9. [17]17. Nal Kalchbrenner and Phil Blunsom. 2013. Recurrent continuous translation models. In Proceedings of the 2013 Conference on Empirical Methods in Natu- ral Language Processing. Association for Computa- tional Linguistics, Seattle, Washington, USA, pages 1700-1709. http://www.aclweb.org/anthology/D13- 1176.
  10. [18]18. Philipp Koehn and Barry Haddow. 2012. Interpolated backoff for factored translation models. In Proceed- ings of the Tenth Conference of the Association for Machine Translation in the Americas (AMTA).
  11. [21]21. Minh-Thang Luong and Christopher D Manning. 2015. Stanford neural machine translation systems for spo- ken language domains. In Proceedings of the In- ternational Workshop on Spoken Language Transla- tion.
  12. [22]22. Thang Luong, Ilya Sutskever, Quoc Le, Oriol Vinyals, and Wojciech Zaremba. 2015. Addressing the rare word problem in neural machine transla- tion. In Proceedings of the 53rd Annual Meet- ing of the Association for Computational Lin- guistics and the 7th International Joint Con- ference on Natural Language Processing (Vol- ume 1: Long Papers). Association for Computa- tional Linguistics, Beijing, China, pages 11-19. http://www.aclweb.org/anthology/P15-1002.
  13. [23]23. Jean Pouget-Abadie, Dzmitry Bahdanau, Bart van Merrienboer, KyungHyun Cho, and Yoshua Ben- gio. 2014. Overcoming the curse of sentence length for neural machine translation using au- tomatic segmentation. CoRR abs/1409.1257. http://arxiv.org/abs/1409.1257. · arXiv
  14. [25]25. Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016a. Edinburgh neural machine trans- lation systems for WMT 16. In Proceedings of the First Conference on Machine Transla- tion (WMT). Association for Computational Linguistics, Berlin, Germany, pages 371-376. http://www.aclweb.org/anthology/W/W16/W16- 2323.
  15. [26]26. Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016b. Neural machine translation of rare words with subword units. In Proceed- ings of the 54th Annual Meeting of the As- sociation for Computational Linguistics (Volume 1: Long Papers). Association for Computational Linguistics, Berlin, Germany, pages 1715-1725. http://www.aclweb.org/anthology/P16-1162.
  16. [28]28. Antonio Toral and Víctor M. Sánchez-Cartagena. 2017. A multifaceted evaluation of neural versus phrase- based machine translation for 9 language directions. In Proceedings of the 15th Conference of the Euro- pean Chapter of the Association for Computational Linguistics: Volume 1, Long Papers. Association for Computational Linguistics, Valencia, Spain, pages 1063-1073. http://www.aclweb.org/anthology/E17- 1100.
  17. [29]29. Zhaopeng Tu, Zhengdong Lu, Yang Liu, Xiao- hua Liu, and Hang Li. 2016. Modeling cov- erage for neural machine translation. In Pro- ceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Vol- ume 1: Long Papers). Association for Computa- tional Linguistics, Berlin, Germany, pages 76-85. http://www.aclweb.org/anthology/P16-1008.
  18. [30]30. Marco Turchi, Tijl De Bie, and Nello Cristianini. 2008. Learning performance of a machine translation system: a statistical and computational analysis. In Proceedings of the Third Workshop on Statistical Machine Translation. Association for Computa- tional Linguistics, Columbus, Ohio, pages 35-43. http://www.aclweb.org/anthology/W/W08/W08- 0305.
  19. [31]31. Philip Williams, Rico Sennrich, Maria Nadejde, Matthias Huck, Barry Haddow, and Ondřej Bojar. 2016. Edinburgh's statistical machine translation systems for wmt16. In Proceed- ings of the First Conference on Machine Translation. Association for Computational Linguistics, Berlin, Germany, pages 399-410. http://www.aclweb.org/anthology/W/W16/W16- 2327.

Cited by

See all 185

Metadata

Authors
Koehn, P., & Knowles, R.
Affiliations
Johns Hopkins University (US)
Venue
Proceedings of the First Workshop on Neural Machine Translation
Published
2017
Pages
28-39
OpenAlex
W2625092622
Cited by
184
References
12
Citations by year
2026: 11 · 2025: 12 · 2024: 10 · 2023: 18 · 2022: 14 · 2021: 24 · 2020: 27 · 2019: 20 · 2018: 44 · 2017: 4
FWCI
11.59
Type
conference paper
Language
en
Open access
Licence
cc-by
Locations
  • arXiv (Cornell University) (submittedVersion)
  • Edinburgh Research Explorer (University of Edinburgh) (submittedVersion, cc-by)
  • arXiv (Cornell University)
Funders
National Science Foundation (DGE-1,232,825, 1232825)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
Keywords
neural machine translation, domain mismatch, training data size, rare words, long sentences, word alignment, beam search, phrase-based statistical machine translation, translation quality

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Six Challenges for Neural Machine Translation. https://app.cochat.ai/papers/10.18653/v1/w17-3204
The paper
Koehn, P., & Knowles, R. (2017). Six Challenges for Neural Machine Translation. Proceedings of the First Workshop on Neural Machine Translation. https://doi.org/10.18653/v1/w17-3204

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.