The DOI matches its Crossref record; 26 of the 26 references it cites were found in a registry or an index.
Neural versus Phrase-Based Machine Translation Quality: a Case Study
Open accessAbstract
Within the field of Statistical Machine Translation (SMT), the neural approach (NMT) has recently emerged as the first technology able to challenge the long-standing dominance of phrase-based approaches (PBMT).In particular, at the IWSLT 2015 evaluation campaign, NMT outperformed well established state-ofthe-art PBMT systems on English-German, a language pair known to be particularly hard because of morphology and syntactic differences.To understand in what respects NMT provides better translation quality than PBMT, we perform a detailed analysis of neural vs. phrase-based SMT outputs, leveraging high quality post-edits performed by professional translators on the IWSLT data.For the first time, our analysis provides useful insights on what linguistic phenomena are best modeled by neural models -such as the reordering of verbs -while pointing out other aspects that remain to be improved.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation
Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1179 · Cited by 25,545
Sequence to Sequence Learning with Neural Networks
Sutskever, I., Vinyals, O., & Le, Q. V. (2014) · arXiv · doi:10.48550/arxiv.1409.3215 · Cited by 17,766
Neural Machine Translation by Jointly Learning to Align and Translate
Bahdanau, D., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1409.0473 · Cited by 16,213
A Study of Translation Edit Rate with Targeted Human Annotation
Snover, M., Dorr, B. J., Schwartz, R. M., Micciulla, L., & Makhoul, J. I. (2006) · Conference of the Association for Machine Translation in the Americas · Cited by 3,442
A Simple, Fast, and Effective Reparameterization of IBM Model 2
Dyer, C., Chahuneau, V., & Smith, N. A. (2018) · doi:10.1184/r1/6472985 · Cited by 767
Effective Approaches to Attention-based Neural Machine Translation
Luong, M.-T., Pham, H., & Manning, C. D. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1508.04025 · Cited by 655
WIT3: Web Inventory of Transcribed and Translated Talks
Cettolo, M., Girardi, C., & Federico, M. (2012) · Cited by 543
Self-citationStanford neural machine translation systems for spoken language domains.
Luong, M.-T., & Manning, C. D. (2015) · Cited by 402
Findings of the 2015 Workshop on Statistical Machine Translation
Bojar, O., Chatterjee, R., Federmann, C., Haddow, B., Huck, M., Hokamp, C., Koehn, P., Logacheva, V., Monz, C., Negri, M., et al. (2015) · Proceedings of the Tenth Workshop on Statistical Machine Translation · doi:10.18653/v1/w15-3001 · Cited by 277
Error Analysis of Statistical Machine Translation Output
Vilar, D., Xu, J., D’Haro, L. F., & Ney, H. (2006) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/453fcbexxngk · Cited by 257
Accelerated DP based search for statistical translation
Tillmann, C., Vogel, S., Ney, H., Zubiaga, A., & Sawaf, H. (1997) · 5th European Conference on Speech Communication and Technology (Eurospeech 1997) · doi:10.21437/eurospeech.1997-673 · Cited by 232
An “AI readability ” formula for French as a foreign language
François, T., & Fairon, C. (2013) · Cited by 110
Towards Automatic Error Analysis of Machine Translation Output
Popović, M., & Ney, H. (2011) · Computational Linguistics · doi:10.1162/coli_a_00072 · Cited by 102
Overcoming the Curse of Sentence Length for Neural Machine Translation using Automatic Segmentation
Pouget-Abadie, J., Bahdanau, D., van Merriënboer, B., Cho, K., & Bengio, Y. (2014) · doi:10.3115/v1/w14-4009 · Cited by 101
The IWSLT 2015 Evaluation Campaign
Cettolo, M., Niehues, J., Stüker, S., Bentivogli, L., Cattoni, R., & Federico, M. (2015) · Repository KITopen (Karlsruhe Institute of Technology) · doi:10.5281/zenodo.1326521 · Cited by 95
Self-citationMeasuring Machine Translation Errors in New Domains
Irvine, A., Morgan, J., Carpuat, M. J., Daumé, H., & Munteanu, D. S. (2013) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00239 · Cited by 75
Metrics for MT evaluation: evaluating reordering
Birch, A., Osborne, M., & Blunsom, P. (2010) · Machine Translation · doi:10.1007/s10590-009-9066-5 · Cited by 58
Neural Reranking Improves Subjective Quality of Machine Translation: NAIST at WAT2015Preprint
Neubig, G., Morishita, M., & Nakamura, S. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1510.05203 · Cited by 51
Sennrich, R., Volk, M., & Schneider, G. (2013) · Zurich Open Repository and Archive (University of Zurich) · doi:10.5167/uzh-81397 · Cited by 40
On the practice of error analysis for machine translation evaluation
Stymne, S., & Ahrenberg, L. (2012) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/4cs6fjcs27qi · Cited by 36
Assessing the Impact of Translation Errors on Machine Translation Quality with Mixed-effects Models
Federico, M., Negri, M., Bentivogli, L., & Turchi, M. (2014) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1172 · Cited by 35
Self-citationAnalyzing Error Types in English-Czech Machine Translation
Bojar, O. (2011) · The Prague Bulletin of Mathematical Linguistics · doi:10.2478/v10108-011-0005-2 · Cited by 28
Reordering metrics for statistical machine translation
Birch, A. (2011) · ERA · Cited by 23
The KIT Translation Systems for IWSLT 2015
Ha, T.-L., Niehues, J., Cho, E., Mediani, M., & Waibel, A. (2015) · doi:10.5445/ir/1000051636 · Cited by 21
Complexity of spoken versus written language for machine translation
Ruiz, N., & Federico, M. (2014) · Cited by 13
Self-citationLearning from Human Judgments of Machine Translation Output
Popović, M., Avramidis, E., Burchardt, A., Hunsicker, S., Schmeier, S., Tscherwinka, C., & Vilar, D. (2013) · Cited by 9
Cited by
Neural Machine Translation: A Review
Stahlberg, F. (2020) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.12007 · Cited by 397
What do Neural Machine Translation Models Learn about Morphology?
Belinkov, Y., Durrani, N., Dalvi, F., Sajjad, H., & Glass, J. (2017) · Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p17-1080 · Cited by 307
An Analysis of Encoder Representations in Transformer-Based Machine Translation
Raganato, A., & Tiedemann, J. (2018) · Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP · doi:10.18653/v1/w18-5431 · Cited by 277
Analyzing Uncertainty in Neural Machine Translation
Ott, M., Auli, M., Grangier, D., & Ranzato, M. (2018) · arXiv · doi:10.48550/arxiv.1803.00047 · Cited by 140
This is not a Dataset: A Large Negation Benchmark to Challenge Large Language Models
García-Ferrero, I., Altuna, B., Álvez, J., Gonzalez-Dios, I., & Rigau, G. (2023) · arXiv · doi:10.48550/arxiv.2310.15941 · Cited by 18
“Such is the case of negation in machine translation (Bentivogli et al., 2016; Tang et al., 2021), information extraction (Grivas et al., 2020) and sentiment analysis (Barnes et al., 2021) among others.”
Background
Metadata
- Authors
- Bentivogli, L., Bisazza, A., Cettolo, M., & Federico, M.
- Venue
- Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing
- Published
- 2016
- Pages
- 257-267
- OpenAlex
- W2513073850
- Cited by
- 316
- References
- 26
- FWCI
- 24.14
- Type
- conference paper
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.