Beyond Error Propagation in Neural Machine Translation: Characteristics of Language Also Matter
Open accessThe DOI matches its Crossref record; 5 of the 5 references it cites were found in a registry or an index.
Abstract
Neural machine translation usually adopts autoregressive models and suffers from exposure bias as well as the consequent error propagation problem.Many previous works have discussed the relationship between error propagation and the accuracy drop (i.e., the left part of the translated sentence is often better than its right part in left-to-right decoding models) problem.In this paper, we conduct a series of analyses to deeply understand this problem and get several interesting findings.(1) The role of error propagation on accuracy drop is overstated in the literature, although it indeed contributes to the accuracy drop problem.(2) Characteristics of a language play a more important role in causing the accuracy drop: the left part of the translation result in a right-branching language (e.g., English) is more likely to be more accurate than its right part, while the right part is more accurate for a left-branching language (e.g., Japanese).Our discoveries are confirmed on different model structures including Transformer and RNN, and in other sequence generation tasks such as text summarization.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Bengio, Y., Louradour, J., Collobert, R., & Weston, J. (2009) · Proceedings of the 26th Annual International Conference on Machine Learning · doi:10.1145/1553374.1553380 · Cited by 5,366 · cited 1×
“In this section, we study different branching languages from two aspects, including the n-gram statistics of a target language, which has been used as a kind of characterization of hardness of learning (Bengio et al., 2009), and the dependency statistics in parse trees.”
BackgroundA Learning Algorithm for Continually Running Fully Recurrent Neural Networks
Williams, R. J., & Zipser, D. (1989) · Neural Computation · doi:10.1162/neco.1989.1.2.270 · Cited by 4,369 · cited 2×
“Second, to further investigate the influence of error propagation on accuracy drop, we conduct a set of experiments with teacher forcing (Williams and Zipser, 1989) during inference, in which we feed the ground-truth preceding words to predict the next target word.”
BackgroundMethodScheduled Sampling for Sequence Prediction with Recurrent Neural Networks
Bengio, S., Vinyals, O., Jaitly, N., & Shazeer, N. (2015) · arXiv · doi:10.48550/arxiv.1506.03099 · Cited by 1,621
Sequence Level Training with Recurrent Neural Networks
Ranzato, M., Chopra, S., Auli, M., & Zaremba, W. (2015) · arXiv · doi:10.48550/arxiv.1511.06732 · Cited by 1,426 · cited 4×
“Shen et al. (2016); Ranzato et al. (2015) improve the basic maximum likelihood estimation (MLE) with reinforcement learning or minimum risk training and aim to address the limitation of MLE training and exposure bias problem.”
BackgroundInfluentialA Closer Look at Memorization in Deep Networks
Arpit, D., Jastrzębski, S., Ballas, N., Krueger, D., Bengio, E., Kanwal, M. S., Maharaj, T., Fischer, A., Courville, A., Bengio, Y., et al. (2017) · arXiv · doi:10.48550/arxiv.1706.05394 · Cited by 1,307 · cited 1×
“As previous work (Arpit et al., 2017) shows, neural networks are easy to learn and memorize simple patterns but difficult to make a correct prediction on noise examples.”
Background
Cited by
A survey on neural speech synthesis
Tan, X., Qin, T., Soong, F., & Liu, T.-Y. (2021) · arXiv · doi:10.48550/arxiv.2106.15561 · Cited by 216
“Autoregressive sequence generation usually suffers from exposure bias and error propagation [20, 384].”
BackgroundSelf-citationOn Exposure Bias, Hallucination and Domain Shift in Neural Machine Translation
Wang, C., & Sennrich, R. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.326 · Cited by 118
On Exposure Bias, Hallucination and Domain Shift in Neural Machine Translation
Wang, C., & Sennrich, R. (2020) · arXiv · doi:10.48550/arxiv.2005.03642 · Cited by 12
“However, the relevance of error propagation is under debate: Wu et al. (2018) argue that its role is overstated in literature, and that linguistic features explain some of the accuracy drop at higher time steps.”
Background
Metadata
- Authors
- Wu, L., Tan, X., He, D., Tian, F., Qin, T., Lai, J., & Liu, T.-Y.
- Venue
- Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing
- Published
- 2018
- Pages
- 3602-3611
- OpenAlex
- W2889606145
- Cited by
- 54
- References
- 5
- FWCI
- 4.58
- Type
- conference paper
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Beyond Error Propagation in Neural Machine Translation: Characteristics of Language Also Matter. https://app.cochat.ai/papers/10.18653/v1/d18-1396
- The paper
- Wu, L., Tan, X., He, D., Tian, F., Qin, T., Lai, J., & Liu, T.-Y. (2018). Beyond Error Propagation in Neural Machine Translation: Characteristics of Language Also Matter. Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. https://doi.org/10.18653/v1/d18-1396
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.