Imitation Learning for Non-Autoregressive Neural Machine Translation
Open accessThe DOI matches its Crossref record; 5 of the 5 references it cites were found in a registry or an index.
Abstract
Non-autoregressive translation models (NAT) have achieved impressive inference speedup.A potential issue of the existing NAT algorithms, however, is that the decoding is conducted in parallel, without directly considering previous context.In this paper, we propose an imitation learning framework for nonautoregressive machine translation, which still enjoys the fast translation speed but gives comparable translation performance compared to its auto-regressive counterpart.We conduct experiments on the IWSLT16, WMT14 and WMT16 datasets.Our proposed model achieves a significant speedup over the autoregressive models, while keeping the translation quality comparable to the autoregressive models.By sampling sentence length in parallel at inference time, we achieve the performance of 31.85BLEU on WMT16 Ro→En and 30.68 BLEU on IWSLT16 En→De.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Playing Atari with Deep Reinforcement Learning
Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., & Riedmiller, M. (2013) · arXiv · doi:10.48550/arxiv.1312.5602 · Cited by 9,471
Unsupervised deep embedding for clustering analysis
Xie, J., Girshick, R., & Farhadi, A. (2015) · arXiv · doi:10.48550/arxiv.1511.06335 · Cited by 2,476 · cited 1×
“Inspired by Xie et al. (2016), we introduce a regularization method to increase the space utilization.”
MethodSequence-Level Knowledge Distillation
Kim, Y., & Rush, A. M. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1139 · Cited by 836
Deterministic Non-Autoregressive Neural Sequence Modeling by Iterative Refinement
Lee, J., Mansimov, E., & Cho, K. (2018) · Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d18-1149 · Cited by 386 · cited 7×
“(Gu et al., 2017), (Lee et al., 2018) and (Kaiser et al., 2018) respectively. imitate-NAT is our proposed NAT with imitation learning.”
BackgroundMethodInfluential
Cited by
HAT: Hardware-Aware Transformers for Efficient Natural Language Processing
Wang, H., Wu, Z., Liu, Z., Cai, H., Zhu, L., Gan, C., & Han, S. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.686 · Cited by 227
“Another trend is to apply non-or partially-autoregressive models to cut down the iteration number for decoding (Gu et al., 2019; Akoury et al., 2019; Wei et al., 2019; Gu et al., 2018).”
MethodGlancing Transformer for Non-Autoregressive Neural Machine Translation
Qian, L., Zhou, H., Bao, Y., Wang, M., Qiu, L., Zhang, W., Yu, Y., & Li, L. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.155 · Cited by 108
“For GLAT, we first predict m target length candidates, then generate output sequences with argmax decoding for each target length candidate, which is also called length parallel decoding (LPD) (Wei et al., 2019).”
BackgroundMethodInfluentialSelf-citationNon-Autoregressive Line-Level Code Completion
Liu, F., Fu, Z., Li, G., Jin, Z., Liu, H., Hao, Y., & Zhang, L. (2024) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3649594 · Cited by 22
“…is performed in a fully-NAR way, where the input to the decoder H = { h 1 , h 2 , . . . , h n } are copied from the encoder output using soft-copy [52], which maps the encoder embeddings S = { s 1 , s 2 , . . . , s m } into target length H = { h 1 , h 2 , . . . , h n } depending on the distance…”
Method
Metadata
- Authors
- Wei, B., Wang, M., Zhou, H., Lin, J., & Sun, X.
- Venue
- Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics
- Published
- 2019
- Pages
- 1304-1312
- OpenAlex
- W2962915948
- Cited by
- 65
- References
- 5
- FWCI
- 4.01
- Type
- conference paper
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Imitation Learning for Non-Autoregressive Neural Machine Translation. https://app.cochat.ai/papers/10.18653/v1/p19-1125
- The paper
- Wei, B., Wang, M., Zhou, H., Lin, J., & Sun, X. (2019). Imitation Learning for Non-Autoregressive Neural Machine Translation. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. https://doi.org/10.18653/v1/p19-1125
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.