The DOI matches its Crossref record; 25 of the 25 references it cites were found in a registry or an index.
Fully Character-Level Neural Machine Translation without Explicit Segmentation
Open accessAbstract
Most existing machine translation systems operate at the level of words, relying on explicit segmentation to extract tokens. We introduce a neural machine translation (NMT) model that maps a source character sequence to a target character sequence without any segmentation. We employ a character-level convolutional network with max-pooling at the encoder to reduce the length of source representation, allowing the model to be trained at a speed comparable to subword-level models while capturing local regularities. Our character-to-character model outperforms a recently proposed baseline with a subword-level encoder on WMT’15 DE-EN and CS-EN, and gives comparable performance on FI-EN and RU-EN. We then demonstrate that it is possible to share a single character-level encoder across multiple languages by training a model on a many-to-one translation task. In this multilingual setting, the character-level encoder significantly outperforms the subword-level encoder on all the language pairs. We observe that on CS-EN, FI-EN and RU-EN, the quality of the multilingual character-level translation even surpasses the models specifically trained on that language pair alone, both in terms of the BLEU score and human judgment.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation
Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1179 · Cited by 25,545 · cited 3×
“…that builds a single neural network which takes as an input a source sentence X = (x1, . . . , xTX ) and generates its translation Y = (y1, . . . , yTY ), where xt and yt′ are source and target symbols (Bahdanau et al., 2015; Sutskever et al., 2015; Luong et al., 2015; Cho et al., 2014a).”
BackgroundSelf-citationSequence to Sequence Learning with Neural Networks
Sutskever, I., Vinyals, O., & Le, Q. V. (2014) · arXiv · doi:10.48550/arxiv.1409.3215 · Cited by 17,766
Effective Approaches to Attention-based Neural Machine Translation
Luong, T., Pham, H., & Manning, C. D. (2015) · Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d15-1166 · Cited by 8,382
On the Properties of Neural Machine Translation: Encoder–Decoder Approaches
Cho, K., van Merrienboer, B., Bahdanau, D., & Bengio, Y. (2014) · Proceedings of SSST-8, Eighth Workshop on Syntax, Semantics and Structure in Statistical Translation · doi:10.3115/v1/w14-4012 · Cited by 7,099
Self-citationGoogle's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
Wu, Y., Schuster, M., Chen, Z., Le, Q. V., Norouzi, M., Macherey, W., Krikun, M., Cao, Y., Gao, Q., Macherey, K., et al. (2016) · arXiv · doi:10.48550/arxiv.1609.08144 · Cited by 5,319
On the difficulty of training recurrent neural networks
Pascanu, R., Mikolov, T., & Bengio, Y. (2012) · arXiv · doi:10.48550/arxiv.1211.5063 · Cited by 4,310
Character-Aware Neural Language Models
Kim, Y., Jernite, Y., Sontag, D., & Rush, A. M. (2015) · arXiv · doi:10.48550/arxiv.1508.06615 · Cited by 1,530
Character-Aware Neural Language Models
Kim, Y., Jernite, Y., Sontag, D., & Rush, A. (2016) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v30i1.10362 · Cited by 1,491
Cited by
Google’s Multilingual Neural Machine Translation System: Enabling Zero-Shot Translation
Johnson, M., Schuster, M., Le, Q. V., Krikun, M., Wu, Y., Chen, Z., Thorat, N., Viégas, F., Wattenberg, M., Corrado, G., et al. (2017) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00065 · Cited by 1,747
Tacotron: towards end-to-end speech synthesis
Wang, Y., Skerry-Ryan, R. J., Stanton, D., Wu, Y., Weiss, R. J., Jaitly, N., Yang, Z., Xiao, Y., Chen, Z., Bengio, S., et al. (2017) · Interspeech 2017 · doi:10.21437/interspeech.2017-1452 · Cited by 1,746
“CBHG is inspired from work in machine translation [8], where the main differences from [8] include using non-causal convolutions, batch normalization, residual connections, and stride=1 max pooling.”
MethodInfluentialAdapterFusion: Non-Destructive Task Composition for Transfer Learning
Pfeiffer, J., Kamath, A., Rücklé, A., Cho, K., & Gurevych, I. (2021) · Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume · doi:10.18653/v1/2021.eacl-main.39 · Cited by 636
“As has been shown in Lee et al. (2017), these models often overfit on low resource tasks and underfit on high resource tasks.”
BackgroundSelf-citationUnsupervised Neural Machine Translation
Artetxe, M., Labaka, G., Agirre, E., & Cho, K. (2017) · arXiv · doi:10.48550/arxiv.1710.11041 · Cited by 632
Self-citationAdversarial Attacks on Deep-learning Models in Natural Language Processing: A Survey
Zhang, W. E., Sheng, Q. Z., Alhazmi, A., & Li, C. (2020) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3374217 · Cited by 407
Neural Machine Translation: A Review
Stahlberg, F. (2020) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.12007 · Cited by 397
Findings of the 2017 Conference on Machine Translation (WMT17)
Bojar, O., Chatterjee, R., Federmann, C., Graham, Y., Haddow, B., Huang, S., Huck, M., Koehn, P., Liu, Q., Logacheva, V., et al. (2017) · Proceedings of the Second Conference on Machine Translation · doi:10.18653/v1/w17-4717 · Cited by 395
Synthetic and Natural Noise Both Break Neural Machine Translation
Belinkov, Y., & Bisk, Y. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.02173 · Cited by 386
ByT5: Towards a Token-Free Future with Pre-trained Byte-to-Byte Models
Xue, L., Barua, A., Constant, N., Al-Rfou, R., Narang, S., Kale, M., Roberts, A., & Raffel, C. (2022) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00461 · Cited by 330
Meta-Learning for Low-Resource Neural Machine Translation
Gu, J., Wang, Y., Chen, Y., Li, V. O. K., & Cho, K. (2018) · Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d18-1398 · Cited by 325
Self-citationQuasi-Recurrent Neural Networks
Bradbury, J., Merity, S., Xiong, C., & Socher, R. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.01576 · Cited by 299
A Survey of Multilingual Neural Machine Translation
Dabre, R., Chu, C., & Kunchukuttan, A. (2021) · ACM Computing Surveys · doi:10.1145/3406095 · Cited by 294
Massively Multilingual Neural Machine Translation in the Wild: Findings and Challenges
Arivazhagan, N., Bapna, A., Firat, O., Lepikhin, D., Johnson, M., Krikun, M., Chen, M. X., Cao, Y., Foster, G., Cherry, C., et al. (2019) · arXiv · doi:10.48550/arxiv.1907.05019 · Cited by 279
Training Tips for the Transformer Model
Popel, M., & Bojar, O. (2018) · The Prague Bulletin of Mathematical Linguistics · doi:10.2478/pralin-2018-0002 · Cited by 266
Position Information in Transformers: An Overview
Dufter, P., Schmitt, M., & Schütze, H. (2022) · Computational Linguistics · doi:10.1162/coli_a_00445 · Cited by 175
Neural Machine Translation with Byte-Level Subwords
Wang, C., Cho, K., & Gu, J. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i05.6451 · Cited by 147
Self-citationKhanjani, Z., Watson, G., & Janeja, V. P. (2023) · Frontiers in Big Data · doi:10.3389/fdata.2022.1001063 · Cited by 121
Metadata
- Authors
- Lee, J., Cho, K., & Hofmann, T.
- Venue
- Transactions of the Association for Computational Linguistics
- Publisher
- MIT Press - Journals
- Published
- 2017-12
- Volume
- 5
- Pages
- 365-378
- ISSN
- 2307-387X (ISSN-L)
- OpenAlex
- W2531207078
- Cited by
- 394
- References
- 25
- FWCI
- 39.85
- Type
- journal article
- Language
- en
- Open access
- diamond
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.