The DOI matches its Crossref record; 40 of the 40 references it cites were found in a registry or an index.
A Character-level Decoder without Explicit Segmentation for Neural Machine Translation
Open accessAbstract
The existing machine translation systems, whether phrase-based or neural, have relied almost exclusively on word-level modelling with explicit segmentation.In this paper, we ask a fundamental question: can neural machine translation generate a character sequence without any explicit segmentation?To answer this question, we evaluate an attention-based encoderdecoder with a subword-level encoder and a character-level decoder on four language pairs-En-Cs, En-De, En-Ru and En-Fiusing the parallel corpora from WMT'15.Our experiments show that the models with a character-level decoder outperform the ones with a subword-level decoder on all of the four language pairs.Furthermore, the ensembles of neural models with a character-level decoder outperform the state-of-the-art non-neural machine translation systems on En-Cs, En-De and En-Fi and perform comparably on En-Ru.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation
Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1179 · Cited by 25,545
Self-citationSequence to Sequence Learning with Neural Networks
Sutskever, I., Vinyals, O., & Le, Q. V. (2014) · arXiv · doi:10.48550/arxiv.1409.3215 · Cited by 17,766
Neural Machine Translation by Jointly Learning to Align and Translate
Bahdanau, D., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1409.0473 · Cited by 16,213
Self-citationLearning long-term dependencies with gradient descent is difficult
Bengio, Y., Simard, P., & Frasconi, P. (1994) · IEEE Transactions on Neural Networks · doi:10.1109/72.279181 · Cited by 8,542 · cited 1×
“(3) Vanishing Gradient Specifically to neural machine translation, a major reason behind the wide adoption of word-level modelling is due to the difficulty in modelling long-term dependencies with recurrent neural networks (Bengio et al., 1994; Hochreiter, 1998).”
BackgroundSelf-citationEffective Approaches to Attention-based Neural Machine Translation
Luong, T., Pham, H., & Manning, C. D. (2015) · Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d15-1166 · Cited by 8,382
A Neural Probabilistic Language Model
Bengio, Y., Ducharme, R., Vincent, P., Jauvin, C., Jaz, K., Hofmann, T., Poggio, T., & Shawe-Taylor, J. S. (2000) · Cited by 3,479
Self-citationThe Vanishing Gradient Problem During Learning Recurrent Neural Nets and Problem Solutions
Hochreiter, S. (1998) · International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems · doi:10.1142/s0218488598000094 · Cited by 2,939
Generating Text with Recurrent Neural Networks
Sutskever, I., Martens, J., & Hinton, G. E. (2011) · International Conference on Machine Learning · Cited by 1,783
Static Analysis of Shape in TensorFlow Programs
The Theano Development Team, Al‐Rfou, R., Alain, G., Almahairi, A., Angermueller, C., Bahdanau, D., Ballas, N., Bastien, F., Bayer, J., Belikov, A., et al. (2016) · arXiv (Cornell University) · doi:10.4230/lipics.ecoop.2020.15 · Cited by 1,589
Self-citationCharacter-Aware Neural Language Models
Kim, Y., Jernite, Y., Sontag, D., & Rush, A. M. (2015) · arXiv · doi:10.48550/arxiv.1508.06615 · Cited by 1,530
Character-Aware Neural Language Models
Kim, Y., Jernite, Y., Sontag, D., & Rush, A. (2016) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v30i1.10362 · Cited by 1,491
Srivastava, R. K., Greff, K., & Schmidhuber, J. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1507.06228 · Cited by 1,246
Recurrent Continuous Translation Models
Kalchbrenner, N., & Blunsom, P. (2013) · Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d13-1176 · Cited by 1,232
Theano: new features and speed improvements
Bastien, F., Lamblin, P., Pascanu, R., Bergstra, J., Goodfellow, I., Bergeron, A., Bouchard, N., Warde-Farley, D., & Bengio, Y. (2012) · arXiv · doi:10.48550/arxiv.1211.5590 · Cited by 1,005 · cited 1×
“The authors would like to thank the developers of Theano (Bastien et al., 2012).”
BackgroundSelf-citationNeural Machine Translation of Rare Words with Subword Units
Sennrich, R., Haddow, B., & Birch, A. (2016) · Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p16-1162 · Cited by 974 · cited 2×
“We represent the source side as a sequence of subwords extracted using byte-pair encoding from Sennrich et al. (2015), and vary the target side to be either a sequence of subwords or characters.”
MethodNeural Machine Translation of Rare Words with Subword Units
Sennrich, R., Haddow, B., & Birch, A. (2015) · arXiv · doi:10.48550/arxiv.1508.07909 · Cited by 950
On Using Very Large Target Vocabulary for Neural Machine Translation
Jean, S., Cho, K., Memisevic, R., & Bengio, Y. (2015) · Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.3115/v1/p15-1001 · Cited by 862
Self-citationBetter Word Representations with Recursive Neural Networks for Morphology
Luong, M.-T., Socher, R., & Manning, C. D. (2013) · Conference on Computational Natural Language Learning · Cited by 786
Gated Feedback Recurrent Neural Networks
Chung, J., Gulcehre, C., Cho, K., & Bengio, Y. (2015) · arXiv · doi:10.48550/arxiv.1502.02367 · Cited by 583 · cited 2×
“ecurrent neural network with gated recurrent units (GRU). We call this decoder a base decoder. Second, we build a novel two-layer recurrent neural network, inspired by the gated-feedback network from Chung et al. (2015), called a biscale recurrent neural network. We design this network to facilitate capturing two time…”
BackgroundMethodSelf-citationLearning Character-level Representations for Part-of-Speech Tagging
dos Santos, C., & Zadrozny, B. (2014) · Cited by 519
Continuous space language models
Schwenk, H. (2007) · Computer Speech & Language · doi:10.1016/j.csl.2006.09.003 · Cited by 516
The grammar of words : an introduction to linguistic morphology
Booij, G. (2005) · Cited by 344
Unsupervised morpheme segmentation and morphology induction from text corpora using Morfessor 1.0
Creutz, M., & Lagus, K. (2005) · Cited by 263
Character-based Neural Machine Translation
Costa‐jussà, M. R., & Fonollosa, J. A. R. (2016) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p16-2058 · Cited by 261
How to Construct Deep Recurrent Neural Networks
Pascanu, R., Gülçehre, Ç., Cho, K., & Bengio, Y. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1312.6026 · Cited by 228
Self-citationAddressing the Rare Word Problem in Neural Machine TranslationPreprint
Luong, M.-T., Sutskever, I., Le, Q. V., Vinyals, O., & Zaremba, W. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1410.8206 · Cited by 226
Finding Function in Form: Compositional Character Models for Open Vocabulary Word RepresentationPreprint
Ling, W., Luís, T., Marujo, L., Astudillo, R. F., Amir, S., Dyer, C., Black, A. W., & Trancoso, I. M. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1508.02096 · Cited by 186
Character-based Neural Machine Translation
Ling, W., Trancoso, I. M., Dyer, C., & Black, A. W. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1511.04586 · Cited by 179
Efficient Character-level Document Classification by Combining Convolution and Recurrent Layers
Xiao, Y., & Cho, K. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1602.00367 · Cited by 164
Self-citationChinese Word Segmentation: A Decade Review
Zhao, H. Y. (2007) · Zhongwen xinxi xuebao · Cited by 141
Vilar, D., Peter, J.-T., & Ney, H. (2007) · Proceedings of the Second Workshop on Statistical Machine Translation - StatMT '07 · doi:10.3115/1626355.1626360 · Cited by 87
Edinburgh’s Phrase-based Machine Translation Systems for WMT-14
Durrani, N., Haddow, B., Koehn, P., & Heafield, K. (2014) · Conference on Machine Translation (WMT) · doi:10.3115/v1/w14-3309 · Cited by 45
Multilingual Language Processing From BytesPreprint
Gillick, D., Brunk, C., Vinyals, O., et al. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1512.00103 · Cited by 24
Edinburgh's Syntax-Based Systems at WMT 2015
Williams, P. J., Sennrich, R., Nădejde, M., Huck, M., & Koehn, P. (2015) · Conference on Machine Translation (WMT) · doi:10.18653/v1/w15-3024 · Cited by 23
EU-BRIDGE MT: Combined Machine Translation
Freitag, M., Peitz, S., Wuebker, J., Ney, H., Huck, M., Sennrich, R., Durrani, N., Nădejde, M., Williams, P. J., Koehn, P., et al. (2014) · Conference on Machine Translation (WMT) · doi:10.3115/v1/w14-3310 · Cited by 19
The Edinburgh/JHU Phrase-based Machine Translation Systems for WMT~2015
Haddow, B., Huck, M., Birch, A., Bogoychev, N., & Koehn, P. (2015) · Proceedings of the Tenth Workshop on Statistical Machine Translation · doi:10.18653/v1/w15-3013 · Cited by 14
Substring-based machine translation
Neubig, G., Watanabe, T., Mori, S., & Kawahara, T. (2013) · Machine Translation · doi:10.1007/s10590-013-9136-6 · Cited by 12
Abu-MaTran at WMT 2015 Translation Task: Morphological Segmentation and Web Crawling
Rubino, R., Pirinen, T. A., Esplà-Gomis, M., Ljubešić, N., Rojas, S. O., Papavassiliou, V., Prokopidis, P., & Toral, A. (2015) · Conference on Machine Translation (WMT) · doi:10.18653/v1/w15-3022 · Cited by 9
Cited by
A Survey of the Usages of Deep Learning for Natural Language Processing
Otter, D. W., Medina, J. R., & Kalita, J. K. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.2979670 · Cited by 1,836
Massive Exploration of Neural Machine Translation Architectures
Britz, D., Goldie, A., Luong, M.-T., & Le, Q. (2017) · arXiv · doi:10.48550/arxiv.1703.03906 · Cited by 487
Massive Exploration of Neural Machine Translation Architectures
Britz, D., Goldie, A., Luong, M.-T., & Le, Q. (2017) · Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d17-1151 · Cited by 482
Neural Machine Translation in Linear Time
Kalchbrenner, N., Espeholt, L., Simonyan, K., Oord, A. V. D., Graves, A., & Kavukcuoglu, K. (2016) · arXiv · doi:10.48550/arxiv.1610.10099 · Cited by 414
A Convolutional Encoder Model for Neural Machine Translation
Gehring, J., Auli, M., Grangier, D., & Dauphin, Y. (2017) · Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p17-1012 · Cited by 405
Metadata
- Authors
- Chung, J., Cho, K., & Bengio, Y.
- Venue
- Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
- Published
- 2016
- Pages
- 1693-1703
- OpenAlex
- W2311921240
- Cited by
- 309
- References
- 40
- FWCI
- 37.39
- Type
- conference paper
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.