Can Neural Machine Translation be Improved with User Feedback?
Open accessAbstract
Julia Kreutzer, Shahram Khadivi, Evgeny Matusov, Stefan Riezler. Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 3 (Industry Papers). 2018.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Even human labelers and researchers have shown disagreements in annotation.
“Even human labelers and researchers have shown disagreements in annotation [86].”
From RLHF Deciphered: A Critical Analysis of Reinforcement Learning from Human Feedback for LLMs (2026) · cites this paper as [86]
Section 3.1 (page unavailable) reports low agreement among expert annotators rating translations (Fleiss’ κ = 0.12) and an example of disagreement between users and experts. This supports disagreement among human labelers, but the paper does not identify the annotators as researchers or report disagreement among researchers. (LLM, conf=0.83)
Experimental — accuracy not yet measuredRating feedback is easy to specify, but inter-annotator agreement is often low because of the subjective nature of the task.
“While rating feedback is easy to specify, often inter-annotator agreement is low because of the subjective nature of the task [86].”
From RLHF Deciphered: A Critical Analysis of Reinforcement Learning from Human Feedback for LLMs (2026) · cites this paper as [86]
Preference-based feedback provides no further improvement in inter-annotator agreement when compared to rating feedback.
“Moreover, preference-based feedback provides no further improvement in terms of inter-annotator agreement when compared to rating feedback [86].”
From RLHF Deciphered: A Critical Analysis of Reinforcement Learning from Human Feedback for LLMs (2026) · cites this paper as [86]
Section 3.1 reports inter-annotator agreement for five-star ratings and for a binary task in which experts agreed or disagreed with user ratings. Our copy does not report a preference-based feedback agreement measure or compare it with rating feedback (searched: preference-based feedback, pairwise ranking, inter-annotator agreement). (LLM, conf=0.76)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · see citing works
Convolutional Neural Networks for Sentence Classification
Kim, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1181 · Cited by 14,831 · see citing works
Measuring nominal scale agreement among many raters.
Fleiss, J. L. (1971) · Psychological Bulletin · doi:10.1037/h0031619 · Cited by 8,733 · see citing works
Moses: open source toolkit for statistical machine translation
Koehn, P., Zens, R., Dyer, C., Bojar, O., Constantin, A., Herbst, E., Hoang, H., Birch, A., Callison-Burch, C., Federico, M., et al. (2007) · Proceedings of the 45th Annual Meeting of the ACL on Interactive Poster and Demonstration Sessions - ACL '07 · doi:10.3115/1557769.1557821 · Cited by 4,735 · cited 1× · see citing works
“For significance tests we used approximate randomization (Clark et al., 2011), for BLEU score evaluation (lowercased) the multi-bleu script of the Moses decoder (Koehn et al., 2007), for TER computation the tercom tool (Snover et al., 2006).”
A Study of Translation Edit Rate with Targeted Human Annotation
Snover, M., Dorr, B. J., Schwartz, R. M., Micciulla, L., & Makhoul, J. I. (2006) · Cited by 2,380 · see citing works
On Using Very Large Target Vocabulary for Neural Machine Translation
Jean, S., Cho, K., Memisevic, R., & Bengio, Y. (2015) · Annual Meeting of the Association for Computational Linguistics · doi:10.3115/v1/p15-1001 · Cited by 839 · see citing works
News from OPUS — A collection of multilingual parallel corpora with tools and interfaces
Tiedemann, J. (2009) · Amsterdam studies in the theory and history of linguistic science. Series 4, Current issues in linguistic theory · doi:10.1075/cilt.309.19tie · Cited by 541 · see citing works
Better Hypothesis Testing for Statistical Machine Translation: Controlling for Optimizer Instability
Clark, J. H., Dyer, C., Lavie, A., & Smith, N. A. (2018) · Figshare · doi:10.1184/r1/6473090 · Cited by 443 · see citing works
Eligibility Traces for Off-Policy Policy Evaluation
Precup, D., Sutton, R. S., & Singh, S. P. (2000) · ScholarWorks@UMassAmherst (University of Massachusetts Amherst) · Cited by 439 · see citing works
An Actor-Critic Algorithm for Sequence Prediction
Bahdanau, D., Brakel, P., Xu, K., Goyal, A., Lowe, R., Pineau, J., Courville, A., & Bengio, Y. (2016) · arXiv · doi:10.48550/arxiv.1607.07086 · Cited by 423 · see citing works
Stanford neural machine translation systems for spoken language domains.
Luong, M.-T., & Manning, C. D. (2015) · Cited by 402 · see citing works
Doubly Robust Policy Evaluation and LearningPreprint
Dudı́k, M., Langford, J. C., & Li, L. (2011) · arXiv (Cornell University) · doi:10.48550/arxiv.1103.4601 · Cited by 396 · see citing works
Minimum Risk Training for Neural Machine Translation
Shen, S., Cheng, Y., He, Z., He, W. P., Wu, H. R., Sun, M., & Liu, Y. (2016) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p16-1159 · Cited by 377 · see citing works
The self-normalized estimator for counterfactual learning
Swaminathan, A., & Joachims, T. (2015) · Cited by 190 · see citing works
Learning from Logged Implicit Exploration DataPreprint
Strehl, A. L., Langford, J. C., Kakade, S. M., & Li, L. (2010) · arXiv (Cornell University) · doi:10.48550/arxiv.1003.0120 · Cited by 154 · see citing works
Reinforcement Learning for Bandit Neural Machine Translation with Simulated Human Feedback
Nguyen, K., Daumé III, H., & Boyd-Graber, J. (2017) · Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d17-1153 · Cited by 110 · see citing works
Langford, J. C., Strehl, A. L., & Wortman, J. (2008) · International Conference on Machine Learning · doi:10.1145/1390156.1390223 · Cited by 109 · see citing works
Do Convolutional Networks need to be Deep for Text Classification ?Preprint
Le, H. T., Cerisara, C., & Denis, A. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1707.04108 · Cited by 84 · see citing works
Bandit Structured Prediction for Neural Sequence-to-Sequence Learning
Kreutzer, J., Sokolov, A., & Riezler, S. (2017) · Annual Meeting of the Association for Computational Linguistics (ACL) · doi:10.18653/v1/p17-1138 · Cited by 37 · see citing works
Self-citationLearning Structured Predictors from Bandit Feedback for Interactive NLP
Sokolov, A., Kreutzer, J., Lo, C., & Riezler, S. (2016) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p16-1152 · Cited by 28 · see citing works
Self-citationLawrence, C., Sokolov, A., & Riezler, S. (2017) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d17-1272 · Cited by 28 · see citing works
Self-citationUsing Images to Improve Machine-Translating E-Commerce Product Listings.
Calixto, I., Stein, D., Matusov, E., Lohar, P., Castilho, S., & Way, A. (2017) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/e17-2101 · Cited by 24 · see citing works
Self-citationCounterfactual Learning for Machine Translation: Degeneracies and SolutionsPreprint
Lawrence, C., Gajane, P., & Riezler, S. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.08621 · Cited by 6 · see citing works
Self-citation
Flagged first, then by citations
5 of the 28 references couldn’t be matched to a known paper.
Research works printed but not linked: 10
- Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Ben- gio. 2015. Neural machine translation by jointly learning to align and translate. In Third Interna- tional Conference on Learning Representations. San Diego, California.
- Léon Bottou, Jonas Peters, Joaquin Quiñonero- Candela, Denis X. Charles, D. Max Chickering, Elon Portugaly, Dipanakar Ray, Patrice Simard, and Ed Snelson. 2013. Counterfactual reasoning and learning systems: The example of computational ad- vertising. Journal of Machine Learning Research 14:3207-3260.
- Y Gal and Z Ghahramani. 2016. Dropout as a bayesian approximation: Representing model uncertainty in deep learning. In 33rd International Conference on Machine Learning. New York City, NY.
- Nan Jiang and Lihong Li. 2016. Doubly robust off- policy value evaluation for reinforcement learning. In Proceedings of the 33rd International Conference on Machine Learning (ICML). New York, NY.
- Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016a. Edinburgh neural machine translation sys- tems for wmt 16. In Proceedings of the First Con- ference on Machine Translation. Berlin, Germany.
- Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016b. Improving neural machine translation mod- els with monolingual data. In Proceedings of the 54th Annual Meeting of the Association for Compu- tational Linguistics. Berlin, Germany.
- Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016c. Neural machine translation of rare words with subword units. In Proceedings of the 54th An- nual Meeting of the Association for Computational Linguistics. Berlin, Germany.
- Artem Sokolov, Julia Kreutzer, Stefan Riezler, and Christopher Lo. 2016b. Stochastic structured pre- diction under bandit feedback. In Advances in Neural Information Processing Systems. Barcelona, Spain.
- Artem Sokolov, Julia Kreutzer, Kellen Sunder- land, Pavel Danchenko, Witold Szymaniak, Hagen Fürstenau, and Stefan Riezler. 2017. A shared task on bandit learning for machine translation. In Proceedings of the Second Conference on Machine Translation. Copenhagen, Denmark.
- Yonghui Wu, Mike Schuster, Zhifeng Chen, Quoc V Le, Mohammad Norouzi, Wolfgang Macherey, Maxim Krikun, Yuan Cao, Qin Gao, Klaus Macherey, et al. 2016. Google's neural ma- chine translation system: Bridging the gap between human and machine translation. arXiv preprint arXiv:1609.08144 . · arXiv
Cited by
Sorted by citations
A Brief Overview of ChatGPT: The History, Status Quo and Potential Future Development
Wu, T., He, S., Liu, J., Sun, S., Liu, K., Han, Q.-L., & Tang, Y. (2023) · IEEE/CAA Journal of Automatica Sinica · doi:10.1109/jas.2023.123618 · Cited by 1,452 · see citing works
Learning to summarize from human feedback
Stiennon, N., Ouyang, L., Wu, J., Ziegler, D. M., Lowe, R., Voss, C., Radford, A., Amodei, D., & Christiano, P. (2020) · arXiv · doi:10.48550/arxiv.2009.01325 · Cited by 550 · see citing works
Pan, L., Saxon, M., Xu, W., Nathani, D., Wang, X., & Wang, W. Y. (2024) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00660 · Cited by 86 · see citing works
Putting Humans in the Natural Language Processing Loop: A SurveyPreprint
Wang, Z. J., Choi, D., Xu, S., & Yang, D. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.04044 · Cited by 54 · see citing works
Franceschelli, G., & Musolesi, M. (2024) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.15278 · Cited by 52 · see citing works
Bridging the Gap: A Survey on Integrating (Human) Feedback for Natural Language Generation
Fernandes, P., Madaan, A., Liu, E., Farinhas, A., Martins, P. H., Bertsch, A., de Souza, J. G. C., Zhou, S., Wu, T., Neubig, G., et al. (2023) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00626 · Cited by 57 · see citing works
Challenges of Neural Machine Translation for Short Texts
Wan, Y. X., Yang, B., Wong, D. F., Chao, L. S., Yao, L., Zhang, H., & Chen, B. (2022) · Computational Linguistics · doi:10.1162/coli_a_00435 · Cited by 39 · see citing works
Kreutzer, J., Uyheng, J., & Riezler, S. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p18-1165 · Cited by 38 · see citing works
Self-citationA user-study on online adaptation of neural machine translation to human post-edits
Karimova, S., Simianer, P., & Riezler, S. (2018) · Machine Translation · doi:10.1007/s10590-018-9224-8 · Cited by 29 · see citing works
Self-citationExploiting Neural Query Translation into Cross Lingual Information RetrievalPreprint
Yao, L., Yang, B., Zhang, H., Luo, W., & Chen, B. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.13659 · Cited by 29 · see citing works
ChatGPT vs state-of-the-art models: a benchmarking study in keyphrase generation task
Martínez-Cruz, R., López-López, Á. J., & Portela, J. (2024) · Applied Intelligence · doi:10.1007/s10489-024-05901-4 · Cited by 38 · see citing works
Prompt-Driven Neural Machine Translation
Li, Y., Yin, Y., Li, J., & Zhang, Y. (2022) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2022.findings-acl.203 · Cited by 27 · see citing works
Reliability Testing for Natural Language Processing Systems
Tan, S., Joty, S., Baxter, K., Taeihagh, A., Bennett, G. A., & Kan, M. (2021) · doi:10.18653/v1/2021.acl-long.321 · Cited by 27 · see citing works
Active Learning Approaches to Enhancing Neural Machine Translation: An Empirical Study
Zhao, Y.-K., Zhang, H., Zhou, S.-C., & Zhang, Z.-H. (2020) · Findings · doi:10.18653/v1/2020.findings-emnlp.162 · Cited by 45 · see citing works
Chen, Y., Sikka, K., Cogswell, M., Ji, H., & Divakaran, A. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01350 · Cited by 29 · see citing works
Preference-based interactive multi-document summarisation
Gao, Y., Meyer, C. M., & Gurevych, I. (2019) · Information Retrieval · doi:10.1007/s10791-019-09367-8 · Cited by 20 · see citing works
Machine Translation for Machines: the Sentiment Classification Use Case
Tebbifakhr, A., Bentivogli, L., Negri, M., & Turchi, M. (2019) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d19-1140 · Cited by 15 · see citing works
Intelligent Narrative Summaries: From Indicative to Informative Summarization
Ghodratnama, S., Beheshti, A., Zakershahrak, M., & Sobhanmanesh, F. (2021) · Big Data Research · doi:10.1016/j.bdr.2021.100257 · Cited by 16 · see citing works
Diversity driven Query Rewriting in Search Advertising
Mohankumar, A. K., Begwani, N., & Singh, A. P. (2021) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3447548.3467202 · Cited by 16 · see citing works
Crowdsourced Targeted Feedback Collection for Multicriteria Data Source Selection
Ríos, J. C. C., Paton, N. W., Fernandes, A. A. A., Abel, E., & Keane, J. A. (2019) · Journal of Data and Information Quality · doi:10.1145/3284934 · Cited by 15 · see citing works
Tan, X., Shi, S., Qiu, X., Qu, C., Qi, Z., Xu, Y., & Qi, Y. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-industry.62 · Cited by 15 · see citing works
Bandits Don’t Follow Rules: Balancing Multi-Facet Machine Translation with Multi-Armed Bandits
Kreutzer, J., Vilar, D., & Sokolov, A. (2021) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2021.findings-emnlp.274 · Cited by 11 · see citing works
Self-citationOffline Reinforcement Learning from Human Feedback in Real-World Sequence-to-Sequence Tasks
Kreutzer, J., Riezler, S., & Lawrence, C. (2021) · Workshop on Structured Prediction for NLP (SPNLP) · doi:10.18653/v1/2021.spnlp-1.4 · Cited by 10 · see citing works
Self-citationFederated Learning with Noisy User Feedback
Sharma, R. A., Ramakrishna, A., MacLaughlin, A., Rumshisky, A., Majmudar, J., Chung, C., Avestimehr, S., & Gupta, R. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-main.196 · Cited by 9 · see citing works
HumanEvalComm: Benchmarking the Communication Competence of Code Generation for LLMs and LLM Agent
Wu, J. J., & Fard, F. H. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3715109 · Cited by 10 · see citing works
A Survey on the Recent Advancements in Human-Centered Dialog Systems
Oruche, R., Goruganthu, S. K., Akula, R., Cheng, X., Goni, M. A., Shibo, B. W., Kee, K. F., Zampieri, M., & Calyam, P. P. (2025) · ACM Computing Surveys · doi:10.1145/3729220 · Cited by 9 · see citing works
Continual Learning for Grounded Instruction Generation by Observing Human Following Behavior
Kojima, N., Suhr, A., & Artzi, Y. (2021) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00428 · Cited by 9 · see citing works
Ghodratnama, S., Beheshti, A., & Zakershahrak, M. (2023) · IEEE International Conference on Web Services (ICWS) · doi:10.1109/icws60048.2023.00040 · Cited by 8 · see citing works
Modelling Latent Translations for Cross-Lingual TransferPreprint
Ponti, E. M., Kreutzer, J., Vulić, I., & Reddy, S. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2107.11353 · Cited by 14 · see citing works
Self-citationA hierarchical probabilistic underwater image enhancement model with reinforcement tuning
Song, W., Shen, Z., Zhang, M., Wang, Y., & Liotta, A. (2024) · Journal of Visual Communication and Image Representation · doi:10.1016/j.jvcir.2024.104052 · Cited by 6 · see citing works
Learning from Human Feedback: Challenges for Real-World Reinforcement Learning in NLP.Preprint
Kreutzer, J., Riezler, S., & Lawrence, C. (2020) · arXiv (Cornell University) · Cited by 5 · see citing works
Self-citationA Task-Risk Consistency Object Detection Framework Based on Deep Reinforcement Learning
Wen, J., Liu, H., & Li, J. (2023) · Remote Sensing · doi:10.3390/rs15205031 · Cited by 4 · see citing works
Liu, J., Liu, J., Wang, Q., Wang, J., Cai, X., Zhao, D., Wang, R., & Yan, R. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.578 · Cited by 4 · see citing works
Measuring the Influence of Incorrect Code on Test Generation
Huang, D., Zhang, J. M., Harman, M., Du, M., & Cui, H. (2026) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3744916.3764532 · Cited by 7 · see citing works
Lost in Machine Translation: The Sociocultural Implications of Language Technologies in Nigeria
Olojo, S., Zakrzewski, J., Smart, A., van Liemt, E., Miceli, M., Ebinama, A., & Amugongo, L. M. (2025) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3715275.3732153 · Cited by 6 · see citing works
Lodoen, S., & Orchard, A. (2025) · IEEE International Symposium on Ethics in Engineering, Science, and Technology (ETHICS) · doi:10.1109/ethics65148.2025.11098367 · Cited by 4 · see citing works
Optimizing Abstractive Arabic Summarization via RLHF and DPO with Llama 2
Kahla, M., & Yang, Z. G. (2026) · Acta Cybernetica · doi:10.14232/actacyb.316434 · Cited by 1 · see citing works
Holistic Multi-layered System Design for Human-Centered Dialog Systems
Oruche, R., Akula, R., Goruganthu, S. K., & Calyam, P. P. (2024) · IEEE International Conference on Human-Machine Systems (ICHMS) · doi:10.1109/ichms59971.2024.10555807 · Cited by 1 · see citing works
Multi-Action Dialog Policy Learning from Logged User Feedback
Zhang, S., Zhao, J., Wang, P., Wang, T., Liang, Z., Tao, J., Huang, Y., & Feng, J. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i11.26636 · Cited by 1 · see citing works
Topic-time Heatmaps for Human-in-the-loop Topic Detection and TrackingPreprint
Beeferman, D., & Jiang, H. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.07337 · Cited by 1 · see citing works
Machine Translation for Machines: the Sentiment Classification Use CasePreprint
Tebbifakhr, A., Bentivogli, L., Negri, M., & Turchi, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.00478
Quantifying Information of Tokens for Simple and Flexible Simultaneous Machine Translation
Lee, D., Park, M., & Lee, B.-J. (2023) · Conference on Computational Natural Language Learning · doi:10.18653/v1/2023.conll-1.14
Enhancing Neural Machine Translation with Direct Preference Optimization Using Human Feedback
Coroian, P.-S., Kotsis, G., & Khalil, I. (2025) · Communications in computer and information science · doi:10.1007/978-981-96-4291-5_32
Using LoRA and Reinforcement Learning in Interactive Machine Translation
Navarro, Á., & Casacuberta, F. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-99565-1_32
CLAIM: Mitigating Catastrophic Forgetting in Continual Instruction Fine-Tuning Large Language Models
Qiu, X., Shi, S., Hao, T., & Tan, X. (2026) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2026.3674105
Nguyen, T., Yoon, H. S., Luu, T. M., Yoon, E., & Yoo, C. D. (2026) · IEEE Access · doi:10.1109/access.2026.3723028
Haputhanthri, L., & Fernando, M. (2026) · 2026 2nd International Conference on Advanced Machine Learning and Data Science (AMLDS) · doi:10.1109/amlds69611.2026.11723732
Recursively Summarizing Books with Human Feedback
Wu, J., Ouyang, L., Ziegler, D. M., Stiennon, N., Lowe, R., Leike, J., & Christiano, P. (2021) · arXiv · doi:10.48550/arxiv.2109.10862 · Cited by 76 · see citing works
- Authors
- Kreutzer, J., Khadivi, S., Matusov, E., & Riezler, S.
- Affiliations
- Heidelberg University (DE)
- eBay (Ireland) (IE)
- ORCID
- Khadivi, Shahram
- Venue
- Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 3 (Industry Papers)
- Published
- 2018
- Pages
- 92-105
- OpenAlex
- W2798047776
- Cited by
- 49
- References
- 28
- Citations by year
- 2026: 5 · 2025: 6 · 2024: 6 · 2023: 8 · 2022: 3 · 2021: 11 · 2020: 4 · 2019: 4 · 2018: 2
- FWCI
- 2.13
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Deutsche Forschungsgemeinschaft
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Keywords
- neural machine translation, user feedback, human-in-the-loop translation, translation quality improvement
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
