Scientific Credibility of Machine Translation Research: A Meta-Evaluation of 769 Papers
Open accessAbstract
Benjamin Marie, Atsushi Fujita, Raphael Rubino. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 2021.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Heuristic-based metrics such as BLEU and ROUGE have flaws, and underdocumented hyperparameters can present reproducibility challenges.
“Nevertheless these heuristicbased metrics have flaws (Callison-Burch et al., 2006) and underdocumented hyperparameters can present reproducibility challenges (Marie et al., 2021).”
From Lessons from the Trenches on Reproducible Evaluation of Language Models (2024) · cites this paper as Marie et al., 2021
Most works on language modeling do not perform statistical significance testing (Marie et al., 2021).
“Most works on language modeling do not perform statistical significance testing (Marie et al., 2021).”
From Lessons from the Trenches on Reproducible Evaluation of Language Models (2024) · cites this paper as Marie et al., 2021
Comparing results across papers can be misleading due to a wide range of experimental differences, including prompts, sample size, metric calculation, and more (Marie et al., 2021).
“Comparing results across papers can be misleading due to a wide range of experimental differences, including prompts, sample size, metric calculation, and more (Marie et al., 2021).”
From Lessons from the Trenches on Reproducible Evaluation of Language Models (2024) · cites this paper as Marie et al., 2021
Cites
BLEU: a method for automatic evaluation of machine translation
Papineni, K., Roukos, S., Ward, T., & Zhu, W.-J. (2001) · Proceedings of the 40th Annual Meeting on Association for Computational Linguistics - ACL '02 · doi:10.3115/1073083.1073135 · Cited by 21,344 · see citing works
(2005) · Cited by 4,152 · see citing works
Moving to a World Beyond “ <i>p</i> < 0.05”
Wasserstein, R. L., Schirm, A. L., & Lazar, N. A. (2019) · The American Statistician · doi:10.1080/00031305.2019.1583913 · Cited by 2,976 · see citing works
A Study of Translation Edit Rate with Targeted Human Annotation
Snover, M., Dorr, B. J., Schwartz, R. M., Micciulla, L., & Makhoul, J. I. (2006) · Cited by 2,380 · see citing works
A Call for Clarity in Reporting BLEU Scores
Post, M. (2018) · Proceedings of the Third Conference on Machine Translation: Research Papers · doi:10.18653/v1/w18-6319 · Cited by 2,112 · see citing works
Statistical Significance Tests for Machine Translation Evaluation.
Koehn, P. (2004) · Cited by 1,456 · see citing works
chrF: character n-gram F-score for automatic MT evaluation
Popović, M. (2015) · Conference on Machine Translation (WMT) · doi:10.18653/v1/w15-3049 · Cited by 1,026 · see citing works
Re-evaluating the Role of Bleu in Machine Translation Research
Callison-Burch, C., Osborne, M., & Koehn, P. (2006) · Edinburgh Research Explorer (University of Edinburgh) · Cited by 632 · see citing works
A Structured Review of the Validity of BLEU
Reiter, E. (2018) · Computational Linguistics · doi:10.1162/coli_a_00322 · Cited by 386 · see citing works
The Hitchhiker’s Guide to Testing Statistical Significance in Natural Language Processing
Dror, R., Baumer, G., Shlomov, S., & Reichart, R. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p18-1128 · Cited by 384 · see citing works
Automatic Evaluation of Translation Quality for Distant Language Pairs
Isozaki, H., Hirao, T., Duh, K., Sudoh, K., & Tsukada, H. (2010) · Cited by 286 · see citing works
Tangled up in BLEU: Reevaluating the Evaluation of Automatic Machine Translation Evaluation Metrics
Mathur, N., Baldwin, T., & Cohn, T. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.448 · Cited by 206 · see citing works
(2005) · Cited by 189 · see citing works
Findings of the 2020 Conference on Machine Translation (WMT20)
Barrault, L., Biesialska, M., Bojar, O., Costa‐jussà, M. R., Federmann, C., Graham, Y., Grundkiewicz, R., Haddow, B., Huck, M., Joanis, E., et al. (2020) · Conference on Machine Translation (WMT) · doi:10.18653/v1/2020.wmt-1.1 · Cited by 178 · see citing works
Selective Attention for Context-aware Neural Machine Translation
Maruf, S., Martins, A. F. T., & Haffari, G. (2019) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n19-1313 · Cited by 168 · see citing works
Stronger Baselines for Trustable Results in Neural Machine Translation
Denkowski, M., & Neubig, G. (2017) · doi:10.18653/v1/w17-3203 · Cited by 111 · see citing works
Dynamic Context Selection for Document-level Neural Machine Translation via Reinforcement Learning
Kang, X., Zhao, Y., Zhang, J., & Zong, C. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.175 · Cited by 54 · see citing works
A Set of Recommendations for Assessing Human–Machine Parity in Language Translation
Läubli, S., Castilho, S., Neubig, G., Sennrich, R., Shen, Q., & Toral, A. (2020) · Zurich Open Repository and Archive (University of Zurich) · doi:10.5167/uzh-188217 · Cited by 54 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Leakage and the reproducibility crisis in machine-learning-based science
Kapoor, S., & Narayanan, A. (2023) · Patterns · doi:10.1016/j.patter.2023.100804 · Cited by 1,387 · see citing works
The Flores-101 Evaluation Benchmark for Low-Resource and Multilingual Machine Translation
Goyal, N., Gao, C., Chaudhary, V., Chen, P., Wenzek, G., Ju, D., Krishnan, S., Ranzato, M., Guzmán, F., & Fan, A. (2022) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00474 · Cited by 209 · see citing works
To Ship or Not to Ship: An Extensive Evaluation of Automatic Metrics for Machine TranslationPreprint
Kocmi, T., Federmann, C., Grundkiewicz, R., Junczys-Dowmunt, M., Matsushita, H., & Menezes, A. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2107.10821 · Cited by 114 · see citing works
Democratizing neural machine translation with OPUS-MT
Tiedemann, J., Aulamo, M., Bakshandaeva, D., Boggia, M., Grönroos, S.-A., Nieminen, T., Raganato, A., Scherrer, Y., Vázquez, R., & Virpioja, S. (2023) · Language Resources and Evaluation · doi:10.1007/s10579-023-09704-w · Cited by 81 · see citing works
x comet : Transparent Machine Translation Evaluation through Fine-grained Error Detection
Guerreiro, N. M., Rei, R., van Stigt, D., Coheur, L., Colombo, P., & Martins, A. F. T. (2024) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00683 · Cited by 108 · see citing works
A global analysis of metrics used for measuring performance in natural language processing
Blagec, K., Dorffner, G., Moradi, M., Ott, S., & Matthias Samwald, A. (2022) · doi:10.18653/v1/2022.nlppower-1.6 · Cited by 42 · see citing works
An Evaluation of ChatGPT's Translation Accuracy Using BLEU Score
Ghassemiazghandi, M. (2024) · Theory and Practice in Language Studies · doi:10.17507/tpls.1404.07 · Cited by 30 · see citing works
Considerations for meaningful sign language machine translation based on glosses
Müller, M., Jiang, Z., Moryossef, A., Rios, A., & Ebling, S. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-short.60 · Cited by 31 · see citing works
DEMETR: Diagnosing Evaluation Metrics for Translation
Karpinska, M., Raj, N., Thai, K., Song, Y., Gupta, A., & Iyyer, M. (2022) · Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2022.emnlp-main.649 · Cited by 22 · see citing works
A Framework for Cluster and Classifier Evaluation in the Absence of Reference Labels
Joyce, R. J., Raff, E., & Nicholas, C. (2021) · ACM Workshop on Artificial Intelligence and Security (AISec) · doi:10.1145/3474369.3486867 · Cited by 19 · see citing works
Bidimensional Leaderboards: Generate and Evaluate Language Hand in Hand
Kasai, J., Sakaguchi, K., Bras, R. J. L., Dunagan, L., Morrison, J., Fabbri, A. R., Choi, Y., & Smith, N. A. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-main.259 · Cited by 19 · see citing works
Grusky, M. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.107 · Cited by 16 · see citing works
UScore: An Effective Approach to Fully Unsupervised Evaluation Metrics for Machine Translation
Belouadi, J., & Eger, S. (2023) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2023.eacl-main.27 · Cited by 15 · see citing works
Multilingual Domain Adaptation for NMT: Decoupling Language and Domain Information with AdaptersPreprint
Stickland, A. C., Bérard, A., & Nikoulina, V. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.09574 · Cited by 16 · see citing works
Few-shot Controllable Style Transfer for Low-Resource Multilingual Settings
Krishna, K., Nathani, D., Garcia, X., Samanta, B., & Talukdar, P. P. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.514 · Cited by 20 · see citing works
Reproducibility in NLP: What Have We Learned from the Checklist?
Magnusson, I., Smith, N. A., & Dodge, J. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.809 · Cited by 14 · see citing works
Beyond Triplet: Leveraging the Most Data for Multimodal Machine Translation
Zhu, Y., Sun, Z., Cheng, S., Huang, L., Wu, L., & Wang, M. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.168 · Cited by 14 · see citing works
Global Explainability of BERT-Based Evaluation Metrics by Disentangling along Linguistic Factors
Kaster, M., Zhao, W., & Eger, S. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.701 · Cited by 13 · see citing works
DataLab: A Platform for Data Analysis and Intervention
Yang, X., Fu, J., Yuan, W., Viswanathan, V., Liu, Z., Liu, Y., Neubig, G., & Liu, P. (2022) · Annual Meeting of the Association for Computational Linguistics: System Demonstrations (ACL Demo) · doi:10.18653/v1/2022.acl-demo.18 · Cited by 10 · see citing works
Reproducibility Issues for BERT-based Evaluation Metrics
Chen, Y., Belouadi, J., & Eger, S. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.192 · Cited by 10 · see citing works
Toward Human-Like Evaluation for Natural Language Generation with Error Analysis
Lu, Q., Ding, L., Xie, L., Zhang, K., Wong, D. F., & Tao, D. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.324 · Cited by 13 · see citing works
BLEURT Has Universal Translations: An Analysis of Automatic Metrics by Minimum Risk Training
Yan, Y., Wang, T., Zhao, C., Huang, S., Chen, J., & Wang, M. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.297 · Cited by 16 · see citing works
Non-Autoregressive Neural Machine Translation: A Call for Clarity
Schmidt, R. M., Pires, T., Peitz, S., & Lööf, J. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.179 · Cited by 10 · see citing works
Al-Saggaf, M. A. (2025) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3373-0060-3.ch007 · Cited by 8 · see citing works
Lu, S.-C., Xu, C., Kaur, M. N., Edelen, M. O., Pusic, A. L., & Sidey-Gibbons, C. J. (2025) · Journal of Patient-Reported Outcomes · doi:10.1186/s41687-025-00926-w · Cited by 13 · see citing works
HintedBT: Augmenting Back-Translation with Quality and Transliteration Hints
Ramnath, S., Johnson, M., Gupta, A., & Raghuveer, A. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.129 · Cited by 9 · see citing works
Diab, N. (2021) · CDELT Occasional Papers in the Development of English Education · doi:10.21608/opde.2021.208437 · Cited by 8 · see citing works
Evaluating Multiway Multilingual NMT in the Turkic LanguagesPreprint
Mirzakhalov, J., Babu, A. J. G., Kunafin, A., Wahab, A., Moydinboyev, B., Ivanova, S., Uzokova, M., Pulatova, S., Ataman, D., Kreutzer, J., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2109.06262 · Cited by 8 · see citing works
The use of residual analysis to improve the error rate accuracy of machine translation
Benko, Ľ., Munková, D., Munk, M., Benkova, L., & Hájek, P. (2024) · Scientific Reports · doi:10.1038/s41598-024-59524-3 · Cited by 6 · see citing works
The Six Conundrums of Building and Deploying Language Technologies for Social Good
Diddee, H., Bali, K., Choudhury, M., & Mukhija, N. (2022) · ACM SIGCAS/SIGCHI Conference on Computing and Sustainable Societies (COMPASS) · doi:10.1145/3530190.3534792 · Cited by 7 · see citing works
Evaluate & Evaluation on the Hub: Better Best Practices for Data and Model Measurements
von Werra, L., Tunstall, L., Thakur, A., Luccioni, A. S., Thrush, T., Piktus, A., Marty, F., Rajani, N. F., Mustar, V., & Ngo, H. (2022) · Conference on Empirical Methods in Natural Language Processing: System Demonstrations (EMNLP) · doi:10.18653/v1/2022.emnlp-demos.13 · Cited by 7 · see citing works
Automatic Scoring for Translations Based on Language Models
Wu, D., Wang, M., & Li, X. (2022) · Computational Intelligence and Neuroscience · doi:10.1155/2022/2171206 · Cited by 6 · see citing works
Experimental Standards for Deep Learning in Natural Language Processing Research
Ulmer, D., Bassignana, E., Müller-Eberstein, M., Varab, D., Zhang, M., van der Goot, R., Hardmeier, C., & Plank, B. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.196 · Cited by 6 · see citing works
Lessons from the Trenches on Reproducible Evaluation of Language Models
Biderman, S., Schoelkopf, H., Sutawika, L., Gao, L., Tow, J., Abbasi, B., Aji, A. F., Ammanamanchi, P. S., Black, S., Clive, J., et al. (2024) · arXiv · doi:10.48550/arxiv.2405.14782 · Cited by 9 · see citing works
“Most works on language modeling do not perform statistical significance testing (Marie et al., 2021).”
Morpheme-Based Neural Machine Translation Models for Low-Resource Fusion Languages
Gezmu, A. M., & Nürnberger, A. (2023) · ACM Transactions on Asian and Low-Resource Language Information Processing · doi:10.1145/3610773 · Cited by 5 · see citing works
Validity, Reliability, and Significance: Empirical Methods for NLP and Data Science
Riezler, S., & Hagmann, M. (2021) · Synthesis lectures on human language technologies · doi:10.2200/s01137ed1v01y202110hlt055 · Cited by 5 · see citing works
It Takes Two to Tango: Navigating Conceptualizations of NLP Tasks and Measurements of Performance
Subramonian, A., Yuan, X., Daumé, H., & Blodgett, S. L. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.202 · Cited by 7 · see citing works
Is NLP Ready for Standardization?
Aufrant, L. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.202 · Cited by 4 · see citing works
Post-editing of English to Russian machine translation: Issues, methods, and optimization
Khromova, A. A., & Lukmanova, R. R. (2024) · Philology Theory & Practice · doi:10.30853/phil20240138 · Cited by 4 · see citing works
Initial exploration into sarcasm and irony through machine translation
Chia, Z. L., Ptaszynski, M. E., Karpinska, M., Eronen, J., & Masui, F. (2024) · Natural Language Processing Journal · doi:10.1016/j.nlp.2024.100106 · Cited by 7 · see citing works
EffEval: A Comprehensive Evaluation of Efficiency for MT Evaluation Metrics
Larionov, D., Grünwald, J., Leiter, C., & Eger, S. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.7 · Cited by 5 · see citing works
Did AI get more negative recently?
Beese, D., Altunbaş, B., Güzeler, G., & Eger, S. (2023) · Royal Society Open Science · doi:10.1098/rsos.221159 · Cited by 4 · see citing works
Bidirectional Translation of ASL and English Using Machine Vision and CNN and Transformer Networks
Amiruzzaman, S., Amiruzzaman, M., Batchu, R. M., Dracup, J., Pham, A., Crocker, B., Ngo, L. B., & Dewan, M. A. A. (2026) · Computers · doi:10.3390/computers15010020 · Cited by 3 · see citing works
When does Parameter-Efficient Transfer Learning Work for Machine Translation?
Üstün, A., & Stickland, A. C. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.540 · Cited by 4 · see citing works
Ahmed, N., Khuhro, M. A., Khan, A., Dawood, M., Dootio, M. A., & Jan, N. U. (2023) · Pakistan Journal of Engineering Technology & Science · doi:10.22555/pjets.v11i2.1039 · Cited by 3 · see citing works
Few-shot Controllable Style Transfer for Low-Resource Settings: A Study in Indian Languages.Preprint
Krishna, K., Nathani, D., García, X., Samanta, B., & Talukdar, P. P. (2021) · arXiv (Cornell University) · Cited by 3 · see citing works
Han, L., Sorokina, I., Erofeev, G., & Gladkoff, S. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2108.09484 · Cited by 3 · see citing works
Experimental Standards for Deep Learning in Natural Language Processing ResearchPreprint
Ulmer, D., Bassignana, E., Müller-Eberstein, M., Varab, D., Zhang, M., van der Goot, R., Hardmeier, C., & Plank, B. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2204.06251 · Cited by 5 · see citing works
Quantifying Variance in Evaluation Benchmarks
Madaan, L., Singh, A. K., Schaeffer, R., Poulton, A., Koyejo, S., Stenetorp, P., Narang, S., & Hupkes, D. (2024) · arXiv · doi:10.48550/arxiv.2406.10229 · Cited by 2 · see citing works
“Before the era of chat large language models, Marie et al. (2021) conducted a large scale meta-analysis of 769 research papers published from 2010 to 2020 and identified troubling trends, including one that partially motivates our work: models are frequently declared superior to competitors based…”
Improving the Performance of Image Captioning Models Trained on Small Datasets
du Plessis, M., & Brink, W. (2022) · Communications in computer and information science · doi:10.1007/978-3-030-95070-5_6 · Cited by 2 · see citing works
National Institute of Information and Communications Technology
Sumita, E., Ohtake, K., & Uchimoto, K. (2022) · Journal of Natural Language Processing · doi:10.5715/jnlp.29.748 · Cited by 2 · see citing works
Towards Automatic Evaluation of NLG Tasks Using Conversational Large Language Models
Riyadh, M., & Shafiq, M. O. (2023) · IFIP advances in information and communication technology · doi:10.1007/978-3-031-34107-6_34 · Cited by 3 · see citing works
Investigating Data Variance in Evaluations of Automatic Machine Translation Metrics
Xiang, J., Li, H., Liu, Y., Liu, L., Huang, G., Lian, D., & Shi, S. (2022) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2022.findings-acl.14 · Cited by 2 · see citing works
Guidance to Pre-tokeniztion for SacreBLEU: Meta-Evaluation in KoreanPreprint
Kim, A., & Kim, J. (2022) · Preprints.org · doi:10.20944/preprints202201.0018.v1 · Cited by 2 · see citing works
Riezler, S., & Hagmann, M. (2024) · Synthesis lectures on human language technologies · doi:10.1007/978-3-031-57065-0_4 · Cited by 1 · see citing works
Vacillating Human Correlation of SacreBLEU in Unprotected Languages
Kim, A., & Kim, J. (2022) · doi:10.18653/v1/2022.humeval-1.1 · Cited by 1 · see citing works
Moniz, H., & Escartín, C. P. (2023) · Machine translation · doi:10.1007/978-3-031-14689-3_1 · Cited by 1 · see citing works
TeCS: A Dataset and Benchmark for Tense Consistency of Machine Translation
Ai, Y., He, Z., Yu, K., & Wang, R. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-short.164 · Cited by 1 · see citing works
A review of existing Machine Translation Approaches, their Challenges and Evaluation Metrics
Ahmed, N. (2023) · Pakistan Journal of Engineering Technology & Science · doi:10.22555/pjets.v11i1.1002 · Cited by 1 · see citing works
On Degrees of Freedom in Defining and Testing Natural Language Understanding
Sugawara, S., & Tsugita, S. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.861 · Cited by 1 · see citing works
Overview of NLPCC Shared Task 2: Multi-perspective Scientific Machine Reading Comprehension
Zhang, X., Zheng, H., Nie, Y., & Mao, X.-L. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-44699-3_10 · Cited by 1 · see citing works
Scientific Credibility of Machine Translation Research: A Meta-Evaluation of Evaluation Methodology
Marie, B., Fujita, A., & Rubino, R. (2021) · Journal of Natural Language Processing · doi:10.5715/jnlp.28.1312
Self-citationGroup Assignments for Project-Based Learning Using Natural Language Processing—A Feasibility Study
Kim, W., & Yoo, Y. (2022) · Applied Sciences · doi:10.3390/app12136321
Report on the 16th Symposiums of Young Researcher Association for NLP Studies
Takase, S. (2022) · Journal of Natural Language Processing · doi:10.5715/jnlp.29.1010
Interpreting Positional Information in Perspective of Word Order
Xilong, Z., Ruochen, L., Liu, J., & Xuefeng, L. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.534 · Cited by 1 · see citing works
Domain-adapting and evaluating machine translation for institutional German in South Tyrol
Contarino, A. G., & Camillis, F. D. (2023) · Studies in corpus linguistics · doi:10.1075/scl.113.10con
Learning Shortcuts: On the Misleading Promise of NLU in Language Models
Bihani, G., & Rayz, J. T. (2025) · Springer eBooks · doi:10.1007/978-3-031-67886-8_9 · Cited by 1 · see citing works
Sharou, K. A., Jamei, M. K., Khalifa, I. B., Missaoui, S., Messaoud, M. B., & Moorkens, J. (2025) · Lecture notes in computer science · doi:10.1007/978-981-96-7423-7_10
RSDiX: Lightweight and Data-Efficient VLMs for Remote Sensing through Self-Distillation
Terlizzi, A., Nazzaro, A., Bernardi, L., Bardozzo, F., & Tagliaferri, R. (2025) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn64981.2025.11228553
U-DPO: A Unified Approach to Hallucination Reduction in LLM-Based Translation
Thai, N. V., Tien, N. B. N., & Dang, N. Q. (2025) · International Conference on Mechatronics Technology (ICMT) · doi:10.1109/icmt67823.2025.11298744
Neural Approaches to Historical Words Reconstruction
Fourrier, C. (2022) · theses.fr (ABES) · doi:10.70675/647b8a58zeea0z48cdz8229zf1d7ee8b0742 · Cited by 2 · see citing works
Haq, S. U., Castilho, S., & Graham, Y. (2026) · Information · doi:10.3390/info17070689
The influence of socratic dialogue within GenAI on critical thinking and technology perception
Bereznikova, A., Theophilou, E., & Hernández‐Leo, D. (2026) · Educational Technology Research and Development · doi:10.1007/s11423-026-10693-0
Dehghani, M., Tay, Y., Gritsenko, A. A., Zhao, Z., Houlsby, N., Diaz, F., Metzler, D., & Vinyals, O. (2021) · arXiv · doi:10.48550/arxiv.2107.07002 · Cited by 48 · see citing works
- Authors
- Marie, B., Fujita, A., & Rubino, R.
- Affiliations
- National Institute of Information and Communications Technology (JP)
- ORCID
- Fujita, Atsushi
- Venue
- Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)
- Published
- 2021
- Pages
- 7297-7306
- OpenAlex
- W3174292387
- Cited by
- 75
- References
- 18
- Citations by year
- 2026: 3 · 2025: 6 · 2024: 7 · 2023: 23 · 2022: 24 · 2021: 12
- FWCI
- 7.62
- Type
- conference paper
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University) (cc-by)
- Funders
- Japan Society for the Promotion of Science (20K19879, 19H05660)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › scientometrics and bibliometrics research
- Social Sciences › Arts and Humanities › Language and Linguistics › Translation Studies and Practices
- Keywords
- machine translation, research credibility, meta-evaluation, publication quality, research quality assessment
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review