The DOI matches its Crossref record; 10 of the 10 references it cites were found in a registry or an index.
Meteor: an automatic metric for MT evaluation with high levels of correlation with human judgments
Abstract
Meteor is an automatic metric for Machine Translation evaluation which has been demonstrated to have high levels of correlation with human judgments of translation quality, significantly outperforming the more commonly used Bleu metric. It is one of several automatic metrics used in this year's shared task within the ACL WMT-07 workshop. This paper recaps the technical details underlying the metric and describes recent improvements in the metric. The latest release includes improved metric parameters and extends the metric to support evaluation of MT output in Spanish, French and German, in addition to English.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Popular evaluation methods for natural language generation such as BLEU-460, ROUGE-L61, and METEOR62 primarily compare n-grams between two documents and may not effectively capture semantic similarities.”
From Generating dermatopathology reports from gigapixel whole slide images with HistoGPT (2025) · cites this paper as [62]
In the citing paper
“BLEU computes n-gram overlap and is representative for the family of text overlap-based natural language generation metrics such as CIDEr,31 METEOR,30 and ROUGE.32”
From Evaluating progress in automatic chest X-ray radiology report generation (2023) · cites this paper as [30]
Citations
Cites
BLEU: a method for automatic evaluation of machine translation
Papineni, K., Roukos, S., Ward, T., & Zhu, W.-J. (2001) · Proceedings of the 40th Annual Meeting on Association for Computational Linguistics - ACL '02 · doi:10.3115/1073083.1073135 · Cited by 21,344
(2005) · Cited by 4,152
Self-citationMinimum error rate training in statistical machine translation
Och, F. J. (2003) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.3115/1075096.1075117 · Cited by 2,728
A Study of Translation Edit Rate with Targeted Human Annotation
Snover, M., Dorr, B. J., Schwartz, R. M., Micciulla, L., & Makhoul, J. I. (2006) · Cited by 2,380
Confidence estimation for machine translation
Blatz, J., Fitzgerald, E., Foster, G. F., Gandrabur, S., Goutte, C., Kulesza, A., Sanchís, A., & Ueffing, N. (2004) · International Conference on Computational Linguistics · doi:10.3115/1220355.1220401 · Cited by 368
Lattice-based minimum error rate training for statistical machine translation
Macherey, W., Och, F. J., Thayer, I., & Uszkoreit, J. (2008) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/1613715.1613807 · Cited by 347
Precision and recall of machine translation
Melamed, I. D., Green, R., & Turian, J. P. (2003) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.3115/1073483.1073504 · Cited by 228
The Significance of Recall in Automatic Metrics for MT Evaluation
Lavie, A., Sagae, K., & Jayaraman, S. (2004) · Lecture notes in computer science · doi:10.1007/978-3-540-30194-3_16 · Cited by 123
Self-citationCDER: Efficient MT Evaluation Using Block Movements.
Leusch, G., Ueffing, N., & Ney, H. (2006) · RWTH Publications (RWTH Aachen) · Cited by 109
(2024) · ACM eBooks · doi:10.1145/3674127 · Cited by 86
Cited by
Sorted by citations
Items 301–400 of 1,003
NOC-REK: Novel Object Captioning with Retrieved Vocabulary from External Knowledge
Vo, D. M., Chen, H., Sugimoto, A., & Nakayama, H. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01747 · Cited by 26
Maity, S., Deroy, A., & Sarkar, S. (2023) · Annual Meeting of the Forum for Information Retrieval Evaluation (FIRE) · doi:10.1145/3632754.3632755 · Cited by 23
NIST 2013 Open Handwriting Recognition and Translation (Open HaRT'13) Evaluation
Tong, A. N., Przybocki, M. A., Märgner, V., & Abed, H. E. (2014) · IAPR International Workshop on Document Analysis Systems (DAS) · doi:10.1109/das.2014.43 · Cited by 23
Neural Automatic Post-Editing Using Prior Alignment and Reranking
Pal, S., Naskar, S. K., Vela, M., Liu, Q., & van Genabith, J. (2017) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/e17-2056 · Cited by 23
Qualitative Analysis of Post-Editing for High Quality Machine Translation
Blain, F., Sénellart, J., Schwenk, H., Plitt, M., & Roturier, J. (2011) · Wolverhampton Intellectual Repository and E-Theses (University of Wolverhampton) · Cited by 23
Adaptive Spatial Location With Balanced Loss for Video Captioning
Li, L., Zhang, Y., Tang, S., Xie, L., Li, X., & Tian, Q. (2020) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2020.3045735 · Cited by 23
Khurana, K., & Deshpande, U. A. (2021) · IEEE Access · doi:10.1109/access.2021.3058248 · Cited by 26
Ahmed, S., Saif, A. F. M. S., Hanif, M. I., Shakil, M. M. N., Jaman, M. M., Mazid-Ul-Haque, M., Shawkat, S. B., Hasan, J., Sonok, B. S., Rahman, F. M. N., et al. (2021) · Applied Sciences · doi:10.3390/app12010317 · Cited by 24
A Survey on Neural Data-to-Text Generation
Lin, Y., Ruan, T., Liu, J., & Wang, H. (2023) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2023.3304385 · Cited by 25
Deep Learning Based Image Aesthetic Quality Assessment- A Review
Chounchenani, M. D., Shahbahrami, A., Hassanpour, R., & Gaydadjiev, G. N. (2025) · ACM Computing Surveys · doi:10.1145/3716820 · Cited by 27
Spatio-Temporal and Retrieval-Augmented Modeling for Chest X-Ray Report Generation
Yang, Y. Q., You, X., Zhang, K., Fu, Z., Wang, X., Ding, J., Sun, J., Zhou, Y., Huang, Q., Han, W., et al. (2025) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2025.3554498 · Cited by 25
2010 Failures in English-Czech Phrase-Based MT
Bojar, O., & Kos, K. (2010) · Institute of Formal and Applied Linguistics (ÚFAL) · Cited by 22
Code to Comment Translation: A Comparative Study on Model Effectiveness & Errors
Mahmud, J., Faisal, F., Arnob, R. I., Anastasopoulos, A., & Moran, K. (2021) · doi:10.18653/v1/2021.nlp4prog-1.1 · Cited by 27
Semi-Autoregressive Image Captioning
Yan, X., Fei, Z., Li, Z., Wang, S., Huang, Q., & Tian, Q. (2021) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3474085.3475179 · Cited by 22
GRiT: A Generative Region-to-Text Transformer for Object Understanding
Wu, J., Wang, J., Yang, Z., Gan, Z., Liu, Z., Yuan, J., & Wang, L. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72989-8_12 · Cited by 22
Regression for machine translation evaluation at the sentence level
Albrecht, J. S., & Hwa, R. (2008) · Machine Translation · doi:10.1007/s10590-008-9046-1 · Cited by 23
Semantic-filtered Soft-Split-Aware video captioning with audio-augmented feature
Xu, Y., Yang, J., & Mao, K. (2019) · Neurocomputing · doi:10.1016/j.neucom.2019.05.027 · Cited by 22
Unstructured Text Enhanced Open-Domain Dialogue System: A Systematic Survey
Ma, L., Li, M., Zhang, W., Li, J., & Liu, T. (2021) · ACM Transactions on Information Systems · doi:10.1145/3464377 · Cited by 24
Learning to Translate with Multiple Objectives
Duh, K., Sudoh, K., Wu, X., Tsukada, H., & Nagata, M. (2012) · Cited by 21
Multi-Turn Video Question Generation via Reinforced Multi-Choice Attention Network
Guo, Z., Zhao, Z., Jin, W., Wei, Z., Yang, M., Wang, N., & Yuan, N. J. (2020) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2020.3014775 · Cited by 21
Eye Tracking as a Tool for Machine Translation Error Analysis
Stymne, S., Danielsson, H., Bremin, S., Hu, H., Karlsson, J., Lillkull, A. P., & Wester, M. (2012) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/3x28wwqt9mzw · Cited by 21
Generating Easy-to-Understand Referring Expressions for Target Identifications
Tanaka, M. N., Itamochi, T., Narioka, K., Sato, I., Ushiku, Y., & Harada, T. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00589 · Cited by 25
Revisiting Back-Translation for Low-Resource Machine Translation Between Chinese and Vietnamese
Li, H., Sha, J., & Shi, C. (2020) · IEEE Access · doi:10.1109/access.2020.3006129 · Cited by 21
Reordering metrics for statistical machine translation
Birch, A. (2011) · ERA · Cited by 21
Factored neural machine translation
Martínez, M. G. (2018) · theses.fr (ABES) · doi:10.70675/6b041d7cz8d47z4eb0zaefbz4170f95dfd99 · Cited by 21
kizi, M. K. Z., & Suh, Y. (2025) · Electronics · doi:10.3390/electronics14153095 · Cited by 22
Generating Descriptions of Entity Relationships
Voskarides, N., Meij, E., & de Rijke, M. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-56608-5_25 · Cited by 20
Posterior Control of Blackbox Generation
Li, X. L., & Rush, A. M. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.243 · Cited by 20
PIE: A Parallel Idiomatic Expression Corpus for Idiomatic Sentence Generation and Paraphrasing
Zhou, J., Gong, H., & Bhat, S. (2021) · Workshop on Multiword Expressions (MWE) · doi:10.18653/v1/2021.mwe-1.5 · Cited by 20
Weakly Guided Hierarchical Encoder-Decoder Network for Brain CT Report Generation
Yang, S., Ji, J., Zhang, X., Liu, Y., & Wang, Z. (2021) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm52615.2021.9669626 · Cited by 20
Du, Z., & Hashimoto, K. (2025) · Asian-Pacific Journal of Second and Foreign Language Education · doi:10.1186/s40862-025-00334-z · Cited by 20
Generation of Compound Words in Statistical Machine Translation into Compounding Languages
Stymne, S., Cancedda, N., & Ahrenberg, L. (2013) · Computational Linguistics · doi:10.1162/coli_a_00162 · Cited by 20
Seljan, S., Bakarić, M. B., & Kučiš, V. (2011) · Repozitorij Filozofskog fakulteta u Zagrebu' at University of Zagreb (University of Zagreb) · Cited by 20
Costa‐jussà, M. R. (2017) · Workshop on NLP for Similar Languages, Varieties and Dialects (VarDial) · doi:10.18653/v1/w17-1207 · Cited by 20
Dual Learning for Visual Question Generation
Xu, X., Song, J., Lu, H., He, L., Yang, Y., & Shen, F. (2018) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme.2018.8486475 · Cited by 20
Improving Context Modelling in Multimodal Dialogue Generation
Agarwal, S., Dušek, O., Konstas, I., & Rieser, V. (2018) · International Conference on Natural Language Generation · doi:10.18653/v1/w18-6514 · Cited by 23
Neural Generation for Czech: Data and Baselines
Dušek, O., & Jurčíček, F. (2019) · International Conference on Natural Language Generation · doi:10.18653/v1/w19-8670 · Cited by 21
Adaptive Attention-based High-level Semantic Introduction for Image Caption
Liu, X., & Xu, Q. (2020) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3409388 · Cited by 20
Beyond VQA: Generating Multi-word Answers and Rationales to Visual Questions
Dua, R., Kancheti, S. S., & Balasubramanian, V. N. (2021) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw53098.2021.00178 · Cited by 20
Attention Is Indeed All You Need: Semantically Attention-Guided Decoding for Data-to-Text NLG
Juraska, J., & Walker, M. A. (2021) · International Conference on Natural Language Generation · doi:10.18653/v1/2021.inlg-1.45 · Cited by 20
Processing of Swedish Compounds for Phrase-Based Statistical Machine Translation
Stymne, S., & Holmqvist, M. (2008) · Cited by 20
Statistical Machine Translation
Specia, L. (2014) · IGI Global eBooks · doi:10.4018/978-1-4666-6042-7.ch043 · Cited by 20
Exploring Benefits of Transfer Learning in Neural Machine TranslationPreprint
Kocmi, T. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2001.01622 · Cited by 23
Response Generation with Context-Aware Prompt LearningPreprint
Gu, X., Yoo, K. M., & Lee, S.-W. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.02643 · Cited by 26
Video Captioning Based on Channel Soft Attention and Semantic Reconstructor
Zhou, L., & Huang, Y. (2021) · Future Internet · doi:10.3390/fi13020055 · Cited by 19
Goldberg, A. B., Zhu, X., Dyer, C. R., Eldawy, M., & Heng, L. (2008) · Conference on Computational Natural Language Learning · doi:10.3115/1596324.1596345 · Cited by 19
Laith, S., Taghreed, M., & Mohammed, N. (2015) · International Journal of Advanced Computer Science and Applications · doi:10.14569/ijacsa.2015.061128 · Cited by 19
Overview of the seventh Dialog System Technology Challenge: DSTC7
D’Haro, L. F., Yoshino, K., Hori, C., Marks, T. K., Polymenakos, L., Kummerfeld, J. K., Galley, M., & Gao, X. (2020) · Computer Speech & Language · doi:10.1016/j.csl.2020.101068 · Cited by 19
FFCI: A Framework for Interpretable Automatic Evaluation of Summarization
Koto, F., Baldwin, T., & Lau, J. H. (2022) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.13167 · Cited by 25
Combining Confidence Estimation and Reference-based Metrics for Segment-level MT Evaluation
Specia, L., & Giménez, J. (2010) · Conference of the Association for Machine Translation in the Americas · Cited by 19
Machine Translation Evaluation and Optimization
Dorr, B. J., Olive, J. P., McCary, J., & Christianson, C. (2011) · Springer eBooks · doi:10.1007/978-1-4419-7713-7_5 · Cited by 19
Quda: Natural Language Queries for Visual Data AnalyticsPreprint
Fu, S., Xiong, K., Ge, X., Tang, S., Chen, W., & Wu, Y. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2005.03257 · Cited by 28
Control Prefixes for Text Generation.Preprint
Clive, J., Cao, K., & Rei, M. (2021) · arXiv (Cornell University) · Cited by 19
Human and Automatic Evaluation of English to Hindi Machine Translation Systems
Joshi, N., Darbari, H., & Mathur, I. (2012) · Advances in intelligent and soft computing · doi:10.1007/978-3-642-30157-5_42 · Cited by 18
RED: A Reference Dependency Based MT Evaluation Metric
Yu, H., Wu, X., Xie, J., Jiang, W., Liu, Q., & Lin, S. (2014) · Cited by 18
Significance tests of automatic machine translation evaluation metrics
Zhang, Y., & Vogel, S. (2010) · Machine Translation · doi:10.1007/s10590-010-9073-6 · Cited by 18
A Hybrid Rules and Statistical Method for Arabic to English Machine Translation
Alqudsi, A., Omar, N., & Shaker, K. (2019) · International Conference on Computer Applications & Information Security (ICCAIS) · doi:10.1109/cais.2019.8769545 · Cited by 19
An algorithm for cross-lingual sense-clustering tested in a MT evaluation setting
Apidianaki, M., & He, Y. (2010) · Arrow@dit (Dublin Institute of Technology) · Cited by 18
On the Generation of Medical Dialogues for COVID-19Preprint
Yang, W., Zeng, G., Tan, B., Ju, Z., Chakravorty, S., He, X., Chen, S., Yang, X., Wu, Q., Yu, Z., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2005.05442 · Cited by 22
Adapter-Enhanced Hierarchical Cross-Modal Pre-Training for Lightweight Medical Report Generation
Yu, T., Lu, W., Yang, Y. Q., Han, W., Huang, Q., Yu, J., & Zhang, K. (2025) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2025.3535699 · Cited by 18
CAT+: Investigating and Enhancing Audio-Visual Understanding in Large Language Models
Ye, Q., Yu, Z., Shao, R., Cui, Y., Kang, X., Liu, X., Torr, P. H. S., & Cao, X. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2025.3582389 · Cited by 21
CharaParser+EQ: Performance evaluation without gold standard
Cui, H., Dahdul, W., Dececchi, A., Ibrahim, N., Mabee, P., Balhoff, J. P., & Gopalakrishnan, H. (2015) · Proceedings of the Association for Information Science and Technology · doi:10.1002/pra2.2015.145052010020 · Cited by 17
#PraCegoVer: A Large Dataset for Image Captioning in Portuguese
dos Santos, G. O., Colombini, E. L., & Avila, S. (2022) · Data · doi:10.3390/data7020013 · Cited by 18
A Multimodal Framework for Video Caption Generation
Bhooshan, R. S., & Suresh, K. (2022) · IEEE Access · doi:10.1109/access.2022.3202526 · Cited by 18
Leveraging Pre-trained BERT for Audio Captioning
Liu, X., Mei, X., Huang, Q., Sun, J., Zhao, J., Liu, H., Plumbley, M. D., Kılıç, V., & Wang, W. (2022) · European Signal Processing Conference (EUSIPCO) · doi:10.23919/eusipco55093.2022.9909761 · Cited by 22
Cascade Semantic Prompt Alignment Network for Image Captioning
Li, J., Zhang, L., Zhang, K., Hu, B., Xie, H., & Mao, Z. (2023) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2023.3343520 · Cited by 18
Paraphrase Generation and Supervised Learning for Improved Automatic Short Answer Grading
Ouahrani, L., & Bennouar, D. (2024) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-023-00391-w · Cited by 19
Boosting court judgment prediction and explanation using legal entities
Benedetto, I., Koudounas, A., Vaiani, L., Pastor, E., Cagliero, L., Tarasconi, F., & Baralis, E. (2024) · Artificial Intelligence and Law · doi:10.1007/s10506-024-09397-8 · Cited by 17
Habib, M. A., Wadud, M. A. H., Patwary, M. F. K., Rahman, M. M., Mridha, M. F., Okuyama, Y., & Shin, J. (2024) · IEEE Access · doi:10.1109/access.2024.3435541 · Cited by 19
LAYERED: Metric for Machine Translation Evaluation
Gautam, S., & Bhattacharyya, P. (2014) · Conference on Machine Translation (WMT) · doi:10.3115/v1/w14-3350 · Cited by 17
Self-Induced Curriculum Learning in Self-Supervised Neural Machine Translation
Ruiter, D., van Genabith, J., & España-Bonet, C. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.202 · Cited by 17
Automatic image and text-based description for colorectal polyps using BASIC classification
Fonollà, R., van der Zander, Q. E. W., Schreuder, R. M., Subramaniam, S., Bhandari, P., Masclee, A., Schoon, E. J., van der Sommen, F., & de With, P. H. N. (2021) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2021.102178 · Cited by 18
Multiple Communication Channels in Literary Texts
Matricciani, E. (2022) · Open Journal of Statistics · doi:10.4236/ojs.2022.124030 · Cited by 17
Matricciani, E. (2023) · Information · doi:10.3390/info14020068 · Cited by 17
Zahálka, J., Rudinac, S., & Worring, M. (2015) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/2733373.2806279 · Cited by 17
Integrating source-language context into phrase-based statistical machine translation
Haque, R., Naskar, S. K., van den Bosch, A. P. J., & Way, A. (2011) · Machine Translation · doi:10.1007/s10590-011-9100-2 · Cited by 17
Machine Translation for Human Translators
Denkowski, M. (2022) · doi:10.1184/r1/21671609 · Cited by 17
Pivot Machine Translation Using Chinese as Pivot Language
Liu, C.-H., Silva, C. C., Wang, L., & Way, A. (2019) · Communications in computer and information science · doi:10.1007/978-981-13-3083-4_7 · Cited by 17
Decoding with syntactic and non-syntactic phrases in a syntax-based machine translation system
Hanneman, G., & Lavie, A. (2009) · Workshop on Syntax and Structure in Statistical Translation (SSST) · doi:10.3115/1626344.1626345 · Cited by 17
Self-citationMorphological processing of compounds for statistical machine translation
Cap, F. (2014) · OPUS Publication Server of the University of Stuttgart (University of Stuttgart) · doi:10.18419/opus-3474 · Cited by 17
The statistical advantage of automatic NLG metrics at the system level
Wei, J., & Jia, R. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.533 · Cited by 16
“This includes the traditional ROUGE-4 (Lin, 2004) summarization metric, its extension ROUGE-WE (Ng and Abrecht, 2015), and METEOR (Lavie and Agar-wal, 2007).”
MethodSchema-Guided Natural Language Generation
Du, Y., Oraby, S., Perera, V., Shen, M., Narayan-Chen, A., Chung, T., Venkatesh, A., & Hakkani-Tür, D. Z. (2020) · International Conference on Natural Language Generation · doi:10.18653/v1/2020.inlg-1.35 · Cited by 17
Weakly guided attention model with hierarchical interaction for brain CT report generation
Zhang, X., Yang, S., Shi, Y., Ji, J., Liu, Y., Wang, Z., & Xu, H. (2023) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2023.107650 · Cited by 16
Opinerium: Subjective Question Generation Using Large Language Models
Babakhani, P., Lommatzsch, A., Brodt, T., Sacker, D., Sivrikaya, F., & Albayrak, Ş. (2024) · IEEE Access · doi:10.1109/access.2024.3398553 · Cited by 17
Multi-Modal Instruction Tuned LLMs with Fine-Grained Visual Perception
He, J., Wang, Y., Wang, L., Lu, H., He, J.-Y., Lan, J.-P., Luo, B., & Xie, X. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01326 · Cited by 26
Tripathi, S., & Kansal, V. (2019) · International Journal of Pattern Recognition and Artificial Intelligence · doi:10.1142/s0218001420590016 · Cited by 16
Better than Average: Paired Evaluation of NLP systems
Peyrard, M., Zhao, W., Eger, S., & West, R. (2021) · doi:10.18653/v1/2021.acl-long.179 · Cited by 27
Video Description: Datasets & Evaluation Metrics
Rafiq, M. T., Rafiq, G., & Choi, G. S. (2021) · IEEE Access · doi:10.1109/access.2021.3108565 · Cited by 16
A conjoint analysis framework for evaluating user preferences in machine translation
Kirchhoff, K., Daniel Capurro, N., & Turner, A. M. (2013) · Machine Translation · doi:10.1007/s10590-013-9140-x · Cited by 16
The circle of meaning: from translation to paraphrasing and back
Dorr, B. J., & Madnani, N. (2010) · University Libraries (University of Maryland) · Cited by 16
Text2Scene: Generating Abstract Scenes from Textual Descriptions.Preprint
Tan, F., Feng, S., & Ordóñez, V. (2018) · arXiv (Cornell University) · Cited by 16
Template Guided Text Generation for Task-Oriented DialoguePreprint
Kale, M., & Rastogi, A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.15006 · Cited by 26
Are metrics measuring what they should? An evaluation of Image Captioning task metrics
González-Chávez, O., Ruíz, G., Moctezuma, D., & Ramírez-delReal, T. A. (2023) · Signal Processing Image Communication · doi:10.1016/j.image.2023.117071 · Cited by 15
End to End Urdu Abstractive Text Summarization With Dataset and Improvement in Evaluation Metric
Raza, H., & Shahzad, W. (2024) · IEEE Access · doi:10.1109/access.2024.3377463 · Cited by 17
Taming Data and Transformers for Audio Generation
Haji-Ali, M., Menapace, W., Siarohin, A., Balakrishnan, G., & Ordóñez, V. (2026) · International Journal of Computer Vision · doi:10.1007/s11263-025-02632-y · Cited by 31
Learning to Translate: A Statistical and Computational Analysis
Turchi, M., Bie, T. D., Goutte, C., & Cristianini, N. (2012) · Advances in Artificial Intelligence · doi:10.1155/2012/484580 · Cited by 15
Referential translation machines for predicting semantic similarity
Biçici, E., & Way, A. (2015) · Language Resources and Evaluation · doi:10.1007/s10579-015-9322-7 · Cited by 15
Neural Machine Translation by Generating Multiple Linguistic Factors
García-Martínez, M., Barrault, L., & Bougares, F. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-68456-7_2 · Cited by 15
Improving LEO Robot Conversational Ability via Deep Learning Algorithms for Children with Autism
She, T., Kang, X., Nishide, S., & Ren, F. (2018) · IEEE International Conference on Cloud Computing and Intelligence Systems (CCIS) · doi:10.1109/ccis.2018.8691393 · Cited by 15
Adaptive Syncretic Attention for Constrained Image Captioning
Yang, L., & Hu, H. (2019) · Neural Processing Letters · doi:10.1007/s11063-019-10045-5 · Cited by 15
Metadata
- Authors
- Lavie, A., & Agarwal, A.
- Affiliations
- Carnegie Mellon University (US)
- ORCID
- Lavie, Alon
- Venue
- Proceedings of the Second Workshop on Statistical Machine Translation - StatMT '07
- Published
- 2007
- Pages
- 228-231
- OpenAlex
- W2133512280
- Cited by
- 1,047
- References
- 10
- Citations by year
- 2026: 50 · 2025: 104 · 2024: 91 · 2023: 80 · 2022: 78 · 2021: 124 · 2020: 98 · 2019: 84 · 2018: 43 · 2017: 24 · 2016: 29 · 2015: 32 · 2014: 23 · 2013: 23 · 2012: 30
- FWCI
- 21.44
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- Conference on Machine Translation (WMT) (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Social Sciences › Arts and Humanities › Language and Linguistics › Translation Studies and Practices
- Keywords
- meteors, machine translation evaluation, automatic evaluation metrics, translation quality, BLEU, human judgment correlation, multilingual evaluation, Spanish language, French language, German
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review