The DOI matches its Crossref record; 10 of the 10 references it cites were found in a registry or an index.
Meteor: an automatic metric for MT evaluation with high levels of correlation with human judgments
Abstract
Meteor is an automatic metric for Machine Translation evaluation which has been demonstrated to have high levels of correlation with human judgments of translation quality, significantly outperforming the more commonly used Bleu metric. It is one of several automatic metrics used in this year's shared task within the ACL WMT-07 workshop. This paper recaps the technical details underlying the metric and describes recent improvements in the metric. The latest release includes improved metric parameters and extends the metric to support evaluation of MT output in Spanish, French and German, in addition to English.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Popular evaluation methods for natural language generation such as BLEU-460, ROUGE-L61, and METEOR62 primarily compare n-grams between two documents and may not effectively capture semantic similarities.”
From Generating dermatopathology reports from gigapixel whole slide images with HistoGPT (2025) · cites this paper as [62]
In the citing paper
“BLEU computes n-gram overlap and is representative for the family of text overlap-based natural language generation metrics such as CIDEr,31 METEOR,30 and ROUGE.32”
From Evaluating progress in automatic chest X-ray radiology report generation (2023) · cites this paper as [30]
Citations
Cites
BLEU: a method for automatic evaluation of machine translation
Papineni, K., Roukos, S., Ward, T., & Zhu, W.-J. (2001) · Proceedings of the 40th Annual Meeting on Association for Computational Linguistics - ACL '02 · doi:10.3115/1073083.1073135 · Cited by 21,344
(2005) · Cited by 4,152
Self-citationMinimum error rate training in statistical machine translation
Och, F. J. (2003) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.3115/1075096.1075117 · Cited by 2,728
A Study of Translation Edit Rate with Targeted Human Annotation
Snover, M., Dorr, B. J., Schwartz, R. M., Micciulla, L., & Makhoul, J. I. (2006) · Cited by 2,380
Confidence estimation for machine translation
Blatz, J., Fitzgerald, E., Foster, G. F., Gandrabur, S., Goutte, C., Kulesza, A., Sanchís, A., & Ueffing, N. (2004) · International Conference on Computational Linguistics · doi:10.3115/1220355.1220401 · Cited by 368
Lattice-based minimum error rate training for statistical machine translation
Macherey, W., Och, F. J., Thayer, I., & Uszkoreit, J. (2008) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/1613715.1613807 · Cited by 347
Precision and recall of machine translation
Melamed, I. D., Green, R., & Turian, J. P. (2003) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.3115/1073483.1073504 · Cited by 228
The Significance of Recall in Automatic Metrics for MT Evaluation
Lavie, A., Sagae, K., & Jayaraman, S. (2004) · Lecture notes in computer science · doi:10.1007/978-3-540-30194-3_16 · Cited by 123
Self-citationCDER: Efficient MT Evaluation Using Block Movements.
Leusch, G., Ueffing, N., & Ney, H. (2006) · RWTH Publications (RWTH Aachen) · Cited by 109
(2024) · ACM eBooks · doi:10.1145/3674127 · Cited by 86
Cited by
Sorted by citations
Prefix-Tuning: Optimizing Continuous Prompts for Generation
Li, X. L., & Liang, P. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.353 · Cited by 2,473
Generation and Comprehension of Unambiguous Object Descriptions
Mao, J., Huang, J., Toshev, A., Camburu, O., Yuille, A., & Murphy, K. P. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.9 · Cited by 1,281
DIALOGPT : Large-Scale Generative Pre-training for Conversational Response Generation
Zhang, Y., Sun, S., Galley, M., Chen, Y.-C., Brockett, C., Gao, X., Gao, J., Liu, J., & Dolan, B. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: System Demonstrations · doi:10.18653/v1/2020.acl-demos.30 · Cited by 1,076
Gatt, A., & Krahmer, E. (2018) · Journal of Artificial Intelligence Research · doi:10.1613/jair.5477 · Cited by 776
Variational Autoencoder for Deep Learning of Images, Labels and CaptionsPreprint
Pu, Y., Gan, Z., Henao, R., Yuan, X., Li, C., Stevens, A., & Carin, L. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1609.08976 · Cited by 669
SummEval: Re-evaluating Summarization Evaluation
Fabbri, A. R., Kryściński, W., McCann, B., Xiong, C., Socher, R., & Radev, D. (2021) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00373 · Cited by 587
ECO: Efficient Convolutional Network for Online Video Understanding
Zolfaghari, M., Singh, K., & Brox, T. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01216-8_43 · Cited by 541
MoverScore: Text Generation Evaluating with Contextualized Embeddings and Earth Mover Distance
Zhao, W., Peyrard, M., Liu, F., Gao, Y., Meyer, C. M., & Eger, S. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1053 · Cited by 490
Guiding the Long-Short Term Memory Model for Image Caption Generation
Jia, X., Gavves, E., Fernando, B., & Tuytelaars, T. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.277 · Cited by 480
Are You Talking to a Machine? Dataset and Methods for Multilingual Image Question AnsweringPreprint
Gao, H., Mao, J., Zhou, J., Huang, Z., Wang, L., & Xu, W. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1505.05612 · Cited by 441
Why We Need New Evaluation Metrics for NLG
Novikova, J., Dušek, O., Cercas Curry, A., & Rieser, V. (2017) · Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d17-1238 · Cited by 422
Improving automatic source code summarization via deep reinforcement learning
Wan, Y., Zhao, Z., Yang, M., Xu, G., Ying, H., Wu, J., & Yu, P. S. (2018) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/3238147.3238206 · Cited by 404
nocaps: novel object captioning at scale
Agrawal, H., Desai, K., Wang, Y., Chen, X., Jain, R., Johnson, M., Batra, D., Parikh, D., Lee, S., & Anderson, P. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00904 · Cited by 419
(Meta-) evaluation of machine translation
Callison-Burch, C., Fordyce, C. S., Koehn, P., Monz, C., & Schroeder, J. (2007) · Conference on Machine Translation (WMT) · doi:10.3115/1626355.1626373 · Cited by 381
In Defense of Grid Features for Visual Question Answering
Jiang, H., Misra, I., Rohrbach, M., Miller, E. G., & Chen, X. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.01028 · Cited by 384
(2020) · doi:10.18653/v1/2020.acl-main.450 · Cited by 356
The Meteor metric for automatic evaluation of machine translation
Lavie, A., & Denkowski, M. (2009) · Machine Translation · doi:10.1007/s10590-009-9059-4 · Cited by 351
Self-citationFindings of the 2015 Workshop on Statistical Machine Translation
Bojar, O., Chatterjee, R., Federmann, C., Haddow, B., Huck, M., Hokamp, C., Koehn, P., Logacheva, V., Monz, C., Negri, M., et al. (2015) · Conference on Machine Translation (WMT) · doi:10.18653/v1/w15-3001 · Cited by 287
The E2E Dataset: New Challenges For End-to-End Generation
Novikova, J., Dušek, O., & Rieser, V. (2017) · Annual Meeting of the Special Interest Group on Discourse and Dialogue · doi:10.18653/v1/w17-5525 · Cited by 275
How Much Can CLIP Benefit Vision-and-Language Tasks?Preprint
Shen, S., Li, L. H., Tan, H. X., Bansal, M., Rohrbach, A., Chang, K.-W., Yao, Z., & Keutzer, K. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2107.06383 · Cited by 382
DialoGPT: Large-Scale Generative Pre-training for Conversational Response GenerationPreprint
Zhang, Y., Sun, S., Galley, M., Chen, Y.-C., Brockett, C., Gao, X., Gao, J., Liu, J., & Dolan, B. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.00536 · Cited by 258
Neural Paraphrase Generation with Stacked Residual LSTM NetworksPreprint
Prakash, A., Hasan, S. A., Lee, K., Datla, V. V., Qadir, A., Liu, J., & Farri, O. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1610.03098 · Cited by 257
Adversarial text-to-image synthesis: A review
Frolov, S., Hinz, T., Raue, F., Hees, J., & Dengel, A. R. (2021) · Neural Networks · doi:10.1016/j.neunet.2021.07.019 · Cited by 228
Paraphrase Generation with Deep Reinforcement Learning
Li, Z., Jiang, X., Shang, L., & Li, H. (2018) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d18-1421 · Cited by 229
Evaluation of Text Generation: A Survey
Celikyilmaz, A., Clark, E., & Gao, J. (2020) · arXiv · doi:10.48550/arxiv.2006.14799 · Cited by 207
A Review on Explainability in Multimodal Deep Neural Nets
Joshi, G., Walambe, R., & Kotecha, K. (2021) · IEEE Access · doi:10.1109/access.2021.3070212 · Cited by 204
Evaluating progress in automatic chest X-ray radiology report generation
Yu, F., Endo, M., Krishnan, R., Pan, I., Tsai, A., Reis, E. P., Fonseca, E. K. U. N., Lee, H. M. H., Abad, Z. S. H., Ng, A. Y., et al. (2023) · Patterns · doi:10.1016/j.patter.2023.100802 · Cited by 200
Understanding Factuality in Abstractive Summarization with FRANK: A Benchmark for Factuality Metrics
Pagnoni, A., Balachandran, V., & Tsvetkov, Y. (2021) · Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2021.naacl-main.383 · Cited by 197
A Deep Generative Framework for Paraphrase Generation
Gupta, A., Agarwal, A. K., Singh, P., & Rai, P. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11956 · Cited by 197
Self-citationEvaluating the state-of-the-art of End-to-End Natural Language Generation: The E2E NLG challenge
Dušek, O., Novikova, J., & Rieser, V. (2019) · Computer Speech & Language · doi:10.1016/j.csl.2019.06.009 · Cited by 209
Explainable Deep Learning Methods in Medical Image Classification: A Survey
Patrício, C., Neves, J. C., & Teixeira, L. F. (2023) · ACM Computing Surveys · doi:10.1145/3625287 · Cited by 207
Liu, C., Zhao, R., Chen, H., Zou, Z., & Shi, Z. (2022) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2022.3218921 · Cited by 203
Know More Say Less: Image Captioning Based on Scene Graphs
Li, X., & Jiang, S. (2019) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2019.2896516 · Cited by 181
MedDialog: Large-scale Medical Dialogue Datasets
Zeng, G., Yang, W., Ju, Z., Yang, Y., Wang, S., Zhang, R., Zhou, M., Zeng, J., Dong, X., Zhang, R., et al. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.743 · Cited by 196
Transformer models used for text-based question answering systems
Nassiri, K., & Akhloufi, M. A. (2022) · Applied Intelligence · doi:10.1007/s10489-022-04052-8 · Cited by 194
ILDC for CJPE: Indian Legal Documents Corpus for Court Judgment Prediction and Explanation
Malik, V., Sanjay, R., Nigam, S. K., Ghosh, K., Guha, S. K., Bhattacharya, A., & Modi, A. (2021) · Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2021.acl-long.313 · Cited by 198
QuestEval: Summarization Asks for Fact-based Evaluation
Scialom, T., Dray, P.-A., Lamprier, S., Piwowarski, B., Staiano, J., Wang, A., & Gallinari, P. (2021) · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2021.emnlp-main.529 · Cited by 175
Visual Question Generation as Dual Task of Visual Question Answering
Li, Y., Duan, N., Zhou, B., Chu, X., Ouyang, W., Wang, X., & Zhou, M. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00640 · Cited by 177
Selective Attention for Context-aware Neural Machine Translation
Maruf, S., Martins, A. F. T., & Haffari, G. (2019) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n19-1313 · Cited by 168
TER-Plus: paraphrase, semantic, and alignment enhancements to Translation Edit Rate
Snover, M., Madnani, N., Dorr, B. J., & Schwartz, R. M. (2009) · Machine Translation · doi:10.1007/s10590-009-9062-9 · Cited by 167
Machine translation evaluation versus quality estimation
Specia, L., Raj, D., & Turchi, M. (2010) · Machine Translation · doi:10.1007/s10590-010-9077-2 · Cited by 165
Assessing The Factual Accuracy of Generated Text
Goodrich, B., Rao, V., Liu, P. J., & Saleh, M. (2019) · Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining · doi:10.1145/3292500.3330955 · Cited by 162
Ground Truth for Grammaticality Correction Metrics
Napoles, C., Sakaguchi, K., Post, M., & Tetreault, J. (2015) · doi:10.3115/v1/p15-2097 · Cited by 151
Active Learning and Crowd-Sourcing for Machine Translation
Ambati, V., Vogel, S., & Carbonell, J. (2010) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/5dqoxv96xk95 · Cited by 150
Document Context Neural Machine Translation with Memory Networks
Maruf, S., & Haffari, G. (2018) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p18-1118 · Cited by 156
A Survey on Document-level Neural Machine Translation
Maruf, S., Saleh, F., & Haffari, G. (2021) · ACM Computing Surveys · doi:10.1145/3441691 · Cited by 172
Learning Like a Child: Fast Novel Visual Concept Learning from Sentence Descriptions of Images
Mao, J., Xu, W., Yang, Y., Wang, J., Huang, Z., & Yuille, A. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.291 · Cited by 149
Approaches to Human and Machine Translation Quality Assessment
Castilho, S., Doherty, S., Gaspari, F., & Moorkens, J. (2018) · Machine translation · doi:10.1007/978-3-319-91241-7_2 · Cited by 165
Text-to-Text Pre-Training for Data-to-Text Tasks
Kale, M., & Rastogi, A. (2020) · International Conference on Natural Language Generation · doi:10.18653/v1/2020.inlg-1.14 · Cited by 145
CharacTer: Translation Edit Rate on Character Level
Wang, W., Peter, J.-T., Rosendahl, H., & Ney, H. (2016) · Conference on Machine Translation (WMT) · doi:10.18653/v1/w16-2342 · Cited by 134
Region-Object Relation-Aware Dense Captioning via Transformer
Shao, Z., Han, J., Marnerides, D., & Debattista, K. (2022) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3152990 · Cited by 134
Ramanishka, V., Das, A., Park, D. H., Venugopalan, S., Hendricks, L. A., Rohrbach, M., & Saenko, K. (2016) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/2964284.2984066 · Cited by 132
Grounded Conversation Generation as Guided Traverses in Commonsense Knowledge Graphs
Zhang, H., Liu, Z., Xiong, C., & Liu, Z. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.184 · Cited by 137
Sezgın, E., Sirrianni, J., & Linwood, S. L. (2022) · JMIR Medical Informatics · doi:10.2196/32875 · Cited by 129
A Neural Architecture for Generating Natural Language Descriptions from Source Code Changes
Loyola, P., Marrese-Taylor, E., & Matsuo, Y. (2017) · Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) · doi:10.18653/v1/p17-2045 · Cited by 128
A long video caption generation algorithm for big video data retrieval
Ding, S., Qu, S., Xi, Y., & Wan, S. (2018) · Future Generation Computer Systems · doi:10.1016/j.future.2018.10.054 · Cited by 128
Lattice Minimum Bayes-Risk decoding for statistical machine translation
Tromble, R. W., Kumar, S., Och, F. J., & Macherey, W. (2008) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/1613715.1613792 · Cited by 127
Discriminative corpus weight estimation for machine translation
Matsoukas, S., Rosti, A.-V. I., & Zhang, B. (2009) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/1699571.1699605 · Cited by 123
Large Language Model Assisted Software Engineering: Prospects, Challenges, and a Case Study
Belzner, L., Gabor, T., & Wirsing, M. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-46002-9_23 · Cited by 130
FINDINGS OF THE IWSLT 2020 EVALUATION CAMPAIGN
Ansari, E., Axelrod, A. E., Bách, N., Bojar, O., Cattoni, R., Dalvi, F., Durrani, N., Federico, M., Federmann, C., Gu, J., et al. (2020) · International Conference on Spoken Language Translation (IWSLT) · doi:10.18653/v1/2020.iwslt-1.1 · Cited by 111
Changes to Captions: An Attentive Network for Remote Sensing Change Captioning
Chang, S., & Ghamisi, P. (2023) · IEEE Transactions on Image Processing · doi:10.1109/tip.2023.3328224 · Cited by 127
High-Resolution Remote Sensing Image Captioning Based on Structured Attention
Zhao, R., Shi, Z., & Zou, Z. (2021) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2021.3070383 · Cited by 117
Agarwal, A., & Lavie, A. (2008) · Conference on Machine Translation (WMT) · doi:10.3115/1626394.1626406 · Cited by 109
Self-citationA Survey on Deep Learning and Explainability for Automatic Report Generation from Medical Images
Messina, P., Pino, P., Parra, D., Soto, Á., Besa, C., Uribe, S. A., Andía, M. E., Cristián Tejos, N., Prieto, C., & Daniel Capurro, N. (2022) · ACM Computing Surveys · doi:10.1145/3522747 · Cited by 111
Automated audio captioning with recurrent neural networks
Drossos, K., Adavanne, S., & Virtanen, T. I. (2017) · IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA) · doi:10.1109/waspaa.2017.8170058 · Cited by 102
deltaBLEU: A Discriminative Metric for Generation Tasks with Intrinsically Diverse Targets
Galley, M., Brockett, C., Sordoni, A., Ji, Y., Auli, M., Quirk, C., Mitchell, M., Gao, J., & Dolan, B. (2015) · doi:10.3115/v1/p15-2073 · Cited by 101
Image Captioning: A Comprehensive Survey
Sharma, H., Agrahari, M., Singh, S. K., Firoj, M., & Mishra, R. K. (2020) · International Conference on Power Electronics & IoT Applications in Renewable Energy and its Control (PARC) · doi:10.1109/parc49193.2020.236619 · Cited by 102
Learning to rank with partially-labeled data
Duh, K., & Kirchhoff, K. (2008) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/1390334.1390379 · Cited by 101
Toward Subgraph-Guided Knowledge Graph Question Generation With Graph Neural Networks
Chen, Y., Wu, L., & Zaki, M. J. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3264519 · Cited by 106
Conversing by Reading: Contentful Neural Conversation with On-demand Machine Reading
Qin, L., Galley, M., Brockett, C., Liu, X., Gao, X., Dolan, B., Choi, Y., & Gao, J. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1539 · Cited by 100
Neural data-to-text generation: A comparison between pipeline and end-to-end architectures
Ferreira, T. C., van der Lee, C., van Miltenburg, E., & Krahmer, E. (2019) · Tilburg University Research Portal · Cited by 107
Automatic Source Code Summarization with Extended Tree-LSTM
Shido, Y., Kobayashi, Y., Yamamoto, A., Miyamoto, A., & Matsumura, T. (2019) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2019.8851751 · Cited by 102
Sampling alignment structure under a Bayesian translation model
DeNero, J., Bouchard‐Côté, A., & Klein, D. (2008) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/1613715.1613758 · Cited by 97
TrustRank: Inducing Trust in Automatic Translations via Ranking
Soricut, R., & Echihabi, A. (2010) · Cited by 95
Text2Scene: Generating Compositional Scenes From Textual Descriptions
Tan, F., Feng, S., & Ordóñez, V. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00687 · Cited by 95
High-Order Interaction Learning for Image Captioning
Wang, Y., Xu, N., Liu, A.-A., Li, W., & Zhang, Y. (2021) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2021.3121062 · Cited by 95
POINTER: Constrained Progressive Text Generation via Insertion-based Generative Pre-training
Zhang, Y., Wang, G., Li, C., Gan, Z., Brockett, C., & Dolan, B. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.698 · Cited by 110
Large Language Models in Mental Health Care: A Scoping Review
Hua, Y., Liu, F., Yang, K., Li, Z., Na, H., Sheu, Y.-H., Zhou, P., Moran, L. V., Ananiadou, S., Clifton, D. A., et al. (2025) · Current Treatment Options in Psychiatry · doi:10.1007/s40501-025-00363-y · Cited by 92
Global Visual Feature and Linguistic State Guided Attention for Remote Sensing Image Captioning
Zhang, Z., Zhang, W., Yan, M., Gao, X., Fu, K., & Sun, X. (2021) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2021.3132095 · Cited by 96
Multimodal machine translation through visuals and speech
Sulubacak, U., Çağlayan, O., Grönroos, S.-A., Rouhe, A., Elliott, D., Specia, L., & Tiedemann, J. (2020) · Machine Translation · doi:10.1007/s10590-020-09250-0 · Cited by 104
A Decoupling Paradigm With Prompt Learning for Remote Sensing Image Change Captioning
Liu, C., Zhao, R., Chen, J., Qi, Z., Zou, Z., & Shi, Z. (2023) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2023.3321752 · Cited by 100
BEER: BEtter Evaluation as Ranking
Stanojević, M., & Sima’an, K. (2014) · Conference on Machine Translation (WMT) · doi:10.3115/v1/w14-3354 · Cited by 91
Inducing Sentence Structure from Parallel Corpora for Reordering
DeNero, J., & Uszkoreit, J. (2011) · Cited by 90
deltaBLEU: A Discriminative Metric for Generation Tasks with Intrinsically Diverse TargetsPreprint
Galley, M., Brockett, C., Sordoni, A., Ji, Y., Auli, M., Quirk, C., Mitchell, M., Gao, J., & Dolan, B. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1506.06863 · Cited by 90
Very Deep Transformers for Neural Machine TranslationPreprint
Liu, X., Duh, K., Liu, L., & Gao, J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2008.07772 · Cited by 120
Semantic Noise Matters for Neural Natural Language Generation
Dušek, O., Howcroft, D. M., & Rieser, V. (2019) · Proceedings of the 12th International Conference on Natural Language Generation · doi:10.18653/v1/w19-8652 · Cited by 85
Semantic Graphs for Generating Deep Questions
Pan, L., Xie, Y., Feng, Y., Chua, T., & Kan, M. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.135 · Cited by 92
A survey of types of text noise and techniques to handle noisy text
Subramaniam, L. V., Roy, S., Faruquie, T. A., & Negi, S. (2009) · Analytics for Noisy Unstructured Text Data · doi:10.1145/1568296.1568315 · Cited by 85
Large-Scale Transfer Learning for Natural Language Generation
Golovanov, S., Kurbanov, R., Nikolenko, S., Truskovskyi, K., Tselousov, A., & Wolf, T. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1608 · Cited by 86
Ensemble-based deep reinforcement learning for chatbots
Cuayáhuitl, H., Lee, D., Ryu, S., Cho, Y. J., Choi, S., Indurthi, S., Yu, S., Choi, H., Hwang, I.-C., & Kim, J. (2019) · Neurocomputing · doi:10.1016/j.neucom.2019.08.007 · Cited by 87
Change-Agent: Toward Interactive Comprehensive Remote Sensing Change Interpretation and Analysis
Liu, C., Chen, K., Zhang, H., Qi, Z., Zou, Z., & Shi, Z. (2024) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2024.3425815 · Cited by 96
Automatic medical image interpretation: State of the art and future directions
Ayesha, H., Iqbal, S., Tariq, M., Abrar, M., Sanaullah, M., Abbas, I., Rehman, A. U., Niazi, M. F. K., & Hussain, S. (2021) · Pattern Recognition · doi:10.1016/j.patcog.2021.107856 · Cited by 81
A Review on the Use of Large Language Models as Virtual Tutors
García-Méndez, S., de Arriba-Pérez, F., & Somoza-López, M. D. C. (2024) · Science & Education · doi:10.1007/s11191-024-00530-2 · Cited by 95
Diagnosing meaning errors in short answers to reading comprehension questions
Bailey, S. M., & Meurers, D. (2008) · doi:10.3115/1631836.1631849 · Cited by 78
Multimodal Attention for Neural Machine TranslationPreprint
Çağlayan, O., Barrault, L., & Bougares, F. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1609.03976 · Cited by 87
Non-Monotonic Sequential Text GenerationPreprint
Welleck, S., Brantley, K., Daumé, H., & Cho, K. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.02192 · Cited by 89
Scheduled Multi-Task Learning: From Syntax to Translation
Kiperwasser, E., & Ballesteros, M. (2018) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00017 · Cited by 80
Deep learning for ultrasound image caption generation based on object detection
Zeng, X., Wen, L., Liu, B., & Qi, X. (2019) · Neurocomputing · doi:10.1016/j.neucom.2018.11.114 · Cited by 79
Spatio-Temporal Memory Attention for Image Captioning
Ji, J., Xu, C., Zhang, X., Wang, B., & Song, X. (2020) · IEEE Transactions on Image Processing · doi:10.1109/tip.2020.3004729 · Cited by 79
MAXSIM: A Maximum Similarity Metric for Machine Translation Evaluation
Chan, Y. S., & Ng, H. T. (2008) · Cited by 76
Metadata
- Authors
- Lavie, A., & Agarwal, A.
- Affiliations
- Carnegie Mellon University (US)
- ORCID
- Lavie, Alon
- Venue
- Proceedings of the Second Workshop on Statistical Machine Translation - StatMT '07
- Published
- 2007
- Pages
- 228-231
- OpenAlex
- W2133512280
- Cited by
- 1,047
- References
- 10
- Citations by year
- 2026: 50 · 2025: 104 · 2024: 91 · 2023: 80 · 2022: 78 · 2021: 124 · 2020: 98 · 2019: 84 · 2018: 43 · 2017: 24 · 2016: 29 · 2015: 32 · 2014: 23 · 2013: 23 · 2012: 30
- FWCI
- 21.44
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- Conference on Machine Translation (WMT) (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Social Sciences › Arts and Humanities › Language and Linguistics › Translation Studies and Practices
- Keywords
- meteors, machine translation evaluation, automatic evaluation metrics, translation quality, BLEU, human judgment correlation, multilingual evaluation, Spanish language, French language, German
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review