Measuring machine translation quality as semantic equivalence: A metric based on entailment features
The DOI matches its Crossref record; 31 of the 32 references it cites were found in a registry or an index.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
BLEU: a method for automatic evaluation of machine translation
Papineni, K., Roukos, S., Ward, T., & Zhu, W.-J. (2001) · Proceedings of the 40th Annual Meeting on Association for Computational Linguistics - ACL '02 · doi:10.3115/1073083.1073135 · Cited by 21,344
Moses: open source toolkit for statistical machine translation
Koehn, P., Zens, R., Dyer, C., Bojar, O., Constantin, A., Herbst, E., Hoang, H., Birch, A., Callison-Burch, C., Federico, M., et al. (2007) · Proceedings of the 45th Annual Meeting of the ACL on Interactive Poster and Demonstration Sessions - ACL '07 · doi:10.3115/1557769.1557821 · Cited by 4,735
SRILM - an extensible language modeling toolkit
Stolcke, A. (2002) · 7th International Conference on Spoken Language Processing (ICSLP 2002) · doi:10.21437/icslp.2002-303 · Cited by 4,326
Introduction to WordNet: An On-line Lexical Database*
Miller, G. A., Beckwith, R., Fellbaum, C., Gross, D., & Miller, K. J. (1990) · International Journal of Lexicography · doi:10.1093/ijl/3.4.235 · Cited by 4,197
(2005) · Cited by 4,152
A Systematic Comparison of Various Statistical Alignment Models
Och, F. J., & Ney, H. (2003) · Computational Linguistics · doi:10.1162/089120103321337421 · Cited by 3,854
A Study of Translation Edit Rate with Targeted Human Annotation
Snover, M., Dorr, B. J., Schwartz, R. M., Micciulla, L., & Makhoul, J. I. (2006) · Conference of the Association for Machine Translation in the Americas · Cited by 3,442
Statistical phrase-based translation
Koehn, P., Och, F. J., & Marcu, D. (2003) · Proceedings of the 2003 Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology - NAACL '03 · doi:10.3115/1073445.1073462 · Cited by 3,197
Minimum error rate training in statistical machine translation
Och, F. J. (2003) · Proceedings of the 41st Annual Meeting on Association for Computational Linguistics - ACL '03 · doi:10.3115/1075096.1075117 · Cited by 2,725
Generating Typed Dependency Parses from Phrase Structure Parses
de Marneffe, M.-C., MacCartney, B., & Manning, C. D. (2006) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/3wtkaidk6ea8 · Cited by 2,258
Self-citation
Research works printed but not linked: 4
- de Marneffe M-C, Grenager T, MacCartney B, Cer D, Ramage D, Kiddon C, Manning CD (2007) Aligning semantic graphs for textual inference and machine reading. In: Proceedings of the AAAI spring symposium on machine reading, pp 36–42
- Likert R (1932) A technique for the measurement of attitudes. Arch Psychol 22(140): 1–55
- Lin D (1998) Extracting collocations from text corpora. In: First workshop on computational terminology, pp 57–63
- Takayama Y, Flournoy R, Kaufmann S, Peters S (1999) Information retrieval based on domain-specific word associations. In: Proceedings of PACLING, pp 155–161
Cited by
Detecting hallucinations in large language models using semantic entropy
Farquhar, S., Kossen, J., Kuhn, L., & Gal, Y. (2024) · Nature · doi:10.1038/s41586-024-07421-0 · Cited by 934
“We operationalize E ( ⋅ , ⋅ ) using the idea of bidirectional entailment, which has a long history in linguistics 52 and natural language processing 28,53,54 .”
Challenges in building intelligent open-domain dialog systems
Huang, M., Zhu, X., & Gao, J. (2020) · ACM Transactions on Information Systems · doi:10.1145/3383123 · Cited by 288
Analyzing Uncertainty in Neural Machine Translation
Ott, M., Auli, M., Grangier, D., & Ranzato, M. (2018) · arXiv · doi:10.48550/arxiv.1803.00047 · Cited by 140
Approaches to Human and Machine Translation Quality Assessment
Castilho, S., Doherty, S., Gaspari, F., & Moorkens, J. (2018) · Machine translation · doi:10.1007/978-3-319-91241-7_2 · Cited by 168
Neural Approaches to Conversational AI
Gao, J., Galley, M., & Li, L. (2019) · now publishers, Inc. eBooks · doi:10.1561/9781680835533 · Cited by 107
Self-citation
Metadata
- Authors
- Padó, S., Cer, D., Galley, M., Jurafsky, D., & Manning, C. D.
- Affiliations
- University of Stuttgart (DE)
- Stanford University (US)
- Corresponding author
- Padó, Sebastian
- Venue
- Machine Translation
- Published
- 2009-09
- Volume
- 23
- Issue
- 2-3
- Pages
- 181-193
- ISSN
- 0922-6567 (ISSN-L)
- 1573-0573
- OpenAlex
- W2040103855
- Semantic Scholar
- 982923236f3bc380e5ebdd8f56c46686029dc66a
- Cited by
- 47
- References
- 32
- Citations by year
- 2024: 2 · 2023: 1 · 2022: 4 · 2021: 3 · 2020: 2 · 2019: 4 · 2018: 5 · 2017: 1 · 2016: 2 · 2015: 3 · 2014: 1 · 2013: 5 · 2012: 6
- FWCI
- 5.05
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- www.springer.com/tdm (tdm)
- Locations
- Machine Translation (publishedVersion)
- Funders
- Defense Advanced Research Projects Agency
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Social Sciences › Arts and Humanities › Language and Linguistics › Translation Studies and Practices
- Keywords
- machine translation quality assessment, semantic equivalence, natural language inference, evaluation metrics, MERT, translation quality
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Measuring machine translation quality as semantic equivalence: A metric based on entailment features. https://app.cochat.ai/papers/10.1007/s10590-009-9060-y
- The paper
- Padó, S., Cer, D., Galley, M., Jurafsky, D., & Manning, C. D. (2009). Measuring machine translation quality as semantic equivalence: A metric based on entailment features. Machine Translation. https://doi.org/10.1007/s10590-009-9060-y
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.