Human Effort and Machine Learnability in Computer Aided Translation
Open accessAbstract
Spence Green, Sida I. Wang, Jason Chuang, Jeffrey Heer, Sebastian Schuster, Christopher D. Manning. Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP). 2014.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In translation, when translators used an interface that displayed machine-suggested recommendations, they ceded agency and would conform to the machine's recommended phrasing, even when it did not match their usual style.
“When translators used an interface that displayed machine-suggested recommendations, they noted that they ceded agency and would conform to the machine's recommended phrasing, even when it did not match their usual style [15].”
From Assessing the Impact of Automated Suggestions on Decision Making: Domain Experts Mediate Model Errors but Take Less Initiative (2021) · cites this paper as [15]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
BLEU: a method for automatic evaluation of machine translation
Papineni, K., Roukos, S., Ward, T., & Zhu, W.-J. (2001) · Proceedings of the 40th Annual Meeting on Association for Computational Linguistics - ACL '02 · doi:10.3115/1073083.1073135 · Cited by 21,344 · see citing works
Random effects structure for confirmatory hypothesis testing: Keep it maximal
Barr, D. J., Levy, R., Scheepers, C., & Tily, H. J. (2013) · Journal of Memory and Language · doi:10.1016/j.jml.2012.11.001 · Cited by 10,328 · see citing works
Mixed-effects modeling with crossed random effects for subjects and items
Baayen, R. H., Davidson, D. J., & Bates, D. M. (2008) · Journal of Memory and Language · doi:10.1016/j.jml.2007.12.005 · Cited by 8,540 · see citing works
(2010) · Cited by 8,261 · see citing works
The Stanford CoreNLP Natural Language Processing Toolkit
Manning, C. D., Surdeanu, M., Bauer, J., Finkel, J. R., Bethard, S. J., & McClosky, D. (2014) · Annual Meeting of the Association for Computational Linguistics: System Demonstrations (ACL Demo) · doi:10.3115/v1/p14-5010 · Cited by 7,171 · see citing works
Self-citationMoses: open source toolkit for statistical machine translation
Koehn, P., Zens, R., Dyer, C., Bojar, O., Constantin, A., Herbst, E., Hoang, H., Birch, A., Callison-Burch, C., Federico, M., et al. (2007) · Proceedings of the 45th Annual Meeting of the ACL on Interactive Poster and Demonstration Sessions - ACL '07 · doi:10.3115/1557769.1557821 · Cited by 4,735 · see citing works
Minimum error rate training in statistical machine translation
Och, F. J. (2003) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.3115/1075096.1075117 · Cited by 2,728 · see citing works
A Study of Translation Edit Rate with Targeted Human Annotation
Snover, M., Dorr, B. J., Schwartz, R. M., Micciulla, L., & Makhoul, J. I. (2006) · Cited by 2,380 · see citing works
Moses : open source toolkit for statistical machine translation
Koehn, P., Hoang, H., Birch, A., Callison-Burch, C., Federico, M., Bertoldi, N., Cowan, B., Shen, W., Moran, C., Zens, R., et al. (2007) · Edinburgh Research Explorer (University of Edinburgh) · Cited by 1,547 · see citing works
Statistical machine translation
Koehn, P. (2010) · Choice Reviews Online · doi:10.5860/choice.47-6293 · Cited by 1,309 · see citing works
The Alignment Template Approach to Statistical Machine Translation
Och, F. J., & Ney, H. (2004) · Computational Linguistics · doi:10.1162/0891201042544884 · Cited by 927 · see citing works
Statistical Machine Translation
Koehn, P. (2009) · Cambridge University Press eBooks · doi:10.1017/cbo9780511815829 · Cited by 874 · see citing works
Findings of the 2014 Workshop on Statistical Machine Translation
Bojar, O., Buck, C., Federmann, C., Haddow, B., Koehn, P., Leveling, J., Monz, C., Pecina, P., Post, M., Saint-Amand, H., et al. (2014) · Conference on Machine Translation (WMT) · doi:10.3115/v1/w14-3302 · Cited by 705 · see citing works
Re-evaluating the Role of Bleu in Machine Translation Research
Callison-Burch, C., Osborne, M., & Koehn, P. (2006) · Edinburgh Research Explorer (University of Edinburgh) · Cited by 632 · see citing works
Scalable Modified Kneser-Ney Language Model Estimation
Heafield, K., Pouzyrevsky, I., Clark, J. H., & Koehn, P. (2013) · Cited by 442 · see citing works
Liang, P., Taskar, B., & Klein, D. (2006) · doi:10.3115/1220835.1220849 · Cited by 433 · see citing works
Lattice-based minimum error rate training for statistical machine translation
Macherey, W., Och, F. J., Thayer, I., & Uszkoreit, J. (2008) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/1613715.1613807 · Cited by 347 · see citing works
Batch Tuning Strategies for Statistical Machine Translation
Cherry, C. A., & Foster, G. F. (2012) · NPARC · Cited by 329 · see citing works
The Present Status of Automatic Translation of Languages
Bar‐Hillel, Y. (1960) · Advances in computers · doi:10.1016/s0065-2458(08)60607-5 · Cited by 318 · see citing works
A simple and effective hierarchical phrase reordering model
Galley, M., & Manning, C. D. (2008) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/1613715.1613824 · Cited by 291 · see citing works
Self-citationForest Rescoring: Faster Decoding with Integrated Language Models
Huang, L., & Chiang, D. (2007) · Cited by 288 · see citing works
Findings of the 2015 Workshop on Statistical Machine Translation
Bojar, O., Chatterjee, R., Federmann, C., Haddow, B., Huck, M., Hokamp, C., Koehn, P., Logacheva, V., Monz, C., Negri, M., et al. (2015) · Conference on Machine Translation (WMT) · doi:10.18653/v1/w15-3001 · Cited by 287 · see citing works
Findings of the 2013 Workshop on Statistical Machine Translation
Bojar, O., Buck, C., Callison-Burch, C., Federmann, C., Haddow, B., Koehn, P., Monz, C., Post, M., Soricut, R., & Specia, L. (2013) · UvA-DARE (University of Amsterdam) · Cited by 236 · see citing works
Statistical Approaches to Computer-Assisted Translation
Barrachina, S., Bender, O., Casacuberta, F., et al. (2008) · Computational Linguistics · doi:10.1162/coli.2008.07-055-r2-06-29 · Cited by 228 · see citing works
Snover, M., Madnani, N., Dorr, B. J., & Schwartz, R. (2009) · Conference on Machine Translation (WMT) · doi:10.3115/1626431.1626480 · Cited by 195 · see citing works
The efficacy of human post-editing for language translation
Green, S., Heer, J., & Manning, C. D. (2013) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2470654.2470718 · Cited by 191 · see citing works
Self-citation(2005) · Cited by 189 · see citing works
Evaluating translational correspondence using annotation projection
Hwa, R., Resnik, P., Weinberg, A. S., et al. (2001) · Annual Meeting of the Association for Computational Linguistics (ACL) · doi:10.3115/1073083.1073149 · Cited by 114 · see citing works
A process study of computer-aided translation
Koehn, P. (2009) · Machine Translation · doi:10.1007/s10590-010-9076-3 · Cited by 93 · see citing works
Langlais, P., Foster, G., & Lapalme, G. (2000) · doi:10.3115/1117586.1117593 · Cited by 80 · see citing works
User-friendly text prediction for translators
Foster, G., Langlais, P., & Lapalme, G. (2002) · doi:10.3115/1118693.1118713 · Cited by 73 · see citing works
Online Learning for Interactive Statistical Machine Translation
Ortiz-Martínez, D., García-Varea, I., & Casacuberta, F. (2010) · RiuNet (Universitat Politècnica de València) · Cited by 66 · see citing works
Extending the METEOR Machine Translation Evaluation Metric to the Phrase Level
Denkowski, M., & Lavie, A. (2010) · Cited by 61 · see citing works
Efficient Elicitation of Annotations for Human Evaluation of Machine Translation
Sakaguchi, K., Post, M., & Durme, B. V. (2014) · Workshop on Statistical Machine Translation · doi:10.3115/v1/w14-3301 · Cited by 57 · see citing works
Learning from Post-Editing: Online Model Adaptation for Statistical Machine Translation
Denkowski, M., Dyer, C., & Lavie, A. (2014) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.3115/v1/e14-1042 · Cited by 56 · see citing works
Efficient search for interactive statistical machine translation
Och, F. J., Zens, R., & Ney, H. (2003) · doi:10.3115/1067807.1067858 · Cited by 55 · see citing works
Green, S., Chuang, J., Heer, J., & Manning, C. D. (2014) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/2642918.2647408 · Cited by 52 · see citing works
Self-citationA web-based interactive computer aided translation tool
Koehn, P. (2009) · doi:10.3115/1667872.1667877 · Cited by 51 · see citing works
The Limits of N-Gram Translation Evaluation Metrics
Culy, C., & Riehemann, S. Z. (2003) · Cited by 50 · see citing works
The Best Lexical Metric for Phrase-Based Statistical MT System Optimization
Cer, D. M., Manning, C. D., & Jurafsky, D. S. (2010) · Cited by 48 · see citing works
Self-citationOnline adaptation strategies for statistical machine translation in post-editing scenarios
Martínez-Gómez, P., Sanchis-Trilles, G., & Casacuberta, F. (2012) · Pattern Recognition · doi:10.1016/j.patcog.2012.01.011 · Cited by 42 · see citing works
Online Structured Prediction via Coactive LearningPreprint
Shivaswamy, P. K., & Joachims, T. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1205.4213 · Cited by 41 · see citing works
Phrasal: A Toolkit for New Directions in Statistical Machine Translation
Green, S., Cer, D. M., & Manning, C. D. (2014) · Conference on Machine Translation (WMT) · doi:10.3115/v1/w14-3311 · Cited by 38 · see citing works
Self-citationAppraise: An Open-Source Toolkit for Manual Phrase-Based Evaluation of Translations
Federmann, C. (2010) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/5aj2568ubi3f · Cited by 33 · see citing works
Fast and Adaptive Online Training of Feature-Rich Translation Models
Green, S., Wang, S. I., Cer, D. M., & Manning, C. D. (2013) · Cited by 33 · see citing works
Self-citationEvaluating Translational Correspondence Using Annotation Projection
Hwa, R., Resnik, P., Weinberg, A. S., et al. (2003) · doi:10.21236/ada455137 · Cited by 32 · see citing works
The New Thot Toolkit for Fully-Automatic and Interactive Statistical Machine Translation
Ortiz-Martínez, D., & Casacuberta, F. (2014) · doi:10.3115/v1/e14-2012 · Cited by 24 · see citing works
Real Time Adaptive Machine Translation for Post-Editing with cdec and TransCenter
Denkowski, M., Lavie, A., Lacruz, I., et al. (2014) · doi:10.3115/v1/w14-0311 · Cited by 23 · see citing works
An Empirical Comparison of Features and Tuning for Phrase-based Machine Translation
Green, S., Cer, D. M., & Manning, C. D. (2014) · Workshop on Statistical Machine Translation · doi:10.3115/v1/w14-3360 · Cited by 17 · see citing works
Self-citationGenerative and Discriminative Methods for Online Adaptation in SMT
Wäschle, K., Simianer, P., Bertoldi, N., et al. (2013) · Cited by 15 · see citing works
Predicting Human-Targeted Translation Edit Rate via Untrained Human Annotators
Zaidan, O. F., & Callison-Burch, C. (2010) · Cited by 14 · see citing works
Online Word Alignment for Online Adaptive Machine Translation
Farajian, M. A., Bertoldi, N., & Federico, M. (2014) · doi:10.3115/v1/w14-0313 · Cited by 12 · see citing works
Advanced Computer Aided Translation with a Web-Based Workbench
Alabau, V., Bonk, R., Buck, C., et al. (2013) · Cited by 10 · see citing works
Flagged first, then by citations
Research works printed but not linked: 2
- M. Carl. 2010. A computational framework for a cogni- tive model of human translation processes. In Aslib Translating and the Computer Conference.
- P. Langlais, G. Foster, and G. Lapalme. 2000. TransType: a computer-aided translation typing sys- tem. In Workshop on Embedded Machine Transla- tion Systems.
Cited by
Sorted by citations
Levy, A., Agrawal, M., Satyanarayan, A., & Sontag, D. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445522 · Cited by 73 · see citing works
Kreutzer, J., Uyheng, J., & Riezler, S. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p18-1165 · Cited by 38 · see citing works
QuickEdit: Editing Text and Translations by Crossing Words Out
Grangier, D., & Auli, M. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1025 · Cited by 20 · see citing works
- Authors
- Green, S., Wang, S. I., Chuang, J., Heer, J., Schuster, S., & Manning, C. D.
- Affiliations
- Stanford University (US)
- University of Washington (US)
- Venue
- Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP)
- Published
- 2014
- Pages
- 1225-1236
- OpenAlex
- W2251171258
- Cited by
- 68
- References
- 53
- Citations by year
- 2026: 2 · 2025: 1 · 2024: 2 · 2023: 5 · 2022: 4 · 2021: 10 · 2020: 6 · 2019: 8 · 2018: 6 · 2017: 5 · 2016: 12 · 2015: 7
- FWCI
- 4.71
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Conference on Empirical Methods in Natural Language Processing (EMNLP) (publishedVersion, cc-by)
- Funders
- National Science Foundation
- Defense Advanced Research Projects Agency
- Advanced Research Projects Agency
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Social Sciences › Arts and Humanities › Language and Linguistics › Translation Studies and Practices
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Keywords
- computer-assisted translation, human effort, translation quality, human-machine interaction, machine translation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
