Findings of the 2016 Conference on Machine Translation
Open accessAbstract
Ondřej Bojar, Rajen Chatterjee, Christian Federmann, Yvette Graham, Barry Haddow, Matthias Huck, Antonio Jimeno Yepes, Philipp Koehn, Varvara Logacheva, Christof Monz, Matteo Negri, Aurélie Névéol, Mariana Neves, Martin Popel, Matt Post, Raphael Rubino, Carolina Scarton, Lucia Specia, Marco Turchi, Karin Verspoor, Marcos Zampieri. Proceedings of the First Conference on Machine Translation: Volume 2, Shared Task Papers. 2016.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
LLMs are particularly good at translating some low-resource language texts to English, such as in Romanian-English translation of WMT'16, where zero-shot or few-shot LLMs can perform better than SOTA fine-tuned model.
“LLMs are particularly good at translating some low-resource language texts to English texts, such as in the Romanian-English translation of WMT'16 [13], zeroshot or few-shot LLMs can perform better than SOTA fine-tuned model [25].”
From Harnessing the power of llms in practice: a survey on chatGPT and beyond (2024) · cites this paper as [13]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
The Measurement of Observer Agreement for Categorical Data
Landis, J. R., & Koch, G. G. (1977) · Biometrics · doi:10.2307/2529310 · Cited by 81,148 · see citing works
Scikit-learn: Machine Learning in Python
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Müller, A., Nothman, J., Louppe, G., et al. (2012) · arXiv · doi:10.48550/arxiv.1201.0490 · Cited by 71,016 · see citing works
A Coefficient of Agreement for Nominal Scales
Cohen, J. (1960) · Educational and Psychological Measurement · doi:10.1177/001316446002000104 · Cited by 42,398 · see citing works
BLEU: a method for automatic evaluation of machine translation
Papineni, K., Roukos, S., Ward, T., & Zhu, W.-J. (2001) · Proceedings of the 40th Annual Meeting on Association for Computational Linguistics - ACL '02 · doi:10.3115/1073083.1073135 · Cited by 21,344 · see citing works
Neural Machine Translation by Jointly Learning to Align and Translate
Bahdanau, D., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1409.0473 · Cited by 16,213 · see citing works
Gaussian Processes for Machine Learning
Rasmussen, C. E., & Williams, C. K. I. (2005) · doi:10.7551/mitpress/3206.001.0001 · Cited by 11,705 · see citing works
Measuring nominal scale agreement among many raters.
Fleiss, J. L. (1971) · Psychological Bulletin · doi:10.1037/h0031619 · Cited by 8,733 · see citing works
Moses: open source toolkit for statistical machine translation
Koehn, P., Zens, R., Dyer, C., Bojar, O., Constantin, A., Herbst, E., Hoang, H., Birch, A., Callison-Burch, C., Federico, M., et al. (2007) · Proceedings of the 45th Annual Meeting of the ACL on Interactive Poster and Demonstration Sessions - ACL '07 · doi:10.3115/1557769.1557821 · Cited by 4,735 · see citing works
Self-citationA Systematic Comparison of Various Statistical Alignment Models
Och, F. J., & Ney, H. (2003) · Computational Linguistics · doi:10.1162/089120103321337421 · Cited by 3,855 · see citing works
Linguistic Regularities in Continuous Space Word Representations
Mikolov, T., Yih, W.-T., & Zweig, G. (2013) · Cited by 2,913 · see citing works
Minimum error rate training in statistical machine translation
Och, F. J. (2003) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.3115/1075096.1075117 · Cited by 2,728 · see citing works
A Study of Translation Edit Rate with Targeted Human Annotation
Snover, M., Dorr, B. J., Schwartz, R. M., Micciulla, L., & Makhoul, J. I. (2006) · Cited by 2,380 · see citing works
Moses : open source toolkit for statistical machine translation
Koehn, P., Hoang, H., Birch, A., Callison-Burch, C., Federico, M., Bertoldi, N., Cowan, B., Shen, W., Moran, C., Zens, R., et al. (2007) · Edinburgh Research Explorer (University of Edinburgh) · Cited by 1,547 · see citing works
Self-citationStatistical Significance Tests for Machine Translation Evaluation.
Koehn, P. (2004) · Cited by 1,456 · see citing works
Self-citationKenLM: Faster and Smaller Language Model Queries
Heafield, K. (2011) · Cited by 1,145 · see citing works
Findings of the 2014 Workshop on Statistical Machine Translation
Bojar, O., Buck, C., Federmann, C., Haddow, B., Koehn, P., Leveling, J., Monz, C., Pecina, P., Post, M., Saint-Amand, H., et al. (2014) · Conference on Machine Translation (WMT) · doi:10.3115/v1/w14-3302 · Cited by 705 · see citing works
Self-citationNews from OPUS — A collection of multilingual parallel corpora with tools and interfaces
Tiedemann, J. (2009) · Amsterdam studies in the theory and history of linguistic science. Series 4, Current issues in linguistic theory · doi:10.1075/cilt.309.19tie · Cited by 541 · see citing works
Koehn, P., & Hoang, H. (2007) · Edinburgh Research Explorer (University of Edinburgh) · Cited by 509 · see citing works
Self-citationLiang, P., Taskar, B., & Klein, D. (2006) · doi:10.3115/1220835.1220849 · Cited by 433 · see citing works
(Meta-) evaluation of machine translation
Callison-Burch, C., Fordyce, C. S., Koehn, P., Monz, C., & Schroeder, J. (2007) · Conference on Machine Translation (WMT) · doi:10.3115/1626355.1626373 · Cited by 381 · see citing works
Self-citationMore accurate tests for the statistical significance of result differences
Yeh, A. (2000) · Proceedings of the 18th conference on Computational linguistics - · doi:10.3115/992730.992783 · Cited by 378 · see citing works
Parallel implementations of word alignment tool
Gao, Q., & Vogel, S. (2008) · doi:10.3115/1622110.1622119 · Cited by 363 · see citing works
Findings of the 2009 workshop on statistical machine translation
Callison-Burch, C., Koehn, P., Monz, C., & Schroeder, J. (2009) · Conference on Machine Translation (WMT) · doi:10.3115/1626431.1626433 · Cited by 347 · see citing works
Self-citationPolyglot: Distributed Word Representations for Multilingual NLPPreprint
Al‐Rfou, R., Perozzi, B., & Skiena, S. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1307.1662 · Cited by 320 · see citing works
Findings of the 2013 Workshop on Statistical Machine Translation
Buck, C., Callison-Burch, C., Federmann, C., et al. (2013) · Cited by 306 · see citing works
Self-citationManual and automatic evaluation of machine translation between European languages
Koehn, P., & Monz, C. (2006) · Workshop on Statistical Machine Translation · doi:10.3115/1654650.1654666 · Cited by 272 · see citing works
Self-citationFurther meta-evaluation of machine translation
Callison-Burch, C., Fordyce, C. S., Koehn, P., Monz, C., & Schroeder, J. (2008) · Conference on Machine Translation (WMT) · doi:10.3115/1626394.1626403 · Cited by 268 · see citing works
Self-citationFindings of the 2013 Workshop on Statistical Machine Translation
Bojar, O., Buck, C., Callison-Burch, C., Federmann, C., Haddow, B., Koehn, P., Monz, C., Post, M., Soricut, R., & Specia, L. (2013) · UvA-DARE (University of Amsterdam) · Cited by 236 · see citing works
Self-citationA Shared Task on Multimodal Machine Translation and Crosslingual Image Description
Specia, L., Frank, S., Sima’an, K., & Elliott, D. (2016) · Conference on Machine Translation (WMT) · doi:10.18653/v1/w16-2346 · Cited by 228 · see citing works
Self-citationCallison-Burch, C., Koehn, P., Monz, C., Peterson, K., Przybocki, M. A., & Zaidan, O. F. (2010) · UvA-DARE (University of Amsterdam) · Cited by 180 · see citing works
Self-citationStatistical Phrase-Based Post-Editing
Simard, M., Goutte, C., & Isabelle, P. (2007) · CogPrints (University of Southampton) · Cited by 166 · see citing works
Turning on the Turbo: Fast Third-Order Non-Projective Turbo Parsers
Martins, A. F. T., Almeida, M., & Smith, N. A. (2018) · KiltHub Repository · doi:10.1184/r1/6473801 · Cited by 156 · see citing works
N-gram Counts and Language Models from the Common Crawl
Buck, C., Heafield, K., & van Ooyen, B. (2014) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/3uyduafd26my · Cited by 154 · see citing works
Can machine translation systems be evaluated by the crowd alone
Graham, Y., Baldwin, T., Moffat, A., & Zobel, J. (2015) · Natural Language Engineering · doi:10.1017/s1351324915000339 · Cited by 148 · see citing works
Self-citationContinuous Measurement Scales in Human Evaluation of Machine Translation
Graham, Y., Baldwin, T., Moffat, A., & Zobel, J. (2013) · Trinity's Access to Research Output (TARA) (Trinity College Dublin) · Cited by 146 · see citing works
Self-citationPET: a Tool for Post-editing and Assessing Machine Translation
Aziz, W. F., Castilho, S., & Specia, L. (2012) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/5cnb4whoygrr · Cited by 138 · see citing works
Self-citationResults of the WMT16 Metrics Shared Task
Bojar, O., Graham, Y., Kamran, A., & Stanojević, M. (2016) · Conference on Machine Translation · doi:10.18653/v1/w16-2302 · Cited by 128 · see citing works
Self-citationMulti-level Translation Quality Prediction with QuEst++
Specia, L., Paetzold, G. H., & Scarton, C. (2015) · doi:10.3115/v1/p15-4020 · Cited by 121 · see citing works
Self-citationA Joint Sequence Translation Model with Integrated Reordering
Durrani, N., Schmid, H., & Fraser, A. (2011) · Cited by 105 · see citing works
Results of the WMT14 Metrics Shared Task
Macháċek, M., & Bojar, O. (2014) · Conference on Machine Translation (WMT) · doi:10.3115/v1/w14-3336 · Cited by 105 · see citing works
Self-citationIntroducing the SPMRL 2014 Shared Task on Parsing Morphologically-rich Languages
Seddah, D., Kübler, S., & Tsarfaty, R. (2014) · Cited by 93 · see citing works
Asiya: An Open Toolkit for Automatic Machine Translation (Meta-)Evaluation
Giménez, J., & Màrquez, L. (2010) · The Prague Bulletin of Mathematical Linguistics · doi:10.2478/v10108-010-0022-6 · Cited by 91 · see citing works
Is Machine Translation Getting Better over Time?
Graham, Y., Baldwin, T. J., Moffat, A., et al. (2014) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.3115/v1/e14-1047 · Cited by 66 · see citing works
Self-citationKreutzer, J., Schamoni, S., & Riezler, S. (2015) · Conference on Machine Translation (WMT) · doi:10.18653/v1/w15-3037 · Cited by 65 · see citing works
CzEng 1.6: Enlarged Czech-English Parallel Corpus with Processing Tools Dockered
Bojar, O., Dušek, O., Kocmi, T., et al. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-45510-5_27 · Cited by 61 · see citing works
Self-citationEfficient Elicitation of Annotations for Human Evaluation of Machine Translation
Sakaguchi, K., Post, M., & Durme, B. V. (2014) · Workshop on Statistical Machine Translation · doi:10.3115/v1/w14-3301 · Cited by 57 · see citing works
Self-citationStatistical Post-Editing for a Statistical MT System
Béchara, H., Ma, Y., & van Genabith, J. (2011) · Cited by 55 · see citing works
Junczys-Dowmunt, M., Dwojak, T., & Sennrich, R. (2016) · Conference on Machine Translation (WMT) · doi:10.18653/v1/w16-2316 · Cited by 52 · see citing works
Improving Evaluation of Machine Translation Quality Estimation
Graham, Y. (2015) · Annual Meeting of the Association for Computational Linguistics · doi:10.3115/v1/p15-1174 · Cited by 51 · see citing works
Self-citationChatterjee, R., Weller, M., Negri, M., & Turchi, M. (2015) · doi:10.3115/v1/p15-2026 · Cited by 51 · see citing works
Self-citationFindings of the WMT 2016 Bilingual Document Alignment Shared Task
Buck, C., & Koehn, P. (2016) · Conference on Machine Translation (WMT) · doi:10.18653/v1/w16-2347 · Cited by 50 · see citing works
Self-citationFBK-UPV-UEdin participation in the WMT14 Quality Estimation shared-task
de Souza, J. G. C., González-Rubio, J., Buck, C., et al. (2014) · Workshop on Statistical Machine Translation · doi:10.3115/v1/w14-3340 · Cited by 45 · see citing works
Findings of the 2016 WMT Shared Task on Cross-lingual Pronoun Prediction
Guillou, L., Hardmeier, C., Nakov, P., et al. (2016) · doi:10.18653/v1/w16-2345 · Cited by 45 · see citing works
Unbabel's Participation in the WMT16 Word-Level Translation Quality Estimation Shared Task
Martins, A. F. T., Astudillo, R. F., Hokamp, C., et al. (2016) · doi:10.18653/v1/w16-2387 · Cited by 41 · see citing works
MultiVec: a Multilingual and Multilevel Representation Learning Toolkit for NLP
Bérard, A., Servan, C., Pietquin, O., & Besacier, L. (2016) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/52gcepr8aezz · Cited by 38 · see citing works
“This sentence is wrong.” Detecting errors in machine-translated sentences
Raybaud, S., Langlois, D., & Smaı̈li, K. (2011) · Machine Translation · doi:10.1007/s10590-011-9094-9 · Cited by 35 · see citing works
Latent Domain Translation Models in Mix-of-Domains Haystack
Hoang, C., & Sima’an, K. (2014) · International Conference on Computational Linguistics · Cited by 35 · see citing works
The Scielo Corpus: a Parallel Corpus of Scientific Publications for Biomedicine
Neves, M., Yepes, A. J., & Névéol, A. (2016) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/2c6qrwhwmb33 · Cited by 33 · see citing works
Self-citationFBK-UEdin Participation to the WMT13 Quality Estimation Shared Task
de Souza, J. G. C., Buck, C., Turchi, M., & Negri, M. (2013) · Cited by 29 · see citing works
Self-citationAppraise: an Open-Source Toolkit for Manual Evaluation of MT Output
Federmann, C. (2012) · The Prague Bulletin of Mathematical Linguistics · doi:10.2478/v10108-012-0006-9 · Cited by 29 · see citing works
Self-citationRecurrent Neural Network based Translation Quality Estimation
Kim, H. J., & Lee, J.-H. (2016) · doi:10.18653/v1/w16-2384 · Cited by 29 · see citing works
The FBK Participation in the WMT15 Automatic Post-editing Shared Task
Chatterjee, R., Turchi, M., & Negri, M. (2015) · Conference on Machine Translation (WMT) · doi:10.18653/v1/w15-3025 · Cited by 29 · see citing works
Self-citationYSDA Participation in the WMT'16 Quality Estimation Shared Task
Kozlova, A., Shmatova, M., & Frolov, A. (2016) · doi:10.18653/v1/w16-2385 · Cited by 27 · see citing works
Herrmann, T., Niehues, J., & Waibel, A. (2013) · doi:10.5445/ir/1000037725 · Cited by 26 · see citing works
LIG System for Word Level QE task at WMT14
Luong, N.-Q., Besacier, L., & Lecouteux, B. (2014) · Conference on Machine Translation (WMT) · doi:10.3115/v1/w14-3342 · Cited by 24 · see citing works
Sánchez-Cartagena, V. M., & Toral, A. (2016) · doi:10.18653/v1/w16-2322 · Cited by 24 · see citing works
MARMOT: A Toolkit for Translation Quality Estimation at the Word Level
Logacheva, V., Hokamp, C., & Specia, L. (2016) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/3f97od6zyuwm · Cited by 22 · see citing works
Self-citationSearching for Context: a Study on Document-Level Labels for Translation Quality Estimation
Scarton, C., Zampieri, M., Vela, M., et al. (2015) · Cited by 21 · see citing works
Self-citationSHEF-LIUM-NN: Sentence level Quality Estimation with Neural Network Features
Shah, K., Bougares, F., Barrault, L., et al. (2016) · doi:10.18653/v1/w16-2392 · Cited by 20 · see citing works
Edinburgh's Statistical Machine Translation Systems for WMT16
Williams, P. J., Sennrich, R., Nădejde, M., Huck, M., Haddow, B., & Bojar, O. (2016) · Conference on Machine Translation (WMT) · doi:10.18653/v1/w16-2327 · Cited by 19 · see citing works
Self-citationThe JHU Machine Translation Systems for WMT 2016
Ding, S., Duh, K., Khayrallah, H., Koehn, P., & Post, M. (2016) · Conference on Machine Translation (WMT) · doi:10.18653/v1/w16-2310 · Cited by 19 · see citing works
Self-citationTranslation Quality Estimation using Recurrent Neural Network
Patel, R. N., & Sasikumar, M. (2016) · doi:10.18653/v1/w16-2389 · Cited by 18 · see citing works
The FBK Participation in the WMT 2016 Automatic Post-editing Shared Task
Chatterjee, R., de Souza, J. G. C., Negri, M., et al. (2016) · doi:10.18653/v1/w16-2377 · Cited by 18 · see citing works
Self-citationThe JHU Machine Translation Systems for WMT 2017
Ding, S., Khayrallah, H., Koehn, P., et al. (2017) · Conference on Machine Translation (WMT) · doi:10.18653/v1/w17-4724 · Cited by 17 · see citing works
Self-citationMetaMind Neural Machine Translation System for WMT 2016
Bradbury, J., & Socher, R. (2016) · Conference on Machine Translation (WMT) · doi:10.18653/v1/w16-2308 · Cited by 16 · see citing works
UAlacant word-level machine translation quality estimation system at WMT 2015
Esplà-Gomis, M., Sánchez-Martínez, F., & Forcada, M. L. (2015) · Workshop on Statistical Machine Translation · doi:10.18653/v1/w15-3036 · Cited by 15 · see citing works
Referential translation machines for predicting semantic similarity
Biçici, E., & Way, A. (2015) · Language Resources and Evaluation · doi:10.1007/s10579-015-9322-7 · Cited by 15 · see citing works
Tiedemann, J., Cap, F., Kanerva, J., et al. (2016) · doi:10.18653/v1/w16-2326 · Cited by 15 · see citing works
Document-level translation quality estimation: exploring discourse and pseudo-references
Scarton, C., & Specia, L. (2014) · Cited by 15 · see citing works
Self-citationUSAAR-SAPE: An English–Spanish Statistical Automatic Post-Editing System
Pal, S., Vela, M., Naskar, S. K., & van Genabith, J. (2015) · Conference on Machine Translation (WMT) · doi:10.18653/v1/w15-3026 · Cited by 14 · see citing works
USAAR: An Operation Sequential Model for Automatic Statistical Post-Editing
Pal, S., Zampieri, M., & van Genabith, J. (2016) · Conference on Machine Translation (WMT) · doi:10.18653/v1/w16-2379 · Cited by 14 · see citing works
Self-citationCosta‐jussà, M. R., España-Bonet, C., Madhyastha, P., et al. (2016) · doi:10.18653/v1/w16-2336 · Cited by 14 · see citing works
The QT21/HimL Combined Machine Translation System
Peter, J.-T., Alkhouli, T., Ney, H., Huck, M., Braune, F., Fraser, A., Tamchyna, A., Bojar, O., Haddow, B., Sennrich, R., et al. (2016) · Conference on Machine Translation (WMT) · doi:10.18653/v1/w16-2320 · Cited by 13 · see citing works
Self-citationCohere: A Toolkit for Local Coherence
Smith, K., Aziz, W. F., & Specia, L. (2016) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/4dxkavwdfm5d · Cited by 11 · see citing works
Self-citationThe Edit Distance Transducer in Action: The University of Cambridge English-German System at WMT16
Stahlberg, F., Hasler, E., & Byrne, B. (2016) · doi:10.18653/v1/w16-2324 · Cited by 10 · see citing works
LIMSI$@$WMT’16: Machine Translation of News
Allauzen, A., Aufrant, L., Burlot, F., et al. (2016) · doi:10.18653/v1/w16-2304 · Cited by 10 · see citing works
The Karlsruhe Institute of Technology Systems for the News Translation Task in WMT 2018
Ha, T.-L., Niehues, J., & Waibel, A. (2018) · Repository KITopen (Karlsruhe Institute of Technology) · doi:10.5445/ir/1000145062 · Cited by 10 · see citing works
NYU-MILA Neural Machine Translation Systems for WMT’16
Chung, J., Cho, K., & Bengio, Y. (2016) · doi:10.18653/v1/w16-2309 · Cited by 9 · see citing works
NRC Russian-English Machine Translation System for WMT 2016
Lo, C.-K., Cherry, C., Foster, G., et al. (2016) · doi:10.18653/v1/w16-2317 · Cited by 9 · see citing works
CUNI-LMU Submissions in WMT2016: Chimera Constrained and Beaten
Tamchyna, A., Sudarikov, R., Bojar, O., et al. (2016) · doi:10.18653/v1/w16-2325 · Cited by 9 · see citing works
Self-citationThe Karlsruhe Institute of Technology Systems for the News Translation Task in WMT 2017
Pham, N.-Q., Niehues, J., Ha, T.-L., Cho, E., Sperber, M., & Waibel, A. H. (2017) · Conference on Machine Translation (WMT) · doi:10.18653/v1/w17-4736 · Cited by 9 · see citing works
Dušek, O., Gomes, L., Novák, M., et al. (2015) · Workshop on Statistical Machine Translation · doi:10.18653/v1/w15-3009 · Cited by 8 · see citing works
Word embeddings and discourse information for Quality Estimation
Scarton, C., Beck, D., Shah, K., et al. (2016) · doi:10.18653/v1/w16-2391 · Cited by 8 · see citing works
Self-citationDFKI's system for WMT16 IT-domain task, including analysis of systematic errors
Avramidis, E., Burchardt, A., Macketanz, V., & Srivastava, A. (2016) · Conference on Machine Translation (WMT) · doi:10.18653/v1/w16-2329 · Cited by 8 · see citing works
Data Selection for IT Texts using Paragraph Vector
Duma, M.-S., & Menzel, W. (2016) · doi:10.18653/v1/w16-2331 · Cited by 8 · see citing works
LIMSI's Contribution to the WMT'16 Biomedical Translation Task
Ive, J., Max, A., & Yvon, F. (2016) · doi:10.18653/v1/w16-2337 · Cited by 8 · see citing works
Bilingual Embeddings and Word Alignments for Translation Quality Estimation
Abdelsalam, A., Bojar, O., & El-Beltagy, S. R. (2016) · doi:10.18653/v1/w16-2380 · Cited by 8 · see citing works
Self-citationSimpleNets: Quality Estimation with Resource-Light Neural Networks
Paetzold, G. H., & Specia, L. (2016) · doi:10.18653/v1/w16-2388 · Cited by 7 · see citing works
Self-citationThe Edinburgh/LMU Hierarchical Machine Translation System for WMT 2016
Huck, M., Fraser, A., & Haddow, B. (2016) · doi:10.18653/v1/w16-2315 · Cited by 7 · see citing works
Self-citationThe AFRL-MITLL WMT16 News-Translation Task Systems
Gwinnup, J., Anderson, T., Erdmann, G., Young, K., Kazi, M., Salesky, E., & Thompson, B. J. (2016) · Conference on Machine Translation (WMT) · doi:10.18653/v1/w16-2313 · Cited by 7 · see citing works
ParFDA for Instance Selection for Statistical Machine Translation
Biçici, E. (2016) · doi:10.18653/v1/w16-2306 · Cited by 7 · see citing works
USHEF and USAAR-USHEF participation in the WMT15 QE shared task
Scarton, C., Tan, L., & Specia, L. (2015) · Workshop on Statistical Machine Translation · doi:10.18653/v1/w15-3040 · Cited by 6 · see citing works
Self-citationThe UU Submission to the Machine Translation Quality Estimation Task
Sagemo, O., & Stymne, S. (2016) · Conference on Machine Translation (WMT) · doi:10.18653/v1/w16-2390 · Cited by 6 · see citing works
Aires, J., Lopes, G. P., & Gomes, L. (2016) · doi:10.18653/v1/w16-2335 · Cited by 6 · see citing works
IXA Biomedical Translation System at WMT16 Biomedical Translation Task
Perez-de-Viñaspre, O., & Labaka, G. (2016) · doi:10.18653/v1/w16-2338 · Cited by 6 · see citing works
Metrics for Evaluation of Word-level Machine Translation Quality Estimation
Logacheva, V., Łukasik, M., & Specia, L. (2016) · Annual Meeting of the Association for Computational Linguistics (ACL) · doi:10.18653/v1/p16-2095 · Cited by 6 · see citing works
Self-citationResults of the WMT16 Tuning Shared Task
Jawaid, B., Kamran, A., Stanojević, M., et al. (2016) · doi:10.18653/v1/w16-2303 · Cited by 6 · see citing works
SMT and Hybrid systems of the QTLeap project in the WMT16 IT-task
Gaudio, R., Labaka, G., Agirre, E., et al. (2016) · doi:10.18653/v1/w16-2332 · Cited by 5 · see citing works
UAlacant word-level and phrase-level machine translation quality estimation systems at WMT 2016
Esplà-Gomis, M., Sánchez-Martínez, F., & Forcada, M. L. (2016) · doi:10.18653/v1/w16-2383 · Cited by 5 · see citing works
Dictionary-based Domain Adaptation of MT Systems without Retraining
Rosa, R., Sudarikov, R., Novák, M., et al. (2016) · doi:10.18653/v1/w16-2334 · Cited by 5 · see citing works
Referential Translation Machines for Predicting Translation Performance
Biçici, E. (2016) · doi:10.18653/v1/w16-2382 · Cited by 5 · see citing works
ILLC-UvA Adaptation System (Scorpio) at WMT'16 IT-DOMAIN Task
Cuong, H., Frank, S., & Sima’an, K. (2016) · doi:10.18653/v1/w16-2330 · Cited by 4 · see citing works
SHEF-MIME: Word-level Quality Estimation Using Imitation Learning
Beck, D., Vlachos, A., Paetzold, G. H., et al. (2016) · doi:10.18653/v1/w16-2381 · Cited by 4 · see citing works
Hybrid Morphological Segmentation for Phrase-Based Machine Translation
Grönroos, S.-A., Virpioja, S., & Kurimo, M. (2016) · doi:10.18653/v1/w16-2312 · Cited by 4 · see citing works
Merged bilingual trees based on Universal Dependencies in Machine Translation
Mareček, D. (2016) · doi:10.18653/v1/w16-2318 · Cited by 4 · see citing works
The RWTH Aachen University English-Romanian Machine Translation System for WMT 2016
Peter, J.-T., Alkhouli, T., Guta, A., et al. (2016) · doi:10.18653/v1/w16-2321 · Cited by 4 · see citing works
UGENT-LT3 SCATE Submission for WMT16 Shared Task on Quality Estimation
Tezcan, A., Hoste, V., & Macken, L. (2016) · doi:10.18653/v1/w16-2393 · Cited by 3 · see citing works
Dvorkovich, A., Gubanov, S., & Galinskaya, I. (2016) · doi:10.18653/v1/w16-2311 · Cited by 3 · see citing works
PROMT Translation Systems for WMT 2016 Translation Tasks
Molchanov, A. P., & Bykov, F. (2016) · doi:10.18653/v1/w16-2319 · Cited by 3 · see citing works
Sheffield Systems for the English-Romanian WMT Translation Task
Blain, F., Song, X., & Specia, L. (2016) · doi:10.18653/v1/w16-2307 · Cited by 2 · see citing works
Self-citationJU-USAAR: A Domain Adaptive MT System
Pahari, K., Kuila, A., Pal, S., Naskar, S. K., Bandyopadhyay, S., & van Genabith, J. (2016) · Conference on Machine Translation (WMT) · doi:10.18653/v1/w16-2333 · Cited by 2 · see citing works
PJAIT Systems for the WMT 2016
Wołk, K., & Marasek, K. P. (2016) · doi:10.18653/v1/w16-2328 · Cited by 1 · see citing works
Flagged first, then by citations
Research works printed but not linked: 20
- Allauzen, A., Aufrant, L., Burlot, F., Lacroix, O., Knyazeva, E., Lavergne, T., Wisniewski, G., and Yvon, F. (2016). LIMSI@WMT16: Ma- chine Translation of News. In Proceedings of the First Conference on Machine Transla- tion, Berlin, Germany. Association for Compu- tational Linguistics.
- Bektas ¸, E., Yilmaz, E., Mermer, C., and Dur- gar El-Kahlout, . (2016). T ÜBTAK SMT Sys- tem Submission for WMT2016. In Proceed- ings of the First Conference on Machine Trans- lation, Berlin, Germany. Association for Com- putational Linguistics.
- Bic ¸ici, E. and Way, A. (2015). Referential trans- lation machines for predicting semantic similar- ity. Language Resources and Evaluation, pages 1-27.
- Bojar, O., Chatterjee, R., Federmann, C., Had- dow, B., Huck, M., Hokamp, C., Koehn, P., Logacheva, V., Monz, C., Negri, M., Post, M., Scarton, C., Specia, L., and Turchi, M. (2015). Findings of the 2015 workshop on statistical machine translation. In Proceedings of the Tenth Workshop on Statistical Machine Transla- tion, pages 1-46, Lisbon, Portugal. Association for Computational Linguistics.
- Bojar, O., Ercegovčević, M., Popel, M., and Zaidan, O. (2011). A grain of salt for the wmt manual evaluation. In Proceedings of the Sixth Workshop on Statistical Machine Translation, pages 1-11, Edinburgh, Scotland. Association for Computational Linguistics.
- Callison-Burch, C., Koehn, P., Monz, C., and Zaidan, O. (2011). Findings of the 2011 work- shop on statistical machine translation. In Pro- ceedings of the Sixth Workshop on Statistical Machine Translation, pages 22-64, Edinburgh, Scotland. Association for Computational Lin- guistics.
- Chung, J., Cho, K., and Bengio, Y. (2016). NYU- MILA Neural Machine Translation Systems for WMT16. In Proceedings of the First Confer- ence on Machine Translation, Berlin, Germany. Association for Computational Linguistics.
- Junczys-Dowmunt, M. and Grundkiewicz, R. (2016). Log-linear Combinations of Mono- lingual and Bilingual Neural Machine Trans- lation Models for Automatic Post-Editing. In Proceedings of the First Conference on Ma- chine Translation. Association for Computa- tional Linguistics.
- Kim, H. and Lee, J.-H. (2016). Recurrent Neu- ral Network based Translation Quality Estima- tion. In Proceedings of the First Conference on Machine Translation, Berlin, Germany. Associ- ation for Computational Linguistics.
- Liang, P., Taskar, B., and Klein, D. (2006). Align- ment by agreement. In Proceedings of the main conference on Human Language Technol- ogy Conference of the North American Chapter of the Association of Computational Linguis- tics, pages 104-111. Association for Computa- tional Linguistics.
- Libovický, J., Helcl, J., Tlustý, M., Bojar, O., and Pecina, P. (2016). CUNI at Post-editing and Multimodal Translation Tasks. In Proceedings of the First Conference on Machine Translation. Association for Computational Linguistics.
- Logacheva, V., , and Specia, L. (2015). Phrase- level Quality Estimation for Machine Transla- tion. In Proceedings of the 12th International Workshop on Spoken Language Translation, Da Nang, Vietnam.
- Logacheva, V., Blain, F., and Specia, L. (2016a). USFD Phrase-level Quality Estimation Sys- tems. In Proceedings of the First Conference on Machine Translation, Berlin, Germany. As- sociation for Computational Linguistics.
- Och, F. J. and Ney, H. (2003). A Systematic Com- parison of Various Statistical Alignment Mod- els. Comput. Linguist., 29(1):19-51.
- Rosa, R., Sudarikov, R., Novák, M., Popel, M., and Bojar, O. (2016). Dictionary-based Do- main Adaptation of MT Systems without Re- training. In Proceedings of the First Conference on Machine Translation, Berlin, Germany. As- sociation for Computational Linguistics.
- Sagemo, O. and Stymne, S. (2016). The UU Sub- mission to the Machine Translation Quality Es- timation Task. In Proceedings of the First Con- ference on Machine Translation, Berlin, Ger- many. Association for Computational Linguis- tics.
- Sennrich, R., Haddow, B., and Birch, A. (2015). Neural machine translation of rare words with subword units. arXiv preprint arXiv:1508.07909. · arXiv
- Sennrich, R., Haddow, B., and Birch, A. (2016). Edinburgh Neural Machine Translation Sys- tems for WMT 16. In Proceedings of the First Conference on Machine Translation, Berlin, Germany. Association for Computational Lin- guistics.
- Sudarikov, R., Popel, M., Bojar, O., Burchardt, A., and Klejch, O. (2016). Using MT-ComparEval. In Translation Evaluation: From Fragmented Tools and Data Sets to an Integrated Ecosystem, pages 76-82.
- Williams, P., Sennrich, R., Nadejde, M., Huck, M., Haddow, B., and Bojar, O. (2016). Ed- inburgh's Statistical Machine Translation Sys- tems for WMT16. In Proceedings of the First Conference on Machine Translation, Berlin, Germany. Association for Computational Lin- guistics.
Cited by
Sorted by citations
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929 · see citing works
Convolutional Sequence to Sequence Learning
Gehring, J., Auli, M., Grangier, D., Yarats, D., & Dauphin, Y. N. (2017) · arXiv · doi:10.48550/arxiv.1705.03122 · Cited by 2,850 · see citing works
Efficient Memory Management for Large Language Model Serving with PagedAttention
Kwon, W., Li, Z., Zhuang, S., Sheng, Y., Zheng, L., Yu, C. H., Gonzalez, J., Zhang, H., & Stoica, I. (2023) · Proceedings of the 29th Symposium on Operating Systems Principles · doi:10.1145/3600006.3613165 · Cited by 1,817 · see citing works
Efficient Memory Management for Large Language Model Serving with PagedAttention
Kwon, W., Li, Z., Zhuang, S., Sheng, Y., Zheng, L., Yu, C. H., Gonzalez, J. E., Zhang, H., & Stoica, I. (2023) · arXiv · doi:10.48550/arxiv.2309.06180 · Cited by 1,802 · see citing works
Harnessing the power of llms in practice: a survey on chatGPT and beyond
Yang, J., Jin, H., Tang, R., Han, X., Feng, Q., Jiang, H., Zhong, S., Yin, B., & Hu, X. (2024) · ACM Transactions on Knowledge Discovery from Data · doi:10.1145/3649506 · Cited by 644 · see citing works
Experts, Errors, and Context: A Large-Scale Study of Human Evaluation for Machine Translation
Freitag, M., Foster, G., Grangier, D., Ratnakar, V., Tan, Q., & Macherey, W. (2021) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00437 · Cited by 271 · see citing works
Best practices for the human evaluation of automatically generated text
van der Lee, C., Gatt, A., van Miltenburg, E., Wubben, S., & Krahmer, E. (2019) · Proceedings of the 12th International Conference on Natural Language Generation · doi:10.18653/v1/w19-8643 · Cited by 205 · see citing works
“…et al., 2016a), recent editions of the Conference on Machine Translation (WMT), have used Direct Assessment, whereby participants compare an output to a reference text on a continuous (0-100) scale (Graham et al., 2017; Bojar et al., 2016b), similar to Magnitude Estimation (Bard et al., 1996).”
MethodA Survey of Evaluation Metrics Used for NLG Systems
Sai, A. B., Mohankumar, A. K., & Khapra, M. M. (2023) · ACM Computing Surveys · doi:10.1145/3485766 · Cited by 235 · see citing works
Accelerating Transformer Inference for Translation via Parallel Decoding
Santilli, A., Severino, S., Postolache, E., Maiorca, V., Mancusi, M., Marin, R., & Rodola, E. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2023.acl-long.689 · Cited by 29 · see citing works
Efficient Benchmarking (of Language Models)
Perlitz, Y., Bandel, E., Gera, A., Arviv, O., Ein-Dor, L., Shnarch, E., Slonim, N., Shmueli-Scheuer, M., & Choshen, L. (2024) · Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers) · doi:10.18653/v1/2024.naacl-long.139 · Cited by 7 · see citing works
Kondo, T., Donkers, J., Nishigori, H., Rovers, S., & Heeneman, S. (2026) · JMIR Medical Education · doi:10.2196/90064 · Cited by 1 · see citing works
MesaNet: Sequence Modeling by Locally Optimal Test-Time Training
von Oswald, J., Scherrer, N., Kobayashi, S., Versari, L., Yang, S., Mittal, S., Schlegel, M., Maile, K., Schimpf, Y., Sieberling, O., et al. (2025) · arXiv
“We evaluate on two GPT3 word scrambling tasks (cycle letters in word, anagrams of all but first and last two characters) [17] and three translation tasks (WMT-14 FR-EN [14] , WMT-16 DE-EN and RO-EN [15] ).”
Efficient Benchmarking (of Language Models)
Perlitz, Y., Bandel, E., Gera, A., Arviv, O., Ein-Dor, L., Shnarch, E., Slonim, N., Shmueli-Scheuer, M., & Choshen, L. (2023) · arXiv · doi:10.48550/arxiv.2308.11696
- Authors
- Bojar, O., Chatterjee, R., Federmann, C., Graham, Y., Haddow, B., Huck, M., Jimeno Yepes, A., Koehn, P., Logacheva, V., Monz, C., Negri, M., Neveol, A., Neves, M., Popel, M., Post, M., Rubino, R., Scarton, C., Specia, L., Turchi, M., Verspoor, K., & Zampieri, M.
- Affiliations
- Ludwig-Maximilians-Universität München (DE)
- Microsoft (United States) (US)
- Dublin City University (IE)
- University of Edinburgh (GB)
- University of Sheffield (GB)
- University of Amsterdam (NL)
- Fondazione Bruno Kessler (IT)
- Hasso Plattner Institute (DE)
- Charles University (CZ)
- Johns Hopkins University (US)
- The University of Melbourne (AU)
- ORCID
- Venue
- Proceedings of the First Conference on Machine Translation: Volume 2, Shared Task Papers
- Published
- 2016
- Pages
- 131-198
- OpenAlex
- W2512924740
- Cited by
- 539
- References
- 122
- Citations by year
- 2026: 7 · 2025: 12 · 2024: 24 · 2023: 57 · 2022: 54 · 2021: 81 · 2020: 62 · 2019: 64 · 2018: 89 · 2017: 75 · 2016: 13
- FWCI
- 43.29
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Conference on Machine Translation (WMT) (publishedVersion, cc-by)
- RWTH Publications (RWTH Aachen) (publishedVersion, other-oa)
- Edinburgh Research Explorer (University of Edinburgh) (submittedVersion, cc-by)
- UvA-DARE (University of Amsterdam) (submittedVersion, cc-by)
- Figshare (submittedVersion, cc-by)
- Institutional Research Information System (Università degli Studi di Trento) (submittedVersion, cc-by-nc-sa)
- Data Archiving and Networked Services (DANS) (submittedVersion, other-oa)
- Funders
- European Commission (H2020)
- Horizon 2020 Framework Programme
- Air Force Research Laboratory
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
- Keywords
- machine translation, shared task, translation quality assessment, low-resource languages, multilingual translation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review