CoChat
Sign Up
Cited by 89Natural Language Processing TechniquesSpeech Recognition and SynthesisTranslation Studies and Practices

The DOI matches its Crossref record; 1 of the 1 references it cites were found in a registry or an index.

Integrating an Unsupervised Transliteration Model into Statistical Machine Translation

Open access
Durrani, Nadir, Sajjad, Hassan, Hoang, Hieu, Koehn, Philipp·2014·Proceedings of the 14th Conference of the European Chapter of the Association for Computational Linguistics, volume 2: Short Papers
DOI 10.3115/v1/e14-4029
Open this paper in CoChat — free accountOpen PDF

Abstract

We investigate three methods for integrating an unsupervised transliteration model into an end-to-end SMT system.We induce a transliteration model from parallel data and use it to translate OOV words.Our approach is fully unsupervised and language independent.In the methods to integrate transliterations, we observed improvements from 0.23-0.75(∆ 0.41) BLEU points across 7 language pairs.We also show that our mined transliteration corpora provide better rule coverage and translation quality compared to the gold standard transliteration corpora.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.3115/v1/e14-4029

Cites

1 of 1 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • If alphabets differ, transliteration is required.

    “If alphabets differ, transliteration is required (Durrani et al., 2014).”

    From Neural Machine Translation of Rare Words with Subword Units (2016) · cites this paper as Durrani et al., 2014

    The Introduction says that transliteration can translate some out-of-vocabulary words and improve machine translation quality. The paper applies it to several language pairs, but does not say that differing alphabets make transliteration required; its discussion concerns particular words that can be transliterated, not every translation between different alphabets. (LLM, conf=0.82)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Flagged first, then by citations

Research works printed but not linked: 34

  1. Al-Onaizan, Y. and Knight, K. (2002). Translating named entities using monolingual and bilingual resources. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics.
  2. Azab, M., Bouamor, H., Mohit, B., and Oflazer, K. (2013). Dudley north visits north london: Learning when to transliterate to arabic. In Pro- ceedings of the 2013 Conference of the North American Chapter of the Association for Com- putational Linguistics: Human Language Tech- nologies, pages 439-444, Atlanta, Georgia. As- sociation for Computational Linguistics.
  3. Birch, A., Durrani, N., and Koehn, P. (2013). Ed- inburgh SLT and MT system description for the IWSLT 2013 evaluation. In Proceedings of the 10th International Workshop on Spoken Lan- guage Translation, pages 40-48, Heidelberg, Germany.
  4. Bojar, O., Buck, C., Callison-Burch, C., Feder- mann, C., Haddow, B., Koehn, P., Monz, C., Post, M., Soricut, R., and Specia, L. (2013). Findings of the 2013 Workshop on Statistical Machine Translation. In Eighth Workshop on Statistical Machine Translation, WMT-2013, pages 1-44, Sofia, Bulgaria.
  5. Cettolo, M., Girardi, C., and Federico, M. (2012). Wit 3 : Web inventory of transcribed and trans- lated talks. In Proceedings of the 16 th Confer- ence of the European Association for Machine Translation (EAMT), pages 261-268, Trento, Italy.
  6. Cherry, C. and Foster, G. (2012). Batch tun- ing strategies for statistical machine transla- tion. In Proceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Lan- guage Technologies, pages 427-436, Montréal, Canada. Association for Computational Lin- guistics.
  7. Darwish, K. (2010). Transliteration mining with phonetic conflation and iterative training. In Proceedings of the 2010 Named Entities Work- shop, Uppsala, Sweden.
  8. Durrani, N., Fraser, A., Schmid, H., Hoang, H., and Koehn, P. (2013a). Can Markov Models Over Minimal Translation Units Help Phrase- Based SMT? In Proceedings of the 51st An- nual Meeting of the Association for Computa- tional Linguistics, Sofia, Bulgaria. Association for Computational Linguistics.
  9. Durrani, N., Haddow, B., Heafield, K., and Koehn, P. (2013b). Edinburgh's machine translation systems for European language pairs. In Pro- ceedings of the Eighth Workshop on Statistical Machine Translation, Sofia, Bulgaria. Associa- tion for Computational Linguistics.
  10. Durrani, N. and Hussain, S. (2010). Urdu word segmentation. In Human Language Technolo- gies: The 2010 Annual Conference of the North American Chapter of the Association for Com- putational Linguistics, pages 528-536, Los An- geles, California. Association for Computa- tional Linguistics.
  11. Durrani, N., Sajjad, H., Fraser, A., and Schmid, H. (2010). Hindi-to-urdu machine translation through transliteration. In Proceedings of the 48th Annual Conference of the Association for Computational Linguistics, Uppsala, Sweden.
  12. Goldwasser, D. and Roth, D. (2008). Active sam- ple selection for named entity transliteration. In Proceedings of ACL-08: HLT, Short Papers, pages 53-56, Columbus, Ohio. Association for Computational Linguistics.
  13. Habash, N. (2009). Remoov: A tool for on- line handling of out-of-vocabulary words in ma- chine translation. In Proceedings of the Sec- ond International Conference on Arabic Lan- guage Resources and Tools, Cairo, Egypt. The MEDAR Consortium.
  14. Habash, N. and Sadat, F. (2006). Arabic prepro- cessing schemes for statistical machine transla- tion. In Proceedings of the Human Language Technology Conference of the NAACL, Com- panion Volume: Short Papers, pages 49-52, New York City, USA. Association for Compu- tational Linguistics.
  15. Hasler, E., Haddow, B., and Koehn, P. (2012). Sparse lexicalised features and topic adaptation for smt. In Proceedings of the seventh Interna- tional Workshop on Spoken Language Transla- tion (IWSLT), pages 268-275.
  16. Heafield, K. (2011). Kenlm: Faster and smaller language model queries. In Proceedings of the Sixth Workshop on Statistical Machine Trans- lation, pages 187-197, Edinburgh, Scotland, United Kingdom.
  17. Hermjakob, U., Knight, K., and Daumé III, H. (2008). Name translation in statistical machine translation -learning when to transliterate. In Proceedings of the 46th Annual Meeting of the Association for Computational Linguistics: Hu- man Language Technologies, Columbus, Ohio.
  18. Huang, L. and Chiang, D. (2007). Forest rescor- ing: Faster decoding with integrated language models. In Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics, pages 144-151, Prague, Czech Re- public. Association for Computational Linguis- tics.
  19. Jiampojamarn, S., Dwyer, K., Bergsma, S., Bhar- gava, A., Dou, Q., Kim, M.-Y., and Kondrak, G. (2010). Transliteration generation and mining with limited training resources. In Proceedings of the 2010 Named Entities Workshop, Uppsala, Sweden.
  20. Kahki, A. E., Darwish, K., El Din, A. S., and El- Wahab, M. A. (2012). Transliteration mining using large training and test sets. In Proceed- ings of the 2012 Conference of the North Amer- ican Chapter of the Association for Computa- tional Linguistics: Human Language Technolo- gies, NAACL HLT '12.
  21. Kashani, M. M., Joanis, E., Kuhn, R., Foster, G., and Popowich, F. (2007). Integration of an ara- bic transliteration module into a statistical ma- chine translation system. In Proceedings of the Second Workshop on Statistical Machine Trans- lation, Prague, Czech Republic.
  22. Kirschenbaum, A. and Wintner, S. (2009). Lightly supervised transliteration for machine transla- tion. In Proceedings of the 12th Conference of the European Chapter of the ACL (EACL 2009), pages 433-441, Athens, Greece. Asso- ciation for Computational Linguistics.
  23. Klementiev, A. and Roth, D. (2006). Named en- tity transliteration and discovery from multilin- gual comparable corpora. In Proceedings of the Human Language Technology Conference of the NAACL, Main Conference, pages 82-88, New York City, USA. Association for Computational Linguistics.
  24. Kumar, S. and Byrne, W. J. (2004). Mini- mum bayes-risk decoding for statistical ma- chine translation. In HLT-NAACL, pages 169- 176.
  25. Kumaran, A., M. Khapra, M., and Li, H. (2010). Whitepaper of news 2010 shared task on transliteration mining. In Proceedings of the 2010 Named Entities Workshop, pages 29- 38, Uppsala, Sweden. Association for Compu- tational Linguistics.
  26. Lee, J.-S. and Choi, K.-S. (1998). English to ko- rean statistical transliteration for information re- trieval. Computer Processing of Oriental Lan- guages, 12(1):17-37.
  27. Lin, W.-P., Snover, M., and Ji, H. (2011). Un- supervised language-independent name trans- lation mining from wikipedia infoboxes. In Proceedings of the First workshop on Unsu- pervised Learning in NLP, pages 43-52, Edin- burgh, Scotland. Association for Computational Linguistics.
  28. Nakov, P. and Tiedemann, J. (2012). Combining word-level and character-level models for ma- chine translation between closely-related lan- guages. In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 301-305, Jeju Island, Korea. Association for Computational Linguistics.
  29. Papineni, K., Roukos, S., Ward, T., and Zhu, W.-J. (2002). BLEU: a method for automatic evaluation of machine translation. In Proceed- ings of the 40th Annual Meeting on Association for Computational Linguistics, ACL '02, pages 311-318, Morristown, NJ, USA.
  30. Post, M., Callison-Burch, C., and Osborne, M. (2012). Constructing parallel corpora for six in- dian languages via crowdsourcing. In Proceed- ings of the Seventh Workshop on Statistical Ma- chine Translation, pages 401-409, Montréal, Canada. Association for Computational Lin- guistics.
  31. Sajjad, H., Fraser, A., and Schmid, H. (2011). An algorithm for unsupervised transliteration min- ing with an application to word alignment. In Proceedings of the 49th Annual Conference of the Association for Computational Linguistics, Portland, USA.
  32. Sajjad, H., Fraser, A., and Schmid, H. (2012). A statistical model for unsupervised and semi- supervised transliteration mining. In Proceed- ings of the 50th Annual Conference of the Asso- ciation for Computational Linguistics, Jeju, Ko- rea.
  33. Sherif, T. and Kondrak, G. (2007). Bootstrap- ping a stochastic transducer for arabic-english transliteration extraction. In Proceedings of the 45th Annual Meeting of the Association for Computational Linguistics, Prague, Czech Re- public.
  34. Zhao, B., Bach, N., Lane, I., and Vogel, S. (2007). A log-linear block transliteration model based on bi-stream hmms. In Human Language Tech- nologies 2007: The Conference of the North American Chapter of the Association for Com- putational Linguistics, Rochester, New York.

Cited by

Sorted by citations

Metadata

Authors
Durrani, N., Sajjad, H., Hoang, H., & Koehn, P.
Affiliations
University of Edinburgh (GB)
ORCID
Venue
Proceedings of the 14th Conference of the European Chapter of the Association for Computational Linguistics, volume 2: Short Papers
Publisher
Association for Computational Linguistics
Published
2014
Pages
148-153
DOI
10.3115/v1/e14-4029
OpenAlex
W2119202242
Cited by
89
References
1
Citations by year
2026: 1 · 2025: 1 · 2023: 3 · 2022: 2 · 2021: 3 · 2020: 4 · 2019: 4 · 2018: 14 · 2017: 18 · 2016: 17 · 2015: 15 · 2014: 6
FWCI
10.85
Type
conference paper
Language
en
Open access
Licence
cc-by
Locations
  • Conference of the European Chapter of the Association for Computational Linguistics (publishedVersion, cc-by)
  • Edinburgh Research Explorer (University of Edinburgh) (submittedVersion, other-oa)
Funders
European Commission (2007–2013, 287658, FP7 2007-2013, FP7/2007)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
  • Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
  • Social Sciences › Arts and Humanities › Language and Linguistics › Translation Studies and Practices
Keywords
transliteration, unsupervised learning, statistical machine translation, out-of-vocabulary words, rule coverage, end-to-end translation

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review