CoChat
Sign Up
PreprintCited by 187Topic ModelingNatural Language Processing TechniquesDomain Adaptation and Few-Shot Learning

Unsupervised Domain Clusters in Pretrained Language Models

Open access
Aharoni, Roee, Goldberg, Yoav·2020·arXiv
DOI 10.48550/arxiv.2004.02105arXiv 2004.02105
Open this paper in CoChat — free accountOpen PDF

Abstract

The notion of "in-domain data" in NLP is often over-simplistic and vague, as textual data varies in many nuanced linguistic aspects such as topic, style or level of formality. In addition, domain labels are many times unavailable, making it challenging to build domain-specific systems. We show that massive pre-trained language models implicitly learn sentence representations that cluster by domains without supervision -- suggesting a simple data-driven definition of domains in textual data. We harness this property and propose domain data selection methods based on such models, which require only a small set of in-domain monolingual data. We evaluate our data selection methods for neural machine translation across five diverse domains, where they outperform an established approach as measured by both BLEU and by precision and recall of sentence selection with respect to an oracle.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2004.02105

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • It is usually inefficient or prohibitive to update large-scale generative AI models such as LLMs with task-oriented external data through model pre-training or supervised fine-tuning (SFT).

    “It is usually inefficient or prohibitive to update largescale generative AI models such as LLMs with task-oriented external data through model pre-training or supervised fine-tuning (SFT) [81-83].”

    From Foundations of GenIR (2025) · cites this paper as [82]

    The Related Work section says training large language models may be prohibitive without pre-training, which supports a narrower concern about training cost. It does not say that updating models with task-oriented data through pre-training or supervised fine-tuning is usually inefficient or prohibitive; the Introduction instead proposes fine-tuning a pretrained model for domain classification. Page numbers are unavailable in our copy. (LLM, conf=0.86)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

No references could be resolved yet.

Research works printed but not linked: 68

  1. Roee Aharoni and Yoav Goldberg. 2018. Split and rephrase: Better evaluation and stronger baselines. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 719-724, Melbourne, Aus- tralia. Association for Computational Linguistics. · DOI
  2. Mikko Aulamo and Jörg Tiedemann. 2019. The OPUS resource repository: An open package for creating parallel corpora and machine translation services. In Proceedings of the 22nd Nordic Conference on Com- putational Linguistics, pages 389-394, Turku, Fin- land. Linköping University Electronic Press.
  3. Amittai Axelrod. 2017. Cynical selection of lan- guage model training data. arXiv preprint arXiv:1709.02279. · arXiv
  4. Amittai Axelrod, Xiaodong He, and Jianfeng Gao. 2011. Domain adaptation via pseudo in-domain data selection. In Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing, pages 355-362, Edinburgh, Scotland, UK. Associa- tion for Computational Linguistics.
  5. Ankur Bapna and Orhan Firat. 2019. Non-parametric adaptation for neural machine translation. In Pro- ceedings of the 2019 Conference of the North Amer- ican Chapter of the Association for Computational Linguistics: Human Language Technologies, Vol- ume 1 (Long and Short Papers), pages 1921-1931, Minneapolis, Minnesota. Association for Computa- tional Linguistics. · DOI
  6. Loïc Barrault, Ondřej Bojar, Marta R. Costa-jussà, Christian Federmann, Mark Fishel, Yvette Gra- ham, Barry Haddow, Matthias Huck, Philipp Koehn, Shervin Malmasi, Christof Monz, Mathias Müller, Santanu Pal, Matt Post, and Marcos Zampieri. 2019. Findings of the 2019 conference on machine transla- tion (WMT19). In Proceedings of the Fourth Con- ference on Machine Translation (Volume 2: Shared Task Papers, Day 1), pages 1-61, Florence, Italy. As- sociation for Computational Linguistics. · DOI
  7. David M Blei, Andrew Y Ng, and Michael I Jordan. 2003. Latent dirichlet allocation. Journal of ma- chine Learning research, 3(Jan):993-1022.
  8. Chenhui Chu and Rui Wang. 2018. A survey of do- main adaptation for neural machine translation. In Proceedings of the 27th International Conference on Computational Linguistics, pages 1304-1319, Santa Fe, New Mexico, USA. Association for Computa- tional Linguistics.
  9. Kevin Clark, Urvashi Khandelwal, Omer Levy, and Christopher D Manning. 2019. What does BERT look at? an analysis of BERT's attention. arXiv preprint arXiv:1906.04341. · arXiv
  10. Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettle- moyer, and Veselin Stoyanov. 2019. Unsupervised cross-lingual representation learning at scale. arXiv preprint arXiv:1911.02116. · arXiv
  11. Alexis Conneau and Guillaume Lample. 2019. Cross- lingual language model pretraining. In Advances in Neural Information Processing Systems, pages 7057-7067.
  12. Hoang Cuong, Khalil Sima'an, and Ivan Titov. 2016. Adapting to all domains at once: Rewarding domain invariance in SMT. Transactions of the Association for Computational Linguistics, 4:99-112. · DOI
  13. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language under- standing. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171-4186, Minneapolis, Minnesota. Associ- ation for Computational Linguistics. · DOI · Paper page
  14. Zi-Yi Dou, Junjie Hu, Antonios Anastasopoulos, and Graham Neubig. 2019a. Unsupervised domain adap- tation for neural machine translation with domain- aware feature embeddings. In Proceedings of the 2019 Conference on Empirical Methods in Natu- ral Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 1417-1422, Hong Kong, China. Association for Computational Linguistics. · DOI
  15. Zi-Yi Dou, Xinyi Wang, Junjie Hu, and Graham Neu- big. 2019b. Domain differential adaptation for neu- ral machine translation. In Proceedings of the 3rd Workshop on Neural Generation and Translation, Hong Kong. Association for Computational Linguis- tics.
  16. Kevin Duh, Graham Neubig, Katsuhito Sudoh, and Ha- jime Tsukada. 2013. Adaptation data selection us- ing neural language models: Experiments in ma- chine translation. In Proceedings of the 51st Annual Meeting of the Association for Computational Lin- guistics (Volume 2: Short Papers), pages 678-683, Sofia, Bulgaria. Association for Computational Lin- guistics.
  17. Mirela-Stefania Duma and Wolfgang Menzel. 2016. Data selection for IT texts using paragraph vector. In Proceedings of the First Conference on Machine Translation: Volume 2, Shared Task Papers, pages 428-434, Berlin, Germany. Association for Compu- tational Linguistics. · DOI
  18. Sauleh Eetemadi, William Lewis, Kristina Toutanova, and Hayder Radha. 2015. Survey of data-selection methods in statistical machine translation. Machine Translation, 29(3-4):189-223.
  19. M. Amin Farajian, Marco Turchi, Matteo Negri, and Marcello Federico. 2017. Multi-domain neural machine translation through unsupervised adapta- tion. In Proceedings of the Second Conference on Machine Translation, pages 127-137, Copenhagen, Denmark. Association for Computational Linguis- tics. · DOI
  20. Guillem Gascó, Martha-Alicia Rocha, Germán Sanchis-Trilles, Jesús Andrés-Ferrer, and Francisco Casacuberta. 2012. Does more data always yield better translations? In Proceedings of the 13th Conference of the European Chapter of the Associa- tion for Computational Linguistics, pages 152-161, Avignon, France. Association for Computational Linguistics.
  21. Yoav Goldberg. 2019. Assessing BERT's syntactic abilities. arXiv preprint arXiv:1901.05287. · arXiv
  22. Suchin Gururangan, Ana Marasovi, Swabha Swayamdipta, Kyle Lo, Iz Beltagy, Doug Downey, and Noah A. Smith. 2020. Don't stop pretraining: Adapt language models to domains and tasks. ACL.
  23. Gholamreza Haffari, Maxim Roy, and Anoop Sarkar. 2009. Active learning for statistical phrase-based machine translation. In Proceedings of Human Language Technologies: The 2009 Annual Confer- ence of the North American Chapter of the Associa- tion for Computational Linguistics, pages 415-423, Boulder, Colorado. Association for Computational Linguistics.
  24. Eva Hasler, Phil Blunsom, Philipp Koehn, and Barry Haddow. 2014. Dynamic topic adaptation for phrase-based MT. In Proceedings of the 14th Conference of the European Chapter of the Asso- ciation for Computational Linguistics, pages 328- 337, Gothenburg, Sweden. Association for Compu- tational Linguistics. · DOI
  25. Kenneth Heafield. 2011. KenLM: Faster and smaller language model queries. In Proceedings of the Sixth Workshop on Statistical Machine Translation, pages 187-197, Edinburgh, Scotland. Association for Computational Linguistics.
  26. Junjie Hu, Sebastian Ruder, Aditya Siddhant, Gra- ham Neubig, Orhan Firat, and Melvin Johnson. 2020. Xtreme: A massively multilingual multi-task benchmark for evaluating cross-lingual generaliza- tion. arXiv preprint arXiv:2003.11080. · arXiv
  27. Junjie Hu, Mengzhou Xia, Graham Neubig, and Jaime Carbonell. 2019. Domain adaptation of neural ma- chine translation by lexicon induction. In Proceed- ings of the 57th Annual Meeting of the Association for Computational Linguistics, Florence, Italy. Asso- ciation for Computational Linguistics. · DOI
  28. Alon Jacovi, Gang Niu, Yoav Goldberg, and Masashi Sugiyama. 2019. Scalable evaluation and im- provement of document set expansion via neu- ral positive-unlabeled learning. arXiv preprint arXiv:1910.13339. · arXiv
  29. Marcin Junczys-Dowmunt. 2018. Dual conditional cross-entropy filtering of noisy parallel corpora. In Proceedings of the Third Conference on Machine Translation: Shared Task Papers, pages 888-895, Belgium, Brussels. Association for Computational Linguistics. · DOI · Paper page
  30. Yunsu Kim, Miguel Grac ¸a, and Hermann Ney. 2020. When and why is unsupervised neural machine trans- lation useless? arXiv preprint arXiv:2004.10581. · arXiv
  31. Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980. · arXiv
  32. Philipp Koehn, Hieu Hoang, Alexandra Birch, Chris Callison-Burch, Marcello Federico, Nicola Bertoldi, Brooke Cowan, Wade Shen, Christine Moran, Richard Zens, Chris Dyer, Ondřej Bojar, Alexandra Constantin, and Evan Herbst. 2007. Moses: Open source toolkit for statistical machine translation. In Proceedings of the 45th Annual Meeting of the As- sociation for Computational Linguistics Companion Volume Proceedings of the Demo and Poster Ses- sions, pages 177-180, Prague, Czech Republic. As- sociation for Computational Linguistics.
  33. Philipp Koehn and Rebecca Knowles. 2017. Six chal- lenges for neural machine translation. In Proceed- ings of the First Workshop on Neural Machine Trans- lation, pages 28-39, Vancouver. Association for Computational Linguistics. · DOI · Paper page
  34. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Man- dar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining ap- proach. arXiv preprint arXiv:1907.11692. · arXiv
  35. Xiaofei Ma, Peng Xu, Zhiguo Wang, Ramesh Nalla- pati, and Bing Xiang. 2019. Domain adaptation with BERT-based domain classification and data se- lection. In Proceedings of the 2nd Workshop on Deep Learning Approaches for Low-Resource NLP (DeepLo 2019), pages 76-83, Hong Kong, China. Association for Computational Linguistics. · DOI
  36. Christopher D Manning, Prabhakar Raghavan, and Hin- rich Schütze. 2008. Introduction to information re- trieval. Cambridge university press.
  37. Kelly Marchisio, Kevin Duh, and Philipp Koehn. 2020. When does unsupervised machine translation work? arXiv preprint arXiv:2004.05516. · arXiv
  38. Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Cor- rado, and Jeff Dean. 2013. Distributed representa- tions of words and phrases and their compositional- ity. In Advances in neural information processing systems, pages 3111-3119.
  39. Robert C. Moore and William Lewis. 2010. Intelligent selection of language model training data. In Pro- ceedings of the ACL 2010 Conference Short Papers, pages 220-224, Uppsala, Sweden. Association for Computational Linguistics.
  40. Nathan Ng, Kyra Yee, Alexei Baevski, Myle Ott, Michael Auli, and Sergey Edunov. 2019. Facebook FAIR's WMT19 news translation task submission. In Proceedings of the Fourth Conference on Ma- chine Translation (Volume 2: Shared Task Papers, Day 1), pages 314-319, Florence, Italy. Association for Computational Linguistics. · DOI
  41. Xing Niu, Marianna Martindale, and Marine Carpuat. 2017. A study of style in machine translation: Con- trolling the formality of machine translation output. In Proceedings of the 2017 Conference on Empiri- cal Methods in Natural Language Processing, pages 2814-2819, Copenhagen, Denmark. Association for Computational Linguistics. · DOI
  42. Myle Ott, Sergey Edunov, Alexei Baevski, Angela Fan, Sam Gross, Nathan Ng, David Grangier, and Michael Auli. 2019. fairseq: A fast, extensible toolkit for sequence modeling. In Proceedings of the 2019 Conference of the North American Chap- ter of the Association for Computational Linguistics (Demonstrations), pages 48-53, Minneapolis, Min- nesota. Association for Computational Linguistics. · DOI
  43. F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duch- esnay. 2011. Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12:2825-2830.
  44. Álvaro Peris, Mara Chinea-Ríos, and Francisco Casacuberta. 2017. Neural networks classifier for data selection in statistical machine translation. The Prague Bulletin of Mathematical Linguistics, 108(1):283-294.
  45. Matthew Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word rep- resentations. In Proceedings of the 2018 Confer- ence of the North American Chapter of the Associ- ation for Computational Linguistics: Human Lan- guage Technologies, Volume 1 (Long Papers), pages 2227-2237, New Orleans, Louisiana. Association for Computational Linguistics. · DOI · Paper page
  46. Alberto Poncelas, Gideon Maillette de Buy Wenniger, and Andy Way. 2018. Data selection with feature decay algorithms using an approximated target side. arXiv preprint arXiv:1811.03039. · arXiv
  47. Matt Post. 2018. A call for clarity in reporting BLEU scores. In Proceedings of the Third Conference on Machine Translation: Research Papers, pages 186- 191, Belgium, Brussels. Association for Computa- tional Linguistics. · DOI · Paper page
  48. Ofir Press and Lior Wolf. 2017. Using the output em- bedding to improve language models. In Proceed- ings of the 15th Conference of the European Chap- ter of the Association for Computational Linguistics: Volume 2, Short Papers, pages 157-163, Valencia, Spain. Association for Computational Linguistics.
  49. Sebastian Ruder, Matthew E. Peters, Swabha Swayamdipta, and Thomas Wolf. 2019. Trans- fer learning in natural language processing. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Com- putational Linguistics: Tutorials, pages 15-18, Minneapolis, Minnesota. Association for Computa- tional Linguistics. · DOI
  50. Sebastian Ruder and Barbara Plank. 2017. Learning to select data for transfer learning with Bayesian opti- mization. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 372-382, Copenhagen, Denmark. Association for Computational Linguistics. · DOI
  51. Hassan Sajjad, Nadir Durrani, Fahim Dalvi, Yonatan Belinkov, and Stephan Vogel. 2017. Neural ma- chine translation training in a multi-domain scenario. arXiv preprint arXiv:1708.08712. · arXiv
  52. Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. 2019. Distilbert, a distilled version of BERT: smaller, faster, cheaper and lighter. arXiv preprint arXiv:1910.01108. · arXiv
  53. Lucía Santamaría and Amittai Axelrod. 2019. Data selection with cluster-based language difference models and cynical selection. arXiv preprint arXiv:1904.04900. · arXiv
  54. Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. Neural machine translation of rare words with subword units. In Proceedings of the 54th An- nual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1715- 1725, Berlin, Germany. Association for Computa- tional Linguistics. · DOI · Paper page
  55. Catarina Cruz Silva, Chao-Hong Liu, Alberto Poncelas, and Andy Way. 2018. Extracting in-domain training corpora for neural machine translation using data se- lection methods. In Proceedings of the Third Con- ference on Machine Translation: Research Papers, pages 224-231, Belgium, Brussels. Association for Computational Linguistics. · DOI
  56. Ian Tenney, Dipanjan Das, and Ellie Pavlick. 2019. BERT rediscovers the classical NLP pipeline. In Proceedings of the 57th Annual Meeting of the Asso- ciation for Computational Linguistics, pages 4593- 4601, Florence, Italy. Association for Computational Linguistics. · DOI · Paper page
  57. Jörg Tiedemann. 2012. Parallel data, tools and inter- faces in OPUS. In Proceedings of the Eighth In- ternational Conference on Language Resources and Evaluation (LREC-2012), pages 2214-2218, Istan- bul, Turkey. European Languages Resources Associ- ation (ELRA).
  58. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Gar- nett, editors, Advances in Neural Information Pro- cessing Systems 30, pages 5998-6008.
  59. Alex Wang, Yada Pruksachatkun, Nikita Nangia, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R Bowman. 2019a. Super- glue: A stickier benchmark for general-purpose language understanding systems. arXiv preprint arXiv:1905.00537. · arXiv
  60. Alex Wang, Amanpreet Singh, Julian Michael, Fe- lix Hill, Omer Levy, and Samuel Bowman. 2018. GLUE: A multi-task benchmark and analysis plat- form for natural language understanding. In Pro- ceedings of the 2018 EMNLP Workshop Black- boxNLP: Analyzing and Interpreting Neural Net- works for NLP, pages 353-355, Brussels, Belgium. Association for Computational Linguistics. · DOI · Paper page
  61. Rui Wang, Andrew Finch, Masao Utiyama, and Ei- ichiro Sumita. 2017. Sentence embedding for neural machine translation domain adaptation. In Proceed- ings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Pa- pers), pages 560-566, Vancouver, Canada. Associa- tion for Computational Linguistics. · DOI
  62. Wei Wang, Isaac Caswell, and Ciprian Chelba. 2019b. Dynamically composing domain-data selection with clean-data selection by "co-curricular learning" for neural machine translation. In Proceedings of the 57th Annual Meeting of the Association for Com- putational Linguistics, pages 1282-1292, Florence, Italy. Association for Computational Linguistics. · DOI
  63. Marlies van der Wees. 2017. What's in a Domain? To- wards Fine-Grained Adaptation for Machine Trans- lation. Ph.D. thesis, University of Amsterdam.
  64. Marlies van der Wees, Arianna Bisazza, Wouter Weerkamp, and Christof Monz. 2015. What's in a domain? analyzing genre and topic differences in statistical machine translation. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 2: Short Papers), pages 560-566, Beijing, China. Association for Computational Linguistics. · DOI
  65. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pier- ric Cistac, Tim Rault, R'emi Louf, Morgan Funtow- icz, and Jamie Brew. 2019. Huggingface's trans- formers: State-of-the-art natural language process- ing. ArXiv, abs/1910.03771. · arXiv
  66. Shijie Wu, Alexis Conneau, Haoran Li, Luke Zettle- moyer, and Veselin Stoyanov. 2019. Emerging cross-lingual structure in pretrained language mod- els. arXiv preprint arXiv:1911.01464. · arXiv
  67. Zhilin Yang, Zihang Dai, Yiming Yang, Jaime Car- bonell, Ruslan Salakhutdinov, and Quoc V Le. 2019. XLNet: Generalized autoregressive pretrain- ing for language understanding. arXiv preprint arXiv:1906.08237. · arXiv
  68. Xuan Zhang, Pamela Shapiro, Gaurav Kumar, Paul McNamee, Marine Carpuat, and Kevin Duh. 2019. Curriculum learning for domain adaptation in neu- ral machine translation. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 1903-1915, Minneapolis, Minnesota. Association for Computational Linguistics. · DOI

Cited by

Sorted by citations

Authors
Aharoni, R., & Goldberg, Y.
Affiliations
  • Google (United States) (US)
  • Bar-Ilan University (IL)
ORCID
Goldberg, Yoav
Venue
arXiv
Publisher
arXiv
Published
2020
Pages
7747-7763
ISSN
0736-587X (ISSN-L)
DOI
10.48550/arxiv.2004.02105
arXiv
2004.02105
OpenAlex
W3014390292
Cited by
187
References
0
Citations by year
2026: 9 · 2025: 15 · 2024: 24 · 2023: 52 · 2022: 34 · 2021: 42 · 2020: 11
FWCI
13.01
Type
conference paper
Language
en
Open access
Licence
Locations
  • Proceedings of the conference - Association for Computational Linguistics. Meeting (publishedVersion, cc-by)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
  • Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
Keywords
pre-trained language models, domain adaptation, neural machine translation, domain-specific translation, sentence selection, monolingual data

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review