CoChat
Sign Up
PreprintCited by 1Text and Document Classification TechnologiesLibrary Science and Information SystemsTopic Modeling

An Extreme Multi-label Text Classification (XMTC) Library Dataset: What if we took "Use of Practical AI in Digital Libraries" seriously?

Open access
D'Souza, Jennifer, Sadruddin, Sameer, Kähler, Maximilian, Salfinger, Andrea, Zaccagna, Luca, Incitti, Francesca, Snidaro, Lauro, Suominen, Osma·2026·arXiv
DOI 10.48550/arxiv.2603.10876arXiv 2603.10876
Open this paper in CoChat — free account

Abstract

Subject indexing is vital for discovery but hard to sustain at scale and across languages. We release a large bilingual (English/German) corpus of catalog records annotated with the Integrated Authority File (GND), plus a machine-actionable GND taxonomy. The resource enables ontology-aware multi-label classification, mapping text to authority terms, and agent-assisted cataloging with reproducible, authority-grounded evaluation. We provide a brief statistical profile and qualitative error analyses of three systems. We invite the community to assess not only accuracy but usefulness and transparency, toward authority-anchored AI co-pilots that amplify catalogers' work.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2603.10876

Cites

1 of 1 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

  • The Llama 3 Herd of Models

    Grattafiori, A., Dubey, A., Jauhri, A., Pandey, A., Kadian, A., Al-Dahle, A., Letman, A., Mathur, A., Schelten, A., Vaughan, A., et al. (2024) · arXiv · doi:10.48550/arxiv.2407.21783 · Cited by 505 · cited 1× · see citing works

    “Overall, System 2 establishes a baseline for untrained LLM-based meth-ods (nDCG@5 = 0.4919), and System 3 sets the benchmark for hybrid, model-trained approaches (nDCG@5 = 0.6020).”
    Influential

Flagged first, then by citations

Research works printed but not linked: 49

  1. Hamed Babaei Giglou, Jennifer D'Souza, Oliver Karras, and Sören Auer. 2025. Ontoaligner: A comprehensive modular and robust python toolkit for ontology alignment. In European Semantic Web Conference , pages 174-191. Springer.
  2. Rohit Babbar and Bernhard Schölkopf. 2017. Dismec: Distributed sparse machines for extreme multi-label classification. In Proceedings of the tenth ACM international conference on web search and data mining , pages 721-729.
  3. Kush Bhatia, Himanshu Jain, Purushottam Kar, Manik Varma, and Prateek Jain. 2015. Sparse local embeddings for extreme multi-label classification. Advances in neural information processing systems , 28.
  4. Kathi Canese and Sarah Weis. 2013. Pubmed: the bibliographic database. The NCBI handbook , 2(1):2013.
  5. Caterina Caracciolo, Armando Stellato, Ahsan Morshed, Gudrun Johannsen, Sachit Rajbhandari, Yves Jaques, and Johannes Keizer. 2013. The agrovoc linked dataset. Semantic Web , 4(3):341348.
  6. Wei-Cheng Chang, Hsiang-Fu Yu, Kai Zhong, Yiming Yang, and Inderjit S Dhillon. 2020. Taming pretrained transformers for extreme multilabel text classification. In Proceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining , pages 31633171.
  7. Jianlv Chen, Shitao Xiao, Peitian Zhang, Kun Luo, Defu Lian, and Zheng Liu. 2024. Bge m3-embedding: Multi-lingual, multi-functionality, multi-granularity text embeddings through self-knowledge distillation. arXiv preprint arXiv:2402.03216 . · arXiv
  8. Maria Dermentzi, Mike Bryant, Fabio Rovigo, and Herminio García-González. 2025b. Multilingual Automated Subject Indexing: a comparative study of LLMs vs alternative approaches in the context of the EHRI project. Working paper or preprint.
  9. Aleksei Dorkin and Kairit Sirts. 2025. Tartunlp at semeval-2025 task 5: Subject tagging as twostage information retrieval. In Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025) , pages 2449-2454.
  10. Jennifer D'Souza, Sameer Sadruddin, Holger Israel, Mathias Begoin, and Diana Slawig. 2025. SemEval-2025 task 5: LLMs4Subjects - LLMbased automated subject tagging for a national technical library's open-access catalog. In Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025) , pages 2570-2583, Vienna, Austria. Association for Computational Linguistics.
  11. Luis Gasco, Anastasios Nentidis, Anastasia Krithara, Darryl Estrada-Zavala, Renato Toshiyuki Murasaki, Elena Primo-Peña, Cristina Bojo Canales, Georgios Paliouras, Martin Krallinger, et al. 2021. Overview of bioasq 2021-mesinesp track. evaluation of advance hierarchical classification techniques for scientific literature, patents and clinical trials. CEUR Workshop Proceedings .
  12. Jim Hahn. 2025. Jim at semeval-2025 task 5: Multilingual bert ensemble. In Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025) , pages 2407-2412.
  13. Clara Wan Ching Ho. 2025. UBFFM at the GermEval-2025 LLMs4Subjects task: What if we take 'you are an expert in subject indexing' seriously? In Proceedings of the 21st Conference on Natural Language Processing (KONVENS 2025): Workshops , pages 471-478, Hannover, Germany. HsH Applied Academics.
  14. Antoine Isaac and Bernhard Haslhofer. 2013. Europeana linked open data-data. europeana. eu. Semantic Web , 4(3):291-297.
  15. Baharul Islam, Nasim Ahmad, Ferdous Barbhuiya, and Kuntal Dey. 2025. Nbf at semeval-2025 task 5: Light-burst attention enhanced system for multilingual subject recommendation. In Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025) , pages 953958.
  16. Paul Jaccard. 1912. The distribution of the flora in the alpine zone. 1. New phytologist , 11(2):37-50.
  17. Kalervo Järvelin and Jaana Kekäläinen. 2002. Cumulated gain-based evaluation of ir techniques. ACM Transactions on Information Systems (TOIS) , 20(4):422-446.
  18. Hong Jiang, Jin Wang, and Xuejie Zhang. 2025. Ynu-hpcc at semeval-2025 task 5: Contrastive learning for gnd subject tagging with multilingual sentence-bert. In Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025) , pages 2443-2448.
  19. Maximilian Kähler, Lisa Kluge, and Katja Konermann. 2025. DNB-AI-project at the GermEval2025 LLMs4Subjects task: KIFSPrompt -knowledge-injected few-shot prompting. In Proceedings of the 21st Conference on Natural Language Processing (KONVENS 2025): Workshops , pages 455-464, Hannover, Germany. HsH Applied Academics.
  20. Sujay Khandagale, Han Xiao, and Rohit Babbar. 2020. Bonsai: diverse and shallow trees for extreme multi-label classification. Machine Learning , 109(11):2099-2119.
  21. Anastasia Krithara, Anastasios Nentidis, Konstantinos Bougiatiotis, and Georgios Paliouras. 2023b. Bioasq-qa: A manually curated corpus for biomedical question answering. Scientific Data , 10(1):170.
  22. Solomon Kullback and Richard A Leibler. 1951. On information and sufficiency. The annals of mathematical statistics , 22(1):79-86.
  23. Jingzhou Liu, Wei-Cheng Chang, Yuexin Wu, and Yiming Yang. 2017. Deep learning for extreme multi-label text classification. In Proceedings of the 40th international ACM SIGIR conference on research and development in information retrieval , pages 115-124.
  24. Yu A Malkov and Dmitry A Yashunin. 2018. Efficient and robust approximate nearest neighbor search using hierarchical navigable small world graphs. IEEE transactions on pattern analysis and machine intelligence , 42(4):824-836.
  25. Olena Medelyan. 2009. Human-competitive automatic topic indexing . Ph.D. thesis, The University of Waikato.
  26. Isabelle Mohr, Markus Krimmel, Saba Sturua, Mohammad Kalim Akram, Andreas Koukounas, Michael Günther, Georgios Mastrapas, Vinit Ravishankar, Joan Fontanals Martínez, Feng Wang, et al. 2024. Multi-task contrastive learning for 8192-token bilingual text embeddings. arXiv preprint arXiv:2402.17016 . · arXiv
  27. Hercules Panoutsopoulos and Christopher Brewster. 2022. Data-driven update of agrovoc using agricultural text corpora. In HAICTA , pages 260265.
  28. Karl Pearson. 1900. X. on the criterion that a given system of deviations from the probable in the case of a correlated system of variables is such that it can be reasonably supposed to have arisen from random sampling. The London, Edinburgh, and Dublin Philosophical Magazine and Journal of Science , 50(302):157-175.
  29. Yashoteja Prabhu, Anil Kag, Shrutendra Harsola, Rahul Agrawal, and Manik Varma. 2018. Parabel: Partitioned label trees for extreme classification with application to dynamic search advertising. In Proceedings of the 2018 World Wide Web Conference , pages 993-1002.
  30. Yashoteja Prabhu and Manik Varma. 2014. Fastxml: A fast, accurate and stable tree-classifier for extreme multi-label learning. In Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining , pages 263-272.
  31. Stephen Robertson, Hugo Zaragoza, et al. 2009. The probabilistic relevance framework: Bm25 and beyond. Foundations and Trends® in Information Retrieval , 3(4):333-389.
  32. Andrea Salfinger, Luca Zaccagna, Francesca Incitti, Gianluca De Nardi, Lorenzo Dal Fabbro, and Lauro Snidaro. 2025. LA2I2F at SemEval2025 task 5: Reasoning in embedding space fusing analogical and ontology-based reasoning for document subject tagging. In Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025) , pages 2413-2423, Vienna, Austria. Association for Computational Linguistics.
  33. Hinrich Schütze, Christopher D Manning, and Prabhakar Raghavan. 2008. Introduction to information retrieval , volume 39. Cambridge University Press Cambridge.
  34. Sumit Singh, Pankaj Goyal, and Uma Tiwary. 2025. silp\_nlp at semeval-2025 task 5: Subject recommendation with sentence transformer. In Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025) , pages 2455-2460.
  35. Kaitao Song, Xu Tan, Tao Qin, Jianfeng Lu, and Tie-Yan Liu. 2020. Mpnet: Masked and permuted pre-training for language understanding. Advances in neural information processing systems , 33:16857-16867.
  36. Osma Suominen. 2019. Annif: Diy automated subject indexing using multiple algorithms. LIBER Quarterly: The Journal of the Association of European Research Libraries , 29(1):1-25.
  37. Osma Suominen, Juho Inkinen, and Mona Lehtinen. 2025. Annif at the GermEval-2025 LLMs4Subjects task: Traditional XMTC augmented by efficient LLMs. In Proceedings of the 21st Conference on Natural Language Processing (KONVENS 2025): Workshops , pages 447-454, Hannover, Germany. HsH Applied Academics.
  38. Yukihiro Tagami. 2017. Annexml: Approximate nearest neighbor search for extreme multi-label classification. In Proceedings of the 23rd ACM SIGKDD international conference on knowledge discovery and data mining , pages 455-464.
  39. Hadi Bayrami Asl Tekanlou, Jafar Razmara, Mahsa Sanaei, Mostafa Rahgouy, and Hamed Babaei Giglou. 2025. Homa at semeval-2025 task 5: Aligning librarian records with ontoaligner for subject tagging. In Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025) , pages 2400-2406.
  40. Xia Tian, Yang Xin, Wu Jing, Xiu Heng, Zhang Xin, Li Yu, Gao Tong, Tan Xi, Hu Dong, Chen Tao, et al. 2025a. Ruc team at semeval-2025 task 5: Fast automated subject indexing: A method based on similar records matching and related subject ranking. In Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025) , pages 2437-2442.
  41. Yicen Tian, Erchen Yu, Yanan Wang, Dailin Li, Jiaqi Yao, Hongfei Lin, Linlin Zong, and Bo Xu. 2025b. Dutir831 at semeval-2025 task 5: A multi-stage llm approach to gnd subject assignment for tibkat records. In Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval2025) , pages 363-372.
  42. George Tsatsaronis, Georgios Balikas, Prodromos Malakasiotis, Ioannis Partalas, Matthias Zschunke, Michael R Alvers, Dirk Weissenborn, Anastasia Krithara, Sergios Petridis, Dimitris Polychronopoulos, Yannis Almirantis, John Pavlopoulos, Nicolas Baskiotis, Patrick Gallinari, Thierry Artiéres, Axel-Cyrille Ngonga Ngomo, Norman Heino, Eric Gaussier, Liliana BarrioAlvers, Michael Schroeder, Ion Androutsopoulos, and Georgios Paliouras. 2015. An overview of the bioasq large-scale biomedical semantic indexing and question answering competition. BMC Bioinformatics , 16(1):138.
  43. Henrique Schechter Vera, Sahil Dua, Biao Zhang, Daniel Salz, Ryan Mullins, Sindhu Raghuram Panyam, Sara Smoot, Iftekhar Naim, Joe Zou, Feiyang Chen, et al. 2025. Embeddinggemma: Powerful and lightweight text representations. arXiv preprint arXiv:2509.20354 . · arXiv
  44. Liang Wang, Nan Yang, Xiaolong Huang, Linjun Yang, Rangan Majumder, and Furu Wei. 2024. Multilingual e5 text embeddings: A technical report. arXiv preprint arXiv:2402.05672 . · arXiv
  45. Ian EH Yen, Xiangru Huang, Wei Dai, Pradeep Ravikumar, Inderjit Dhillon, and Eric Xing. 2017. Ppdsparse: A parallel primal-dual sparse method for extreme classification. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining , pages 545-553.
  46. Ronghui You, Zihan Zhang, Ziye Wang, Suyang Dai, Hiroshi Mamitsuka, and Shanfeng Zhu. 2019. Attentionxml: Label tree-based attentionaware deep model for high-performance extreme multi-label text classification. Advances in neural information processing systems , 32.
  47. Hsiang-Fu Yu, Kai Zhong, Jiong Zhang, Wei-Cheng Chang, and Inderjit S Dhillon. 2022. Pecos: Prediction for enormous and correlated output spaces. Journal of Machine Learning Research , 23(98):1-32.
  48. Jiong Zhang, Wei-Cheng Chang, Hsiang-Fu Yu, and Inderjit Dhillon. 2021. Fast multi-resolution transformer fine-tuning for extreme multi-label text classification. Advances in Neural Information Processing Systems , 34:7267-7280.
  49. Ruohong Zhang, Yau-Shian Wang, Yiming Yang, Donghan Yu, Tom Vu, and Likun Lei. 2023. Longtailed extreme multi-label text classification by the retrieval of generated pseudo label descriptions. In Findings of the Association for Computational Linguistics: EACL 2023 , pages 1092-1106.

Cited by

Sorted by citations

Authors
D'Souza, J., Sadruddin, S., Kähler, M., Salfinger, A., Zaccagna, L., Incitti, F., Snidaro, L., & Suominen, O.
Affiliations
  • Technische Informationsbibliothek (TIB) (DE)
  • Deutsche Nationalbibliothek (DE)
  • University of Udine (IT)
  • National Library of Finland (FI)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2026
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2603.10876
arXiv
2603.10876
OpenAlex
W7135049228
Cited by
0
References
1
Type
posted content
Language
en
Open access
Licence
creativecommons.org/licenses/by-sa/4.0/legalcode
Locations
arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Text and Document Classification Technologies
  • Physical Sciences › Computer Science › Information Systems › Library Science and Information Systems
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
Keywords
multi-label text classification, subject indexing, authority control, library catalog records, bilingual corpus, digital libraries, AI copilots

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review