Automatic subject indexing with large language model fine-tuning and controlled vocabulary retrieval

Abstract

Purpose This study enhances automatic subject indexing techniques by addressing the limitations of conventional methods, which typically generate index terms that appear explicitly in the target documents or the training corpus. Design/methodology/approach We propose a subject indexing method based on large language model fine-tuning and controlled vocabulary retrieval (LLMFT-CVR). First, the large language model (LLM) is fine-tuned on training data to generate initial terms for a given document. Next, a text embedding model retrieves a set of candidate terms from the controlled vocabulary that are most similar to each of the initial terms. Finally, a term re-ranking model is trained to score the candidate terms, and the top-ranked terms are selected as the final recommended terms. Findings Experimental results on real-world bibliographic data demonstrate that LLMFT-CVR significantly outperforms the best-performing models of the three traditional approaches, i.e. lexical, associative and fusion approaches, by 22.55, 16.58 and 15.18% points in F1 score, respectively. Moreover, it achieves F1 score improvements of 14.69 and 4.09% points over the retrieval-augmented generation (RAG) and the fine-tuned large language model, respectively. Originality/value We introduce domain knowledge into LLM by leveraging training data and controlled vocabularies, thereby enhancing the quality of subject indexing and effectively mitigating hallucinations in the generated terms.