QLoRA Fine-Tuning and Retrieval-Augmented Generation for Turkish Medical Question Answering: An Evaluation of Corpus Mismatch and Clinical Safety
Abstract
Large language models (LLMs) have demonstrated remarkable capabilities in clinical knowledge tasks; however, existing systems are predominantly English-centric, leaving a significant gap for low-resource languages such as Turkish. This paper presents a hallucination-aware evaluation framework for Turkish medical question answering (QA), combining Quantized Low-Rank Adaptation (QLoRA)-based parameter-efficient fine-tuning with a Retrieval-Augmented Generation (RAG) pipeline. We fine-tuned Llama-3.1-8B-Instruct on Turkish medical QA data using 4-bit NF4 quantization (v1: 2,675 MedTurkQuAD examples; v2: 13,230 augmentation-expanded examples), and constructed a RAG corpus from 18,577 Turkish hospital articles indexed with multilingual-E5-large embeddings and FAISS. Two model variants (v1 and v2) are evaluated under two modes: Mode A (original context provided) and Mode B (RAG-based retrieval). v1 achieves a semantic similarity of 0.966 (Mode A) and 0.874 (Mode B), alongside a grounding score of 0.715. Under a consistent evaluation methodology (n=50, semantic similarity-based inconsistency), both models exhibit low hallucination rates (v1: 0.051, v2: 0.022), both satisfying the clinical safety threshold of ≤0.20. A systematic corpus mismatch between academic gold answers and clinical web-based RAG sources is identified as the primary driver of low ROUGE-L/Recall in Mode B-a finding supported by Pearson correlation analysis (r=0.940 in Mode A vs. r=0.658 in Mode B) and with broad i…
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Cites
Large language models encode clinical knowledge
Singhal, K., Azizi, S., Tu, T., Mahdavi, S. S., Wei, J., Chung, H. W., Scales, N., Tanwani, A., Cole-Lewis, H., Pfohl, S., et al. (2023) · Nature · doi:10.1038/s41586-023-06291-2 · Cited by 4,083 · see citing works
Dense passage retrieval for open-domain question answering
Karpukhin, V., Oguz, B., Min, S., Lewis, P., Wu, L., Edunov, S., Chen, D., & Yih, W.-T. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.550 · Cited by 3,575 · see citing works
Huang, L., Yu, W., Ma, W., Zhong, W., Feng, Z., Wang, H., Chen, Q., Peng, W., Feng, X., Qin, B., et al. (2025) · ACM Transactions on Information Systems · doi:10.1145/3703155 · Cited by 2,142 · see citing works
Foundation models for generalist medical artificial intelligence
Moor, M., Banerjee, O., Abad, Z. S. H., Krumholz, H. M., Leskovec, J., Topol, E. J., & Rajpurkar, P. (2023) · Nature · doi:10.1038/s41586-023-05881-4 · Cited by 2,028 · cited 1× · see citing works
“Moor et al. formalized the framework of generalist medical AI, positing that foundation models may serve as versatile clinical instruments [13].”
QLoRA: efficient finetuning of quantized LLMs
Dettmers, T., Pagnoni, A., Holtzman, A., & Zettlemoyer, L. (2023) · Advances in Neural Information Processing Systems 36 · doi:10.52202/075280-0441 · Cited by 1,814 · see citing works
Toward expert-level medical question answering with large language models
Singhal, K., Tu, T., Gottweis, J., Sayres, R., Wulczyn, E., Amin, M., Hou, L., Clark, K., Pfohl, S. R., Cole-Lewis, H., et al. (2025) · Nature Medicine · doi:10.1038/s41591-024-03423-7 · Cited by 1,734 · see citing works
Detecting hallucinations in large language models using semantic entropy
Farquhar, S., Kossen, J., Kuhn, L., & Gal, Y. (2024) · Nature · doi:10.1038/s41586-024-07421-0 · Cited by 934 · cited 2× · see citing works
“Farquhar et al.[10] propose semantic entropy as a complementary approach, measuring uncertainty at the semantic rather than token level.”
SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Models
Manakul, P., Liusie, A., & Gales, M. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.557 · Cited by 557 · see citing works
Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Models
Zhang, Y., Li, Y., Cui, L., Cai, D., Liu, L., Fu, T., Huang, X., Zhao, E., Zhang, Y., Chen, Y., et al. (2025) · Computational Linguistics · doi:10.1162/coli.a.16 · Cited by 534 · see citing works
QLoRA: Efficient Finetuning of Quantized LLMs
Dettmers, T., Pagnoni, A., Holtzman, A., & Zettlemoyer, L. (2023) · arXiv · doi:10.48550/arxiv.2305.14314 · Cited by 507 · see citing works
A survey on LoRA of large language models
Mao, Y., Ge, Y., Fan, Y., Xu, W., Mi, Y., Hu, Z., & Gao, Y. (2024) · Frontiers of Computer Science · doi:10.1007/s11704-024-40663-9 · Cited by 137 · see citing works
Biomedical Question Answering: A Survey of Approaches and Challenges
Jin, Q., Yuan, Z. P., Xiong, G., Yu, Q., Ying, H., Tan, C., Chen, M., Huang, S., Liu, X., & Yu, S. (2022) · ACM Computing Surveys · doi:10.1145/3490238 · Cited by 128 · see citing works
Retrieval-Augmented Generation (RAG) in Healthcare: A Comprehensive Review
Neha, F., Bhati, D., & Shukla, D. K. (2025) · AI · doi:10.3390/ai6090226 · Cited by 117 · cited 1× · see citing works
“In medical settings, RAG has been shown to improve factual consistency and reduce hallucinations, though corpus quality remains a critical factor [19].”
Lee, C., Jin, J., Kim, T., Kim, H., & Park, E. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i12.29237 · Cited by 107 · see citing works
Eryiğit, G., Nivre, J., & Oflazer, K. (2008) · Computational Linguistics · doi:10.1162/coli.2008.07-017-r1-06-83 · Cited by 75 · see citing works
FIT-RAG: Black-Box RAG with Factual Information and Token Reduction
Mao, Y., Dong, X., Xu, W., Gao, Y., Wei, B., & Zhang, Y. (2024) · ACM Transactions on Information Systems · doi:10.1145/3676957 · Cited by 29 · see citing works
Türkmen, H., Dikenelli, O., Eraslan, C., et al. (2023) · Journal of Healthcare Informatics Research · doi:10.1007/s41666-023-00140-7 · Cited by 25 · see citing works
Yang, Y., Jin, Q., Zhu, Q., Wang, Z., Álvarez, F. E., Wan, N., Hou, B., & Lu, Z. (2025) · Annual Review of Biomedical Data Science · doi:10.1146/annurev-biodatasci-103123-094851 · Cited by 15 · see citing works
Flagged first, then by citations
Research works printed but not linked: 3
- Lewis, 2020, Retrieval-augmented generation for knowledgeintensive NLP tasks, *Adv. Neural Inf. Process. Syst.*, 33, 9459
- Hu, LoRA: Low-rank adaptation of large language models
- Guo, On calibration of modern neural networks, 1321
- Authors
- Suzgen, E. E., Şahin, M. E., Bakir, M., Arisoy, Ç., Ulutaş, E. G., Günay, R. B., Yildiz, G., & Ulutaş, H.
- Affiliations
- Yozgat Bozok University
- Yozgat Bozok Üniversitesi (TR)
- İzmir Bakirçay University
- Bakırçay Üniversitesi
- Venue
- 2026 2nd International Symposium on AI-Driven Engineering Systems (ISADES)
- Publisher
- IEEE
- Published
- 2026-06
- Pages
- 1-6
- OpenAlex
- W7169881898
- Semantic Scholar
- 6d6a3f24317b60f27dbbd29d3bde60ca3d611a6a
- Cited by
- 0
- References
- 17
- FWCI
- 0.00
- Type
- conference paper
- Open access
- closed
- Licence
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
- Keywords
- QLoRA, retrieval-augmented generation, medical question answering, Turkish language, fine-tuning, clinical safety, low-rank adapters
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
