The DOI matches its Crossref record; 1 of the 8 references it cites were found in a registry or an index, and 7 are not checked yet.
Towards Faithful and Robust LLM Specialists for Evidence-Based Question-Answering
Open accessAbstract
Advances towards more faithful and traceable answers of Large Language Models (LLMs) are crucial for various research and practical endeavors.One avenue in reaching this goal is basing the answers on reliable sources.However, this Evidence-Based QA has proven to work insufficiently with LLMs in terms of citing the correct sources (source quality) and truthfully representing the information within sources (answer attributability).In this work, we systematically investigate how to robustly fine-tune LLMs for better source quality and answer attributability.Specifically, we introduce a data generation pipeline with automated data quality filters, which can synthesize diversified high-quality training and testing data at scale.We further introduce four test sets to benchmark the robustness of fine-tuned specialist models.Extensive evaluation shows that fine-tuning on synthetic data improves performance on both in-and out-of-distribution.Furthermore, we show that data quality, which can be drastically improved by proposed quality filters, matters more than quantity in improving Evidence-Based QA.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Recent NLP studies have emphasized measuring how well model outputs stay supported by provided data (i.e., faithfulness) [2], [5], but this has not been systematically applied to intrusion detection scenarios.”
From Can Large Language Models Be Trusted for AI-Based Intrusion Detection in Cybersecurity? An Evidence-Grounded Log Analysis (2026) · cites this paper as [5]
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“However, existing studies largely evaluate LLMs based on detection performance or qualitative usefulness, without rigorously assessing whether their explanations are grounded in actual log evidence.”
From Can Large Language Models Be Trusted for AI-Based Intrusion Detection in Cybersecurity? An Evidence-Grounded Log Analysis (2026) · cites this paper as [5]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The publisher hasn’t deposited this paper’s reference list.
Research works printed but not linked: 6
- Xuming Hu, Junzhe Chen, Xiaochuan Li, Yufei Guo, Lijie Wen, Philip S. Yu, and Zhijiang Guo. 2023. Do large language models know about facts? ArXiv, abs/2310.05177. · arXiv
- Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. 2020. Dense Passage Retrieval for Open-Domain Question Answering. ArXiv:2004.04906 [cs. · arXiv
- Dongfang Li, Zetian Sun, Xinshuo Hu, Zhenyu Liu, Ziyang Chen, Baotian Hu, Aiguo Wu, and Min Zhang. 2023. A Survey of Large Language Mod- els Attribution. ArXiv:2311.03731 [cs]. · arXiv
- Nelson F. Liu, Tianyi Zhang, and Percy Liang. 2023a. Evaluating Verifiability in Generative Search Engines. ArXiv:2304.09848 [cs]. · arXiv
- Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. Training language models to follow instruc- tions with human feedback. Advances in Neural Information Processing Systems, 35:27730-27744.
- Lewis Tunstall, Edward Beeching, Nathan Lambert, Nazneen Rajani, Kashif Rasul, Younes Belkada, Shengyi Huang, Leandro von Werra, Clémentine Fourrier, Nathan Habib, Nathan Sarrazin, Omar San- seviero, Alexander M. Rush, and Thomas Wolf. 2023. Zephyr: Direct Distillation of LM Alignment. ArXiv:2310.16944 [cs. · arXiv
Cited by
Sorted by citations
Using AI to assess corporate climate transition disclosures
Senni, C. C., Schimanski, T., Bingler, J., Ni, J., & Leippold, M. (2024) · Environmental Research Communications · doi:10.1088/2515-7620/ad9e88 · Cited by 19
Self-citationJiang, H., Shi, S., Yao, Y., Jiang, C., & Li, Q. (2025) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2025.3633887 · Cited by 13
Gehricke, S. A., Leippold, M., & Schimanski, T. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5246456 · Cited by 6
Self-citationGuo, S., Liu, H., Chen, X., Xie, Y., Zhang, L., Han, T., Chen, H., Chang, Y., & Wang, Y.-X. (2025) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3711896.3737254 · Cited by 3
Balancing 3D-model fidelity for training a vehicle detector on simulated data
Eker, T. A., Fokkinga, E. P., Heslinga, F. G., & Klamer Schutte, B. (2024) · SPIE Conference on Artificial Intelligence for Security and Defence Applications · doi:10.1117/12.3031691 · Cited by 1
Metadata
- Authors
- Schimanski, T., Ni, J., Kraus, M., Ash, E., & Leippold, M.
- Affiliations
- University of Zurich (CH)
- University of Regensburg (DE)
- ETH Zurich (CH)
- Venue
- Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
- Published
- 2024
- Pages
- 1913-1931
- ISSN
- 0736-587X (ISSN-L)
- OpenAlex
- W4402671969
- Cited by
- 15
- References
- 0
- Citations by year
- 2026: 5 · 2025: 8 · 2024: 2
- FWCI
- 4.43
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Proceedings of the conference - Association for Computational Linguistics. Meeting (publishedVersion, cc-by)
- Zurich Open Repository and Archive (University of Zurich) (submittedVersion, cc-by)
- Funders
- National Science Foundation
- Schweizerischer Nationalfonds zur Förderung der Wissenschaftlichen Forschung (207800)
- Hasler Stiftung
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Keywords
- large language models, LLM fine-tuning, source quality, synthetic training data, data quality filtering, robustness, out-of-distribution generalization, benchmark dataset
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
