Cited by 78Topic ModelingMeta-analysis and systematic reviewsInformation Retrieval and Search Behavior

Automating Systematic Literature Reviews with Retrieval-Augmented Generation: A Comprehensive Overview

Open access
Han, Binglan, Susnjak, Teo, Mathrani, Anuradha·2024·Applied Sciences

Abstract

This study examines Retrieval-Augmented Generation (RAG) in large language models (LLMs) and their significant application for undertaking systematic literature reviews (SLRs). RAG-based LLMs can potentially automate tasks like data extraction, summarization, and trend identification. However, while LLMs are exceptionally proficient in generating human-like text and interpreting complex linguistic nuances, their dependence on static, pre-trained knowledge can result in inaccuracies and hallucinations. RAG mitigates these limitations by integrating LLMs’ generative capabilities with the precision of real-time information retrieval. We review in detail the three key processes of the RAG framework—retrieval, augmentation, and generation. We then discuss applications of RAG-based LLMs to SLR automation and highlight future research topics, including integration of domain-specific LLMs, multimodal data processing and generation, and utilization of multiple retrieval sources. We propose a framework of RAG-based LLMs for automating SRLs, which covers four stages of SLR process: literature search, literature screening, data extraction, and information synthesis. Future research aims to optimize the interaction between LLM selection, training strategies, RAG techniques, and prompt engineering to implement the proposed framework, with particular emphasis on the retrieval of information from individual scientific papers and the integration of these data to produce outputs addressing various aspects such as current status, existing gaps, and emerging trends.

Integrity signals

Registry

Crossref · 10.3390/app14199103

Cites

37 of 37 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • RAG facilitates dynamic literature review by retrieving and summarizing recent, domain-relevant publications in response to specific clinical or research queries.

    “RAGfacilitates dynamic literature review by retrieving and summarizing recent, domainrelevant publications in response to specific clinical or research queries [67].”

Cites

Flagged first, then by citations

Cited by

Sorted by citations

Authors
Han, B., Susnjak, T., & Mathrani, A.
Affiliations
  • Massey University (NZ)
  • School of Mathematical and Computational Sciences, Massey University, Auckland 0632, New Zealand
Corresponding author
Mathrani, Anuradha
Venue
Applied Sciences
Publisher
MDPI AG
Published
2024-10-09
Volume
14
Issue
19
Pages
9103
ISSN
2076-3417 (ISSN-L)
OpenAlex
W4403248625
Cited by
68
References
36
Citations by year
2026: 38 · 2025: 30
FWCI
2.53
Type
journal article
Language
en
Open access
Locations
  • Applied Sciences (publishedVersion, cc-by)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
  • Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
Keywords
retrieval-augmented generation, systematic reviews, large language models, data extraction, literature screening, information synthesis, hallucinations, prompt engineering, multimodal processing, literature searching

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review