Leveraging artificial intelligence to enhance systematic reviews in health research: advanced tools and challenges
Open accessThe DOI matches its Crossref record; 13 of the 13 references it cites were found in a registry or an index.
Abstract
Artificial Intelligence (AI) is transforming systematic reviews (SRs) in health research by automating processes such as study screening, data extraction, and quality assessment. This perspective highlights recent advancements in AI tools that enhance efficiency and accuracy in SRs. It discusses the benefits, challenges, and future directions of AI integration, emphasising the need for human oversight to ensure the reliability of AI outputs in evidence synthesis and decision-making in healthcare.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Other recent work has demonstrated promise for LLMs in data extraction 7,8”
From Automation of Systematic Reviews with Large Language Models (2025) · cites this paper as [7]
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“though these studies rely on self-defined reference standards and evaluate on small datasets.”
From Automation of Systematic Reviews with Large Language Models (2025) · cites this paper as [7]
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“This approach represented a methodological improvement over prior work that relied on self-created ground truths 7 .”
From Automation of Systematic Reviews with Large Language Models (2025) · cites this paper as [7]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
An open source machine learning framework for efficient and transparent systematic reviews
van de Schoot, R., de Bruin, J., Schram, R., Zahedi, P., de Boer, J., Weijdema, F., Kramer, B., Huijts, M., Hoogerwerf, M., Ferdinands, G., et al. (2021) · Nature Machine Intelligence · doi:10.1038/s42256-020-00287-7 · Cited by 1,157
Collins, G. S., Dhiman, P., Andaur Navarro, C. L., Ma, J., Hooft, L., Reitsma, J. B., Logullo, P., Beam, A. L., Peng, L., Van Calster, B., et al. (2021) · BMJ Open · doi:10.1136/bmjopen-2020-048008 · Cited by 852
Marshall, I. J., & Wallace, B. C. (2019) · Systematic Reviews · doi:10.1186/s13643-019-1074-9 · Cited by 638 · cited 2×
“Tools like RobotReviewer have high accuracy in extracting relevant details from randomised control trials (RCTs) [10].”
BackgroundMethodHarrison, H., Griffin, S. J., Kuhn, I., & Usher-Smith, J. A. (2020) · BMC Medical Research Methodology · doi:10.1186/s12874-020-0897-3 · Cited by 449 · cited 1×
“Several software tools support titles and abstract screening, with Abstrackr and Rayyan being the most validated [5, 6].”
MethodReducing Workload in Systematic Review Preparation Using Automated Citation Classification
Cohen, A. M., Hersh, W. R., Peterson, K., & Yen, P.-Y. (2006) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m1929 · Cited by 411 · cited 1×
“Since 2005, artificial intelligence (AI) tools have gained increasing attention [1] for their ability to automate these processes, offering increased efficiency and accuracy [2].”
BackgroundRobotReviewer: evaluation of a system for automatically assessing bias in clinical trials
Marshall, I. J., Kuiper, J., & Wallace, B. C. (2016) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocv044 · Cited by 317 · cited 1×
“For instance, RobotReviewer uses NLP to identify relevant text and assess the risk of bias in RCTs, achieving an accuracy rate of 71–78.3% [11], though it is less precise than Cochrane reviews.”
MethodInfluentialO’Connor, A. M., Tsafnat, G., Thomas, J., Glasziou, P., Gilbert, S. B., & Hutton, B. (2019) · Systematic Reviews · doi:10.1186/s13643-019-1062-0 · Cited by 167 · cited 1×
“The performance of these tools depends on the effectiveness of the underlying ML paradigms and algorithms, the quality and quantity of data used for training, and the degree of human involvement [9].”
MethodHamel, C., Kelly, S. E., Thavorn, K., Rice, D. B., Wells, G. A., & Hutton, B. (2020) · BMC Medical Research Methodology · doi:10.1186/s12874-020-01129-1 · Cited by 140 · cited 1×
“Several software tools support titles and abstract screening, with Abstrackr and Rayyan being the most validated [5, 6].”
MethodHow to optimize the systematic review process using AI tools
Fabiano, N., Gupta, A., Bhambra, N., Luu, B., Wong, S., Maaz, M., Fiedorowicz, J. G., Smith, A. L., & Solmi, M. (2024) · JCPP Advances · doi:10.1002/jcv2.12234 · Cited by 130 · cited 2×
“Since 2005, artificial intelligence (AI) tools have gained increasing attention [1] for their ability to automate these processes, offering increased efficiency and accuracy [2].”
BackgroundMethodEvaluating the effectiveness of large language models in abstract screening: a comparative analysis
Li, M., Sun, J., & Tan, X. (2024) · Systematic Reviews · doi:10.1186/s13643-024-02609-x · Cited by 125 · cited 1×
“0 API script achieved 96% screening specificity and 93% sensitivity when prompted with inclusion and exclusion criteria [13], showcasing the capability of base models without fine-tuning.”
Background
Cited by
Items 101–111 of 111
Wei, I., Shah, J. N., Miah, A., Hawco, C. S., Coutts, F., Sikstrom, L., Ameis, S. H., Voineskos, A. N., Glatard, T., & Dickie, E. W. (2026) · medRxiv · doi:10.64898/2026.10.02.26364621
Lao, L.-H., Chen, H., Sutcliffe, W. L., Yebyo, H. G., Serra, N., & Puhan, M. A. (2026) · medRxiv · doi:10.64898/2026.10.05.26364746
Weiss, B., Wagner, M., Daikeler, J., & Matthias, K. (2026) · Springer eBooks · doi:10.1007/978-3-658-52312-1_59
Kattimani, V. S. (2026) · Journal of Maxillofacial and Oral Surgery · doi:10.1007/s12663-026-03312-1
Nagendrababu, V., Dufey-Portilla, N., Abraham, S. B., Vinothkumar, T. S., Tzanetakis, G. N., Jaćimović, J., Jakovljević, A., Peña‐Bengoa, F., García‐Font, M., Sans, F. A., et al. (2026) · International Endodontic Journal · doi:10.1111/iej.70297
de Geofroy, B., Lemé, C., Delafontaine, A., Migаud, H., & Micicoi, G. (2026) · Orthopaedics & Traumatology Surgery & Research · doi:10.1016/j.otsr.2026.104979
Mercer, L., Reed, M. K., Tran, H. J., & Garnier, A. J. (2026) · medRxiv · doi:10.64898/2026.10.29.26366847
Singh, B., Kumar, A., Goel, A., Jacob, J., Kishore, A., & Chawla, R. (2026) · Indian Journal of Medical Sciences · doi:10.25259/ijms_82_2026
Uddin, A. H., & Arif, A. S. M. (2026) · Publications · doi:10.3390/publications14040074
Jay, M., ESTTAIFO, M., Samani, A., Wilson, E., Esttaifo, S., Morgan, M., Abi‐Jaoude, J., Reszetnik, G., Chopra, M., Alrashidi, A., et al. (2026) · medRxiv · doi:10.64898/2026.11.06.26367388
The Quality of Systematic Reviews Published in Leading Orthopaedic Surgery Journals
Burger, M. C., Niekerk, J. V., Ferreira, N., & McCaul, M. (2026) · The Journal of Arthroplasty · doi:10.1016/j.arth.2026.11.022
Metadata
- Authors
- Ge, L., Agrawal, R., Singer, M., Kannapiran, P., De Castro Molina, J. A., Teow, K. L., Yap, C. W., & Abisheganaden, J. A.
- Affiliations
- National Skin Centre (SG)
- Tan Tock Seng Hospital (SG)
- Yale University (US)
- Corresponding author
- Ge, Lixia
- Venue
- Systematic Reviews
- Published
- 2024-10-25
- Volume
- 13
- Issue
- 1
- Pages
- 269-269
- ISSN
- 2046-4053 (ISSN-L)
- PMC
- PMC11504244
- OpenAlex
- W4403778417
- Cited by
- 92
- References
- 13
- Citations by year
- 2026: 53 · 2025: 38 · 2024: 1
- FWCI
- 11.61
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by-nc-nd
- creativecommons.org/licenses/by-nc-nd/4.0 (tdm)
- creativecommons.org/licenses/by-nc-nd/4.0 (vor)
- Locations
- Systematic Reviews (publishedVersion, cc-by-nc-nd)
- PubMed Central (publishedVersion, cc-by-nc-nd)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Social Sciences › General Social Sciences › Computational and Text Analysis Methods
- Keywords
- artificial intelligence, automated data extraction, quality assessment, evidence synthesis, human oversight, healthcare decision-making, AI tools
- MeSH terms
- Systematic Reviews as Topic, Artificial Intelligence, Humans, Reproducibility of Results, Biomedical Research
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.