Sensitivity and Specificity of Using GPT-3.5 Turbo Models for Title and Abstract Screening in Systematic Reviews and Meta-analyses
Abstract
BACKGROUND: Systematic reviews are performed manually despite the exponential growth of scientific literature. OBJECTIVE: To investigate the sensitivity and specificity of GPT-3.5 Turbo, from OpenAI, as a single reviewer, for title and abstract screening in systematic reviews. DESIGN: Diagnostic test accuracy study. SETTING: Unannotated bibliographic databases from 5 systematic reviews representing 22 665 citations. PARTICIPANTS: None. MEASUREMENTS: A generic prompt framework to instruct GPT to perform title and abstract screening was designed. The output of the model was compared with decisions from authors under 2 rules. The first rule balanced sensitivity and specificity, for example, to act as a second reviewer. The second rule optimized sensitivity, for example, to reduce the number of citations to be manually screened. RESULTS: Under the balanced rule, sensitivities ranged from 81.1% to 96.5% and specificities ranged from 25.8% to 80.4%. Across all reviews, GPT identified 7 of 708 citations (1%) missed by humans that should have been included after full-text screening at the cost of 10 279 of 22 665 false-positive recommendations (45.3%) that would require reconciliation during the screening process. Under the sensitive rule, sensitivities ranged from 94.6% to 99.8% and specificities ranged from 2.2% to 46.6%. Limiting manual screening to citations not ruled out by GPT could reduce the number of citations to screen from 127 of 6334 (2%) to 1851 of 4077 (45.4%), at the cost of missing from 0 to 1 of 26 citations (3.8%) at the full-text level. LIMITATIONS: Time needed to fine-tune prompt. Retrospective nature of the study, convenient sample of 5 systematic reviews, and GPT performance sensitive to prompt development and time. CONCLUSION: The GPT-3.5 Turbo model may be used as a second reviewer for title and abstract screening, at the cost of additional work to reconcile added false positives. It also showed potential to reduce the number of citations before screening by humans, at the cost of missing some citations at the full-text level. PRIMARY FUNDING SOURCE: None.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Initial studies indicate that LLM performance in identifying relevant publications is highly dependent on the prompt and often results in high differences between sensitivity and specificity, reflecting the tendency of LLMs toward overconfidence or underconfidence.
“Initial studies indicate that LLM performance in identifying relevant publications is highly dependent on the prompt and often results in high differences between sensitivity and speci fi city, re fl ecting the tendency of LLMs toward overcon fi dence or undercon fi dence.”
From AIM review tool: artificial intelligence for smarter systematic review screening (2026) · cites this paper as [37]
Cites
Rayyan-a web and mobile app for systematic reviews.
Ouzzani, M., Hammady, H., Fedorowicz, Z., & Elmagarmid, A. (2016) · Systematic Reviews · doi:10.1186/s13643-016-0384-4 · Cited by 26,092 · see citing works
Kung, T. H., Cheatham, M., Medenilla, A., Sillos, C., De Leon, L., Elepaño, C., Madriaga, M., Aggabao, R., Diaz-Candido, G., Maningo, J., et al. (2023) · PLOS Digital Health · doi:10.1371/journal.pdig.0000198 · Cited by 4,115 · see citing works
Ayers, J. W., Poliak, A., Dredze, M., Leas, E. C., Zhu, Z., Kelley, J. B., Faix, D. J., Goodman, A. M., Longhurst, C. A., Hogarth, M., et al. (2023) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2023.1838 · Cited by 2,613 · cited 1× · see citing works
“These included 1) a review on the ef fi cacy of primary treatment of con fi rmed COVID-19 in outpatient settings (Sommer and colleagues, 2023 [23]), 2) a review on the ef fi cacy of outpatient treatment options for the Omicron variant of SARS-CoV-2 (Sommer and colleagues, 2023 [24]), 3) a…”
MethodAn open source machine learning framework for efficient and transparent systematic reviews
van de Schoot, R., de Bruin, J., Schram, R., Zahedi, P., de Boer, J., Weijdema, F., Kramer, B., Huijts, M., Hoogerwerf, M., Ferdinands, G., et al. (2021) · Nature Machine Intelligence · doi:10.1038/s42256-020-00287-7 · Cited by 1,157 · see citing works
Interactive machine learning for health informatics: when do we need the human-in-the-loop?
Holzinger, A. (2016) · Brain Informatics · doi:10.1007/s40708-016-0042-6 · Cited by 906 · see citing works
Sources of Variation and Bias in Studies of Diagnostic Accuracy
Whiting, P. F., Rutjes, A. W., Reitsma, J. B., Glas, A. S., Bossuyt, P. M., & Kleijnen, J. (2004) · Annals of Internal Medicine · doi:10.7326/0003-4819-140-3-200402030-00010 · Cited by 801 · see citing works
O’Mara-Eves, A., Thomas, J., McNaught, J., Miwa, M., & Ananiadou, S. (2015) · Systematic Reviews · doi:10.1186/2046-4053-4-5 · Cited by 681 · see citing works
Waffenschmidt, S., Knelangen, M., Sieben, W., Bühn, S., & Pieper, D. (2019) · BMC Medical Research Methodology · doi:10.1186/s12874-019-0782-0 · Cited by 626 · see citing works
Bornmann, L., Haunschild, R., & Mutz, R. (2021) · Humanities and Social Sciences Communications · doi:10.1057/s41599-021-00903-w · Cited by 532 · see citing works
How is ChatGPT's behavior changing over time?
Chen, L., Zaharia, M., & Zou, J. (2023) · arXiv · doi:10.48550/arxiv.2307.09009 · Cited by 507 · see citing works
How Is ChatGPT’s Behavior Changing Over Time?
Chen, L., Zaharia, M., & Zou, J. (2024) · Harvard Data Science Review · doi:10.1162/99608f92.5317da47 · Cited by 507 · see citing works
Sarraju, A., Bruemmer, D., Iterson, E. H. V., Cho, L., Rodríguez, F., & Laffin, L. J. (2023) · JAMA · doi:10.1001/jama.2023.1044 · Cited by 455 · see citing works
Reitsma, J. B., Rutjes, A. W. S., Khan, K. S., Coomarasamy, A., & Bossuyt, P. M. (2009) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2009.02.005 · Cited by 405 · see citing works
Khraisha, Q., Put, S., Kappenberg, J., Warraitch, A., & Hadfield, K. (2024) · Research Synthesis Methods · doi:10.1002/jrsm.1715 · Cited by 269 · see citing works
Khraisha, Q., Put, S., Kappenberg, J., Warraitch, A., & Hadfield, K. (2023) · arXiv · doi:10.48550/arxiv.2310.17526 · Cited by 268 · see citing works
Error rates of human reviewers during abstract screening in systematic reviews
Wang, Z., Nayfeh, T., Tetzlaff, J., O’Blenis, P., & Murad, M. H. (2020) · PLOS ONE · doi:10.1371/journal.pone.0227742 · Cited by 217 · see citing works
Qureshı, R., Shaughnessy, D. P., Gill, K., Robinson, K. A., Li, T., & Agai, E. (2023) · Systematic Reviews · doi:10.1186/s13643-023-02243-z · Cited by 217 · see citing works
Want To Reduce Labeling Cost? GPT-3 Can Help
Wang, S., Liu, Y., Xu, Y., Zhu, C., & Zeng, M. (2021) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2021.findings-emnlp.354 · Cited by 208 · see citing works
Gartlehner, G., Affengruber, L., Titscher, V., Noel-Storr, A., Dooley, G., Ballarini, N., & König, F. (2020) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2020.01.005 · Cited by 206 · see citing works
Self-citationAutomated paper screening for clinical reviews using large language models: Data analysis study
Guo, E., Gupta, M., Deng, J., Park, Y.-J., Paget, M., & Naugler, C. (2024) · Journal of Medical Internet Research · doi:10.2196/48996 · Cited by 205 · see citing works
Data extraction for evidence synthesis using a large language model: A proof-of-concept study
Gartlehner, G., Kahwati, L., Hilscher, R., Thomas, I., Kugley, S., Crotty, K., Viswanathan, M., Nussbaumer‐Streit, B., Booth, G., Erskine, N., et al. (2024) · Research Synthesis Methods · doi:10.1002/jrsm.1710 · Cited by 164 · cited 1× · see citing works
“Namely, preliminary results were promising for extracting data elements from published studies with high accuracy and determining adherence to reporting guidelines (19, 37).”
BackgroundSelf-citationEstimating Time to Conduct a Meta-analysis From Number of Citations Retrieved
Allen, I. E. (1999) · JAMA: The Journal of the American Medical Association · doi:10.1001/jama.282.7.634 · Cited by 163 · see citing works
Four principles to make evidence synthesis more useful for policy
Donnelly, C. A., Boyd, I., Campbell, P., Craig, C., Vallance, P., Walport, M., Whitty, C. J. M., Woods, E., & Wormald, C. (2018) · Nature · doi:10.1038/d41586-018-05414-4 · Cited by 157 · see citing works
Is GPT-3 a Good Data Annotator?
Ding, B., Qin, C., Liu, L., Chia, Y. K., Li, B., Joty, S., & Bing, L. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2023.acl-long.626 · Cited by 149 · see citing works
Gates, A., Johnson, C., & Hartling, L. (2018) · Systematic Reviews · doi:10.1186/s13643-018-0707-8 · Cited by 148 · see citing works
Créquit, P., Trinquart, L., Yavchitz, A., & Ravaud, P. (2016) · BMC Medicine · doi:10.1186/s12916-016-0555-0 · Cited by 125 · see citing works
Self-citationStatistical stopping criteria for automated screening in systematic reviews
Callaghan, M. W., & Müller-Hansen, F. (2020) · Systematic Reviews · doi:10.1186/s13643-020-01521-4 · Cited by 121 · cited 1× · see citing works
“…PICOS framework, making it easier for humans to identify false positives and negatives, and 2) their performance is not dependent on the proportion of citations that will be included by researchers, nor is it subject to the lack of appropriate stopping criteria faced by prioritization tools (32).”
BackgroundThe Diagnostic and Triage Accuracy of the GPT-3 Artificial Intelligence Model
Levine, D. M., Tuwani, R., Kompa, B., Varma, A., Finlayson, S. G., Mehrotra, A., & Beam, A. (2023) · doi:10.1101/2023.01.30.23285067 · Cited by 106 · see citing works
Towards Automatic Recognition of Scientifically Rigorous Clinical Research Evidence
Kilicoglu, H., Demner-Fushman, D., Rindflesch, T. C., Wilczynski, N. L., & Haynes, R. B. (2009) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m2996 · Cited by 101 · see citing works
Characteristics and knowledge synthesis approach for 456 network meta-analyses: a scoping review
Zarin, W., Veroniki, A. A., Nincic, V., Vafaei, A., Reynen, E., Motiwala, S. S., Antony, J., Sullivan, S. M., Rios, P. D. L., Daly, C., et al. (2017) · BMC Medicine · doi:10.1186/s12916-016-0764-6 · Cited by 94 · see citing works
Hasan, B., Saadi, S. M., Rajjoub, N. S., Hegazi, M., Al-Kordi, M., Fleti, F., Farah, M. H., Riaz, I. B., Banerjee, I., Wang, Z., et al. (2024) · BMJ Evidence-Based Medicine · doi:10.1136/bmjebm-2023-112597 · Cited by 72 · see citing works
Kiesswetter, E., Stadelmaier, J., Πετροπούλου, Μ., Morze, J., Grummich, K., Roux, I., Lay, R., Himmelsbach, L., Kussmann, M., Roeger, C., et al. (2023) · Advances in Nutrition · doi:10.1016/j.advnut.2023.03.004 · Cited by 49 · see citing works
Self-citationMeasuring the impact of screening automation on meta-analyses of diagnostic test accuracy
Norman, C. R., Leeflang, M., Porcher, R., & Aurélie Névéol, A. (2019) · Systematic Reviews · doi:10.1186/s13643-019-1162-x · Cited by 31 · see citing works
Assessing the Ability of ChatGPT to Screen Articles for Systematic Reviews
Syriani, E., David, I., & Kumar, G. (2023) · arXiv · doi:10.48550/arxiv.2307.06464 · Cited by 27 · see citing works
GPT for RCTs?: Using AI to determine adherence to reporting guidelines
Wrightson, J. G., Blazey, P., Moher, D., Khan, K. M., & Ardern, C. L. (2023) · doi:10.1101/2023.12.14.23299971 · Cited by 24 · see citing works
GPT for RCTs? Using AI to determine adherence to clinical trial reporting guidelines
Wrightson, J. G., Blazey, P., Moher, D., Khan, K. M., & Ardern, C. L. (2025) · BMJ Open · doi:10.1136/bmjopen-2024-088735 · Cited by 24 · see citing works
Sommer, I., Dobrescu, A. I., Ledinger, D., Moser, I., Thaler, K. J., Persad, E., Fangmeyer, M., Emprechtinger, R., Klerings, I., & Gartlehner, G. (2022) · Annals of Internal Medicine · doi:10.7326/m22-2202 · Cited by 17 · see citing works
Self-citationIs GPT-3 a Good Data Annotator?
Ding, B., Qin, C., Liu, L., Chia, Y. K., Joty, S., Li, B., & Bing, L. (2022) · arXiv · doi:10.48550/arxiv.2212.10450 · Cited by 15 · see citing works
O’Hearn, K., MacDonald, C., Tsampalieros, A. K., Kadota, L., Sandarage, R. V., Jayawarden, S. K., Datko, M., Reynolds, J. M., Bui, T., Sultan, S. A., et al. (2021) · BMC Medical Research Methodology · doi:10.1186/s12874-021-01335-5 · Cited by 12 · see citing works
Sommer, I., Ledinger, D., Thaler, K. J., Dobrescu, A. I., Persad, E., Fangmeyer, M., Klerings, I., & Gartlehner, G. (2023) · Annals of Internal Medicine · doi:10.7326/m23-1626 · Cited by 11 · see citing works
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
Scherbakov, D., Hubig, N., Jansari, V., Bakumenko, A., & Lenert, L. A. (2025) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocaf063 · Cited by 154 · see citing works
Generative artificial intelligence use in evidence synthesis: A systematic review
Clark, J., Barton, B., Albarqouni, L., Byambasuren, O., Jowsey, T., Keogh, J., Liang, T., Moro, C., O’Neill, H., & Jones, M. (2025) · Research Synthesis Methods · doi:10.1017/rsm.2025.16 · Cited by 90 · see citing works
Development of Prompt Templates for Large Language Model–Driven Screening in Systematic Reviews
Cao, C., Sang, J., Arora, R., Chen, D., Kloosterman, R., Cecere, M., Gorla, J., Saleh, R., Drennan, I., Teja, B., et al. (2025) · Annals of Internal Medicine · doi:10.7326/annals-24-02189 · Cited by 67 · see citing works
ChatGPT-4o can serve as the second rater for data extraction in systematic reviews
Motzfeldt Jensen, M., Brix Danielsen, M., Riis, J., Assifuah Kristjansen, K., Andersen, S., Okubo, Y., & Jørgensen, M. G. (2025) · PLOS ONE · doi:10.1371/journal.pone.0313401 · Cited by 60 · see citing works
Can generative AI reliably synthesise literature? exploring hallucination issues in ChatGPT
Adel, A., & Saleem, N. H. (2025) · AI & Society · doi:10.1007/s00146-025-02406-7 · Cited by 54 · see citing works
Matsui, K., Utsumi, T., Aoki, Y., Maruki, T., Takeshima, M., & Takaesu, Y. (2024) · Journal of Medical Internet Research · doi:10.2196/52758 · Cited by 50 · see citing works
Gartlehner, G., Kahwati, L. C., Nußbaumer-Streit, B., Crotty, K., Hilscher, R., Kugley, S., Viswanathan, M., Thomas, I., Konet, A., Booth, G., et al. (2024) · BMJ Evidence-Based Medicine · doi:10.1136/bmjebm-2024-113199 · Cited by 37 · see citing works
Self-citationOpportunities, challenges and risks of using artificial intelligence for evidence synthesis
Siemens, W., von Elm, E. B., Binder, H., Böhringer, D., Eisele‐Metzger, A., Gartlehner, G., Hanegraaf, P., Metzendorf, M., Mosselman, J. J., Nowak, A. J., et al. (2025) · BMJ Evidence-Based Medicine · doi:10.1136/bmjebm-2024-113320 · Cited by 34 · see citing works
Self-citationSystematic review and meta-analysis of preclinical studies
Ineichen, B. V., Held, U., Salanti, G., Macleod, M. R., & Wever, K. E. (2024) · Nature Reviews Methods Primers · doi:10.1038/s43586-024-00347-x · Cited by 27 · see citing works
Fleurence, R., Dawoud, D., Bian, J., Higashi, M. K., Wang, X., Xu, H., Chhatwal, J., & Ayer, T. (2025) · Value in Health · doi:10.1016/j.jval.2025.06.018 · Cited by 27 · see citing works
Lee, K., Paek, H., Ofoegbu, N., Rube, S., Higashi, M. K., Dawoud, D., Xu, H., Shi, L., & Wang, X. (2025) · Value in Health · doi:10.1016/j.jval.2025.08.002 · Cited by 20 · see citing works
López‐Pineda, A., Nouni-García, R., Carbonell-Soliva, Á., Gil-Guillén, V. F., Carratalá‐Munuera, C., & Borrás, F. (2025) · Research Synthesis Methods · doi:10.1017/rsm.2025.15 · Cited by 18 · see citing works
Optimal large language models to screen citations for systematic reviews
Oami, T., Okada, Y., & Nakada, T. (2025) · Research Synthesis Methods · doi:10.1017/rsm.2025.10014 · Cited by 16 · see citing works
Rashid, M., Yi, C. S., Sathapanasiri, T., Udayachalerm, S., Boonpattharatthiti, K., Insuk, S., Veettil, S. K., Lai, N. M., Chaiyakunapruk, N., Dhippayom, T., et al. (2025) · Value in Health · doi:10.1016/j.jval.2025.07.001 · Cited by 18 · see citing works
GPT-3.5 Turbo and GPT-4 Turbo in Title and Abstract Screening for Systematic Reviews
Oami, T., Okada, Y., & Nakada, T. (2025) · JMIR Medical Informatics · doi:10.2196/64682 · Cited by 13 · see citing works
Do it faster with PICOS: Generative AI-Assisted systematic review screening
Vallamchetla, S. K., Abdelkader, O., Elnaggar, A., Ramadan, D., Islam Shourav, M. M., Riaz, I. B., & Lin, M. P. (2025) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2025.104860 · Cited by 13 · see citing works
Gartlehner, G., Nußbaumer-Streit, B., Hamel, C., Garritty, C., Griebler, U., King, V. J., Devane, D., & Kamel, C. (2025) · Cochrane Evidence Synthesis and Methods · doi:10.1002/cesm.70063 · Cited by 12 · see citing works
Self-citationFleurence, R., Wang, X., Bian, J., Higashi, M. K., Ayer, T., Xu, H., Dawoud, D., & Chhatwal, J. (2025) · Value in Health · doi:10.1016/j.jval.2025.04.2167 · Cited by 10 · see citing works
Tao, Y., Li, X., Yisha, Z., Yang, S., Zhan, S., & Sun, F. (2025) · Health Data Science · doi:10.34133/hds.0322 · Cited by 10 · see citing works
Shi, X., Zhao, W., Chen, T., Yang, C., & Du, J. (2025) · Nature Communications · doi:10.1038/s41467-025-62783-x · Cited by 8 · see citing works
Sciurti, A., Migliara, G., Siena, L. M., Isonne, C., Blasiis, M. R. D., Sinopoli, A., Iera, J., Marzuillo, C., Vito, C. D., Villari, P., et al. (2025) · Research Synthesis Methods · doi:10.1017/rsm.2025.10044 · Cited by 7 · see citing works
Cassell, K., Ologunowa, A., Rastegar-Mojarad, M., Chun, B., Huang, Y.-L., Wang, D., & Cossrow, N. (2025) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2025.1662202 · Cited by 6 · see citing works
Ramchandani, R., Guo, E., Rakab, E., Rathod, J., Strain, J., Klement, W., Shorr, R., Williams, E., Jones, D., & Gilbert, S. (2025) · PeerJ Computer Science · doi:10.7717/peerj-cs.2822 · Cited by 5 · see citing works
Doneva, S. E., de Viragh, S., Hubarava, H., Schandelmaier, S., Briel, M., & Ineichen, B. V. (2025) · Research Synthesis Methods · doi:10.1017/rsm.2025.10031 · Cited by 5 · see citing works
Janoudi, G., Uzun, M., Disher, T. C., Jurdana, M., Fuzul, E., Ivkovic, J., & Hutton, B. (2025) · Value in Health · doi:10.1016/j.jval.2025.09.008 · Cited by 6 · see citing works
Rich, A. J., McGorray, E. L., Baldwin-SoRelle, C., Cawley, M. A., Grigg, K., Beach, L. B., Phillips, G., & Poteat, T. (2025) · Journal of the Medical Library Association JMLA · doi:10.5195/jmla.2025.1860 · Cited by 5 · see citing works
Bentegeac, R., Guellec, B. L., Leblanc, V., Lenain, R., Dauchet, L., Gauthier, V., Gerard, E., Chazard, E., Amouyel, P., Aymes, E., et al. (2026) · Research Synthesis Methods · doi:10.1017/rsm.2026.10079 · Cited by 5 · see citing works
APOE genotype and the effect of statins: a systematic review and meta-analysisPreprint
Asiimwe, I. G., Gebru, T. G., Jorgensen, A., & Pirmohamed, M. (2024) · medRxiv · doi:10.1101/2024.12.13.24318973 · Cited by 4 · see citing works
Mo, Y., Tang, M., Lin, R., Zhou, B., & Li, X. (2025) · IEEE Access · doi:10.1109/access.2025.3582519 · Cited by 3 · see citing works
Courvoisier, D. S., Buitrago‐García, D., Buclin, C. P., Bürgisser, N., Iudici, M., & Mongin, D. (2025) · Research Synthesis Methods · doi:10.1017/rsm.2025.10054 · Cited by 3 · see citing works
Song, Z., Huang, S., Thapa, N., Zhang, X., Park, B.-K., Lu, J., Li, W., Liu, W., Zhan, B., & Li, J. (2026) · Research Synthesis Methods · doi:10.1017/rsm.2026.10094 · Cited by 3 · see citing works
Cawley, M. A., Carlson, R., Vest, T. A., & Eckel, S. F. (2024) · American Journal of Health-System Pharmacy · doi:10.1093/ajhp/zxae357 · Cited by 3 · see citing works
Janoudi, G., Uzun, M., Jurdana, M., Fuzul, E., & Ivkovic, J. (2024) · medRxiv · doi:10.1101/2024.09.06.24313186 · Cited by 3 · see citing works
Tian, H.-Y., Xu, C., Liu, C.-J., & Li, Y. (2025) · International Archives of Allergy and Immunology · doi:10.1159/000548019 · Cited by 2 · see citing works
Tran, V.-T., Graña, C., Boutron, I., & Ravaud, P. (2025) · Research Synthesis Methods · doi:10.1017/rsm.2025.10034 · Cited by 2 · see citing works
Self-citationFrench-Style Applications of Artificial Intelligence to Human Health in a European Context
Desbois, D. (2024) · IFIP advances in information and communication technology · doi:10.1007/978-3-031-67535-5_10 · Cited by 1 · see citing works
Cheng, P.-J., Hu, F., Chen, L.-Y., Liu, J.-Y., Wu, J., & Chen, W. (2025) · Taiwan Journal of Ophthalmology · doi:10.4103/tjo.tjo-d-25-00072 · Cited by 1 · see citing works
Human-AI collaboration enhances the performance of large language models in risk of bias assessment
Li, Y., Yang, F., Wu, M., & Li, J. (2026) · BMC Medical Research Methodology · doi:10.1186/s12874-025-02763-3 · Cited by 1 · see citing works
Forero, D. A. (2026) · Current Genomics · doi:10.2174/0113892029403403260112075225 · Cited by 1 · see citing works
Fleurence, R., Qureshi, R., Aggarwal, R., Bian, J., Cheyne, S., Dawoud, D., Faden, R., & Chhatwal, J. (2026) · Value in Health · doi:10.1016/j.jval.2026.06.012 · Cited by 2 · see citing works
Bloudek, L. M., Cichewicz, A. B., Patel, K., Sullivan, S. D., & Kallmes, K. M. (2026) · PharmacoEconomics · doi:10.1007/s40273-026-01648-7 · Cited by 1 · see citing works
Shi, X., Zhao, W., Chen, T., Yang, C., & Du, J. (2024) · medRxiv · doi:10.1101/2024.03.18.24304457 · Cited by 1 · see citing works
Gartlehner, G., Banda, S., Callaghan, M., Chase, J. D., Dobrescu, A. I., Eisele‐Metzger, A., Flemyng, E., Gardner, S., Griebler, U., Helfer, B., et al. (2026) · medRxiv · doi:10.64898/2026.04.13.26350802 · Cited by 1 · see citing works
Self-citationAIM review tool: artificial intelligence for smarter systematic review screening
Mena, S., Rituerto-González, E., Coutts, F., von Trott, J., Jacobs, G. R., Bryant, L., Moles, L., Sirbu, N., Promet, L., Oliver, D., et al. (2026) · npj Artificial Intelligence · doi:10.1038/s44387-026-00080-8
Rahgozar, A., & Mortezaagha, P. (2026) · arXiv · doi:10.48550/arxiv.2608.14551
The Heap, the Hype, the Reality: Generative Pretrained Transformer for Systematic Reviews
Li, T., & Chang, S. (2024) · Annals of Internal Medicine · doi:10.7326/m24-0877
Xia, Z., Ye, J., Hu, B., Qiang, Q., & Debnath, R. (2024) · Research Square · doi:10.21203/rs.3.rs-5063165/v1 · Cited by 1 · see citing works
Cheima, M., Altayyar, S., Vergonjeanne, M., Gellé, T., & Preux, P. (2025) · Journal of Epidemiology and Population Health · doi:10.1016/j.jeph.2025.202990
A Parallel Corpus of Chinese-English Legal Judgments with Argumentative Structure Annotations
Zhou, J., Cheng, F., Zhang, X., & Huang, K. (2025) · Data Intelligence · doi:10.3724/2096-7004.di.2025.0056
Literature Discovery and Synthesis With AI
(2026) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3373-9340-7.ch003
Barreto, G., Burke, C., Davies, P. A., Halicka, M., Paterson, C., Swinton, P. A., Saunders, B., & Higgins, J. P. T. (2026) · medRxiv · doi:10.64898/2026.04.08.26350384
Derdouri, A., Oyama, T., Murayama, Y., & Masago, Y. (2026) · Research Square · doi:10.21203/rs.3.rs-9582053/v1
Müller-Eberstein, R., & Barnes, B. (2026) · Research Square · doi:10.21203/rs.3.rs-9789751/v1
Jóźwiak, Á. B., Imre, A., Józwiak‐Hagymásy, J., Tittmann, J., Nagy, Á., Kovács, S., Kardas, P., van Boven, J. F. M., Mommers, I., & Ágh, T. (2026) · Expert Review of Pharmacoeconomics & Outcomes Research · doi:10.1080/14737167.2026.2691995
Melo, P. H. M., Guadagno, E., & Poenaru, D. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-30813-9_73
Yang, J., & Kim, K. (2026) · The Journal of Korean Association of Computer Education · doi:10.32431/kace.2026.29.6.006
Wang, L., & Poenaru, D. (2026) · medRxiv · doi:10.64898/2026.07.23.26358815
Evaluating AI Performance in Systematic Literature Reviews for HEOR: A Case Study
Wang-Silvanto, J., Jajoo, M., Guo, H., Ohri, R., Nelissen, J., & Comabella, C. C. I. (2026) · Journal of health economics and outcomes research · doi:10.36469/001c.165173
Evaluating AI Performance in Systematic Literature Reviews for HEOR: A Case Study
Wang-Silvanto, J., Jajoo, M., Guo, H., Ohri, R., Nelissen, J., & Comabella, C. C. I. (2026) · Journal of health economics and outcomes research · doi:10.36469/jheor.2026.165173
Barragán, A. M., Bonett, S. E. O., Rodríguez-Grande, E.-I., Orjuela-Cañón, Á. D., Perdómo, O., & Sánchez, G. (2026) · Cochrane Evidence Synthesis and Methods · doi:10.1002/cesm.70098
Borovoi, L., Bar, K., Lazar, S., Bernstine, T., Essa-Hadad, J., Kuodi, P., Rhone, N., & Edelstein, M. L. (2026) · doi:10.2196/preprints.109576
Kertész, G., Czere, J. T., Zrubka, Z., Gulàcsi, L., & Péntek, M. (2026) · JMIR Formative Research · doi:10.2196/86647 · Cited by 2 · see citing works
Merchán-Cruz, E. A., Gabelaia, I., Soe, S. P., Yatskiv, I., & Dmitry, P. (2026) · Informatics · doi:10.3390/informatics13100160
Wiedermann, C. J. (2026) · Medical Ethics and Bioethics · doi:10.3390/meb1010003
Başer, O. (2026) · Journal of health economics and outcomes research · doi:10.36469/001c.171937
Başer, O. (2026) · Journal of health economics and outcomes research · doi:10.36469/jheor.2026.171937
Babaei, A., Georgievski, G., Hosseini, B., Li, A. S., & Pelc, J. (2026) · Digital Health · doi:10.1177/20552076261503859
- Authors
- Tran, V.-T., Gartlehner, G., Yaacoub, S., Boutron, I., Schwingshackl, L., Stadelmaier, J., Sommer, I., Alebouyeh, F., Afach, S., Meerpohl, J., & Ravaud, P.
- Affiliations
- Inserm (FR)
- Universität für Weiterbildung Krems (AT)
- University of Freiburg (DE)
- Université Paris-Est Créteil (FR)
- ORCID
- Venue
- Annals of Internal Medicine
- Publisher
- American College of Physicians
- Published
- 2024-06
- Volume
- 177
- Issue
- 6
- Pages
- 791-799
- ISSN
- 0003-4819 (ISSN-L)
- 1539-3704
- OpenAlex
- W4398144060
- Cited by
- 66
- References
- 36
- Citations by year
- 2026: 26 · 2025: 30 · 2024: 10
- FWCI
- 8.33
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Annals of Internal Medicine (publishedVersion)
- PubMed (publishedVersion)
- HAL (submittedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
- Keywords
- GPT-3.5-turbo, large language models, title and abstract screening, systematic reviews, meta-analysis, diagnostic accuracy, sensitivity and specificity, false positives, prompt engineering, full-text screening, AI-assisted literature screening
- MeSH terms
- Abstracting and Indexing, Systematic Reviews as Topic, Artificial Intelligence, Humans, Review Literature as Topic, Sensitivity and Specificity, Meta-Analysis as Topic
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
