Screening articles for systematic reviews with ChatGPT
Open accessPublished version of a preprint
Abstract
Systematic reviews (SRs) provide valuable evidence for guiding new research directions. However, the manual effort involved in selecting articles for inclusion in an SR is error-prone and time-consuming. While screening articles has traditionally been considered challenging to automate, the advent of large language models offers new possibilities. In this paper, we discuss the effect of using ChatGPT on the SR process. In particular, we investigate the effectiveness of different prompt strategies for automating the article screening process using five real SR datasets. Our results show that ChatGPT can reach up to 82% accuracy. The best performing prompts specify exclusion criteria and avoid negative shots. However, prompts should be adapted to different corpus characteristics.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The most effective prompts clearly state exclusion criteria and avoid negative examples, but they should also be tailored to the specific features of the corpus.
“The most effective prompts clearly state exclusion criteria and avoid negative examples, but they should also be tailored to the specific features of the corpus (Syriani et al., 2024).”
From Machine Learning-Assisted Systematic Review: A Case Study in Learning Analytics (2025) · cites this paper as Syriani et al., 2024
Initial studies indicate that LLM performance in identifying relevant publications is highly dependent on the prompt and often results in high differences between sensitivity and specificity, reflecting the tendency of LLMs toward overconfidence or underconfidence.
“Initial studies indicate that LLM performance in identifying relevant publications is highly dependent on the prompt and often results in high differences between sensitivity and speci fi city, re fl ecting the tendency of LLMs toward overcon fi dence or undercon fi dence.”
From AIM review tool: artificial intelligence for smarter systematic review screening (2026) · cites this paper as [36]
Cites
Scikit-learn: Machine Learning in Python
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Müller, A., Nothman, J., Louppe, G., et al. (2012) · arXiv · doi:10.48550/arxiv.1201.0490 · Cited by 71,016 · see citing works
Distributed Representations of Words and Phrases and their Compositionality
Mikolov, T., Sutskever, I., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1310.4546 · Cited by 24,533 · see citing works
Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks
Reimers, N., & Gurevych, I. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1410 · Cited by 12,778 · see citing works
Advances in Neural Information Processing Systems 25
Zhang, Y., Sutton, C. A., Storkey, A., & Zoubin Ghahramani, A. (2012) · Cited by 12,653 · see citing works
Distributed Representations of Sentences and DocumentsPreprint
Le, Q. V., & Mikolov, T. (2014) · International Conference on Machine Learning · doi:10.48550/arxiv.1405.4053 · Cited by 10,307 · see citing works
Text categorization with Support Vector Machines: Learning with many relevant features
Joachims, T. (1998) · Lecture notes in computer science · doi:10.1007/bfb0026683 · Cited by 8,134 · see citing works
(2012) · Cited by 8,009 · see citing works
(2009) · Minds at UW (University of Wisconsin) · Cited by 4,791 · see citing works
Liu, P., Yuan, W., Fu, J., Jiang, Z., Hayashi, H., & Neubig, G. (2023) · ACM Computing Surveys · doi:10.1145/3560815 · Cited by 4,361 · see citing works
Liu, P., Yuan, W., Fu, J., Jiang, Z., Hayashi, H., & Neubig, G. (2021) · arXiv · doi:10.48550/arxiv.2107.13586 · Cited by 4,352 · see citing works
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E., Le, Q., & Zhou, D. (2022) · arXiv · doi:10.48550/arxiv.2201.11903 · Cited by 4,263 · see citing works
GPT-3: Its Nature, Scope, Limits, and Consequences
Floridi, L., & Chiriatti, M. (2020) · Minds and Machines · doi:10.1007/s11023-020-09548-1 · Cited by 2,519 · see citing works
Beyond Accuracy, F-Score and ROC: A Family of Discriminant Measures for Performance Evaluation
Sokolova, M., Japkowicz, N., & Szpakowicz, S. (2006) · Lecture Notes in Computer Science · doi:10.1007/11941439_114 · Cited by 1,974 · see citing works
LaValley, M. P. (2008) · Circulation · doi:10.1161/circulationaha.106.682658 · Cited by 1,163 · see citing works
An open source machine learning framework for efficient and transparent systematic reviews
van de Schoot, R., de Bruin, J., Schram, R., Zahedi, P., de Boer, J., Weijdema, F., Kramer, B., Huijts, M., Hoogerwerf, M., Ferdinands, G., et al. (2021) · Nature Machine Intelligence · doi:10.1038/s42256-020-00287-7 · Cited by 1,157 · see citing works
Page, M. J., Moher, D., Bossuyt, P., Boutron, I., Hoffmann, T., mulrow, C., Shamseer, L., Tetzlaff, J., Akl, E., Brennan, S. E., et al. (2020) · doi:10.31222/osf.io/gwdhk · Cited by 916 · see citing works
A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPT
White, J., Fu, Q., Hays, S., Sandborn, M., Olea, C., Gilbert, H., Elnashar, A., Spencer-Smith, J., & Schmidt, D. C. (2023) · arXiv · doi:10.48550/arxiv.2302.11382 · Cited by 838 · see citing works
Expediting systematic reviews: methods and implications of rapid reviews
Ganann, R., Ciliska, D., & Thomas, H. (2010) · Implementation Science · doi:10.1186/1748-5908-5-56 · Cited by 752 · see citing works
Borah, R., Brown, A. W., Capers, P. L., & Kaiser, K. A. (2017) · BMJ Open · doi:10.1136/bmjopen-2016-012545 · Cited by 737 · see citing works
A comprehensive survey on pretrained foundation models: a history from BERT to ChatGPT
Zhou, C., Li, Q., Li, C., Yu, J., Liu, Y., Wang, G., Zhang, K., Ji, C., Yan, Q., He, L., et al. (2024) · International Journal of Machine Learning and Cybernetics · doi:10.1007/s13042-024-02443-6 · Cited by 704 · see citing works
Role of Chat GPT in public health
Biswas, S. S. (2023) · Annals of Biomedical Engineering · doi:10.1007/s10439-023-03172-7 · Cited by 673 · see citing works
Chicco, D., & Jurman, G. (2023) · BioData Mining · doi:10.1186/s13040-023-00322-4 · Cited by 667 · see citing works
Instruction Tuning for Large Language Models: A Survey
Zhang, S., Dong, L., Li, X., Zhang, S., Sun, X., Wang, S., Li, J., Hu, R., Zhang, T., Wang, G., et al. (2025) · ACM Computing Surveys · doi:10.1145/3777411 · Cited by 640 · see citing works
Marshall, I. J., & Wallace, B. C. (2019) · Systematic Reviews · doi:10.1186/s13643-019-1074-9 · Cited by 638 · see citing works
The Benefits and Challenges of Using Systematic Reviews in International Development Research
Mallett, R., Hagen-Zanker, J., Slater, R., & Duvendack, M. (2012) · Journal of Development Effectiveness · doi:10.1080/19439342.2012.711342 · Cited by 565 · see citing works
Systematic review automation technologies
Tsafnat, G., Glasziou, P., Choong, M. K., Dunn, A., Galgani, F., & Coiera, E. (2014) · Systematic Reviews · doi:10.1186/2046-4053-3-74 · Cited by 472 · see citing works
Evaluation Measures for Models Assessment over Imbalanced Data Sets
Bekkar, M., Djemaa, H. K., & Alitouche, T. A. (2013) · Journals & Books Hosting (International Knowledge Sharing Platform) · Cited by 436 · see citing works
Automation of systematic literature reviews: A systematic literature review
van Dinter, R., Tekinerdogan, B., & Catal, C. (2021) · Information and Software Technology · doi:10.1016/j.infsof.2021.106589 · Cited by 432 · see citing works
Reducing Workload in Systematic Review Preparation Using Automated Citation Classification
Cohen, A. M., Hersh, W. R., Peterson, K., & Yen, P.-Y. (2006) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m1929 · Cited by 411 · see citing works
Potential Use of Chat GPT in Global Warming
Biswas, S. S. (2023) · Annals of Biomedical Engineering · doi:10.1007/s10439-023-03171-8 · Cited by 291 · see citing works
Automating data extraction in systematic reviews: a systematic review
Jonnalagadda, S. R., Goyal, P., & Huffman, M. D. (2015) · Systematic Reviews · doi:10.1186/s13643-015-0066-7 · Cited by 287 · see citing works
Can ChatGPT Write a Good Boolean Query for Systematic Review Literature Search?
Wang, S., Scells, H., Koopman, B., & Zuccon, G. (2023) · Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval · doi:10.1145/3539618.3591703 · Cited by 275 · see citing works
Khraisha, Q., Put, S., Kappenberg, J., Warraitch, A., & Hadfield, K. (2024) · Research Synthesis Methods · doi:10.1002/jrsm.1715 · Cited by 269 · see citing works
Khraisha, Q., Put, S., Kappenberg, J., Warraitch, A., & Hadfield, K. (2023) · arXiv · doi:10.48550/arxiv.2310.17526 · Cited by 268 · see citing works
Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems
Kitamura, Y., Kitamura, Y., Quigley, A., Isbister, K., Igarashi, T., Japan, C. '. C. C. O. H. F. I. C. S. 2. Y., Association for Computing Machinery-Digital Library, ACM Special Interest Group on Computer-Human Interaction, & SIGs, A. (2021) · doi:10.1145/3411763 · Cited by 245 · see citing works
Distribution-balanced stratified cross-validation for accuracy estimation
Zeng, X., & Martinez, T. R. (2000) · Journal of Experimental & Theoretical Artificial Intelligence · doi:10.1080/095281300146272 · Cited by 242 · see citing works
Instruction Tuning for Large Language Models: A Survey
Zhang, S., Dong, L., Li, X., Zhang, S., Sun, X., Wang, S., Li, J., Hu, R., Zhang, T., Wu, F., et al. (2023) · arXiv · doi:10.48550/arxiv.2308.10792 · Cited by 241 · see citing works
Reducing systematic review workload through certainty-based screening
Miwa, M., Thomas, J., O’Mara-Eves, A., & Ananiadou, S. (2014) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2014.06.005 · Cited by 190 · see citing works
PromptMaker: Prompt-based Prototyping with Large Language Models
Jiang, E., Olson, K., Toh, E., Molina, A., Donsbach, A., Terry, M., & Cai, C. J. (2022) · CHI Conference on Human Factors in Computing Systems Extended Abstracts · doi:10.1145/3491101.3503564 · Cited by 157 · see citing works
ChainForge: A Visual Toolkit for Prompt Engineering and LLM Hypothesis Testing
Arawjo, I., Swoopes, C., Vaithilingam, P., WATTENBERG, M. P., & Glassman, E. L. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642016 · Cited by 146 · see citing works
ChainForge: A Visual Toolkit for Prompt Engineering and LLM Hypothesis Testing
Arawjo, I., Swoopes, C., Vaithilingam, P., Wattenberg, M., & Glassman, E. (2023) · arXiv · doi:10.48550/arxiv.2309.09128 · Cited by 146
Bayesian Multinomial Naïve Bayes Classifier to Text Classification
Xu, S., Li, Y., & Wang, Z. (2017) · Lecture notes in electrical engineering · doi:10.1007/978-981-10-5041-1_57 · Cited by 126 · see citing works
AI 2006: Advances in Artificial Intelligence
(2006) · Lecture notes in computer science · doi:10.1007/11941439 · Cited by 119 · see citing works
A new algorithm for reducing the workload of experts in performing systematic reviews
Matwin, S., Kouznetsov, A., Inkpen, D., Frunza, O., & O'Blenis, P. (2010) · Journal of the American Medical Informatics Association · doi:10.1136/jamia.2010.004325 · Cited by 100 · see citing works
Khabsa, M., Elmagarmid, A., Ilyas, I., Hammady, H., & Ouzzani, M. (2016) · Machine Learning · doi:10.1007/s10994-015-5535-7 · Cited by 98 · see citing works
Empirical Standards for Software Engineering ResearchPreprint
Ralph, P., bin Ali, N., Baltes, S., Bianculli, D., Díaz, J., Dittrich, Y., Ernst, N., Felderer, M., FELDT, R. H., Filieri, A., et al. (2020) · arXiv (Cornell University) · doi:10.4230/lipics.esem.2026.10 · Cited by 96 · see citing works
A Semantics Aware Random Forest for Text Classification
Islam, M. Z., Liu, J., Li, J., Liu, L., & Kang, W. (2019) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3357384.3357891 · Cited by 73 · see citing works
Hasan, B., Saadi, S. M., Rajjoub, N. S., Hegazi, M., Al-Kordi, M., Fleti, F., Farah, M. H., Riaz, I. B., Banerjee, I., Wang, Z., et al. (2024) · BMJ Evidence-Based Medicine · doi:10.1136/bmjebm-2023-112597 · Cited by 72 · see citing works
Information and Knowledge Management
(2019) · doi:10.7176/ikm · Cited by 67 · see citing works
Active learning for screening prioritization in systematic reviews - A simulation study
Ferdinands, G., Schram, R., de Bruin, J., Bagheri, A., Oberski, D. L., Tummers, L., & van de Schoot, R. (2020) · doi:10.31219/osf.io/w6qbg · Cited by 51 · see citing works
Ji, X., Ritter, A., & Yen, P. (2017) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2017.03.007 · Cited by 48 · see citing works
A Systematic Mapping Study driven by the margin of error
Kosar, T., Bohra, S., & Mernik, M. (2018) · Journal of Systems and Software · doi:10.1016/j.jss.2018.06.078 · Cited by 47 · see citing works
Paving the way for mature secondary research: the seven types of literature review
Ralph, P., & Baltes, S. (2022) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3540250.3560877 · Cited by 47 · see citing works
Computer Science Papers in Web of Science: A Bibliometric Analysis
Fiala, D., & Tutoky, G. (2017) · Publications · doi:10.3390/publications5040023 · Cited by 46 · see citing works
Multi-paradigm modeling for cyber–physical systems: A systematic mapping review
Barišić, A., Ruchkin, I., Savić, D., Mohamed, M. A., Al-Ali, R., Li, L. W., Mkaouar, H., Eslampanah, R., Challenger, M., Blouin, D., et al. (2021) · Journal of Systems and Software · doi:10.1016/j.jss.2021.111081 · Cited by 42 · see citing works
Automating Systematic Literature Review
Felizardo, K. R., & Carver, J. C. (2020) · Springer eBooks · doi:10.1007/978-3-030-32489-6_12 · Cited by 41 · see citing works
Collaborative Model-Driven Software Engineering: A Systematic Update
Dávid, I., Aslam, K., Faridmoayer, S., Malavolta, I., Syriani, E., & Lago, P. (2021) · ACM/IEEE International Conference on Model Driven Engineering Languages and Systems (MODELS) · doi:10.1109/models50736.2021.00035 · Cited by 33 · see citing works
Self-citationOn the Reliability of the Area Under the ROC Curve in Empirical Software Engineering
Lavazza, L., Morasca, S., & Rotoloni, G. (2023) · International Conference on Evaluation and Assessment in Software Engineering (EASE) · doi:10.1145/3593434.3593456 · Cited by 32 · see citing works
Model Driven Engineering Languages and Systems
Whittle, J., Kühne, T., & Clark, T. (2011) · Lecture notes in computer science · doi:10.1007/978-3-642-24485-8 · Cited by 28 · see citing works
Advanced Multimedia and Ubiquitous Engineering
Park, J., Chen, S., & Choo, K.-K. R. (2017) · Lecture notes in electrical engineering · doi:10.1007/978-981-10-5041-1 · Cited by 28 · see citing works
Towards Better Chain-of-Thought Prompting Strategies: A SurveyPreprint
Yu, Z., He, L., Wu, Z., Dai, X., & Chen, J. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2310.04959 · Cited by 28 · see citing works
Vision for SLR tooling infrastructure: Prioritizing value-added requirements
Al-Zubidy, A., Carver, J. C., Hale, D. P., & Hassler, E. E. (2017) · Information and Software Technology · doi:10.1016/j.infsof.2017.06.007 · Cited by 27 · see citing works
Assessing the Ability of ChatGPT to Screen Articles for Systematic Reviews
Syriani, E., David, I., & Kumar, G. (2023) · arXiv · doi:10.48550/arxiv.2307.06464 · Cited by 27 · see citing works
Self-citationWatanabe, W. M., Felizardo, K. R., Cândido, A., de Souza, É. F., Neto, J. E. D. C., & Vijaykumar, N. L. (2020) · Information and Software Technology · doi:10.1016/j.infsof.2020.106395 · Cited by 25 · see citing works
Kusa, W., Lipani, A., Knoth, P., & Hanbury, A. (2023) · Intelligent Systems with Applications · doi:10.1016/j.iswa.2023.200193 · Cited by 21 · see citing works
Facilitating biomedical systematic reviews using ranked text retrieval and classification
Martínez, D., Karimi, S., Cavedon, L., & Baldwin, T. (2008) · Cited by 20 · see citing works
What Will the United States Look Like in 2050? A Chatgpt Short StoryPreprint
McGee, R. W. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4413442 · Cited by 19 · see citing works
PARMOREL: a framework for customizable model repair
Barriga, A., Heldal, R., Rutle, A., & Iovino, L. (2022) · Software & Systems Modeling · doi:10.1007/s10270-022-01005-0 · Cited by 17 · see citing works
Australasian Document Computing Symposium
(2021) · doi:10.1145/3503516 · Cited by 14 · see citing works
Modeling a Tool for Conducting Systematic Reviews Iteratively
Bigendako, B. M., & Syriani, E. (2018) · International Conference on Model-Driven Engineering and Software Development · doi:10.5220/0006664405520559 · Cited by 13 · see citing works
Self-citationEvaluation and assessment in software engineering 1998
Linkman, S. (1998) · Information and Software Technology · doi:10.1016/s0950-5849(98)00106-2 · Cited by 8 · see citing works
The screening phase in systematic reviews: Can we speed up the process?
Rožanc, I., & Mernik, M. (2021) · Advances in computers · doi:10.1016/bs.adcom.2021.01.006 · Cited by 8 · see citing works
Automated title and abstract screening for scoping reviews using the GPT-4 Large Language ModelPreprint
Wilkins, D. E. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2311.07918 · Cited by 8 · see citing works
Brunschwig, L., Guerra, E., & Lara, J. D. (2021) · Software & Systems Modeling · doi:10.1007/s10270-021-00897-8 · Cited by 4 · see citing works
Model-Driven Engineering and Software Development
(2018) · Communications in computer and information science · doi:10.1007/978-3-319-94764-8 · Cited by 3 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Deploying AI on Edge: Advancement and Challenges in Edge Intelligence
Wang, T., Guo, J., Zhang, B., Yang, G., & Li, D. (2025) · Mathematics · doi:10.3390/math13111878 · Cited by 92 · see citing works
Li, Y., Datta, S., Rastegar-Mojarad, M., Lee, K., Paek, H., Glasgow, J., Liston, C., He, L., Wang, X., & Xu, Y. (2025) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocaf030 · Cited by 69 · see citing works
Accelerating clinical evidence synthesis with large language models
Wang, Z., Cao, L., Danek, B., Jin, Q., Lu, Z., & Sun, J. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-01840-7 · Cited by 69 · see citing works
Mahmoudi, H., Chang, D., Lee, H., Ghaffarzadegan, N., & Jalali, M. S. (2025) · JMIR AI · doi:10.2196/68097 · Cited by 35 · see citing works
García-Torres, D., Vicente Ripoll, M. A., Fernández Peris, C., & Mira Solves, J. J. (2024) · Healthcare · doi:10.3390/healthcare12222241 · Cited by 29 · see citing works
A foundation model for human-AI collaboration in medical literature mining
Wang, Z., Cao, L., Jin, Q., Chan, J., Wan, N., Afzali, B., Cho, H.-J., Choi, C.-I., Emamverdi, M., Gill, M. K., et al. (2025) · Nature Communications · doi:10.1038/s41467-025-62058-5 · Cited by 35 · see citing works
Large Language Model for Qualitative Research: A Systematic Mapping Study
Barros, C., Azevedo, B. B., Neto, V. V. G., Kassab, M., Kalinowski, M., do Nascimento, H. A. D., & Bandeira, M. C. G. S. P. (2025) · IEEE/ACM International Workshop on Methodological Issues with Empirical Studies in Software Engineering (WSESE) · doi:10.1109/wsese66602.2025.00015 · Cited by 24 · see citing works
Zero-shot Generative Large Language Models for Systematic Review Screening Automation
Wang, S., Scells, H., Zhuang, S., Potthast, M., Koopman, B., & Zuccon, G. (2024) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-56027-9_25 · Cited by 26 · see citing works
Bayani, A., Ewane, L. P. E., dos Anjos, D. S. O., Mac-Seing, M., & Nikiema, J. N. (2025) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2025.106048 · Cited by 21 · see citing works
Streamlining evidence based clinical recommendations with large language models
Li, D., Jiang, N., Huang, K., Tu, R., Ouyang, S., Yu, H., Lin, Q., Chen, Y., Zhou, T., Tong, D., et al. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-02273-y · Cited by 20 · see citing works
Sifat, M. S. I., Kabir, M. A., Islam, M. M. M., Rehman, A. U., & Bermak, A. (2025) · IEEE Access · doi:10.1109/access.2025.3600235 · Cited by 15 · see citing works
Generative artificial intelligence for literature reviews
Wagner, G., Prester, J., Mousavi, R., Lukyanenko, R., & Paré, G. (2026) · Journal of Information Technology · doi:10.1177/02683962261425675 · Cited by 15 · see citing works
On the Utility of Domain Modeling Assistance with Large Language Models
Chaaben, M. B., Burgueño, L., Dávid, I., & Sahraoui, H. A. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3744920 · Cited by 11 · see citing works
Zuo, C., Yang, X., Errickson, J., Li, J., Hong, Y., & Wang, R. (2025) · Environmental Evidence · doi:10.1186/s13750-025-00358-5 · Cited by 11 · see citing works
The hunt for the last relevant paper: blending the best of humans and AI
van de Schoot, R., Coimbra, B. M., Evenhuis, T., Lombaers, P., Weijdema, F., de Bruin, L., Neeleman, R. C., Grandfield, E. M., Sijbrandij, M., Teijema, J. J., et al. (2025) · European Journal of Psychotraumatology · doi:10.1080/20008066.2025.2546214 · Cited by 10 · see citing works
Large language models for the screening step in systematic reviews in dentistry
Rokhshad, R., Bagherianlemraski, M., Ehsani, S., Haghighat, S., & Schwendicke, F. (2025) · Journal of Dentistry · doi:10.1016/j.jdent.2025.105877 · Cited by 8 · see citing works
Felizardo, K. R., Steinmacher, I., Lima, M. H. T. D. F., Deizepe, A., Conte, T. U., & Barcellos, M. P. (2024) · Proceedings of the ... ACM-IEEE International Symposium on Empirical Software Engineering and Measurement/International Symposium on Empirical Software Engineering and Measurement · doi:10.1145/3674805.3690743 · Cited by 7 · see citing works
AI Simulation by Digital Twins: Systematic Survey of the State of the Art and a Reference Framework
Liu, X., & Dávid, I. (2024) · ACM/IEEE International Conference on Model Driven Engineering Languages and Systems (MODELS) · doi:10.1145/3652620.3688253 · Cited by 7 · see citing works
Daraio, C. C., Leo, S. D., & Ferazzoli, F. (2025) · International Transactions in Operational Research · doi:10.1111/itor.70054 · Cited by 7 · see citing works
Sciurti, A., Migliara, G., Siena, L. M., Isonne, C., Blasiis, M. R. D., Sinopoli, A., Iera, J., Marzuillo, C., Vito, C. D., Villari, P., et al. (2025) · Research Synthesis Methods · doi:10.1017/rsm.2025.10044 · Cited by 7 · see citing works
Strong, G., Bresnihan, N., & Tangney, B. (2025) · Journal of Computer Languages · doi:10.1016/j.cola.2025.101342 · Cited by 6 · see citing works
Cheng, Y., & Zhang, H. J. (2025) · ACS ES&T Water · doi:10.1021/acsestwater.5c00551 · Cited by 6 · see citing works
Felizardo, K. R., Deizepe, A., COUTINHO, D. J. B., Gomes, G., Meireles, M. A. C., Gerosa, M. A., & Steinmacher, I. (2025) · IEEE/ACM International Workshop on Methodological Issues with Empirical Studies in Software Engineering (WSESE) · doi:10.1109/wsese66602.2025.00010 · Cited by 6 · see citing works
Liu, X.-R., & David, I. (2025) · Journal of Software and Systems Modeling · doi:10.1007/s10270-025-01306-0 · Cited by 17 · see citing works
Nawrath, M., Merlina, A., Knight, J., Welch, S. A., Rashidian, M., & Seifert-Dähnn, I. (2026) · Information · doi:10.3390/info17050501 · Cited by 6 · see citing works
Doneva, S. E., de Viragh, S., Hubarava, H., Schandelmaier, S., Briel, M., & Ineichen, B. V. (2025) · Research Synthesis Methods · doi:10.1017/rsm.2025.10031 · Cited by 5 · see citing works
Vivekanantha, P., Son, H., Bernardini, L., Bouchard, M. D., Ayeni, O. R., & Kay, J. (2025) · Current Reviews in Musculoskeletal Medicine · doi:10.1007/s12178-025-10001-y · Cited by 5 · see citing works
Adam, T., Abosabie, S. A. S., Dittmer, M., Wolf, E., Abosabie, S., Behnke, C., Baier, F., Weickmann, A., Köser, L., Correll, C., et al. (2026) · Research Synthesis Methods · doi:10.1017/rsm.2026.10093 · Cited by 6 · see citing works
Teaching with intelligence; AI-enhanced pedagogies in nursing education: a scoping review
Nugent, L., Murray, B., Moore, Z., Patton, D., O’Connor, T., Watson, C., Walsh, K., Renjith, V., & George, J. (2026) · Nurse Education in Practice · doi:10.1016/j.nepr.2026.104878 · Cited by 4 · see citing works
Sujau, M., Wada, M., Vallée, É., Hillis, N., & Sušnjak, T. (2025) · Machine Learning and Knowledge Extraction · doi:10.3390/make7020028 · Cited by 3 · see citing works
Shereefdeen, H., Thaivalappil, A., Young, I., & MacKay, M. (2025) · INQUIRY The Journal of Health Care Organization Provision and Financing · doi:10.1177/00469580251399374 · Cited by 3 · see citing works
de Araújo, P. R., de Rezende, J. A. M., Faria, D. R. M., & Gomes, O. D. S. M. (2026) · Sensors · doi:10.3390/s26020747 · Cited by 5 · see citing works
Oami, T., Okada, Y., & Nakada, T. (2026) · Hong Kong Journal of Emergency Medicine · doi:10.1002/hkj2.70085 · Cited by 3 · see citing works
Erslev, M. S., & Tretow-Fish, T. A. B. (2025) · Decennial Aarhus Conference · doi:10.1145/3744169.3744195 · Cited by 2 · see citing works
Large language model enhanced framework for systematic reviews and meta-analyses
Shen, J., Luo, Z., Jia, D., Wang, S., Sun, F., & Wu, J. (2025) · BMJ Digital Health & AI · doi:10.1136/bmjdhai-2025-000017 · Cited by 2 · see citing works
How artificial intelligence shapes customer emotional experiences: a systematic literature review
Lytle, S., & Gopinath, M. (2026) · International Journal of Quality and Service Sciences · doi:10.1108/ijqss-08-2025-0179 · Cited by 2 · see citing works
Goals and strategies for the indexing of publication types and study designs
Smalheiser, N. R., Menke, J. D., Holt, A. W., Halil Kilicoglu, B., & Schneider, J. (2026) · Database · doi:10.1093/database/baag039 · Cited by 2 · see citing works
Tran, T. P., & Nguyen, T. H. L. (2025) · Computer-Assisted Language Learning Electronic Journal · doi:10.54855/callej.252637 · Cited by 1 · see citing works
Nordmann, K., & Fischer, F. (2025) · BMC Health Services Research · doi:10.1186/s12913-025-13901-4 · Cited by 1 · see citing works
Köhler, J., Harl, M. V., Westner, M. K., & Strahringer, S. (2025) · IEEE Conference on Business Informatics · doi:10.1109/cbi68102.2025.00012 · Cited by 1 · see citing works
Huang, Y., Xu, Z., Bei, X., & Huang, H. (2025) · Mathematics · doi:10.3390/math14010145 · Cited by 1 · see citing works
Homer, S. T. (2026) · Quality & Quantity · doi:10.1007/s11135-025-02568-y · Cited by 1 · see citing works
Keenan, P., & Heavin, C. (2026) · Lecture notes in business information processing · doi:10.1007/978-3-032-25146-6_20 · Cited by 1 · see citing works
Nunnery, J. A. (2026) · Research Synthesis Methods · doi:10.1017/rsm.2026.10116 · Cited by 1 · see citing works
Collaborative large language models (LLMs) are all you need for screening in systematic reviewsPreprint
Parmar, M., Naqvi, S. A. A., Warraich, K., Saeidi, A., Rawal, S. C., Faisal, K. S., Kazmi, S. Z., Fatima, M., He, H., Safdar, M., et al. (2026) · medRxiv · doi:10.64898/2026.02.07.26345640 · Cited by 1 · see citing works
Musarat, M. A., & Ručevskis, S. (2026) · Automation in Construction · doi:10.1016/j.autcon.2026.107130 · Cited by 1 · see citing works
Machine Learning-Assisted Systematic Review: A Case Study in Learning Analytics
Xu, Z., Zhuang, X., & Ma, S. (2025) · Education Sciences · doi:10.3390/educsci15111488
AIM review tool: artificial intelligence for smarter systematic review screening
Mena, S., Rituerto-González, E., Coutts, F., von Trott, J., Jacobs, G. R., Bryant, L., Moles, L., Sirbu, N., Promet, L., Oliver, D., et al. (2026) · npj Artificial Intelligence · doi:10.1038/s44387-026-00080-8
Sušnjak, T., Palffy, C., Zimina, T., Altynbekova, N., Garg, K., & Gilbert, L. K. (2025) · medRxiv · doi:10.1101/2025.04.03.25325216
Buitrago‐García, D., Courvoisier, D. S., Capderou, S., Iudici, M., & Mongin, D. (2025) · doi:10.2196/preprints.77943
Bridging Oceans: Exploring South America’s Regional Integration Through the Bioceanic Corridor
Vieira, A. D. D. G., da Silva, A. L. R., Frogeri, R. F., Leandro, F. J., & Chen, H. (2025) · Springer eBooks · doi:10.1007/978-981-95-1169-3_28
Brazil and the Geopolitics of Submarine Cables
Bernardino, L. M. B., Frogeri, R. F., & Leandro, F. J. (2025) · Springer eBooks · doi:10.1007/978-981-95-1169-3_27
A Systematic Review of Genes Affecting Endocochlear Potential
Kirwin, D. A., Lewis, M. A., & Steel, K. P. (2026) · Journal of the Association for Research in Otolaryngology · doi:10.1007/s10162-026-01046-y
Recommender Systems for Improving Personalised Research Assistance and Literature Discovery
Ahmad, S., Muhammad, A., Abbas, M. S., & Jan, S. U. (2026) · Engineering Reports · doi:10.1002/eng2.70714
Reporting and Reviewing LLM-Integrated Systems in HCI: Challenges and Considerations
Navarro, K. F., Syriani, E., & Arawjo, I. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3790439 · Cited by 1 · see citing works
Self-citationAng, Z. Y., Shakirah, M., Lim, W. H., & Lee, S. C. (2026) · JMIR Formative Research · doi:10.2196/97813
Farquhar, H. (2026) · Research Square · doi:10.21203/rs.3.rs-9308492/v1
Buitrago‐García, D., Courvoisier, D. S., Capderou, S., Iudici, M., & Mongin, D. (2026) · JMIR Medical Informatics · doi:10.2196/77943
Familia-escuela en contextos vulnerables: Prácticas que reducen desigualdades
Hernández, Á. R., Valdivieso, D. L. M., & Prados, M. Á. H. (2026) · Ediciones Comunicación Científica eBooks · doi:10.52501/cc.422.09
Xiong, Y.-T., Wang, Y.-C., Sun, Y., Zhan, Z., Liu, W. T., Liu, H.-N., Meng, R.-Y., Tang, W., & Liu, C. (2026) · BMJ Evidence-Based Medicine · doi:10.1136/bmjebm-2025-114055 · Cited by 1 · see citing works
Keenan, P. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-21811-7_23
Laskowski, N. M., Hambruch, H. R., Zaiser, C., Schnorr, I., Maier, D., Appel, K. S., Vehreschild, J. J., & Paslakis, G. (2026) · Addiction Science & Clinical Practice · doi:10.1186/s13722-026-00668-0
Barros, C. F., Kalinowski, M., Kassab, M., & Neto, V. V. G. (2026) · IEEE/ACM International Workshop on Methodological Issues with Empirical Studies in Software Engineering (WSESE) · doi:10.1145/3786149.3788298
Blockchain-Based Data Sharing in the Internet of Vehicles: A Survey
Fan, Y., Guo, Y., Sun, Y., & Zhang, Z. (2026) · Mathematics · doi:10.3390/math14111957
Individual agency and sustainability initiatives in STEM careers and education: a scoping review
Alduais, A. M. S., Tarlochan, F., Chaaban, Y., Mecheter, A., Ibrahim, M. I. M., Elfaki, F. A. M., & Du, X. (2026) · International Journal of STEM Education · doi:10.1186/s40594-026-00626-x
Melo, P. H. M., Guadagno, E., & Poenaru, D. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-30813-9_73
Kurian, N., Cherian, J. M., Reji, R. M., & Varghese, K. G. (2026) · Evidence-Based Dentistry · doi:10.1038/s41432-026-01238-8 · Cited by 1 · see citing works
Researchers' View of Using LLMs in Systematic Literature ReviewsPreprint
dos Santos, V., Lago, P., & Nakagawa, E. Y. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.7275147
Reframing workload saving in LLM-assisted screening: A commentary on Sciurti et al.
Lai, H., Liu, J., Ge, L., & Estill, J. (2026) · Research Synthesis Methods · doi:10.1017/rsm.2026.10114
Ensuring trustworthy AI assisted guideline development for clinical practice
Zhang, Q., & Fu, Y. (2026) · npj Digital Medicine · doi:10.1038/s41746-026-03098-z
Marco-Pérez, I., Pérez, B., & Rubio, Á. L. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-36587-3_13
The AI Validation Gap in Scientific Knowledge Production
Rahmouni, M. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3373-8919-6.ch001
Artificial Intelligence and Consumer Well‐Being: A Cross‐Domain Systematic Review
Lytle, S., & Gopinath, M. (2026) · International Journal of Consumer Studies · doi:10.1111/ijcs.70251
Investigación científica en la era de la inteligencia artificial
Pluas, R. E. M., Fernández-Loor, A. O., Mendizábal, V. L. L., Sánchez-Vivanco, J. P., Urbano, A. P. M., & Angulo, R. M. M. (2026) · doi:10.70577/pd8w9r97
Merchán-Cruz, E. A., Gabelaia, I., Soe, S. P., Yatskiv, I., & Dmitry, P. (2026) · Informatics · doi:10.3390/informatics13100160
Walica, D., Zhidchenko, V., Saeed, S. M., Gupta, T. S., Agúndez, A. G., Härkönen, A., Khadim, Q., Nemov, A. S., Ahola, A., Orzechowski, G., et al. (2026) · Results in Engineering · doi:10.1016/j.rineng.2026.113301
Harrison, T. B., Hu, D., Jia, H., Tan, S. H., Chen, F., Zhang, Z., Lu, Q., Wang, J., Wang, L., Huang, M., et al. (2026) · PLOS Digital Health · doi:10.1371/journal.pdig.0001768
김민진, Youn, S., 박지예, & Rew, J. (2026) · Journal of Broadcast Engineering · doi:10.5909/jbe.2026.31.5.985
Looking Back, Moving Forward: A Guiding Framework for Future Research in Supply Chain Management
Mecca, D., Patrucco, P. P., Wieland, A., Durach, C. F., Handfield, R., & Nikookar, E. (2026) · Journal of Business Logistics · doi:10.1111/jbl.70098
Nunnery, J. A. (2026) · Journal of Higher Education Theory and Practice · doi:10.33423/5tg2ts54
Pietrancosta, M., Wang, S., Wu, Y., Brulotte, V., Harutyunyan, R., Hemmerling, T. M., Morisson, L., & Laferrière-Langlois, P. (2026) · Research Square · doi:10.21203/rs.3.rs-10879359/v1
Carvalho, S. A., & Rubio, N. (2026) · Issues in Mental Health Nursing · doi:10.1080/01612840.2026.2739251
An Enhanced Reference Paper Collection System with Generative AI for Paper Relevance Assessment
Fahrudin, T. M., Funabiki, N., Brata, K. C., Kotama, I. N. D., Haz, A. L., Riyantoko, P. A., & Batubulan, K. S. (2026) · Information · doi:10.3390/info17111072
- Authors
- Syriani, E., David, I., & Kumar, G.
- Affiliations
- Université de Montréal (CA)
- Corresponding author
- Syriani, Eugene
- ORCID
- Syriani, Eugene
- Venue
- Journal of Computer Languages
- Publisher
- Elsevier BV
- Published
- 2024-08
- Volume
- 80
- Pages
- 101287
- ISSN
- 2590-1184 (ISSN-L)
- OpenAlex
- W4400411501
- Cited by
- 82
- References
- 74
- Citations by year
- 2026: 44 · 2025: 34 · 2024: 4
- FWCI
- 10.35
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Journal of Computer Languages (publishedVersion, cc-by-nc)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Keywords
- systematic reviews, article screening, ChatGPT, large language models, prompt engineering, exclusion criteria, negative examples
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review