Is artificial intelligence for medical professionals serving the patients?: Protocol for a systematic review on patient-relevant benefits and harms of algorithmic decision-making
Open accessPublished version of a preprint
Abstract
Background: Algorithmic decision-making (ADM) utilises algorithms to collect and process data and develop models to make or support decisions. Advances in artificial intelligence (AI) have led to the development of support systems that can be superior to medical professionals without AI support in certain tasks. However, whether patients can benefit from this remains unclear. The aim of this systematic review is to assess the current evidence on patient-relevant benefits and harms, such as improved survival rates and reduced treatment-related complications, when healthcare professionals use ADM systems (developed using or working with AI) compared to healthcare professionals without AI-related ADM (standard care)—regardless of the clinical issues. Methods: Following the PRISMA statement, MEDLINE and PubMed (via PubMed), Embase (via Elsevier) and IEEE Xplore will be searched using English free text terms in title/abstract, Medical Subject Headings (MeSH) terms and Embase Subject Headings (Emtree fields). Additional studies will be identified by contacting authors of included studies and through reference lists of included studies. Grey literature searches will be conducted in Google Scholar. Risk of bias will be assessed by using Cochrane’s RoB 2 for randomised trials and ROBINS-I for non-randomised trials. Transparent reporting of the included studies will be assessed using the CONSORT-AI extension statement. Two researchers will screen, assess and extract from the studies independently, with a third in case of conflicts that cannot be resolved by discussion. Discussion: It is expected that there will be a substantial shortage of suitable studies that compare healthcare professionals with and without ADM systems concerning patient-relevant endpoints. This can be attributed to the prioritisation of technical quality criteria and, in some cases, clinical parameters over patient-relevant endpoints in the development of study designs. Furthermore, it is anticipated that a significant portion of the identified studies will exhibit relatively poor methodological quality and provide only limited generalisable results. Systematic review registration: This study is registered within PROSPERO (CRD42023412156).
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
The PRISMA 2020 statement: An updated guideline for reporting systematic reviews
Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., Shamseer, L., Tetzlaff, J. M., Akl, E. A., Brennan, S. E., et al. (2021) · BMJ · doi:10.1136/bmj.n71 · Cited by 108,283 · see citing works
Highly accurate protein structure prediction with AlphaFold
Jumper, J., Evans, R., Pritzel, A., Green, T., Figurnov, M., Ronneberger, O., Tunyasuvunakool, K., Bates, R., Žídek, A., Potapenko, A., et al. (2021) · Nature · doi:10.1038/s41586-021-03819-2 · Cited by 43,764 · see citing works
RoB 2: a revised tool for assessing risk of bias in randomised trials
Sterne, J. A. C., Savović, J., Page, M. J., Elbers, R. G., Blencowe, N. S., Boutron, I., Cates, C. J., Cheng, H.-Y., Corbett, M. S., Eldridge, S. M., et al. (2019) · BMJ · doi:10.1136/bmj.l4898 · Cited by 33,583 · see citing works
PRISMA-P Group, Moher, D., Shamseer, L., Clarke, M., Ghersi, D., Liberati, A., Petticrew, M., Shekelle, P., & Stewart, L. A. (2015) · Systematic Reviews · doi:10.1186/2046-4053-4-1 · Cited by 26,223 · cited 1× · see citing works
“In accordance with the Preferred Reporting Items for Systematic Review and Meta-Analysis Protocols (PRISMA-P) statement [26], the study protocol for this systematic is registered on the International Prospective Register of Systematic Reviews (PROSPERO) data-base (CRD42023412156).”
MethodRayyan-a web and mobile app for systematic reviews.
Ouzzani, M., Hammady, H., Fedorowicz, Z., & Elmagarmid, A. (2016) · Systematic Reviews · doi:10.1186/s13643-016-0384-4 · Cited by 26,092 · see citing works
ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions
Sterne, J. A., Hernán, M. A., Reeves, B. C., Savović, J., Berkman, N. D., Viswanathan, M., Henry, D., Altman, D. G., Ansari, M. T., Boutron, I., et al. (2016) · BMJ · doi:10.1136/bmj.i4919 · Cited by 20,329 · see citing works
Standards for Reporting Qualitative Research: A Synthesis of Recommendations
O’Brien, B. C., Harris, I. B., Beckman, T. J., Reed, D. A., & Cook, D. A. (2014) · Academic Medicine · doi:10.1097/acm.0000000000000388 · Cited by 12,211 · see citing works
A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play
Silver, D., Hubert, T., Schrittwieser, J., Antonoglou, I., Lai, M., Guez, A., Lanctot, M., Sifre, L., Kumaran, D., Graepel, T., et al. (2018) · Science · doi:10.1126/science.aar6404 · Cited by 3,562 · see citing works
Liu, X., Cruz Rivera, S., Moher, D., Calvert, M. J., Denniston, A. K., The SPIRIT-AI and CONSORT-AI Working Group, SPIRIT-AI and CONSORT-AI Steering Group, Chan, A.-W., Darzi, A., Holmes, C., et al. (2020) · Nature Medicine · doi:10.1038/s41591-020-1034-x · Cited by 1,421 · see citing works
Exploiting machine learning for end-to-end drug discovery and development
Ekins, S., Puhl, A. C., Zorn, K. M., Lane, T. R., Russo, D. P., Klein, J. J., Hickey, A. J., & Clark, A. M. (2019) · Nature Materials · doi:10.1038/s41563-019-0338-z · Cited by 578 · see citing works
Brocklehurst, P., Field, D., Greene, K., Juszczak, E., Keith, R., Kenyon, S., Linsell, L., Mabey, C., Newburn, M., Plachcinski, R., et al. (2017) · The Lancet · doi:10.1016/s0140-6736(17)30568-8 · Cited by 207 · see citing works
Evaluating artificial intelligence in medicine: phases of clinical research
Park, Y., Jackson, G. P., Foreman, M. A., Gruen, D., Hu, J., & Das, A. K. (2020) · JAMIA Open · doi:10.1093/jamiaopen/ooaa033 · Cited by 150 · see citing works
Li, Q., Zhao, K., Bustamante, C. D., Ma, X., & Wong, W. H. (2019) · Genetics in Medicine · doi:10.1038/s41436-019-0439-8 · Cited by 106 · see citing works
Flagged first, then by citations
- Authors
- Wilhelm, C., Steckelberg, A., & Rebitschek, F. G.
- Affiliations
- University of Potsdam (DE)
- Martin Luther University Halle-Wittenberg (DE)
- Corresponding author
- Wilhelm, Christoph
- Venue
- Systematic Reviews
- Published
- 2024-09-06
- Volume
- 13
- Issue
- 1
- Pages
- 228-228
- ISSN
- 2046-4053 (ISSN-L)
- PMC
- PMC11378383
- OpenAlex
- W4402320302
- Cited by
- 5
- References
- 13
- Citations by year
- 2025: 4 · 2024: 1
- FWCI
- 0.47
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- Systematic Reviews (publishedVersion, cc-by)
- medRxiv (acceptedVersion, cc-by-nc-nd)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Informationszentrum Lebenswissenschaften (ZB MED) (submittedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- publish.UP (University of Potsdam) (submittedVersion, cc-by)
- Digitalen Hochschulbibliothek Sachsen-Anhalt (Universitäts- und Landesbibliothek Sachsen-Anhalt) (cc-by)
- Funders
- Universität Potsdam
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
- Social Sciences › Safety Research › Ethics and Social Impacts of AI
- Keywords
- artificial intelligence in healthcare, algorithmic decision-making, clinical decision support systems, patient-relevant outcomes, survival rate, treatment complications
- MeSH terms
- Clinical Decision-Making, Systematic Reviews as Topic, Artificial Intelligence, Decision Making, Health Personnel, Humans
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
