Artificial intelligence to automate the systematic review of scientific literature

Human interaction is among the least explored AI features, with just one previous study.

“This is among the least explored features with just one previous study ( de la Torre-López et al. 2023).”

De la Torre-López et al. 2023 provide the most comprehensive analysis, utilising all five specified features to examine seven tools.

“( De la Torre-López et al. 2023) provide the most comprehensive analysis, utilising all five specified features to examine seven tools.”

Existing ML classifiers generally require human researchers providing large amounts of sufficiently diverse data for training and active learning.

“However, these tools, sometimes referred to as discriminative artificial intelligence (AI) technologies, generally require human researchers providing large amount of sufficiently diverse data for training and active learning [1, 3, 5, 6].”

Researchers' current lack of trust in the accuracy and quality of SLR automation tools has been identified as a major barrier to uptake, and transparency from developers and users has been suggested as a means to overcome this challenge.

“Researchers’ current lack of trust in the accuracy and quality of SLR automation tools has been identified as a major barrier to uptake, and transparency from the developers and users of these tools has been suggested as a means to overcome this challenge (Bolaños et al., 2024; de la Torre-López et al., 2023; Gates et al., 2019; O’Connor et al., 2019; van Altena et al., 2019).”

The ability of LLMs to document the rationale behind each screening and data extraction decision is a major advantage over earlier AI approaches, improving transparency and aiding interpretation of the models' outputs.

“The ability of LLMs to document the rationale behind each screening and data extraction decision is a major advantage over earlier AI approaches, improving transparency and aiding in the interpretation of the models’ outputs, as well as informing the future development of models and prompts for better performance (Bolaños et al., 2024; de la Torre-López et al., 2023).”

Relatively few studies have explored the role of AI in SLR tools.

“While a significant body of research has examined SLR tools (Carver et al. 2013; Feng et al. 2017; Napoleão et al. 2021; Cierco Jimenez et al. 2022; Jesso et al. 2022; Khalil et al. 2022; Wagner et al. 2022; Ng et al. 2023; Robledo et al. 2023), relatively few studies have explored the role of AI in this domain (Cowie et al. 2022; Burgard and Bittermann 2023; de la Torre-López et al. 2023; Robledo”

Text representation was analysed by two previous surveys.

“This feature was analysed by two previous surveys ( de la Torre-López et al. 2023); Burgard and Bittermann 2023).”

Approach is the most examined AI feature, receiving attention from four studies.

“Approach: dentifies the method used for performing a specific task. This is the most examined feature, receiving attention from four studies (Cowie et al. 2022; Burgard and Bittermann 2023; de la Torre-López et al. 2023; Robledo et al. 2023).”

The meta-review focuses on four surveys that analysed AI features.

“We focus on four surveys that analysed AI features (Cowie et al. 2022; Burgard and Bittermann 2023; de la Torre-López et al. 2023; Robledo et al. 2023).”

Many challenges and barriers must be addressed before AI tools can be routinely used in practice.

“Although we have demonstrated the potential utility of ISLaR in rapidly performing SLR tasks to ease researcher workload, there are many challenges and barriers that must be addressed before AI tools can be routinely used in practice (Bolaños et al., 2024; de la Torre-López et al., 2023; Doyal et al., 2023; O’Connor et al., 2019; van Altena et al., 2019).”

Input is among the least explored AI features, with just one previous study.

“Alongside Human interaction , this is the least explored feature, with just one previous study ( de la Torre-López et al. 2023).”