CoChat

Diffusion of innovation in systematic review methodology: Why is study selection not yet assisted by automation?

Open access
Thomas, JM · 2013 · OA Evidence-Based Medicine

The DOI matches its Crossref record; 15 of the 15 references it cites were found in a registry or an index.

Abstract

Introduction Systematic reviews, the foundation of much Evidence-Based medicine, are suffering from increasing ‘data deluge’: reviewers often need to manually assess many thousands of titles and abstracts to determine their relevance. Automation has been advanced as a potential solution; but given that its efficacy was first demonstrated in 2006, why is it not yet widely used? The Diffusion of Innovations framework by EM Rogers is used to structure an exploration of why this might be the case. Discussion According to Rogers, five characteristics affect the rate of adoption of innovations: those perceived as having greater relative advantage, compatibility, trialability and observability, and less complexity, will be adopted more rapidly than others. The relative advantage of automation has been demonstrated empirically, though usually in narrowly focused reviews in clinical areas, rather than more challenging areas for automation, such as public health. Detailed methods and procedures for their use have yet to be established, addressing transparency, replicability and reporting practices. While issues concerning the compatibility of new technology with existing infrastructure are probably surmountable, the use of automation may challenge contemporary notions of what constitutes a systematic search and how publication bias is addressed using sensitive search techniques. The remaining factors are interrelated. The technologies are complex, both to understand and to deploy. This affects the trialability of automation: technical expertise is required and there are thus few opportunities for reviewers to observe others using these technologies. Conclusion Further technical and empirical work is needed where systematic reviewers work with information and computer scientists to develop solutions which have a demonstrative relative advantage and which are clearly compatible with the needs of systematic reviewers and their users. Such work may have a significant role to play in addressing the deluge of new research publications which threaten to overwhelm systematic review processes.

Overview

Integrity signals

Registry

Crossref · 10.13172/2053-2636-1-2-1109

Cites

15 of 15 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Citations

Cites

Show all 15 references

Cited by

See all 44

Metadata

Authors
Thomas, J.
Corresponding author
Thomas, JM
Venue
OA Evidence-Based Medicine
Published
2013-10
Volume
1
Issue
2
ISSN
2053-2636 (ISSN-L)
OpenAlex
W1997248429
Cited by
44
References
15
Citations by year
2024: 6 · 2023: 2 · 2022: 2 · 2021: 4 · 2020: 8 · 2019: 5 · 2018: 4 · 2017: 4 · 2016: 3 · 2015: 5 · 2013: 1
FWCI
3.27
Type
journal article
Language
en
Open access
Licence
cc-by
Locations
OA Evidence-Based Medicine (publishedVersion, cc-by)
Funders
Medical Research Council (MR/J005037/1)
Topics
  • Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
  • Social Sciences › Library and Information Sciences › scientometrics and bibliometrics research
  • Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
Keywords
diffusion of innovations, study selection, title and abstract screening, technology adoption, relative advantage, compatibility, trialability, observability, complexity, publication bias, systematic search, data deluge

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Diffusion of innovation in systematic review methodology: Why is study selection not yet assisted by automation? https://app.cochat.ai/papers/10.13172/2053-2636-1-2-1109
The paper
Thomas, J. (2013). Diffusion of innovation in systematic review methodology: Why is study selection not yet assisted by automation? OA Evidence-Based Medicine. https://doi.org/10.13172/2053-2636-1-2-1109

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.