Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 85 of the 85 references it cites were found in a registry or an index.

Using text mining for study identification in systematic reviews: a systematic review of current approaches

Open access

Abstract

BACKGROUND: The large and growing number of published studies, and their increasing rate of publication, makes the task of identifying relevant studies in an unbiased way for inclusion in systematic reviews both complex and time consuming. Text mining has been offered as a potential solution: through automating some of the screening process, reviewer time can be saved. The evidence base around the use of text mining for screening has not yet been pulled together systematically; this systematic review fills that research gap. Focusing mainly on non-technical issues, the review aims to increase awareness of the potential of these technologies and promote further collaborative research between the computer science and systematic review communities. METHODS: Five research questions led our review: what is the state of the evidence base; how has workload reduction been evaluated; what are the purposes of semi-automation and how effective are they; how have key contextual problems of applying text mining to the systematic review field been addressed; and what challenges to implementation have emerged? We answered these questions using standard systematic review methods: systematic and exhaustive searching, quality-assured data extraction and a narrative synthesis to synthesise findings. RESULTS: The evidence base is active and diverse; there is almost no replication between studies or collaboration between research teams and, whilst it is difficult to establish any overall conclusions about best approaches, it is clear that efficiencies and reductions in workload are potentially achievable. On the whole, most suggested that a saving in workload of between 30% and 70% might be possible, though sometimes the saving in workload is accompanied by the loss of 5% of relevant studies (i.e. a 95% recall). CONCLUSIONS: Using text mining to prioritise the order in which items are screened should be considered safe and ready for use in 'live' reviews. The use of text mining as a 'second screener' may also be used cautiously. The use of text mining to eliminate studies automatically should be considered promising, but not yet fully proven. In highly technical/clinical areas, it may be used with a high degree of confidence; but more developmental and evaluative work is needed in other disciplines.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1186/2046-4053-4-5

Cites

85 of 85 references verified0 citation errors · 0 not checkable

Statistics

Figures

Figure 1 Brief timeline of developments in the use of text mining technologies for reducing screening burden in systematic reviews.
Figure 1 Brief timeline of developments in the use of text mining technologies for reducing screening burden in systematic reviews.CC BY · PubMed Central

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

34 more claims about this paper

Create a free account to see every claim other papers make about this one.

See all claims — free account

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 82 references

Research works printed but not linked: 12

  1. Lefebvre C, Manheimer E, Glanville J: Searching for studies (chapter 6). In Cochrane Handbook for Systematic Reviews of Interventions Version 510 [updated March 2011]. Edited by: Higgins J, Green S. Oxford: The Cochrane Collaboration; 2011.
  2. Gomersall A, Cooper C: Database selection bias and its affect on systematic reviews: a United Kingdom perspective. In Joint Colloquium of the Cochrane and Campbell Collaborations. Keystone, Colorado: The Campbell Collaboration; 2010.
  3. Malheiros V, Hohn E, Pinho R, Mendonca M: A visual text mining approach for systematic reviews. In Empirical Software Engineering and Measurement, 2007 ESEM 2007 First International Symposium on: 2007 2007. Piscataway: IEEE; 2007:245–54.
  4. Hearst M: Untangling Text Data Mining.Proceedings of the 37th Annual Meeting of the Association for Computational Linguistics (ACL 1999): 1999 1999, 3–10.
  5. Thomas J, Brunton J, Graziosi S: EPPI-Reviewer 4.0: Software for Research Synthesis. London: EPPI-Centre Software, Social Science Research Unit, Institute of Education; 2010.
  6. Sun Y, Yang Y, Zhang H, Zhang W, Wang Q: Towards evidence-based ontology for supporting Systematic Literature Review. In Proceedings of the EASE Conference 2012: 2012 2012. Ciudad Real Spain: IET; 2012.
  7. Martinez D, Karimi S, Cavedon L, Baldwin T: Facilitating biomedical systematic reviews using ranked text retrieval and classification.Proceedings of the 13th Australasian Document Computing Symposium: 2008; Hobart Australia 2008, 53.
  8. Thomas J, O’Mara A: How can we find relevant research more quickly? In NCRM MethodsNews. UK: NCRM; 2011:3.
  9. Felizardo K, Maldonado J, Minghim R, MacDonell S, Mendes E: An extension of the systematic literature review process with visual text mining: a case study on software engineering. 16. Unpublished. Downloadable from: (p.164) http://www.teses.usp.br/teses/disponiveis/55/55134/tde-18072012–102032/publico/Thesis.pdf
  10. Sasaki Y: Automatic text classification.University of Manchester: presentation 2008.
  11. Brinker K: Incorporating diversity in active learning with support vector machines. In Proceedings of the 20th International Conference on Machine Learning: 2003. Palo Alto: AAAI Press; 2003:59–66.
  12. Settles B: Active Learning Literature Survey. Computer Sciences Technical Report 1648. Wisconsin: University of Wisconsin–Madison; 2009.

Cited by

See all 683

Metadata

Authors
O’Mara-Eves, A., Thomas, J., McNaught, J., Miwa, M., & Ananiadou, S.
Affiliations
  • University of London (GB)
  • University of Manchester (GB)
  • Toyota Technological Institute (JP)
Corresponding author
Thomas, James
Venue
Systematic Reviews
Published
2015-01-14
Volume
4
Issue
1
Pages
5-5
ISSN
2046-4053 (ISSN-L)
OpenAlex
W4313371821
Cited by
681
References
82
Citations by year
2026: 60 · 2025: 72 · 2024: 87 · 2023: 80 · 2022: 56 · 2021: 102 · 2020: 64 · 2019: 54 · 2018: 44 · 2017: 27 · 2016: 23 · 2015: 9
FWCI
5.65
Type
journal article
Language
en
Update notices
erratum (2015-04-28)
Open access
Locations
  • Systematic Reviews (publishedVersion, cc-by)
  • Europe PMC (PubMed Central) (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • Research Explorer (The University of Manchester) (publishedVersion, other-oa)
  • UCL Discovery (University College London) (submittedVersion)
Funders
Medical Research Council (MR/J005037/1, MR/L01078X/1)
Topics
  • Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
  • Social Sciences › General Social Sciences › Computational and Text Analysis Methods
  • Social Sciences › Library and Information Sciences › scientometrics and bibliometrics research
Keywords
text mining, systematic review screening, study identification, automated screening, memory retrieval, interdisciplinary collaboration
MeSH terms
Humans, Publications, Databases, Factual, Information Storage and Retrieval, Computational Biology, Evidence-Based Medicine, Data Mining

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Using text mining for study identification in systematic reviews: a systematic review of current approaches. https://app.cochat.ai/papers/10.1186/2046-4053-4-5
The paper
O’Mara-Eves, A., Thomas, J., McNaught, J., Miwa, M., & Ananiadou, S. (2015). Using text mining for study identification in systematic reviews: a systematic review of current approaches. Systematic Reviews. https://doi.org/10.1186/2046-4053-4-5

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.