CoChat

Automation of citation screening in pre-clinical systematic reviews

Open access
Liao, J., Ananiadou, S., Currie, L. G., Howard, B. E., Rice, A., Sena, S. E., Thomas, J., Varghese, A., Macleod, M.R. · 2018 · bioRxiv (Cold Spring Harbor Laboratory)

The DOI matches its Crossref record; 9 of the 9 references it cites were found in a registry or an index.

Abstract

Background: The amount of published in vivo studies and the speed researchers are publishing them make it virtually impossible to follow the recent development in the field. Systematic review emerged as a method to summarise and analyse the studies quantitatively and critically but it is often out-of-date due to its lengthy process. Method: We invited five machine learning and text-mining groups to build classifiers for identifying publications relevant to neuropathic pain (33814 training publications). We kept 1188 publications for the assessment of the performance of different classifiers. Two groups participated in the next stage: testing their algorithm on datasets labeled for psychosis (11777/2944) and datasets labeled for Vitamin D in multiple sclerosis (train/text: 2038/510). Result: The performances (sensitive/specificity) of the most promising classifier built for neuropathic pain are: 95%/84%. The performance for psychosis and Vitamin D in multiple sclerosis datasets are 95%/73% and 100%/45%. Conclusions: Machine learning can significantly reduce the irrelevant publications in a systematic review, and save the scientists’ time and money. Classifier algorithms built for one dataset can be reapplied on another dataset in different field. We are building a machine learning service at the back of Systematic Review & Meta-analysis Facility (SyRF).

Overview

Integrity signals

Registry

Crossref · 10.1101/280131

Cites

9 of 9 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Citations

Cites

Show all 9 references

Research works printed but not linked: 1

  1. Gillian L Currie , Nicki Sherratt , Lesley A Colvin , Andrew SC Rice , Fala Cramond , Malcolm R Macleod , & Emily S Sena (2015). Systematic review and meta-analysis of studies using animal models of neuropathic pain protocol https://drive.google.com/file/d/0B30wfjnG6aEQRS0yMVNLWmdKeDQ/view

Cited by

See all 13

Metadata

Authors
Liao, J., Ananiadou, S., Currie, L. G., Howard, B. E., Rice, A., Sena, S. E., Thomas, J., Varghese, A., & Macleod, M. R.
Affiliations
  • University of Edinburgh (GB)
  • University of Manchester (GB)
  • Imperial College London (GB)
Corresponding author
Liao, J.
Venue
bioRxiv (Cold Spring Harbor Laboratory)
Publisher
openRxiv
Published
2018-03-12
OpenAlex
W2792585084
Cited by
13
References
9
Citations by year
2025: 1 · 2024: 1 · 2023: 1 · 2022: 2 · 2021: 3 · 2020: 1 · 2019: 3 · 2018: 1
Type
posted content
Language
en
Open access
Licence
cc-by-nc-nd
Locations
bioRxiv (Cold Spring Harbor Laboratory) (acceptedVersion, cc-by-nc-nd)
Funders
  • Wellcome Trust (MR/N015665/1)
  • Medical Research Council (MR/N015665/1)
Topics
  • Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
  • Health Sciences › Medicine › Health Informatics › Biomedical Text Mining and Ontologies
  • Social Sciences › Library and Information Sciences › scientometrics and bibliometrics research
Keywords
machine learning classifiers, text mining, neuropathic pain, psychosis, vitamin D, multiple sclerosis

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Automation of citation screening in pre-clinical systematic reviews. https://app.cochat.ai/papers/10.1101/280131
The paper
Liao, J., Ananiadou, S., Currie, L. G., Howard, B. E., Rice, A., Sena, S. E., Thomas, J., Varghese, A., & Macleod, M. R. (2018). Automation of citation screening in pre-clinical systematic reviews. bioRxiv (Cold Spring Harbor Laboratory). https://doi.org/10.1101/280131

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.