Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 21 of the 21 references it cites were found in a registry or an index; 10 of the 11 sources its claims cite were checked for support.

Reducing systematic review burden using Deduklick: a novel, automated, reliable, and explainable deduplication algorithm to foster medical research

Open access
Borissov, Nikolay, Haas, Quentin, Minder, Beatrice, Kopp-Heim, Doris, von Gernler, Marc, Janka, Heidrun, Teodoro, Douglas, Amini, Poorya · 2022 · Systematic Reviews

Abstract

BACKGROUND: Identifying and removing reference duplicates when conducting systematic reviews (SRs) remain a major, time-consuming issue for authors who manually check for duplicates using built-in features in citation managers. To address issues related to manual deduplication, we developed an automated, efficient, and rapid artificial intelligence-based algorithm named Deduklick. Deduklick combines natural language processing algorithms with a set of rules created by expert information specialists. METHODS: Deduklick's deduplication uses a multistep algorithm of data normalization, calculates a similarity score, and identifies unique and duplicate references based on metadata fields, such as title, authors, journal, DOI, year, issue, volume, and page number range. We measured and compared Deduklick's capacity to accurately detect duplicates with the information specialists' standard, manual duplicate removal process using EndNote on eight existing heterogeneous datasets. Using a sensitivity analysis, we manually cross-compared the efficiency and noise of both methods. DISCUSSION: Deduklick achieved average recall of 99.51%, average precision of 100.00%, and average F1 score of 99.75%. In contrast, the manual deduplication process achieved average recall of 88.65%, average precision of 99.95%, and average F1 score of 91.98%. Deduklick achieved equal to higher expert-level performance on duplicate removal. It also preserved high metadata quality and drastically reduced time spent on analysis. Deduklick represents an efficient, transparent, ergonomic, and time-saving solution for identifying and removing duplicates in SRs searches. Deduklick could therefore simplify SRs production and represent important advantages for scientists, including saving time, increasing accuracy, reducing costs, and contributing to quality SRs.

Overview

CoChat Summary

TL;DR

Deduklick is an NLP-based automated deduplication algorithm for systematic review references, benchmarked against standardized manual EndNote deduplication on eight heterogeneous datasets. It matched or exceeded manual performance while reducing average execution time from 70 minutes to under one minute.

Key findings

  • Across eight datasets, Deduklick averaged 99.51% recall, 100.00% precision, and 99.75% F1; manual deduplication averaged 88.65% recall, 99.95% precision, and 91.98% F1.
  • Deduklick’s F1 score was higher than manual deduplication in six of eight cases; the 45-reference clinical trials dataset was deduplicated correctly by both methods.
  • Deduklick found on average 82 more duplicates than information specialists and showed no false-positive cases among all eight datasets; manual deduplication had 11 false-positive references removed unintentionally from one dataset.
  • Average manual execution time was 70 min (4200 s); Deduklick averaged below 1 min, with 18,314 references deduplicated in 109 s and 45 references in 2 s.

Methods

  • Eight real-life systematic search datasets from University of Bern libraries, average 8,078 references (largest 18,314; smallest 45); six contained pooled publication and clinical trial metadata.
  • Manual deduplication used EndNote’s duplicate identification with a multistep, rule-based process and manual eye validation; Deduklick used NLP normalization, Levenshtein-distance similarity scoring, clustering, and rule-based metadata comparisons, with selected threshold θp ≥ 78.
  • Gold-standard datasets were created by validating additional duplicates and erroneously removed unique references using low threshold θg ≥ 58; performance was measured by recall, precision, F1, and execution time, excluding flowchart/report generation and system roundtrip.

Limitations

  • Stated: only eight EndNote-derived datasets were tested; validating additional databases and testing other datasets are required to explore Deduklick’s full potential.
  • Evident: authors QH, NB, and PA work for Risklick AG, the company that developed Deduklick, and the code will not be shared; data are available only upon reasonable request.
  • Evident: the benchmark/gold standard depends on manual deduplication plus validation, and the manual comparator used EndNote-specific import order and configurations.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1186/s13643-022-02045-9

Cites

21 of 21 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Figures

Fig. 1 Example of Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) deduplication flowchart report following Deduklick analysis
Fig. 1 Example of Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) deduplication flowchart report following Deduklick analysisCC BY · PubMed Central
Fig. 2 Illustration of deduplication report record with an identified duplicates and corresponding unique reference
Fig. 2 Illustration of deduplication report record with an identified duplicates and corresponding unique referenceCC BY · PubMed Central

Verification

Claims the paper makes

6 Valid · 1 Not valid · 0 Not checked · 44 Own results and interpretations

25 more claims

Create a free account to see all 51 claims and every source verdict.

See all claims — free account
  • Probably not a substantial claim. 8
  • It may state two separate claims as one. 15
  • The quote only partly states this claim. 2

Create a free account to see these claims.

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

2 of 2

Citations

Cites

Show all 20 references

Cited by

See all 116

Metadata

Authors
Borissov, N., Haas, Q., Minder, B., Kopp-Heim, D., von Gernler, M., Janka, H., Teodoro, D., & Amini, P.
Affiliations
  • University of Bern (CH)
  • University of Geneva (CH)
Corresponding author
Amini, Poorya
Venue
Systematic Reviews
Published
2022-08-17
Volume
11
Issue
1
Pages
172-172
ISSN
2046-4053 (ISSN-L)
OpenAlex
W4292262803
Cited by
116
References
20
Citations by year
2026: 31 · 2025: 40 · 2024: 31 · 2023: 13 · 2022: 1
FWCI
4.18
Type
journal article
Language
en
Open access
Licence
Locations
  • Systematic Reviews (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • Archive ouverte UNIGE (University of Geneva) (publishedVersion, other-oa)
  • Bern Open Repository and Information System (University of Bern) (submittedVersion, cc-by)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • Archive ouverte UNIGE (University of Geneva) (submittedVersion, cc-by)
Funders
Innosuisse - Schweizerische Agentur für Innovationsförderung (41013.1 IP-ICT)
Topics
  • Physical Sciences › Computer Science › Information Systems › Data Quality and Management
  • Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
  • Social Sciences › Library and Information Sciences › scientometrics and bibliometrics research
Keywords
duplicate detection, citation management, EndNote, natural language processing, metadata normalization, similarity scores, information professionals
MeSH terms
Systematic Reviews as Topic, Algorithms, Artificial Intelligence, Humans, Natural Language Processing, Biomedical Research

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-09-29, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-09-29 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Reducing systematic review burden using Deduklick: a novel, automated, reliable, and explainable deduplication algorithm to foster medical research. https://app.cochat.ai/papers/10.1186/s13643-022-02045-9
The paper
Borissov, N., Haas, Q., Minder, B., Kopp-Heim, D., von Gernler, M., Janka, H., Teodoro, D., & Amini, P. (2022). Reducing systematic review burden using Deduklick: a novel, automated, reliable, and explainable deduplication algorithm to foster medical research. Systematic Reviews. https://doi.org/10.1186/s13643-022-02045-9

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.