Cited by 2,103Meta-analysis and systematic reviewsData Quality and Managementscientometrics and bibliometrics research

The DOI matches its Crossref record; 8 of the 8 references it cites were found in a registry or an index; 1 of the 7 sources its claims cite were checked for support.

De-duplication of database search results for systematic reviews in EndNote

Open access
Bramer, Wichor M., Giustini, Dean, de Jonge, Gerdien B., Holland, Leslie, Bekhuis, Tanja·2016·Journal of the Medical Library Association : JMLA

Abstract

When conducting exhaustive searches for systematic reviews, information professionals search multiple databases with overlapping content [1][2][3][4].They typically remove duplicate records to reduce the reviewers' workload associated with screening titles and abstracts; sometimes the reviewers remove the duplicates.Several articles have been published recently on de-duplication methods.In the authors' opinion, these methods are either very time consuming [5] or impractical, as they require uploading large files to an online platform [6,7].A recent overview article compared existing software programs but found that none was truly satisfactory [8].Unique identifiers for journal articles are digital object identifiers (DOIs) and PubMed IDs (PMIDs).However, these identifiers are not present in every database.When they are present, they often cannot be exported easily.Thus, they cannot be relied upon to identify duplicates.An alternative involves using pagination, because the often large page numbers in scientific journals, in combination with other fields, can serve as a type of unique identifier.However, this is complicated by variations in the way page numbers are stored.Most biomedical databases use a long format (e.g.,

Overview

AI snapshot

Written from the paper’s full text.

TL;DR

This paper describes a detailed EndNote-based method for de-duplicating search results in systematic reviews, using page-number normalization and multiple field-comparison passes. The authors argue it is more systematic and reproducible than other approaches, with reduced manual checking and lower error rates, but it is tailored to EndNote and still fairly complex.

Key findings

  • The method has three stages: change field/display settings, import records through a temporary library and adapted export/import filters, then run several duplicate-detection passes with different field settings and remove probable duplicates.
  • The authors state that the time spent de-duplicating references and the error rate are “significantly reduced” because only a small subset of records needs manual assessment.
  • The method relies on pagination because DOIs and PMIDs are not present in every database and often cannot be exported easily; the paper notes that most biomedical databases use long page-number formats (e.g., 1008-1012) while MEDLINE and the Cochrane Library use abbreviated formats (e.g., 1008-12).
  • Table 1 lays out multiple duplicate-removal settings, including field combinations such as Author | Year | Title | Secondary (Journal) | Press, Delete, and other combinations involving Pages, Volume, and Issue.

Methods

  • Design: A methods/tool article describing an EndNote workflow for de-duplication; the method was previously presented in a brief conference paper and is here described in detail for EndNote X7.
  • Data/sample: Not an empirical sample; the paper discusses importing references from databases including PubMed, Ovid MEDLINE, and the Cochrane Library, and notes customized import filters for Embase.com, Web of Science, CINAHL, and Scopus.
  • Main analysis: The workflow compares records across multiple field combinations in EndNote, after normalizing page-number formats via custom export/import filters, and uses repeated duplicate-finding steps plus manual checking of subsets to remove false duplicates.

Limitations

  • The paper states the method is tuned to EndNote; it notes that Reference Manager was tried but “failed” because too many false duplicates were removed, and Reference Manager is no longer sold/support is likely discontinued.
  • The paper says the method is “rather complicated” and has a “steep” learning curve. This is a limitation stated by the authors.
  • Evident from the text: the method depends on databases exporting usable page numbers and journal-title formats, and the authors note that differences across databases and changes such as Cochrane’s switch to abbreviated page numbers and CINAHL’s page-length suffixes complicate the workflow.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.3163/1536-5050.104.3.014

Cites

8 of 8 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Verification

Claims the paper makes

1 of 7 cited sources checked · 1 supported · 0 flagged

  • Premisecitation
    § De-duplication of database search results for systematic reviews in EndNote

    When conducting exhaustive searches for systematic reviews, information professionals search multiple databases with overlapping content.

    “When conducting exhaustive searches for systematic reviews, information professionals search multiple databases with overlapping content [1-4].”
    Quote in the paperSubstantial claim1 of 4 sources checked
    • cites [1] 1. Levay P, Raynor M, Tuvey D. The contributions of MEDLINE, other bibliographic databases and various

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [2]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [3] 3. Lorenzetti DL, Topfer LA, Dennett L, Clement F. Value of databases other than MEDLINE for rapid health technology assessments. Int J Technol Assessment Health Care. 2014;30(02):173-8. source
    • cites [4] 4. Beyer FR, Wright K. Can we prioritise which databases to search? a case study using a systematic review of frozen shoulder management. Health Inf Libr J. 2013; 30(1):49-58. source
  • Findingown data
    § DISCUSSION

    When the authors tried their method in Reference Manager, it failed because too many false duplicates were removed.

    “However, when we tried our method in Reference Manager, it failed: too many false duplicates were removed.”
    Quote in the paperSubstantial claim
  • Premisecitation
    § DISCUSSION

    The only alternative software to EndNote in which the fields that are to be compared in the de-duplication process can be changed is Reference Manager, also provided by Thomson Reuters.

    “The only alternative software to EndNote in which the fields that are to be compared in the de-duplication process can be changed is Reference Manager, also provided by Thomson Reuters [14].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [14] 14. Reference Manager. Version 12. Thomson Reuters; 2008.

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Interpretationreasoning
    § DISCUSSION

    The de-duplication method is tuned to EndNote.

    “The limitation of this method is that it is tuned to EndNote”
    Quote in the paperSubstantial claim
  • Premisecitation
    § DISCUSSION

    Reference Manager allows comparisons by start pages.

    “Reference Manager allows comparisons by start pages.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [14] 14. Reference Manager. Version 12. Thomson Reuters; 2008.

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim. 10
  • It may state two separate claims as one. 5
  • Probably not a substantial claim. The paper’s conclusions may not depend on it.

    Findingreasoning
    § DISCUSSION

    In older versions of EndNote, step 2 in the third column of rows C and D in Table 1 will not work as desired because duplicate references are not highlighted.

    “however, in older versions, step 2 in the third column of rows C and D in Table 1 will not work as desired because duplicate references are not highlighted.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim. The paper’s conclusions may not depend on it.

    Findingnone given
    § DISCUSSION

    The method will also work in earlier versions of EndNote (versions X3 and higher).

    “It will also work in earlier versions (versions X3 and higher)”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds. It may state two separate claims as one.

    Interpretationnone given
    § DISCUSSION

    If performed frequently, librarian-mediated de-duplication services can be faster than current methods and less error prone.

    “If performed frequently, librarian-mediated de-duplication services can be faster than current methods and less error prone.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim.

    Premisecitation
    § De-duplication of database search results for systematic reviews in EndNote

    A recent overview article found that none of the existing software programs for de-duplication was truly satisfactory.

    “A recent overview article compared existing software programs but found that none was truly satisfactory [8].”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [8]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim. The paper may give no reason why it holds.

    Interpretationnone given
    § DISCUSSION

    Simplification of the method (e.g., by reducing the number of different field combinations or by not normalizing page numbers) increases the workload by increasing the number of references to manually assess.

    “However, simplification of the method (e.g., by reducing the number of different field combinations or by not normalizing page numbers) increases the workload by increasing the number of references to manually assess.”
    Quote in the paperNot a substantial claim
  • It may state two separate claims as one.

    Interpretationown data
    § DISCUSSION

    The comparison of start page numbers in Reference Manager appears to be flawed; therefore, tailoring the method for Reference Manager is not a good alternative.

    “The comparison of start page numbers in Reference Manager appears to be flawed; therefore, tailoring by our method for Reference Manager is not a good alternative.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim. The paper may give no reason why it holds. It may state two separate claims as one.

    Interpretationreasoning
    § DISCUSSION

    The time spent de-duplicating references and the error rate are significantly reduced because just a small subset of the search results has to be assessed manually.

    “The time spent de-duplicating references and the error rate are significantly reduced because just a small subset of the search results has to be assessed manually.”
    Quote in the paperNot a substantial claim
  • It may state two separate claims as one.

    Interpretationreasoning
    § DISCUSSION

    The last two steps require some additional manual assessment, but screening by page numbers expedites the work, and the number of references to assess is lower than in other methods.

    “The last two steps require some additional manual assessment, but screening by page numbers expedites the work, and the number of references to assess is lower than in other methods.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim. The paper may give no reason why it holds.

    Interpretationnone given
    § DISCUSSION

    A requisite of this method is that for efficient deduplication, page numbers should include both a start and an end page.

    “A requisite of this method is that for efficient deduplication, page numbers should include both a start and an end page.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.

    Interpretationnone given
    § DISCUSSION

    The method described in the paper is for the most recent version of EndNote for Windows, version X7.

    “The method described in this paper is for the most recent version of EndNote for Windows, version X7.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim. The paper may give no reason why it holds.

    Interpretationnone given
    § DISCUSSION

    The method's strength is based on the specificity of the first two steps, which require no manual assessment.

    “The method's strength is based on the specificity of the first two steps, which require no manual assessment.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim. The paper may give no reason why it holds. It may state two separate claims as one.

    Interpretationreasoning
    § DISCUSSION

    Although using customized import filters is not strictly necessary, it improves the sensitivity of the first step and thus reduces the number of references that have to be checked manually.

    “Although this is not strictly necessary, it improves the sensitivity of the first step and, thus, reduces the number of references that have to be checked manually.”
    Quote in the paperNot a substantial claim
12 of 12
  • The paper relies on this without support.

    Information professionals typically remove duplicate records to reduce the reviewers' workload associated with screening titles and abstracts; sometimes the reviewers remove the duplicates.

    “They typically remove duplicate records to reduce the reviewers' workload associated with screening titles and abstracts; sometimes the reviewers remove the duplicates.”
  • The paper relies on this without support.

    DOIs and PMIDs are not present in every database.

    “However, these identifiers are not present in every database.”
  • The paper relies on this without support.

    When DOIs and PMIDs are present, they often cannot be exported easily.

    “When they are present, they often cannot be exported easily.”
  • The paper relies on this without support.

    Page numbers, in combination with other fields, can serve as a type of unique identifier.

    “An alternative involves using pagination, because the often large page numbers in scientific journals, in combination with other fields, can serve as a type of unique identifier.”
  • The paper relies on this without support.

    Variations in the way page numbers are stored complicate using pagination.

    “However, this is complicated by variations in the way page numbers are stored.”
  • The paper relies on this without support.

    Most biomedical databases use a long page number format (e.g., 1008-1012).

    “Most biomedical databases use a long format (e.g., 1008-1012)”
  • The paper relies on this without support.

    MEDLINE and the Cochrane Library use an abbreviated page number format (e.g., 1008-12).

    “two important databases (MEDLINE and the Cochrane Library) use an abbreviated one (e.g., 1008-12).”
  • The paper relies on this without support.

    The de-duplication method designed for EndNote is more systematic, rigorous, and reproducible than procedures regularly carried out by other information professionals.

    “Although the de-duplication method that we designed for EndNote resembles procedures regularly carried out by other information professionals, it is more systematic, rigorous, and reproducible.”
  • The paper relies on this without support.

    EndNote is commonly used to manage bibliographic records.

    “however, EndNote is commonly used to manage bibliographic records.”
  • The paper relies on this without support.

    Reference Manager is no longer for sale, and support for Reference Manager will likely be discontinued.

    “Additionally, Reference Manager is no longer for sale, and support for Reference Manager will likely be discontinued.”
  • The paper relies on this without support.

    Databases differ in the format of the exported journal titles.

    “Databases also differ in the format of the exported journal titles.”
  • The paper relies on this without support.

    Some databases use abbreviations, while others provide full journal titles.

    “Some databases use abbreviations, while others provide full journal titles.”
  • The paper relies on this without support.

    Cochrane recently switched from exporting full page numbers to abbreviated page numbers.

    “Cochrane recently switched from exporting full page numbers to abbreviated page numbers”
  • The paper relies on this without support.

    CINAHL recently appended the length of the article in the fields for page numbers (e.g., in the format 1008-1012 5p).

    “CINAHL recently appended the length of the article in the fields for page numbers (e.g., in the format 1008-1012 5p).”
14 of 14

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Citations

Cites

Flagged first, then by citations

Research works printed but not linked: 5

  1. [2]2. search techniques to NICE public health guidance. Evid Based Libr Inf Pract. 2015;10(1):50-68.
  2. [9]9. Bramer WM, Holland L, Mollema J, Hannon T, Bekhuis - T. Removing duplicates in retrieval sets from electronic databases: comparing the efficiency and accuracy of the Bramer-method with other methods and software packages. Proceedings of the 14th European Association for Health Information and Libraries (EAHIL) Conference, 57-9; Rome, Italy; 11-13 Jun 2014.
  3. [11]11. National Library of Medicine. PubMed [Internet]. The Library; 2016 [cited 4 Feb 2016]. , http://www.ncbi.nlm. nih.gov/pubmed/ . .
  4. [12]12. Ovid MEDLINE [Internet]. Wolters Kluwer; 2016 [cited 4 Feb 2016]. , http://site.ovid.com/pdf/ OvidMedline\_fs.pdf . .
  5. [13]13. Cochrane. Cochrane Library [Internet]. John Wiley & Sons; 2016 [cited 4 Feb 2016]. , http://www. cochranelibrary.com . .

Cited by

Sorted by citations

Metadata

Authors
Bramer, W. M., Giustini, D., de Jonge, G. B., Holland, L., & Bekhuis, T.
Affiliations
  • Erasmus MC (NL)
  • University of British Columbia (CA)
  • Southern College of Optometry (US)
  • University of Pittsburgh (US)
Venue
Journal of the Medical Library Association : JMLA
Published
2016-07
Volume
104
Issue
3
Pages
240-243
ISSN
  • 1536-5050 (ISSN-L)
  • 1558-9439
OpenAlex
W2471940872
Cited by
2,103
References
8
Citations by year
2026: 346 · 2025: 333 · 2024: 419 · 2023: 340 · 2022: 252 · 2021: 218 · 2020: 113 · 2019: 48 · 2018: 18 · 2017: 13 · 2016: 2
FWCI
5.54
Type
journal article
Language
en
Open access
Licence
cc-by
Locations
  • Journal of the Medical Library Association JMLA (publishedVersion, cc-by)
  • PubMed Central (publishedVersion)
  • PubMed (publishedVersion)
  • EUR Research Repository (Erasmus University Rotterdam) (publishedVersion)
  • EUR Research Repository (Erasmus University Rotterdam) (submittedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Topics
  • Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
  • Physical Sciences › Computer Science › Information Systems › Data Quality and Management
  • Social Sciences › Library and Information Sciences › scientometrics and bibliometrics research
Keywords
data deduplication, duplicate records, database search results, systematic reviews, EndNote, information professionals, bibliographic databases
MeSH terms
Review Literature as Topic, Databases, Bibliographic, Information Storage and Retrieval

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review