CoChat

The DOI matches its Crossref record; 72 of the 72 references it cites were found in a registry or an index; 15 of the 70 sources its claims cite were checked for support.

Semi-automated fact-checking of nucleotide sequence reagents in biomedical research publications: The Seek & Blastn tool

Open access
Labbé, Cyril, Grima, Natalie, Gautier, Thierry, Favier, Bertrand, Byrne, Jennifer A. · 2019 · PLOS ONE

Abstract

Nucleotide sequence reagents are verifiable experimental reagents in biomedical publications, because their sequence identities can be independently verified and compared with associated text descriptors. We have previously reported that incorrectly identified nucleotide sequence reagents are characteristic of highly similar human gene knockdown studies, some of which have been retracted from the literature on account of possible research fraud. Because of the throughput limitations of manual verification of nucleotide sequences, we developed a semi-automated fact checking tool, Seek & Blastn, to verify the targeting or non-targeting status of published nucleotide sequence reagents. From previously described and unknown corpora of 48 and 155 publications, respectively, Seek & Blastn correctly extracted 304/342 (88.9%) and 1066/1522 (70.0%) nucleotide sequences and a predicted targeting/ non-targeting status. Seek & Blastn correctly predicted the targeting/ non-targeting status of 293/304 (96.4%) and 988/1066 (92.7%) of the correctly extracted nucleotide sequences. A total of 38/39 (97.4%) or 31/79 (39.2%) Seek & Blastn predictions of incorrect nucleotide sequence reagent use were correct in the two literature corpora. Combined Seek & Blastn and manual analyses identified a list of 91 misidentified nucleotide sequence reagents, which could be built upon through future studies. In summary, incorrect nucleotide sequence reagents represent an under-recognized source of error within the biomedical literature, and fact checking tools such as Seek & Blastn may help to identify papers and manuscripts affected by these errors.

Overview

CoChat Summary

TL;DR

Seek & Blastn (S&B) is a semi-automated fact-checking tool that extracts nucleotide sequences and their claimed targeting/non-targeting status from biomedical PDFs, then uses blastn to verify whether the claim matches the sequence identity. Applied to two corpora, it performed well on a known problematic corpus but less precisely on a broader unknown corpus, identifying 91 misidentified nucleotide sequence reagents overall.

Key findings

  • S&B correctly extracted 304/342 (88.9%) nucleotide sequences and their targeting/non-targeting status from Corpus P and 1066/1522 (70.0%) from Corpus U.
  • Among correctly extracted sequences, S&B correctly predicted targeting/non-targeting status for 293/304 (96.4%) in Corpus P and 988/1066 (92.7%) in Corpus U.
  • S&B predictions of incorrect nucleotide sequence reagent use were correct for 38/39 (97.4%) Corpus P predictions and 31/79 (39.2%) Corpus U predictions.
  • Combined S&B and manual analyses identified 91 misidentified nucleotide sequence reagents: 26 (28.6%) incorrectly identified sh/siRNA targeting reagents and 65 (71.4%) incorrectly identified PCR primers, including one mutagenesis primer; 48/91 (52.7%) had incorrect targeting/non-targeting status and 43/91 (47.3%) targeted a gene or sequence other than described.

Methods

  • The authors developed S&B to extract nucleotide sequences and associated targeting/non-targeting claims from PDF text, perform blastn analyses against human genomic and transcript databases, and fact-check whether the claimed status matches the blastn result.
  • S&B was tested on Corpus P (48 publications previously reported by Byrne and Labbé, containing 342 sequences) and Corpus U (155 publications containing nucleotide sequences, with 1522 sequences; retrieved via PubMed similar and Google Scholar queries, excluding papers already manually analyzed).
  • Manual sequence extraction and independent blastn analyses cross-checked S&B predictions at the sequence and publication levels; Fisher’s exact test was used for specified proportion comparisons.

Limitations

  • The paper states that S&B was applied to only selected text corpora and an overall small number of papers, that Corpus U retrieval means reported error frequencies are unlikely to represent the total journal literature, and that S&B has not yet been applied to consecutively published or randomly sampled corpora.
  • The paper states that S&B was optimized on Corpus P, and its Corpus U error rate for sequence/status extraction was 30.0% versus 11.1% for Corpus P, with reduced precision of 39.2%; false positives included non-human sequences, incorrectly assigned targeting/non-targeting claims, and other issues.
  • The paper notes that statistical comparisons used non-random, non-independent samples, so confidence intervals may be biased upwards; the text also makes evident that S&B cannot automatically flag targeting sequences that target different genes from those claimed, and that manual analysis remains necessary for some error types.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1371/journal.pone.0213266

Cites

72 of 72 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Figures

Fig 1 Diagrammatic illustration of the key steps of Seek & Blastn (S&B). S&B extracts facts to check from published text (nucleotide sequences with associated targeting/ non-targeting status), and then performs blastn analyses and fact checking.
Fig 1 Diagrammatic illustration of the key steps of Seek & Blastn (S&B). S&B extracts facts to check from published text (nucleotide sequences with associated targeting/ non-targeting status), and then performs blastn analyses and fact checking.CC BY · PubMed Central
Fig 2 Example of Seek & Blastn (S&B) output for a retracted Corpus P paper (Ref. [22]). Columns shown from left to right are: “Tested file”; “Nearest dist”, which provides intertextual distance analysis results [9]; “Genes”, which provides gene and species identifiers extracted from the text; “Cont. CL”, which provides identifiers that correspond to contaminated or misidentified cell lines; and “Sequences”, which lists all nucleotide sequences that were extracted, and their corresponding blastn results. The tested publication forms part of the reference cohort [16], and its closest match is the same publication within the reference cohort. The tested pdf included 6 nucleotide sequences, which were correctly extracted and identified by S&B. Two extracted sequences were recognized as a previously reported sequence (SeqA) [16], and a mismatch was detected between the claimed non-targeting status and the blastn identity, as shown in red hypertext.
Fig 2 Example of Seek & Blastn (S&B) output for a retracted Corpus P paper (Ref. [22]). Columns shown from left to right are: “Tested file”; “Nearest dist”, which provides intertextual distance analysis results [9]; “Genes”, which provides gene and species identifiers extracted from the text; “Cont. CL”, which provides identifiers that correspond to contaminated or misidentified cell lines; and “Sequences”, which lists all nucleotide sequences that were extracted, and their corresponding blastn results. The tested publication forms part of the reference cohort [16], and its closest match is the same publication within the reference cohort. The tested pdf included 6 nucleotide sequences, which were correctly extracted and identified by S&B. Two extracted sequences were recognized as a previously reported sequence (SeqA) [16], and a mismatch was detected between the claimed non-targeting status and the blastn identity, as shown in red hypertext.CC BY · PubMed Central
Fig 3 The proportions of Seek & Blastn (S&B) status predictions for 304 correctly extracted Corpus P sequences that were either confirmed or refuted by manual analyses. Predictions were classified as either true negative, false negative, true positive or false positive outcomes. The numbers of targeting and non-targeting sequences for each of the 4 possible outcomes are listed separately. Where sequences were correctly flagged by S&B as true positives, “Targeting” and “Non-targeting” refer to the incorrect claimed status in the relevant publication.
Fig 3 The proportions of Seek & Blastn (S&B) status predictions for 304 correctly extracted Corpus P sequences that were either confirmed or refuted by manual analyses. Predictions were classified as either true negative, false negative, true positive or false positive outcomes. The numbers of targeting and non-targeting sequences for each of the 4 possible outcomes are listed separately. Where sequences were correctly flagged by S&B as true positives, “Targeting” and “Non-targeting” refer to the incorrect claimed status in the relevant publication.CC BY · PubMed Central
Fig 4 The proportions of Seek & Blastn (S&B) status predictions for 1066 correctly extracted Corpus U sequences that were either confirmed or refuted by manual analyses. Predictions were classified as either true negative, false negative, true positive or false positive outcomes. The numbers of targeting and non-targeting sequences for each of the 4 possible outcomes are listed separately. Where sequences were correctly flagged by S&B as true positives, “Targeting” and “Non-targeting” refer to the incorrect claimed status in the relevant publication.
Fig 4 The proportions of Seek & Blastn (S&B) status predictions for 1066 correctly extracted Corpus U sequences that were either confirmed or refuted by manual analyses. Predictions were classified as either true negative, false negative, true positive or false positive outcomes. The numbers of targeting and non-targeting sequences for each of the 4 possible outcomes are listed separately. Where sequences were correctly flagged by S&B as true positives, “Targeting” and “Non-targeting” refer to the incorrect claimed status in the relevant publication.CC BY · PubMed Central
Fig 5 The three Seek & Blastn automata (A1, A2, A3) and associated stacks (StatStk, NucStack, AllStack). Circles represent states and arrows represent state transitions. The upper part of the label on an arrow specifies the property of the scanned word (W) that causes the transition. The lower part of the label described other actions triggered by the transition. The A1 automaton (shown at left) builds nucleotide sequence encounters in a sentence, extracting a nucleotide sequence (Seq) and building the sentence nucleotide stack (@NucStk). The A2 automaton (shown at centre) tracks the different targeting/ non-targeting status encounters when reading a sentence, which may be unknown (S = ʽU’), targeting (S = ʽT’) or non-targeting (S = ʽNT’). The automaton A3 (shown at right) tracks the use of the word “respectively” in the sentence. The ending state of this automaton is used to determine which stacks need to be used to decide the targeting/ non-targeting status of each nucleotide sequence.
Fig 5 The three Seek & Blastn automata (A1, A2, A3) and associated stacks (StatStk, NucStack, AllStack). Circles represent states and arrows represent state transitions. The upper part of the label on an arrow specifies the property of the scanned word (W) that causes the transition. The lower part of the label described other actions triggered by the transition. The A1 automaton (shown at left) builds nucleotide sequence encounters in a sentence, extracting a nucleotide sequence (Seq) and building the sentence nucleotide stack (@NucStk). The A2 automaton (shown at centre) tracks the different targeting/ non-targeting status encounters when reading a sentence, which may be unknown (S = ʽU’), targeting (S = ʽT’) or non-targeting (S = ʽNT’). The automaton A3 (shown at right) tracks the use of the word “respectively” in the sentence. The ending state of this automaton is used to determine which stacks need to be used to decide the targeting/ non-targeting status of each nucleotide sequence.CC BY · PubMed Central

Verification

Claims the paper makes

11 Valid · 1 Not valid · 26 Not checked · 86 Own results and interpretations

65 more claims · 1 with their cited sources checked

Create a free account to see all 124 claims and to verify the 1 reference still open.

See all claims — free account
  • Probably not a substantial claim. 24
  • It may state two separate claims as one. 35

Create a free account to see these claims.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

17 more claims about this paper

Create a free account to see every claim other papers make about this one.

See all claims — free account

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Self-check results appear here once an independent check has confirmed them.

Cited works that were retracted

Works this paper cites that were later retracted, from the retraction notices publishers deposit. Major when the notice came out before this paper was published.

19 of 19

Citations

Cites

19 retracted

Show all 70 references

Research works printed but not linked: 1

  1. 2018, Retraction. TPD52L2 silencing inhibits lung cancer cell proliferation by G2/M phase arrest, Int J Clin Exp Med, 11, 413

Cited by

See all 58

Metadata

Authors
Labbé, C., Grima, N., Gautier, T., Favier, B., & Byrne, J. A.
Affiliations
  • Institut polytechnique de Grenoble (FR)
  • Children's Hospital at Westmead (AU)
  • Centre National de la Recherche Scientifique (FR)
  • Établissement Français du Sang (FR)
  • The University of Sydney (AU)
Corresponding author
  • Labbé, Cyril
  • Byrne, Jennifer A.
Venue
PLOS ONE
Published
2019-03-01
Volume
14
Issue
3
Pages
e0213266
ISSN
1932-6203 (ISSN-L)
OpenAlex
W2918695271
Cited by
55
References
70
Citations by year
2026: 2 · 2025: 3 · 2024: 6 · 2023: 2 · 2022: 6 · 2021: 20 · 2020: 15 · 2019: 1
FWCI
18.33
Type
journal article
Language
en
Open access
Locations
  • PLoS ONE (publishedVersion, cc-by)
  • Europe PMC (PubMed Central) (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • HAL (submittedVersion, other-oa)
  • RePEc: Research Papers in Economics (submittedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • Figshare (submittedVersion, cc-by)
Funders
  • University of Sydney (Post-Truth Initiative, a Sydney University Research Excellence Initiative (SREI 2020))
  • US Office of Research Integrity (ORIIR180038-01-00)
Topics
  • Social Sciences › Education › Academic integrity and plagiarism
  • Social Sciences › Library and Information Sciences › Academic Publishing and Open Access
  • Health Sciences › Medicine › Health Informatics › Biomedical Text Mining and Ontologies
Keywords
fact-checking, research integrity, gene knockdown, scientific fraud, BLASTN
MeSH terms
Algorithms, Humans, Indicators and Reagents, Publications, Sequence Alignment, Sequence Analysis, DNA, Biomedical Research

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-01, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-01 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Semi-automated fact-checking of nucleotide sequence reagents in biomedical research publications: The Seek & Blastn tool. https://app.cochat.ai/papers/10.1371/journal.pone.0213266
The paper
Labbé, C., Grima, N., Gautier, T., Favier, B., & Byrne, J. A. (2019). Semi-automated fact-checking of nucleotide sequence reagents in biomedical research publications: The Seek & Blastn tool. PLOS ONE. https://doi.org/10.1371/journal.pone.0213266

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.