Cited by 234Genomics and Rare DiseasesGenetic Associations and EpidemiologyGenomics and Phylogenetic Studies

Evaluation of in silico algorithms for use with ACMG/AMP clinical variant interpretation guidelines

Open access
Ghosh, Rajarshi, Oak, Ninad, Plon, Sharon E.·2017·Genome biology

Published version of a preprint

Abstract

BACKGROUND: The American College of Medical Genetics and American College of Pathologists (ACMG/AMP) variant classification guidelines for clinical reporting are widely used in diagnostic laboratories for variant interpretation. The ACMG/AMP guidelines recommend complete concordance of predictions among all in silico algorithms used without specifying the number or types of algorithms. The subjective nature of this recommendation contributes to discordance of variant classification among clinical laboratories and prevents definitive classification of variants. RESULTS: Using 14,819 benign or pathogenic missense variants from the ClinVar database, we compared performance of 25 algorithms across datasets differing in distinct biological and technical variables. There was wide variability in concordance among different combinations of algorithms with particularly low concordance for benign variants. We also identify a previously unreported source of error in variant interpretation (false concordance) where concordant in silico predictions are opposite to the evidence provided by other sources. We identified recently developed algorithms with high predictive power and robust to variables such as disease mechanism, gene constraint, and mode of inheritance, although poorer performing algorithms are more frequently used based on review of the clinical genetics literature (2011-2017). CONCLUSIONS: Our analyses identify algorithms with high performance characteristics independent of underlying disease mechanisms. We describe combinations of algorithms with increased concordance that should improve in silico algorithm usage during assessment of clinically relevant variants using the ACMG/AMP guidelines.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1186/s13059-017-1353-5

Cites

41 of 41 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Score intervals defined by tool developers, and ACMG/AMP recommendations that require the consensus of multiple predictors, lack quantitative support.

    “However, score intervals defined by tool developers, and ACMG/AMP recommendations that require the consensus of multiple predictors, lack quantitative support.”

    From Calibration of computational tools for missense variant pathogenicity classification and ClinGen recommendations for PP3/BP4 criteria (2022) · cites this paper as [3]

    The Discussion says the ACMG/AMP recommendation requires concordance among multiple algorithms without specifying their number or choice, and reports that using more algorithms increases discordant calls. The Results say cutoffs for some algorithms are absent or arbitrarily inferred, but the paper does not establish that developer-defined score intervals generally lack quantitative support. (LLM, conf=0.84)

    Experimental — accuracy not yet measured
  • This is in agreement with previous studies.

    “These fractions were considerably larger than 4.41%, our estimate of the prior probability of pathogenicity (prevalence of pathogenic variants) in the gnomAD set, suggesting a high false positive rate with respect to PP3, in agreement with previous studies.3”
  • Several studies have attempted to benchmark methods to identify tools and tool-threshold combinations that are the most accurate and appropriate for clinically relevant variants.

    “Several studies have attempted to benchmark methods to identify tools and tool-threshold combinations that are the most accurate and appropriate for clinically relevant variants.3,38,39,40,41”

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Cited by

Sorted by citations

Authors
Ghosh, R., Oak, N., & Plon, S. E.
Affiliations
Baylor College of Medicine (US)
Venue
Genome biology
Published
2017-11-28
Volume
18
Issue
1
Pages
225-225
ISSN
  • 1465-6906 (ISSN-L)
  • 1474-760X
OpenAlex
W2949886398
Cited by
233
References
41
Citations by year
2026: 16 · 2025: 22 · 2024: 20 · 2023: 24 · 2022: 31 · 2021: 30 · 2020: 30 · 2019: 36 · 2018: 22 · 2017: 1
FWCI
16.16
Type
journal article
Language
en
Open access
Locations
  • Genome biology (publishedVersion, cc-by)
  • bioRxiv (Cold Spring Harbor Laboratory) (acceptedVersion, cc-by)
  • Europe PMC (PubMed Central) (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
Funders
National Human Genome Research Institute (5 U01 HG007436)
Topics
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Genetics › Genomics and Rare Diseases
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Genetics › Genetic Associations and Epidemiology
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Genomics and Phylogenetic Studies
Keywords
clinical variant interpretation, missense mutations, ClinVar, pathogenic variants, benign variants, disease mechanisms, mode of inheritance, algorithm comparison
MeSH terms
Algorithms, Humans, Software, Genetic Variation, Reproducibility of Results, Genome, Human, Guidelines as Topic, Computational Biology, Databases, Genetic, Genetic Association Studies

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review