CoChat

Assessing the analytical validity of SNP-chips for detecting very rare pathogenic variants: implications for direct-to-consumer genetic testing

Open access
Weedon, Michael N, Jackson, Leigh, Harrison, James W, Ruth, Kate S, Tyrrell, Jessica, Hattersley, Andrew T, Wright, Caroline F · 2019 · bioRxiv (Cold Spring Harbor Laboratory)

The DOI matches its Crossref record; 34 of the 34 references it cites were found in a registry or an index; 19 of the 27 sources its claims cite were checked for support.

Abstract

Objectives: To determine the analytical validity of SNP-chips for genotyping very rare genetic variants. Design: Retrospective study using data from two publicly available resources, the UK Biobank and the Personal Genome Project. Setting: Research biobanks and direct-to-consumer genetic testing in the UK and USA. Participants: 49,908 individuals recruited to UK Biobank, and 21 individuals who purchased consumer genetic tests and shared their data online via the Personal Genomes Project. Main outcome measures: We assessed the analytical validity of genotypes from SNP-chips (index test) with sequencing data (reference standard). We evaluated the genotyping accuracy of the SNP-chips and split the results by variant frequency. We went on to select rare pathogenic variants in the BRCA1 and BRCA2 genes as an exemplar for detailed analysis of clinically-actionable variants in UK Biobank, and assessed BRCA-related cancers (breast, ovarian, prostate and pancreatic) in participants using cancer registry data. Results: SNP-chip genotype accuracy is high overall; sensitivity, specificity and precision are all >99% for 108,574 common variants directly genotyped by the UK Biobank SNP-chips. However, the likelihood of a true positive result reduces dramatically with decreasing variant frequency; for variants with a frequency <0.001% in UK Biobank the precision is very low and only 16% of 4,711 variants from the SNP-chips confirm with sequencing data. Results are similar for SNP-chip data from the Personal Genomes Project, and 20/21 individuals have at least one rare pathogenic variant that has been incorrectly genotyped. For pathogenic variants in the BRCA1 and BRCA2 genes, the overall performance metrics of the SNP-chips in UK Biobank are sensitivity 34.6%, specificity 98.3% and precision 4.2%. Rates of BRCA-related cancers in individuals in UK Biobank with a positive SNP-chip result are similar to age-matched controls (OR 1.28, P=0.07, 95% CI: 0.98 to 1.67), while sequence-positive individuals have a significantly increased risk (OR 3.73, P=3.5×10 −12 , 95% CI: 2.57 to 5.40). Conclusion: SNP-chips are extremely unreliable for genotyping very rare pathogenic variants and should not be used to guide health decisions without validation. SUMMARY BOX: Section 1: What is already known on this topic: SNP-chips are an accurate and affordable method for genotyping common genetic variants across the genome. They are often used by direct-to-consumer (DTC) genetic testing companies and research studies, but there several case reports suggesting they perform poorly for genotyping rare genetic variants when compared with sequencing. Section 2: What this study adds: Our study confirms that SNP-chips are highly inaccurate for genotyping rare, clinically-actionable variants. Using large-scale SNP-chip and sequencing data from UK Biobank, we show that SNP-chips have a very low precision of <16% for detecting very rare variants (i.e. the majority of variants with population frequency of <0.001% are false positives). We observed a similar performance in a small sample of raw SNP-chip data from DTC genetic tests. Very rare variants assayed using SNP-chips should not be used to guide health decisions without validation.

Overview

CoChat Summary

TL;DR

SNP-chips genotype common variants accurately but are unreliable for very rare pathogenic variants, with precision falling to 16.1% for UK Biobank Axiom variants with MAF <0.001% and 4.2% overall for pathogenic BRCA variants. The authors conclude SNP-chip results for very rare pathogenic variants should not guide health decisions without sequencing validation.

Key findings

  • Overall SNP-chip performance was high: across all variants in UK Biobank, sensitivity was 96.3%, specificity 99.9%, and precision 85.9%. For common variants with MAF >1%, Axiom performance was sensitivity 99.8%, specificity 99.7%, precision 99.0%; BiLEVE was 99.7%, 99.7%, and 98.7%.
  • For very rare variants with MAF <0.001%, precision collapsed: only 16.1% of Axiom SNP-chip heterozygous genotypes were confirmed by sequencing (708 true positives in 4,239 individuals at 3,422 variants), and only 9.4% for BiLEVE (46 true positives in 518 individuals at 488 variants). In the Personal Genome Project, precision was 14% for variants with population frequency <0.01% (83 true positives in 21 individuals at 594 variants).
  • For pathogenic BRCA variants in UK Biobank, overall performance was sensitivity 34.6%, specificity 98.3%, and precision 4.2%. Axiom had sensitivity 33.0%, specificity 99.7%, precision 16.9%; BiLEVE had sensitivity 50.0%, specificity 82.7%, precision 0.7%. Of 425 pathogenic BRCA variants in 889 individuals, only 17 variants in 37 individuals were present in sequencing data; 43 pathogenic BRCA variants present in sequencing data of 70 individuals were not detected by either chip.
  • In the Personal Genome Project, 100% (47/47) of rare pathogenic single nucleotide variants and 74% (25/34) of likely protein-truncating insertions/deletions were incorrectly genotyped by SNP-chips; 95% (20/21) of individuals had at least one false positive rare pathogenic variant. BRCA-related cancer risk was similar to controls for SNP-chip-positive individuals (OR 1.28, P=0.07, 95% CI 0.98 to 1.67) but significantly increased for sequencing-positive individuals (OR 3.73, P=3.5x10^-12, 95% CI 2.57 to 5.40).

Methods

  • Retrospective analytical validity study comparing SNP-chip genotypes (index test) with next-generation sequencing (reference standard) in the same individuals.
  • UK Biobank: 49,908 individuals with QC-passed SNP-chip and sequencing data; 45,871 genotyped on the Axiom array and 4,037 on BiLEVE. Personal Genome Project: 21 individuals with direct-to-consumer 23andMe SNP-chip data and Veritas genome sequencing.
  • Analyses compared directly genotyped single nucleotide variants with sequencing positions covered by >15 reads; BRCA pathogenic/likely pathogenic variants were defined by predicted truncation or ClinVar classification. Performance metrics were sensitivity, specificity, and precision; cancer registry data were analyzed with logistic regression.

Limitations

  • The paper does not include a formal limitations section.
  • Evident from the text: the Personal Genome Project replication cohort is very small (21 individuals) and uses raw direct-to-consumer SNP-chip data; the authors note most DTC companies may validate important variants, so raw-data findings may not reflect all DTC reports.
  • Evident from the text: the study is retrospective, and the authors note genotyping batch size can affect accuracy for rare variants. The preprint also states it was not certified by peer review.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1101/696799

Cites

34 of 34 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Verification

Claims the paper makes

10 Valid · 1 Not valid · 7 Not checked · 31 Own results and interpretations

24 more claims

Create a free account to see all 49 claims and every source verdict.

See all claims — free account
  • It may state two separate claims as one. 16
  • Probably not a substantial claim. 7

Create a free account to see these claims.

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

2 of 2

Citations

Cites

Cited by

See all 23

Metadata

Authors
Weedon, M. N., Jackson, L., Harrison, J. W., Ruth, K. S., Tyrrell, J., Hattersley, A. T., & Wright, C. F.
Affiliations
University of Exeter (GB)
Corresponding author
Wright, Caroline F
Venue
bioRxiv (Cold Spring Harbor Laboratory)
Publisher
openRxiv
Published
2019-07-09
OpenAlex
W3100960594
Cited by
19
References
34
Citations by year
2026: 1 · 2022: 2 · 2021: 4 · 2020: 10 · 2019: 2
Type
posted content
Language
en
Open access
Licence
cc-by
Locations
bioRxiv (Cold Spring Harbor Laboratory) (acceptedVersion, cc-by)
Funders
  • Wellcome Trust
  • National Institute for Health and Care Research
  • University of Exeter
Topics
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Genetics › Genomics and Rare Diseases
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Genetics › Genetic Associations and Epidemiology
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Genetics › BRCA gene mutations in cancer
Keywords
SNP array, genotyping accuracy, rare pathogenic variants, direct-to-consumer genetic testing, BRCA1, BRCA2, variant frequency, precision medicine, UK Biobank, Personal Genome Project, cancer risk prediction, analytical validity, breast cancer, ovarian cancer, prostate cancer, pancreatic cancer

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-09-28, under version 3 of the claim definition (prompt extract-v3).

A second model labelled each claim on 2026-09-28 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.