Assessing the analytical validity of SNP-chips for detecting very rare pathogenic variants: implications for direct-to-consumer genetic testing
Open accessThe DOI matches its Crossref record; 34 of the 34 references it cites were found in a registry or an index; 19 of the 27 sources its claims cite were checked for support.
Abstract
Objectives: To determine the analytical validity of SNP-chips for genotyping very rare genetic variants. Design: Retrospective study using data from two publicly available resources, the UK Biobank and the Personal Genome Project. Setting: Research biobanks and direct-to-consumer genetic testing in the UK and USA. Participants: 49,908 individuals recruited to UK Biobank, and 21 individuals who purchased consumer genetic tests and shared their data online via the Personal Genomes Project. Main outcome measures: We assessed the analytical validity of genotypes from SNP-chips (index test) with sequencing data (reference standard). We evaluated the genotyping accuracy of the SNP-chips and split the results by variant frequency. We went on to select rare pathogenic variants in the BRCA1 and BRCA2 genes as an exemplar for detailed analysis of clinically-actionable variants in UK Biobank, and assessed BRCA-related cancers (breast, ovarian, prostate and pancreatic) in participants using cancer registry data. Results: SNP-chip genotype accuracy is high overall; sensitivity, specificity and precision are all >99% for 108,574 common variants directly genotyped by the UK Biobank SNP-chips. However, the likelihood of a true positive result reduces dramatically with decreasing variant frequency; for variants with a frequency <0.001% in UK Biobank the precision is very low and only 16% of 4,711 variants from the SNP-chips confirm with sequencing data. Results are similar for SNP-chip data from the Personal Genomes Project, and 20/21 individuals have at least one rare pathogenic variant that has been incorrectly genotyped. For pathogenic variants in the BRCA1 and BRCA2 genes, the overall performance metrics of the SNP-chips in UK Biobank are sensitivity 34.6%, specificity 98.3% and precision 4.2%. Rates of BRCA-related cancers in individuals in UK Biobank with a positive SNP-chip result are similar to age-matched controls (OR 1.28, P=0.07, 95% CI: 0.98 to 1.67), while sequence-positive individuals have a significantly increased risk (OR 3.73, P=3.5×10 −12 , 95% CI: 2.57 to 5.40). Conclusion: SNP-chips are extremely unreliable for genotyping very rare pathogenic variants and should not be used to guide health decisions without validation. SUMMARY BOX: Section 1: What is already known on this topic: SNP-chips are an accurate and affordable method for genotyping common genetic variants across the genome. They are often used by direct-to-consumer (DTC) genetic testing companies and research studies, but there several case reports suggesting they perform poorly for genotyping rare genetic variants when compared with sequencing. Section 2: What this study adds: Our study confirms that SNP-chips are highly inaccurate for genotyping rare, clinically-actionable variants. Using large-scale SNP-chip and sequencing data from UK Biobank, we show that SNP-chips have a very low precision of <16% for detecting very rare variants (i.e. the majority of variants with population frequency of <0.001% are false positives). We observed a similar performance in a small sample of raw SNP-chip data from DTC genetic tests. Very rare variants assayed using SNP-chips should not be used to guide health decisions without validation.
Overview
CoChat Summary
TL;DR
SNP-chips genotype common variants accurately but are unreliable for very rare pathogenic variants, with precision falling to 16.1% for UK Biobank Axiom variants with MAF <0.001% and 4.2% overall for pathogenic BRCA variants. The authors conclude SNP-chip results for very rare pathogenic variants should not guide health decisions without sequencing validation.
Key findings
- Overall SNP-chip performance was high: across all variants in UK Biobank, sensitivity was 96.3%, specificity 99.9%, and precision 85.9%. For common variants with MAF >1%, Axiom performance was sensitivity 99.8%, specificity 99.7%, precision 99.0%; BiLEVE was 99.7%, 99.7%, and 98.7%.
- For very rare variants with MAF <0.001%, precision collapsed: only 16.1% of Axiom SNP-chip heterozygous genotypes were confirmed by sequencing (708 true positives in 4,239 individuals at 3,422 variants), and only 9.4% for BiLEVE (46 true positives in 518 individuals at 488 variants). In the Personal Genome Project, precision was 14% for variants with population frequency <0.01% (83 true positives in 21 individuals at 594 variants).
- For pathogenic BRCA variants in UK Biobank, overall performance was sensitivity 34.6%, specificity 98.3%, and precision 4.2%. Axiom had sensitivity 33.0%, specificity 99.7%, precision 16.9%; BiLEVE had sensitivity 50.0%, specificity 82.7%, precision 0.7%. Of 425 pathogenic BRCA variants in 889 individuals, only 17 variants in 37 individuals were present in sequencing data; 43 pathogenic BRCA variants present in sequencing data of 70 individuals were not detected by either chip.
- In the Personal Genome Project, 100% (47/47) of rare pathogenic single nucleotide variants and 74% (25/34) of likely protein-truncating insertions/deletions were incorrectly genotyped by SNP-chips; 95% (20/21) of individuals had at least one false positive rare pathogenic variant. BRCA-related cancer risk was similar to controls for SNP-chip-positive individuals (OR 1.28, P=0.07, 95% CI 0.98 to 1.67) but significantly increased for sequencing-positive individuals (OR 3.73, P=3.5x10^-12, 95% CI 2.57 to 5.40).
Methods
- Retrospective analytical validity study comparing SNP-chip genotypes (index test) with next-generation sequencing (reference standard) in the same individuals.
- UK Biobank: 49,908 individuals with QC-passed SNP-chip and sequencing data; 45,871 genotyped on the Axiom array and 4,037 on BiLEVE. Personal Genome Project: 21 individuals with direct-to-consumer 23andMe SNP-chip data and Veritas genome sequencing.
- Analyses compared directly genotyped single nucleotide variants with sequencing positions covered by >15 reads; BRCA pathogenic/likely pathogenic variants were defined by predicted truncation or ClinVar classification. Performance metrics were sensitivity, specificity, and precision; cancer registry data were analyzed with logistic regression.
Limitations
- The paper does not include a formal limitations section.
- Evident from the text: the Personal Genome Project replication cohort is very small (21 individuals) and uses raw direct-to-consumer SNP-chip data; the authors note most DTC companies may validate important variants, so raw-data findings may not reflect all DTC reports.
- Evident from the text: the study is retrospective, and the authors note genotyping batch size can affect accuracy for rare variants. The preprint also states it was not certified by peer review.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
10 Valid · 1 Not valid · 7 Not checked · 31 Own results and interpretations
Some cited sources support this claim and some don’t — read each source’s verdict below.
In the paper
“False positive results for rare clinically-actionable variants detected by DTC SNP-chips have been described in practice guidance [13], several case reports [14,15] and two small case series [16,17].”
What the sources say
- Supportedcites [13] source
The cited BMJ paper is practice guidance on direct-to-consumer genetic testing. Its sections on SNP-chip testing and false positives explain that very rare disease-causing variants detected by SNP-chips are often false positives and give a clinical example. (LLM, conf=0.91)
- Disputedcites [14] sourceExperimental — accuracy not yet measured
The case in “First, do no harm: direct-to-consumer genetic testing” (page not supplied) describes a woman with a clinically identified pathogenic BRCA2 variant whose DTC test reported low breast-cancer risk and no detected BRCA markers—the opposite of a false-positive result. The paper discusses discrepant DTC results reported by Tandy-Connor et al., but its own case is not a false-positive case report. (LLM, conf=0.88)
Checked by a reasoning model — may be wrong.
- Supportedcites [15] source
The Introduction describes four case vignettes from cardiovascular genetics clinics. The Themes section identifies them as analytical false positives detected by DTC genotyping SNP arrays; Cases 2 and 3 document clinically unconfirmed MYBPC3 results. This supports citing the paper as one of the small case series described. (LLM, conf=0.91)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“As a result, although the performance of chips from different manufacturers may differ somewhat due to different underlying chemistries [5], our findings are likely to be generalisable to any SNP-chip, and indeed the results are similar across different SNP-chips used in both our UKB and PGP datasets.”
What the sources say
- Supportedcites [5] source
The citation is attached to the qualification about manufacturers’ underlying chemistries, not to the citing authors’ conclusion that their findings generalise to any SNP-chip. The source’s “UNDERLYING TECHNOLOGY AND SNP GENOTYPING” section explicitly says Affymetrix and Illumina arrays use different chemistries (page unavailable). (LLM, conf=0.90)
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“A rare pathogenic variant in ABCC8 that causes congenital hyperinsulinism [29] is incorrectly genotyped in 43% (9/21) of individuals.”
What the sources say
- Supportedcites [29] source
The citation is attached to the statement that an ABCC8 variant causes congenital hyperinsulinism, not to the citing authors’ 43% genotyping-error result. The Results section, “Analysis of ABCC8 gene mutations,” identifies ABCC8 mutations in patients with congenital hyperinsulinism; this supports the attributed point. (LLM, conf=0.91)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Single gene disorders are usually caused by genetic variants that are very rare in the population (<1 in 10,000 individuals) [1].”
What the sources say
- Partly supportedcites [1] sourceExperimental — accuracy not yet measured
The Introduction (page unavailable) says severe Mendelian disease-causing variants must be rare, supporting the claim’s general point. It does not establish a usual cutoff of fewer than 1 in 10,000 individuals; the Application to recessive diseases section (page unavailable) calculates a possible variant frequency of 2.42 × 10⁻³ for one disorder, above that cutoff. (LLM, conf=0.86)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“Amongst the genetics community, it is generally well recognised that SNP-chips perform poorly for genotyping rare genetic variants [9-11] due to their reliance upon data clustering (Figure 1).”
What the sources say
- Supportedcites [9] source
The Discussion reports that rarer variants on the UK Biobank genotyping arrays were more likely to be poor quality and that examining genotype clusters separately by batch proved impossible for many clinically important variants because of their rarity. The Methods describe assessing genotype quality through the separation of clusters in intensity plots. This supports the cited point about poor performance for very rare variants and the role of clustering, while the paper also notes that many rare variants were well genotyped. (LLM, conf=0.89)
- Supportedcites [10] source
The Whole-genome genotyping section says rare variants pose a particular challenge for array genotype calling because a true minor-allele sample can be hard to distinguish from the tail of the major-homozygote intensity cluster. It also reports that a larger fraction of rare markers fail quality-control tests than common markers, while noting that 84% of rare markers still pass in all batches. (LLM, conf=0.88)
- Supportedcites [11] source
The Introduction says that SNPs with very low minor allele frequency are difficult to call because their heterozygote and rare-homozygote clusters are small. This supports the claim’s point that cluster-based genotyping performs poorly for rare variants. (LLM, conf=0.91)
boostedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
24 more claims
Create a free account to see all 49 claims and every source verdict.
- It may state two separate claims as one. 16
- Probably not a substantial claim. 7
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ ABSTRACT
“sensitivity, specificity and precision are all >99% for 108,574 common variants directly genotyped by the UK Biobank SNP-chips.”
§ Performance of SNP-chips appears excellent when assessed for all variants
“In the 49,908 UK Biobank individuals, we compared genotypes for 108,574 single nucleotide variants that were classified as heterozygous by the SNP-chip to reference standard sequencing data.”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ ABSTRACT
“OR 1.28, P=0.07, 95% CI: 0.98 to 1.67”
§ Risk of BRCA-related hereditary cancer in individuals in UKB with a pathogenic BRCA variant genotyped by SNP-chips is similar to population risk
“OR 1.28, P=0.07, 95% CI: 0.9 to 1.67”
Citations
Cites
Analysis of protein-coding genetic variation in 60,706 humans
Exome Aggregation Consortium, Lek, M., Karczewski, K. J., Minikel, E. V., Samocha, K. E., Banks, E., Fennell, T., O’Donnell-Luria, A. H., Ware, J. S., Hill, A. J., et al. (2016) · Nature · doi:10.1038/nature19057 · Cited by 9,587
Integrative Genomics Viewer (IGV): high-performance genomics data visualization and exploration
Thorvaldsdottir, H., Robinson, J. T., & Mesirov, J. P. (2013) · Briefings in Bioinformatics · doi:10.1093/bib/bbs017 · Cited by 9,049
The UK Biobank resource with deep phenotyping and genomic data
Bycroft, C., Freeman, C., Petkova, D., Band, G., Elliott, L. T., Sharp, K., Motyer, A., Vukcevic, D., Delaneau, O., O’Connell, J., et al. (2018) · Nature · doi:10.1038/s41586-018-0579-z · Cited by 8,876
An integrated map of genetic variation from 1,092 human genomes
The 1000 Genomes Project Consortium (2012) · Nature · doi:10.1038/nature11632 · Cited by 7,711
ClinVar: public archive of interpretations of clinically relevant variants
Landrum, M. J., Lee, J. M., Benson, M., Brown, G., Chao, C., Chitipiralla, S., Gu, B., Hart, J., Hoffman, D., Hoover, J., et al. (2016) · Nucleic Acids Research · doi:10.1093/nar/gkv1222 · Cited by 2,758
STARD 2015 guidelines for reporting diagnostic accuracy studies: explanation and elaboration
Cohen, J. F., Korevaar, D. A., Altman, D. G., Bruns, D. E., Gatsonis, C. A., Hooft, L., Irwig, L., Levine, D., Reitsma, J. B., de Vet, H. C. W., et al. (2016) · BMJ Open · doi:10.1136/bmjopen-2016-012799 · Cited by 2,570
Data quality control in genetic case-control association studies
Anderson, C. A., Pettersson, F. H., Clarke, G. M., Cardon, L. R., Morris, A. P., & Zondervan, K. T. (2010) · Nature Protocols · doi:10.1038/nprot.2010.116 · Cited by 1,318
Border, R., Johnson, E. C., Evans, L. M., Smolen, A., Berley, N., Sullivan, P. F., & Keller, M. C. (2019) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2018.18070881 · Cited by 629
Clark, M. M., Stark, Z., Farnaes, L., Tan, T. Y., White, S. M., Dimmock, D., & Kingsmore, S. F. (2018) · npj Genomic Medicine · doi:10.1038/s41525-018-0053-8 · Cited by 619
Guidelines for the clinical management of Lynch syndrome (hereditary non-polyposis cancer)
Vasen, H. F. A., Möslein, G., Alonso, A., Bernstein, I., Bertario, L., Blanco, I., Burn, J., Capella, G., Engel, C., Frayling, I., et al. (2007) · Journal of Medical Genetics · doi:10.1136/jmg.2007.048991 · Cited by 554
Cited by
Towards clinical utility of polygenic risk scores
Lambert, S. A., Abraham, G., & Inouye, M. (2019) · Human Molecular Genetics · doi:10.1093/hmg/ddz187 · Cited by 575
Genotyping Array Design and Data Quality Control in the Million Veteran Program
Hunter-Zinck, H., Shi, Y., Li, M., Gorman, B. R., Ji, S.-G., Sun, N., Webster, T., Liem, A., Hsieh, P., Devineni, P., et al. (2020) · The American Journal of Human Genetics · doi:10.1016/j.ajhg.2020.03.004 · Cited by 198
A Fast and Simple Method for Detecting Identity-by-Descent Segments in Large-Scale Data
Zhou, Y., Browning, S. R., & Browning, B. L. (2020) · The American Journal of Human Genetics · doi:10.1016/j.ajhg.2020.02.010 · Cited by 176
AutoMap is a high performance homozygosity mapping tool using next-generation sequencing data
Quinodoz, M., Peter, V. G., Bedoni, N., Royer Bertrand, B., Cisarova, K., Salmaninejad, A., Sepahi, N., Rodrigues, R., Piran, M., Mojarrad, M., et al. (2021) · Nature Communications · doi:10.1038/s41467-020-20584-4 · Cited by 164
Chami, N., Preuss, M., Walker, R. W., Moscati, A., & Loos, R. J. F. (2020) · PLOS Medicine · doi:10.1371/journal.pmed.1003196 · Cited by 84
Metadata
- Authors
- Weedon, M. N., Jackson, L., Harrison, J. W., Ruth, K. S., Tyrrell, J., Hattersley, A. T., & Wright, C. F.
- Affiliations
- University of Exeter (GB)
- Corresponding author
- Wright, Caroline F
- ORCID
- Venue
- bioRxiv (Cold Spring Harbor Laboratory)
- Publisher
- openRxiv
- Published
- 2019-07-09
- OpenAlex
- W3100960594
- Cited by
- 19
- References
- 34
- Citations by year
- 2026: 1 · 2022: 2 · 2021: 4 · 2020: 10 · 2019: 2
- Type
- posted content
- Language
- en
- Open access
- green
- doi.org
- Licence
- cc-by
- Locations
- bioRxiv (Cold Spring Harbor Laboratory) (acceptedVersion, cc-by)
- Funders
- Wellcome Trust
- National Institute for Health and Care Research
- University of Exeter
- Topics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Genetics › Genomics and Rare Diseases
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Genetics › Genetic Associations and Epidemiology
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Genetics › BRCA gene mutations in cancer
- Keywords
- SNP array, genotyping accuracy, rare pathogenic variants, direct-to-consumer genetic testing, BRCA1, BRCA2, variant frequency, precision medicine, UK Biobank, Personal Genome Project, cancer risk prediction, analytical validity, breast cancer, ovarian cancer, prostate cancer, pancreatic cancer
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-09-28, under version 3 of the claim definition (prompt extract-v3).
A second model labelled each claim on 2026-09-28 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.