CoChat
Sign Up
Cited by 2,840Race, Genetics, and SocietyGenetic Associations and EpidemiologyForensic and Genetic Research

Clinical use of current polygenic risk scores may exacerbate health disparities

Open access
Martin, Alicia R., Kanai, Masahiro, Kamatani, Yoichiro, Okada, Yukinori, Neale, Benjamin M., Daly, Mark J.·2019·Nature Genetics
DOI 10.1038/s41588-019-0379-xPMC PMC6563838

Published version of a preprint

Open this paper in CoChat — free accountOpen PDF

Abstract

Polygenic risk scores (PRS) are poised to improve biomedical outcomes via precision medicine. However, the major ethical and scientific challenge surrounding clinical implementation of PRS is that those available today are several times more accurate in individuals of European ancestry than other ancestries. This disparity is an inescapable consequence of Eurocentric biases in genome-wide association studies, thus highlighting that-unlike clinical biomarkers and prescription drugs, which may individually work better in some populations but do not ubiquitously perform far better in European populations-clinical uses of PRS today would systematically afford greater improvement for European-descent populations. Early diversifying efforts show promise in leveling this vast imbalance, even when non-European sample sizes are considerably smaller than the largest studies to date. To realize the full and equitable potential of PRS, greater diversity must be prioritized in genetic studies, and summary statistics must be publically disseminated to ensure that health disparities are not increased for those individuals already most underserved.

Integrity signals

Registry

Crossref · 10.1038/s41588-019-0379-x

Cites

81 of 81 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

78 of 110 cited sources checked · 10 supported · 1 flagged · accuracy not yet measured

  • Premisecitation

    Existing findings demonstrate a large reduction in prediction utility in non-European populations.

    “Those existing findings indeed demonstrate a large reduction in prediction utility in non-European populations11,”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites [11] Khera, A. V. et al. Genetic risk, adherence to a healthy lifestyle, and coronary disease. N. Engl. J. Med. 375, 2349–2358 (2016). source

      The Results (page unavailable) report similar findings among black and white participants in the ARIC cohort, rather than a large reduction in non-European populations. The Discussion (page unavailable) says the risk score was associated with coronary events among black participants, while cautioning that generalizability needs further testing; it does not report a comparative measure of prediction utility. (LLM, conf=0.79)

      Checked by a reasoning model — may be wrong.

      Experimental — accuracy not yet measured
  • Premisecitation

    Population genetics theory predicts that genetic risk prediction accuracy will decay with increasing genetic divergence between the original GWAS sample and the target of prediction, as a function of population history.

    “population genetics theory predicts that genetic risk prediction accuracy will decay with increasing genetic divergence between the original GWAS sample and target of prediction, a function of population history13,14.”
    Quote in the paperSubstantial claim1 of 2 sources checked
    • cites [13] Martin, A. R. et al. Human demographic history impacts genetic risk prediction across diverse populations. Am. J. Hum. Genet. 100, 635–649 (2017). source
    • cites [14] Scutari, M., Mackay, I. & Balding, D. Using genetic distance to infer the accuracy of genomic prediction. PLoS Genet. 12, e1006288 (2016). source
  • Interpretationcitation

    Even when non-European cohorts are only a fraction the size of the largest European study, they are likely to have disproportionate value for predicting polygenic traits in other individuals of similar ancestry.

    “These studies have shown that even when non-European cohorts are only a fraction the size of the largest European study, they are likely to have disproportionate value for predicting polygenic traits in other individuals of similar ancestry.”
    Quote in the paperSubstantial claim2 of 3 sources checked
    • cites [23] Akiyama, M. et al. Genome-wide association study identifies 112 new loci for body mass index in the Japanese population. Nat. Genet. 49, 1458–1467 (2017). source

      The Variance explained by variants in the GWAS section (p. 12) reports that variants selected using the Japanese GWAS explained BMI variance in Japanese cohorts better than variants selected using a European GWAS—1.81 times better on average. This supports the value of ancestry-matched data, but the paper tests BMI in Japanese cohorts, not prediction of polygenic traits generally. The passages also do not establish the claimed size comparison with the largest European study. (LLM, conf=0.84)

      Experimental — accuracy not yet measured
    • cites [24] Li, Z. et al. Genome-wide association analysis identifies 30 new susceptibility loci for schizophrenia. Nat. Genet. 49, 1576–1583 (2017). source

      The polygenic risk score results (pp. 17–18) show that adding the smaller Chinese cohort improved prediction of schizophrenia case-control status in a Chinese target sample; at one threshold, variance explained rose from 2.34% with European-only training to 7.86% with combined training. The sample-size comparison (p. 9) supports the cohort-size premise. The paper studies schizophrenia in Chinese ancestry, however, not polygenic traits generally or non-European cohorts generally. (LLM, conf=0.88)

      Experimental — accuracy not yet measured
    • cites [40] Lam, M. et al. Comparative genetic architectures of schizophrenia in East Asian and European populations. Preprint at https://www.biorxiv.org/content/10.1101/445874v2 (2018).

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Premisecitation

    Evidence of polygenic adaptation and/or its relative magnitude for height is under question.

    “These effects are particularly challenging to disentangle, as has clearly been demonstrated for height, where evidence of polygenic adaptation and/or its relative magnitude is under question43,44.”
    Quote in the paperNot a substantial claim1 of 2 sources checked
    • cites [43] Sohail, M. et al. Signals of polygenic adaptation on height have been overestimated due to uncorrected population structure in genome-wide association studies. Preprint at https://www.biorxiv.org/content/10.1101/355057v3 (2018). source
    • cites [44] Berg, J. J. et al. Reduced signal for polygenic adaptation of height in UK Biobank. Preprint at https://www.biorxiv.org/content/10.1101/354951v4 (2018).

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Premisereasoning

    The most significant associations tend to be more common in the populations in which they are discovered than elsewhere.

    “This dependence means that the most significant associations tend to be more common in the populations in which they are discovered than elsewhere13,30.”
    Quote in the paperSubstantial claim1 of 2 sources checked
    • cites [13] Martin, A. R. et al. Human demographic history impacts genetic risk prediction across diverse populations. Am. J. Hum. Genet. 100, 635–649 (2017). source
    • cites [30] 1000 Genomes Project Consortium. et al. A global reference for human genetic variation. Nature 526, 68–74 (2015). source

      In “Insights about demography” (page unavailable), the paper reports variants that are much more common in particular populations and suggests that these variants may reveal associations difficult to identify in larger global samples. It does not establish that the most significant associations generally have higher variant frequencies in their discovery populations than elsewhere. (LLM, conf=0.76)

      Experimental — accuracy not yet measured
  • Classified as probably not a substantial claim. 18
  • May combine two claims. 30

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Return of genetic risk of complex disease to at-risk patients does not induce significant self-reported negative behavior or psychological function, and some potentially positive behavioral changes have been detected.

    “Promisingly, return of genetic risk of complex disease to at-risk patients does not induce significant self-reported negative behavior or psychological function, and some potentially positive behavioral changes have been detected10.”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites [10] Frieser, M. J., Wilson, S. & Vrieze, S. Behavioral impact of return of genetic test results for complex disease: systematic review and meta-analysis. Health Psychol. 37, 1134–1144 (2018). source

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationcitation

    Cardiovascular research has led the charge towards clinical translation of PRS.

    “Cardiovascular research, for example, has led the charge towards clinical translation of PRS1.”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites [1] Knowles, J. W. & Ashley, E. A. Cardiovascular disease: the rise of the genetic risk score. PLoS Med. 15, e1002546–e1002547 (2018). source

      The Summary points describe a cardiovascular pilot study that showed the feasibility of delivering a genetic risk score in a clinical setting, and the authors propose incorporating such scores into practice. They also say few studies have addressed practical implementation. The paper does not establish that cardiovascular research has led other fields in clinical translation. (LLM, conf=0.83)

      Experimental — accuracy not yet measured

  • May combine two claims.

    Interpretationcitation

    PRS are many-fold more accurate in European ancestry individuals than in others, and this disparity is the major ethical and scientific challenge surrounding clinical implementation.

    “However, the major ethical and scientific challenge surrounding clinical implementation is that they are many-fold more accurate in European ancestry individuals than others.”
    • Also stated in: § Abstract · boosted
      “the consistent observation that they are currently of far greater predictive value in individuals of recent European descent than in others to be the major ethical and scientific challenge surrounding clinical translation and, at present, the most critical limitation to genetics in precision medicine.”
    • Also stated in: § Abstract · boosted
      “All studies to date using well-powered genome-wide association studies (GWAS) to assess the predictive value of PRS across a range of traits and populations have made a consistent observation: PRS predict individual risk far more accurately in Europeans. than non-Europeans15,16,18–24.”
    Quote in the paperSubstantial claim9 of 10 sources checked
    • cites [11] Khera, A. V. et al. Genetic risk, adherence to a healthy lifestyle, and coronary disease. N. Engl. J. Med. 375, 2349–2358 (2016). source

      The Results (page not provided) note a paucity of well-validated genetic loci in black populations, and the Discussion (page not provided) calls for testing the score in more diverse populations. But the paper reports similar findings among black and white ARIC participants; it does not show that scores are many-fold more accurate in people of European ancestry or identify that disparity as the major ethical and scientific challenge to clinical implementation. (LLM, conf=0.82)

      Experimental — accuracy not yet measured
    • cites [12] Khera, A. V. & Kathiresan, S. Genetics of coronary artery disease: discovery, biology and clinical translation. Nat. Rev. Genet. 18, 331–344 (2017). source

      The Genomic medicine section describes polygenic risk scores and their potential clinical use, but does not compare their accuracy across ancestry groups. The Common variant association studies section mentions a genotyping chip designed for individuals of European ancestry; it does not establish a many-fold accuracy difference or identify that disparity as the major ethical and scientific challenge. Searches for ancestry-related accuracy and clinical-implementation challenges found no passage addressing the claim. (LLM, conf=0.76)

      Experimental — accuracy not yet measured
    • cites [13] Martin, A. R. et al. Human demographic history impacts genetic risk prediction across diverse populations. Am. J. Hum. Genet. 100, 635–649 (2017). source
    • cites [14] Scutari, M., Mackay, I. & Balding, D. Using genetic distance to infer the accuracy of genomic prediction. PLoS Genet. 12, e1006288 (2016). source

      The Introduction (page unavailable) says genomic prediction accuracy tends to fall as target populations become more genetically distant from training populations. But the human simulation uses an Asian training population (Materials and Methods, page unavailable); it does not establish that PRS are many-fold more accurate in European-ancestry individuals or identify this disparity as the major ethical and scientific challenge to clinical implementation. (LLM, conf=0.88)

      Experimental — accuracy not yet measured
    • cites [15] Vilhjálmsson, B. J. et al. Modeling linkage disequilibrium increases accuracy of polygenic risk scores. Am. J. Hum. Genet. 97, 576–592 (2015). source

      The Results (p. 8) report lower absolute prediction accuracy in African- and Asian-ancestry schizophrenia validation samples, and the Discussion (p. 9) calls for improving prediction in diverse populations. The paper does not establish a many-fold difference across PRSs generally or characterize that disparity as the major ethical and scientific challenge to clinical implementation. (LLM, conf=0.88)

      Experimental — accuracy not yet measured
    • cites [16] Ware, E. B. et al. Heterogeneity in polygenic scores for common human traits. Preprint at https://www.biorxiv.org/content/10.1101/106062v1 (2017). source

      The Discussion reports that European-ancestry-weighted scores explained approximately twice as much trait variation in the HRS non-Hispanic White sample as in the non-Hispanic Black sample. That supports a performance disparity, but not a general claim about all other ancestries or that the scores are “many-fold more accurate.” The paper does not identify this disparity as the major ethical and scientific challenge to clinical implementation. (LLM, conf=0.89)

      Experimental — accuracy not yet measured
    • cites [17] Curtis, D. Polygenic risk score for schizophrenia is more strongly associated with ancestry than with schizophrenia. Psychiatr. Genet. 28, 85–89 (2018). source

      The Discussion (page unavailable) warns that ancestry differences complicate clinical risk prediction and that a PRS derived largely from European samples may not apply to people of African ancestry. It also raises the possibility that non-European populations may be disadvantaged. But its roughly tenfold figure compares differences in PRS values, not predictive accuracy; the paper does not show that PRS are many-fold more accurate in Europeans or rank this as the major implementation challenge. (LLM, conf=0.87)

      Experimental — accuracy not yet measured
    • cites [18] Schizophrenia Working Group of the Psychiatric Genomics Consortium. Biological insights from 108 schizophrenia-associated genetic loci. Nature 511, 421–427 (2014). source

      The Polygenic risk score profiling section (page unavailable) reports limited sensitivity and specificity and cautions against using the scores as a predictive test. The 108 independent associated loci section (page unavailable) identifies European- and East Asian-ancestry samples, but the passages found do not compare score accuracy across ancestries or call such a disparity the major ethical and scientific challenge. Searches for ancestry-related prediction differences and clinical implementation found no supporting passage. (LLM, conf=0.76)

      Experimental — accuracy not yet measured
    • cites [19] Belsky, D. W. et al. Development and evaluation of a genetic risk score for obesity. Biodemography Soc. Biol. 59, 85–100 (2013). source

      The Results and Discussion (page unavailable) show that this obesity genetic risk score was substantially less predictive in African American than in white ARIC participants. But the study tested one obesity score in those two groups, not PRS generally across all ancestries. Its Discussion does not identify that disparity as the major ethical and scientific challenge to clinical implementation; it instead says the score's modest added predictive value was insufficient to recommend it for clinical risk assessment. (LLM, conf=0.87)

      Experimental — accuracy not yet measured
    • cites [20] Domingue, B. W., Belsky, D., Conley, D., Harris, K. M. & Boardman, J. D. Polygenic influence on educational attainment: new evidence from The National Longitudinal Study of Adolescent to Adult Health. AERA Open 1, 1–13 (2015). source

      The Materials > Sample section says that polygenic scores derived from European-ancestry samples may be less precise in African-American samples, and the Results report a smaller association with educational attainment among African Americans. The paper does not establish a many-fold accuracy difference across all other ancestries or characterize that disparity as the major ethical and scientific challenge to clinical implementation. No page numbers are available in our copy. (LLM, conf=0.88)

      Experimental — accuracy not yet measured

  • May combine two claims.

    Interpretationcitation

    Some machine learning approaches may modestly improve PRS accuracy beyond current approaches for some phenotypes, but most likely for atypical traits with simpler architectures, known interactions, and poor prediction generalizability across populations, such as skin pigmentation.

    “Some machine learning approaches may thus modestly improve PRS accuracy beyond current approaches for some phenotypes52, but most likely for atypical traits with simpler architectures, known interactions, and poor prediction generalizability across populations, such as skin pigmentation53.”
    Quote in the paperNot a substantial claim2 of 2 sources checked
    • cites [52] Paré, G., Mao, S. & Deng, W. Q. A machine-learning heuristic to improve gene score prediction of polygenic traits. Sci. Rep. 7, 12665 (2017). source

      Split into 2 atoms — 1 confirmed, 1 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.

      Experimental — accuracy not yet measured
    • cites [53] Martin, A. R. et al. An unexpectedly complex architecture for skin pigmentation in Africans. Cell 171, 1340–1353.e14 (2017). source

      Split into 2 atoms — 0 confirmed, 2 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.

      Experimental — accuracy not yet measured

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Non-linear genetic factors explain little variation in complex traits beyond a purely additive model.

    “While non-linear genetic factors explain little variation in complex traits beyond a purely additive model51,”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites [51] Zhu, Z. et al. Dominance genetic variation contributes little to the missing heritability for human complex traits. Am. J. Hum. Genet. 96, 377–385 (2015). source

      The Discussion (page not supplied) reports that dominance variance averaged about one-fifth of additive variance across 79 quantitative traits. That supports the claim for dominance, but the study did not estimate all non-linear genetic factors, including epistasis, so the broader wording goes beyond its findings. (LLM, conf=0.84)

      Experimental — accuracy not yet measured

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationcitation

    Assessments of PRS across diverse populations are becoming increasingly feasible with the growth and public availability of global biobanks as well as diversifying priorities from funding agencies.

    “These assessments are becoming increasingly feasible with the growth and public availability of global bio banks as well as diversifying priorities from funding agencies55,56.”
    Quote in the paperNot a substantial claim2 of 2 sources checked
    • cites [55] H3Africa Consortium. et al. Enabling the genomic revolution in Africa. Science 344, 1346–1348 (2014). source

      In “Enabling the genomic revolution in Africa” (page not provided), the paper describes funding for H3Africa and plans to make its genomic data publicly accessible through an access committee. It does not discuss polygenic risk score (PRS) assessments or establish that growing global biobanks and diversifying funding priorities are making those assessments increasingly feasible. (LLM, conf=0.80)

      Experimental — accuracy not yet measured
    • cites [56] Hindorff, L. A. et al. Prioritizing diversity in human genomics research. Nat. Rev. Genet. 19, 175–185 (2018). source

      The sections on foundational genomic data resources and dedicated funding support describe expanding diverse reference datasets, benefits of publicly accessible reference populations, and funding programmes that prioritize diversity (page numbers unavailable). They do not specifically establish that global biobanks are growing and publicly available, or that these developments are making assessments of polygenic risk scores across populations increasingly feasible. (LLM, conf=0.83)

      Experimental — accuracy not yet measured

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    A polygenic burden of LDL-increasing SNPs can confer monogenic-equivalent risk of cardiovascular disease, with polygenic scores improving clinical models for risk assessment and statin prescription that can reduce coronary heart disease and improve healthcare delivery efficiency.

    “This enthusiasm is driven by observations that a polygenic burden of LDL-increasing SNPs can confer monogenic-equivalent risk of cardiovascular disease, with polygenic scores improving clinical models for risk assessment and statin prescription that can reduce coronary heart disease and improve healthcare delivery efficiency5–7.”
    Quote in the paperNot a substantial claim3 of 3 sources checked
    • cites [5] Khera, A. V. et al. Genome-wide polygenic scores for common diseases identify individuals with risk equivalent to monogenic mutations. Nat. Genet. 50, 1219–1224 (2018). source

      Split into 2 atoms — 0 confirmed, 2 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.

      Experimental — accuracy not yet measured
    • cites [6] Kullo, I. J. et al. Incorporating a genetic risk score into coronary heart disease risk estimates: effect on low-density lipoprotein cholesterol levels (the MI-GENES Clinical Trial). Circulation 133, 1181–1188 (2016). source

      Split into 2 atoms — 0 confirmed, 2 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.

      Experimental — accuracy not yet measured
    • cites [7] Natarajan, P. et al. Polygenic risk score identifies subgroup with higher burden of atherosclerosis and greater relative benefit from statin therapy in the primary prevention setting. Circulation 135, 2091–2101 (2017). source

      Split into 2 atoms — 0 confirmed, 2 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.

      Experimental — accuracy not yet measured

  • May combine two claims.

    Interpretationcitation

    Differences in environmental exposure, gene-gene interactions, gene-environment interactions, historical population size dynamics, statistical noise, some potential causal effect differences, and/or other factors will further limit generalizability for genetic risk scores in an unpredictable, trait-specific fashion.

    “Differences in environmental exposure, gene-gene interactions, gene-environment interactions, historical population size dynamics, statistical noise, some potential causal effect differences, and/or other factors will further limit generalizability for genetic risk scores in an unpredictable, trait-specific fashion46–49.”
    Quote in the paperSubstantial claim4 of 4 sources checked
    • cites [46] Novembre, J. & Barton, N. H. Tread lightly interpreting polygenic tests of selection. Genetics 208, 1351–1355 (2018). source

      In the paper’s discussion of transferring effect sizes across space and time (page unavailable), the authors say that differences in genetic and environmental background, including gene–gene and gene–environment interactions, may affect polygenic scores across populations. They also discuss environmental differences and changes in effects between populations. Our copy does not establish historical population-size dynamics or statistical noise as limits on genetic-risk-score generalizability, nor does it characterize those limits as unpredictable and trait-specific. (LLM, conf=0.85)

      Experimental — accuracy not yet measured
    • cites [47] Henn, B. M., Botigué, L. R., Bustamante, C. D., Clark, A. G. & Gravel, S. Estimating the mutation load in human genomes. Nat. Rev. Genet. 16, 333–343 (2015). source

      The Conclusions (page unavailable) discuss epistasis, adaptation to different environments, and differences in complex-disease architecture across populations. Demographic Simulations of Mutation Load (page unavailable) discusses bottlenecks and founder effects. The paper does not establish that these factors limit genetic risk-score generalizability in an unpredictable, trait-specific way, and the cited passages do not address statistical noise or gene–environment interactions as risk-score limitations. (LLM, conf=0.82)

      Experimental — accuracy not yet measured
    • cites [48] Brown, B. C., Asian Genetic Epidemiology Network Type 2 Diabetes Consortium, Ye, C. J., Price, A. L. & Zaitlen, N. Transethnic genetic-correlation estimates from summary statistics. Am. J. Hum. Genet. 99, 76–88 (2016). source

      The Discussion (page not provided) identifies gene-gene and gene-environment interactions and statistical noise as possible reasons genetic effects differ between populations; the Introduction says such differences may make disease-risk prediction from current GWAS results problematic. The paper does not establish historical population-size dynamics or environmental exposure as specific limits on risk-score generalizability, nor does it conclude that these limits will be unpredictable and trait-specific. (LLM, conf=0.85)

      Experimental — accuracy not yet measured
    • cites [49] Galinsky, K. J. et al. Estimating cross-population genetic correlations of causal effect sizes. Genet. Epidemiol. 43, 180–188 (2019). source

      Split into 2 atoms — 0 confirmed, 2 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.

      Experimental — accuracy not yet measured

  • May combine two claims.

    Premisecitation

    Imputation reference panels share the same study biases as in GWAS, creating challenges for imputing sites that are rare in European populations but common elsewhere when the catalog of non-European haplotypes is substantially smaller.

    “Furthermore, imputation reference panels share the same study biases as in GWAS41, creating challenges for imputing sites that are rare in European populations but common elsewhere when the catalog of non-European haplotypes is substantially smaller.”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites [41] McCarthy, S. et al. A reference panel of 64,976 haplotypes for genotype imputation. Nat. Genet. 48, 1279–1283 (2016). source

      The Abstract (page not provided) says the HRC panel draws mainly on studies of predominantly European ancestry and proposes increasing its ethnic diversity. It also links larger, more diverse panels to better imputation accuracy. Online Methods > Site filtering (page not provided) notes variants frequent in non-European populations that were called only in 1000 Genomes. The paper does not explicitly establish that reference panels share GWAS study biases or assess the claimed imputation difficulty for variants rare in Europeans but common elsewhere. (LLM, conf=0.82)

      Experimental — accuracy not yet measured

  • May combine two claims.

    Premisecitation

    Africa retains the most genetic diversity, and Africans are understudied but disproportionately informative for genetic analyses and evolutionary history.

    “These considerations notwithstanding, there are critical needs and challenges for expanding the scale of genetic studies of heritable traits in diverse populations; this is especially apparent in Africa where humans originated and retain the most genetic diversity, as Africans are understudied but disproportionately informative for genetic analyses and evolutionary history27,73.”
    Quote in the paperNot a substantial claim2 of 2 sources checked
    • cites [27] Morales, J. et al. A standardized framework for representation of ancestry data in genomics studies, with application to the NHGRI-EBI GWAS Catalog. Genome Biol. 19, 21 (2018). source

      The Results (Improving data analysis and assessing diversity; page not provided) show that African-ancestry participants account for 2.4% of individuals in the GWAS Catalog but contribute 7% of associations. The Discussion (Assessing diversity in genomics; page not provided) links this yield to higher genetic diversity and reduced linkage disequilibrium. Our copy does not state that Africa has the most genetic diversity, that humans originated there, or that African populations are disproportionately informative for evolutionary history. (LLM, conf=0.86)

      Experimental — accuracy not yet measured
    • cites [73] Martin, A. R., Teferra, S., Möller, M., Hoal, E. G. & Daly, M. J. The critical needs and challenges for genetic architecture studies in Africa. Curr. Opin. Genet. Dev. 53, 113–120 (2018). source

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The H3Africa Initiative has increased genomics research capacity in Africa through more than $216 million in funding from the NIH (USA) and Wellcome Trust (UK) for genetics research led by African investigators.

    “The most notable investment here comes from the Human Heredity and Health in Africa (H3Africa) Initiative, increasing genomics research capacity in Africa through more than $216 million in funding from the NIH (USA) and Wellcome Trust (UK) for genetics research led by African investigators55,74.”
    Quote in the paperNot a substantial claim2 of 2 sources checked
    • cites [55] H3Africa Consortium. et al. Enabling the genomic revolution in Africa. Science 344, 1346–1348 (2014). source

      In “Enabling the genomic revolution in Africa” (page unavailable), the paper says the NIH and Wellcome Trust fund H3Africa, which focuses on capacity building and awards research grants directly to African institutions with African principal investigators. It gives the funding amount as $76 million over five years, not more than $216 million. (LLM, conf=0.90)

      Experimental — accuracy not yet measured
    • cites [74] Coles, E. & Mensah, G. A. Geography of genetics and genomics research funding in Africa. Glob. Heart 12, 173–176 (2017). source

      The Introduction says substantial NIH and Wellcome Trust investment in H3Africa gave genetics and genomics research in sub-Saharan Africa momentum. But the Results and Summary and Conclusions identify approximately $216 million as 2015 funding for 185 genetics and genomics projects across Africa, not funding for H3Africa alone; the paper also does not establish that those projects were led by African investigators. (LLM, conf=0.89)

      Experimental — accuracy not yet measured

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    The U.S. leads other middle- and high-income countries in both real and perceived healthcare disparities.

    “This is a major concern globally and especially in the U.S., which already leads other middle-and high-income countries in both real and perceived healthcare disparities69,70.”
    Quote in the paperNot a substantial claim1 of 2 sources checked
    • cites [69] Hero, J. O., Zaslavsky, A. M. & Blendon, R. J. The United States leads other nations in differences by income in perceptions of health and health care. Health Aff. (Millwood) 36, 1032–1040 (2017). source
    • cites [70] Williams, D. R., Priest, N. & Anderson, N. B. Understanding associations among race, socioeconomic status, and health: Patterns and prospects. Health Psychol. 35, 407–411 (2016). source

      The opening section describes poorer quality of care for racial and ethnic minorities in the U.S. and globally. It does not say the U.S. leads middle- and high-income countries in healthcare disparities or compare perceived disparities across countries. (LLM, conf=0.85)

      Experimental — accuracy not yet measured

  • May combine two claims.

    Premisecitation

    Guidelines state that as few as 120 individuals define reference intervals for clinical factors (though often smaller numbers from only one subpopulation are used) and there is no clear definition of who is "normal".

    “Currently, guidelines state that as few as 120 individuals define reference intervals for clinical factors (though often smaller numbers from only one subpopulation are used) and there is no clear definition of who is “normal”64.”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites [64] Manrai, A. K., Patel, C. J. & Ioannidis, J. P. A. In the era of precision medicine and big data, who is normal? JAMA 319, 1981–1982 (2018). source

      In “Defining Normality,” the paper says CLSI guidelines call for 120 reference individuals, that laboratories may use fewer, and that health lacks a universal definition. The “Multiplicity” section mentions sampling from the same subpopulation only as a simulation example; it does not establish that smaller samples are often drawn from just one subpopulation. Page numbers are unavailable in our copy. (LLM, conf=0.85)

      Experimental — accuracy not yet measured

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    H3ABioNet provides cost-effective examples for training local scientists in LMICs.

    “Frameworks such as the H3ABioNet, a pan-African bioinformatics network designed to build capacity to enable H3Africa researchers to analyze their data in Africa, provide cost-effective examples for training local scientists in LMICs75.”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites [75] Mulder, N. J. et al. Development of bioinformatics infrastructure for genomics research. Glob. Heart 12, 91–98 (2017). source

      The Approach and implementation section describes H3ABioNet as a pan-African network building local capacity, and Genomics skills training describes its courses and other training for African scientists. Our copy does not establish that the training is cost-effective or assess it as a model across LMICs; the cited evidence is specific to Africa. Page numbers were not available in the indexed passages. (LLM, conf=0.86)

      Experimental — accuracy not yet measured

  • May combine two claims.

    Interpretationcitation

    Individual clinical tests, biomarkers, and prescription drug efficacy may vary across populations in their utility, but are fundamentally informed by the same underlying biology.

    “Individual clinical tests, biomarkers, and prescription drug efficacy may vary across populations in their utility, but are fundamentally informed by the same underlying biology64,65.”
    Quote in the paperSubstantial claim1 of 2 sources checked
    • cites [64] Manrai, A. K., Patel, C. J. & Ioannidis, J. P. A. In the era of precision medicine and big data, who is normal? JAMA 319, 1981–1982 (2018). source

      The paper’s discussion of population variation in laboratory tests and its Potential Solutions section (page not provided) support the point that a clinical test’s utility can differ across populations. The copy does not establish population differences in prescription-drug efficacy or the claim that all three are informed by the same underlying biology. (LLM, conf=0.86)

      Experimental — accuracy not yet measured
    • cites [65] Plenge, R. M., Scolnick, E. M. & Altshuler, D. Validating therapeutic targets through human genetics. Nat. Rev. Drug Discov. 12, 581–594 (2013). source

  • May combine two claims.

    Findingown data

    Prediction accuracy was far lower in individuals of African descent in the UK Biobank using GWAS summary statistics from either European or Japanese ancestry individuals, consistent with reduced prediction accuracy with increasing genetic divergence.

    “Prediction accuracy was far lower in individuals of African descent in the UK Bio bank (Supplementary Figures 4 and 11) using GWAS summary statistics from either European or Japanese ancestry individuals, consistent with reduced prediction accuracy with increasing genetic divergence (Figures 3 and 4).”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites Supplementary Figures 4 and 11

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites Figures 3 and 4

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Findingown data

    Average sample sizes of GWAS in Europeans continue to grow, while they have stagnated and remain several-fold smaller in other populations.

    “However, whereas the average sample sizes of GWAS in Europeans continue to grow, they have stagnated and remain several-fold smaller in other populations (Supplementary Figure 1).”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Supplementary Figure 1

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Findingown data

    Trans-ethnic correlation estimates indicate that effect sizes were mostly highly correlated across ancestries, with a few traits somewhat lower than expected (e.g. height and BMI, with ρge=0.69 and 0.75, respectively).

    “These trans-ethnic correlation estimates indicate that effect sizes were mostly highly correlated across ancestries, with a few traits that were somewhat lower than excepted (e.g. height and BMI, with ρge=0.69 and 0.75, respectively).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Relative to European prediction accuracy, genetic prediction accuracy using European-derived summary statistics was far lower in other populations in the UK Biobank: 1.6-fold lower in Hispanic/Latino Americans, 1.7-fold lower in South Asians, 2.5-fold lower in East Asians, and 4.9-fold lower in Africans on average.

    “Consistent with previous studies, we find that relative to European prediction accuracy, genetic prediction accuracy was far lower in other populations: 1.6-fold lower in Hispanic/Latino Americans, 1.7-fold lower in South Asians, 2.5-fold lower in East Asians, and 4.9-fold lower in Africans on average (Figure 3).”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Figure 3

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Findingnone given

    Diversifying progress in GWAS has stalled over the last five years, with current imbalance.

    “An equal investment in GWAS across all major ancestries and global populations is the most obvious solution to generate a substrate for equally informative risk scores but is not likely to occur any time soon absent a dramatic priority shift given the current imbalance and stalled diversifying progress over the last five years (Figure 1, Supplementary Figure 1).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Interpretationnone given

    Most PRS methods do not explicitly address recent admixture and none consider recently admixed individuals' unique local mosaic of ancestry; further methods development is needed.

    “Related to stratification, most PRS methods do not explicitly address recent admixture and none consider recently admixed individuals’ unique local mosaic of ancestry; further methods development is needed.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationcitation

    The issues of Eurocentric GWAS biases and imputation panel biases are insurmountable through statistical methods alone, but motivate substantial investments in more diverse populations to produce similar-sized GWAS of biomedical phenotypes in other populations.

    “These issues are insurmountable through statistical methods alone13, but rather motivate substantial investments in more diverse populations to produce similar-sized GWAS of biomedical phenotypes in other populations.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [13] Martin, A. R. et al. Human demographic history impacts genetic risk prediction across diverse populations. Am. J. Hum. Genet. 100, 635–649 (2017). source

  • May combine two claims.

    Interpretationreasoning

    Some unrecognized nonlinearities and gene-gene interactions can induce genetic risk prediction challenges, as pairwise interactions are likely to vary more across populations than individual SNPs.

    “some unrecognized nonlinearities and gene-gene interactions can also induce genetic risk prediction challenges, as pairwise interactions are likely to vary more across populations than individual SNPs.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationnone given

    Early diversifying efforts show promise in levelling the imbalance in PRS accuracy, even when non-European sample sizes are considerably smaller than the largest studies to date.

    “Early diversifying efforts show promise in levelling this vast imbalance, even when non-European sample sizes are considerably smaller than the largest studies to date.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning

    The phenomena of GWAS favoring common variants and LD differentiating effect size estimates degrade prediction performance across populations substantially even when there exist no biological, environmental, or diagnostic differences.

    “Notably, the first two phenomena degrade prediction performance across populations substantially even when there exist no biological, environmental, or diagnostic differences,”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationcitation

    Eurocentric GWAS biases mean that variants associated with risk are disproportionately common and discovered in European populations, accounting for a larger fraction of the phenotypic variance there.

    “Eurocentric GWAS biases mean that variants associated with risk are disproportionately common and discovered in European populations, accounting for a larger fraction of the phenotypic variance there13.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [13] Martin, A. R. et al. Human demographic history impacts genetic risk prediction across diverse populations. Am. J. Hum. Genet. 100, 635–649 (2017). source
    • The authors state this as an assumption: Causal genetic variants have an equal effect across all populations.

  • Probably not a substantial claim: may not pass S4 (based).

    Premisenone given

    Environment and demography differ across populations.

    “3) environment and demography differ across populations.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning

    Complex traits do not behave in a genetically deterministic manner, with some environmental factors dwarfing individual genetic effects, creating outsized issues of comparability across globally diverse populations.

    “Complex traits do not behave in a genetically deterministic manner, with some environmental factors dwarfing individual genetic effects, creating outsized issues of comparability across globally diverse populations.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationreasoning

    Clinical uses of PRS today would systematically afford greater improvement to European descent populations than clinical biomarkers and prescription drugs, which may individually work better in some populations but do not ubiquitously perform far better in European populations.

    “This highlights that—unlike clinical biomarkers and prescription drugs, which may individually work better in some populations but do not ubiquitously perform far better in European populations—clinical uses of PRS today would systematically afford greater improvement to European descent populations.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationreasoning

    Environment and demography may interact to drive differential forces of natural selection that in turn drive differences in causal genetic architecture.

    “whereas the environment and demography may interact to drive differential forces of natural selection that in turn drive differences in causal genetic architecture.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning

    Many cardiovascular PRS studies have been conducted exclusively in European descent populations, with few studies rigorously evaluating population-level applicability to non-Europeans.

    “However, many of these studies have been conducted exclusively in European descent populations, with few studies rigorously evaluating population-level applicability to non-Europeans.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationreasoning

    Differences in effect size estimates due to LD differences may typically be small for most regions of the genome, but PRS sum across these effects, also aggregating these population differences.

    “Differences in effect size estimates due to LD differences may typically be small for most regions of the genome (Figure 2C–E), but PRS sum across these effects, also aggregating these population differences.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Figure 2C–E

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Interpretationreasoning

    Residual uncorrected population stratification may impact risk prediction accuracy across populations, but the magnitude of its effect is currently unclear.

    “Additionally, residual uncorrected population stratification may impact risk prediction accuracy across populations, but the magnitude of its effect is currently unclear.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationnone given

    The disparity in PRS accuracy between European ancestry individuals and others is an inescapable consequence of Eurocentric genome-wide association study biases.

    “This disparity is an inescapable consequence of Eurocentric genome-wide association study biases.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based). May combine two claims.

    Interpretationnone given

    Methodological improvements that better define risk scores by accounting for population allele frequency, LD, and/or admixture differences are underway and may help considerably but will not by themselves bring equality.

    “Methodological improvements that better define risk scores by accounting for population allele frequency, LD, and/or admixture differences appropriately are underway and may help considerably but will not by themselves bring equality.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Interpretationnone given

    Vastly increasing the diversity of participants included and analyzed in genetic studies will improve PRS utility for all and most rapidly for underrepresented groups.

    “The single most important step towards parity in PRS accuracy is by vastly increasing the diversity of participants included and analyzed in genetic studies, which will improve utility for all and most rapidly for underrepresented groups.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationreasoning

    Finer scale continental ancestries and phenotypes in large, multi-ethnic projects enable ancestry-matched analyses not possible with a single set of summary statistics.

    “Existing GWAS consortia also need to carefully consider the granularity of summary statistics they release, as finer scale continental ancestries and phenotypes in large, multi-ethnic projects enable ancestry-matched analyses not possible with a single set of summary statistics.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning

    No analogous solution of defining ethnicity-specific reference intervals would ameliorate health disparities implications for PRS or fundamentally aid interpretability in non-European populations.

    “No analogous solution of defining ethnicity-specific reference intervals would ameliorate health disparities implications for PRS or fundamentally aid interpretability in non-European populations.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).

    Interpretationnone given

    Unifying phenotype definitions, normalization approaches, and GWAS methods among studies will improve comparability.

    “Efforts to unify phenotype definitions, normalization approaches, and GWAS methods among studies will also improve comparability.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based). May combine two claims.

    Interpretationnone given

    PRS have potential to improve health outcomes for all individuals and many complex diseases.

    “Given the acute recent attention on clinical use of PRS, we believe it is paramount to recognize their potential to improve health outcomes for all individuals and many complex diseases.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationcitation

    Some biomarkers or clinical tests scale directly with health outcomes independent of ancestry, and many others may have distributional differences by ancestry but are equally valid after centering with respect to a readily collected population reference.

    “Simply put, some biomarkers or clinical tests scale directly with health outcomes independent of ancestry, and many others may have distributional differences by ancestry but are equally valid after centering with respect to a readily collected population reference.”
    Quote in the paperNot a substantial claim
41 of 41

  • The authors state this as an assumption.

    Causal genetic variants have an equal effect across all populations.

    “If we assume that causal genetic variants have an equal effect across all populations—an assumption with some empirical support that offers the best case scenario for transferability35–40—”

  • The authors state this as an assumption.

    All causal loci have the same impact and are under equivalent selective pressure in all populations.

    “these issues actually make many favorable assumptions that all causal loci have the same impact and are under equivalent selective pressure in all populations.”

  • The paper relies on this without support.

    The prerequisite data for dramatically increasing diversity exist in large-scale publicly funded datasets such as the Million Veterans Project and TOPMed, but data access issues prevent even GWAS summary data within and across populations from being publicly shared.

    “The prerequisite data for dramatically increasing diversity also hypothetically exist in several large-scale publicly funded datasets such as the Million Veterans Project and Trans-Omics for Precision Medicine (TOP Med), but with problematic data access issues in which even GWAS summary data within and across populations are not publicly shared.”

  • The paper relies on this without support.

    African descent populations already endure many of the largest health disparities globally.

    “Conversely, African descent populations, which already endure many of the largest health disparities globally, are often predicted marginally better, if at all, compared to random (Figure 4F).”

  • The paper relies on this without support.

    Human population history and study biases cause African descent populations to be predicted marginally better, if at all, compared to random by PRS.

    “They are therefore least likely to benefit from improvements in precision healthcare delivery from genetic risk scores with existing data due to human population history and study biases.”
5 of 5

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • PGIs constructed from GWAS in European genetic ancestries generally have much lower predictive power in samples with other genetic ancestries; on average across phenotypes, relative accuracy in African-ancestry to European-ancestry samples has been 22% and 36%.

    “PGIs like ours that are constructed from GWAS in samples of European genetic ancestries are generally found to have much lower predictive power in samples with other genetic ancestries; for example, on average across phenotypes, estimates of relative accuracy (ratio of R2) in African-genetic-ancestry to European-genetic-ancestry samples have been 22%23 and 36%24.”

    From Polygenic prediction of educational attainment within and between families from genome-wide association analyses in 3 million individuals (2022) · cites this paper as [23]

    Split into 2 atoms — 1 confirmed, 1 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.

    Experimental — accuracy not yet measured

  • Previous studies have shown that while polygenic risk scores and eQTLs have limited cross-population portability, biochemically active functional regulatory elements tend to be under evolutionary constraint within the human lineage.

    “Previous studies have shown that, while polygenic risk scores [65] and eQTLs [66] have limited cross-population portability, biochemically active functional regulatory elements tend to be under evolutionary constraint within the human lineage [67, 68].”

    From Denisovan introgression has shaped the immune system of present-day Papuans (2022) · cites this paper as [65]

  • For most diseases, large-scale GWAS aggregation efforts have been disproportionately powered by information from individuals of European descent.

    “For most diseases, large-scale GWAS-aggregation efforts have been disproportionately powered by information from individuals of European descent55.”

    From A brief history of human disease genetics (2020) · cites this paper as [55]

  • The historical concentration of genetic discovery on populations of European descent has adverse consequences for both discovery and translation.

    “An elegant distillation of the ways in which the historical focus of genetic discovery among populations of European descent has adverse consequences for both discovery and translation.”

    From A brief history of human disease genetics (2020) · cites this paper as [55]

Cites

Flagged first, then by citations

Cited by

Sorted by citations

Related

Authors
Martin, A. R., Kanai, M., Kamatani, Y., Okada, Y., Neale, B. M., & Daly, M. J.
Affiliations
  • Broad Institute (US)
  • Kyoto University (JP)
  • RIKEN Center for Integrative Medical Sciences (JP)
Corresponding author
Martin, Alicia R.
ORCID
Venue
Nature Genetics
Publisher
Springer Science and Business Media LLC
Published
2019-03-29
Volume
51
Issue
4
Pages
584-591
ISSN
  • 1061-4036 (ISSN-L)
  • 1546-1718
DOI
10.1038/s41588-019-0379-x
PMC
PMC6563838
OpenAlex
W2932671560
Cited by
2,831
References
77
Citations by year
2026: 417 · 2025: 419 · 2024: 400 · 2023: 454 · 2022: 412 · 2021: 396 · 2020: 230 · 2019: 96 · 2018: 3 · 2017: 1
FWCI
154.40
Type
journal article
Language
en
Update notices
correction (2021-01-28)
Open access
Licence
Locations
  • Nature Genetics (publishedVersion)
  • bioRxiv (Cold Spring Harbor Laboratory) (acceptedVersion, cc-by)
  • PubMed (publishedVersion)
  • Europe PMC (PubMed Central) (submittedVersion)
  • PubMed Central (submittedVersion)
Funders
  • Japan Agency for Medical Research and Development
  • Masason Foundation
  • National Institutes of Health (31063, K99MH117229)
  • National Institute of Mental Health (K99 MH117229)
  • National Institute of Diabetes and Digestive and Kidney Diseases (P30 DK043351)
Topics
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Genetics › Race, Genetics, and Society
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Genetics › Genetic Associations and Epidemiology
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Genetics › Forensic and Genetic Research
Keywords
polygenic risk score, health disparities, ancestry bias, genome-wide association study, precision medicine, underrepresented groups
MeSH terms
Genotype, Humans, Risk Factors, Genetic Predisposition to Disease, Multifactorial Inheritance, White People, Healthcare Disparities, Genome-Wide Association Study, Precision Medicine

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-29.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-29.

Cited sources are checked by gpt-6-sol.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review