CoChat

CAGI, the Critical Assessment of Genome Interpretation, establishes progress and prospects for computational genetic variant interpretation methods

Open access
The Critical Assessment of Genome Interpretation Consortium, Jain, Shantanu, Bakolitsa, Constantina, Brenner, Steven E., Radivojac, Predrag, Moult, John, Repo, Susanna, Hoskins, Roger A., Andreoletti, Gaia, Barsky, Daniel, Chellapan, Ajithavalli, Chu, Hoyin, Dabbiru, Navya, Kollipara, Naveen K., Ly, Melissa, Neumann, Andrew J., Pal, Lipika R., Odell, Eric, Pandey, Gaurav, Peters-Petrulewicz, Robin C., Srinivasan, Rajgopal, Yee, Stephen F., Yeleswarapu, Sri Jyothsna, Zuhl, Maya, Adebali, Ogun, Patra, Ayoti, Beer, Michael A., Hosur, Raghavendra, Peng, Jian, Bernard, Brady M., Berry, Michael, Dong, Shengcheng, Boyle, Alan P., Adhikari, Aashish, Chen, Jingqi, Hu, Zhiqiang, Wang, Robert, Wang, Yaqiong, Miller, Maximilian, Wang, Yanran, Bromberg, Yana, Turina, Paola, Capriotti, Emidio, Han, James J., Ozturk, Kivilcim, Carter, Hannah, Babbi, Giulia, Bovo, Samuele, Di Lena, Pietro, Martelli, Pier Luigi, Savojardo, Castrense, Casadio, Rita, Cline, Melissa S., De Baets, Greet, Bonache, Sandra, Díez, Orland, Gutiérrez-Enríquez, Sara, Fernández, Alejandro, Montalban, Gemma, Ootes, Lars, Özkan, Selen, Padilla, Natàlia, Riera, Casandra, De la Cruz, Xavier, Diekhans, Mark, Huwe, Peter J., Wei, Qiong, Xu, Qifang, Dunbrack, Roland L., Gotea, Valer, Elnitski, Laura, Margolin, Gennady, Fariselli, Piero, Kulakovskiy, Ivan V., Makeev, Vsevolod J., Penzar, Dmitry D., Vorontsov, Ilya E., Favorov, Alexander V., Forman, Julia R., Hasenahuer, Marcia, Fornasari, Maria S., Parisi, Gustavo, Avsec, Ziga, Çelik, Muhammed H., Nguyen, Thi Yen Duong, Gagneur, Julien, Shi, Fang-Yuan, Edwards, Matthew D., Guo, Yuchun, Tian, Kevin, Zeng, Haoyang, Gifford, David K., Göke, Jonathan, Zaucha, Jan, Gough, Julian, Ritchie, Graham R. S., Frankish, Adam, Mudge, Jonathan M., Harrow, Jennifer, Young, Erin L., Yu, Yao, Huff, Chad D., Murakami, Katsuhiko, Nagai, Yoko, Imanishi, Tadashi, Mungall, Christopher J., Jacobsen, Julius O. B., Kim, Dongsup, Jeong, Chan-Seok, Jones, David T., Li, Mulin Jun, Guthrie, Violeta Beleva, Bhattacharya, Rohit, Chen, Yun-Ching, Douville, Christopher, Fan, Jean, Kim, Dewey, Masica, David, Niknafs, Noushin, Sengupta, Sohini, Tokheim, Collin, Turner, Tychele N., Yeo, Hui Ting Grace, Karchin, Rachel, Shin, Sunyoung, Welch, Rene, Keles, Sunduz, Li, Yue, Kellis, Manolis, Corbi-Verge, Carles, Strokach, Alexey V., Kim, Philip M., Klein, Teri E., Mohan, Rahul, Sinnott-Armstrong, Nicholas A., Wainberg, Michael, Kundaje, Anshul, Gonzaludo, Nina, Mak, Angel C. Y., Chhibber, Aparna, Lam, Hugo Y. K., Dahary, Dvir, Fishilevich, Simon, Lancet, Doron, Lee, Insuk, Bachman, Benjamin, Katsonis, Panagiotis, Lua, Rhonald C., Wilson, Stephen J., Lichtarge, Olivier, Bhat, Rajendra R., Sundaram, Laksshman, Viswanath, Vivek, Bellazzi, Riccardo, Nicora, Giovanna, Rizzo, Ettore, Limongelli, Ivan, Mezlini, Aziz M., Chang, Ray, Kim, Serra, Lai, Carmen, O’Connor, Robert, Topper, Scott, van den Akker, Jeroen, Zhou, Alicia Y., Zimmer, Anjali D., Mishne, Gilad, Bergquist, Timothy R., Breese, Marcus R., Guerrero, Rafael F., Jiang, Yuxiang, Kiga, Nikki, Li, Biao, Mort, Matthew, Pagel, Kymberleigh A., Pejaver, Vikas, Stamboulian, Moses H., Thusberg, Janita, Mooney, Sean D., Teerakulkittipong, Nuttinee, Cao, Chen, Kundu, Kunal, Yin, Yizhou, Yu, Chen-Hsin, Kleyman, Michael, Lin, Chiao-Feng, Stackpole, Mary, Mount, Stephen M., Eraslan, Gökcen, Mueller, Nikola S., Naito, Tatsuhiko, Rao, Aliz R., Azaria, Johnathan R., Brodie, Aharon, Ofran, Yanay, Garg, Aditi, Pal, Debnath, Hawkins-Hooker, Alex, Kenlay, Henry, Reid, John, Mucaki, Eliseos J., Rogan, Peter K., Schwarz, Jana M., Searls, David B., Lee, Gyu Rie, Seok, Chaok, Krämer, Andreas, Shah, Sohela, Huang, ChengLai V., Kirsch, Jack F., Shatsky, Maxim, Cao, Yue, Chen, Haoran, Karimi, Mostafa, Moronfoye, Oluwaseyi, Sun, Yuanfei, Shen, Yang, Shigeta, Ron, Ford, Colby T., Nodzak, Conor, Uppal, Aneeta, Shi, Xinghua, Joseph, Thomas, Kotte, Sujatha, Rana, Sadhna, Rao, Aditya, Saipradeep, V. G., Sivadasan, Naveen, Sunderam, Uma, Stanke, Mario, Su, Andrew, Adzhubey, Ivan, Jordan, Daniel M., Sunyaev, Shamil, Rousseau, Frederic, Schymkowitz, Joost, Van Durme, Joost, Tavtigian, Sean V., Carraro, Marco, Giollo, Manuel, Tosatto, Silvio C. E., Adato, Orit, Carmel, Liran, Cohen, Noa E., Fenesh, Tzila, Holtzer, Tamar, Juven-Gershon, Tamar, Unger, Ron, Niroula, Abhishek, Olatubosun, Ayodeji, Väliaho, Jouni, Yang, Yang, Vihinen, Mauno, Wahl, Mary E., Chang, Billy, Chong, Ka Chun, Hu, Inchi, Sun, Rui, Wu, William Ka Kei, Xia, Xiaoxuan, Zee, Benny C., Wang, Maggie H., Wang, Meng, Wu, Chunlei, Lu, Yutong, Chen, Ken, Yang, Yuedong, Yates, Christopher M., Kreimer, Anat, Yan, Zhongxia, Yosef, Nir, Zhao, Huying, Wei, Zhipeng, Yao, Zhaomin, Zhou, Fengfeng, Folkman, Lukas, Zhou, Yaoqi, Daneshjou, Roxana, Altman, Russ B., Inoue, Fumitaka, Ahituv, Nadav, Arkin, Adam P., Lovisa, Federica, Bonvini, Paolo, Bowdin, Sarah, Gianni, Stefano, Mantuano, Elide, Minicozzi, Velia, Novak, Leonore, Pasquo, Alessandra, Pastore, Annalisa, Petrosino, Maria, Puglisi, Rita, Toto, Angelo, Veneziano, Liana, Chiaraluce, Roberta, Ball, Mad P., Bobe, Jason R., Church, George M., Consalvi, Valerio, Cooper, David N., Buckley, Bethany A., Sheridan, Molly B., Cutting, Garry R., Scaini, Maria Chiara, Cygan, Kamil J., Fredericks, Alger M., Glidden, David T., Neil, Christopher, Rhine, Christy L., Fairbrother, William G., Alontaga, Aileen Y., Fenton, Aron W., Matreyek, Kenneth A., Starita, Lea M., Fowler, Douglas M., Löscher, Britt-Sabina, Franke, Andre, Adamson, Scott I., Graveley, Brenton R., Gray, Joe W., Malloy, Mary J., Kane, John P., Kousi, Maria, Katsanis, Nicholas, Schubach, Max, Kircher, Martin, Mak, Angel C. Y., Tang, Paul L. F., Kwok, Pui-Yan, Lathrop, Richard H., Clark, Wyatt T., Yu, Guoying K., LeBowitz, Jonathan H., Benedicenti, Francesco, Bettella, Elisa, Bigoni, Stefania, Cesca, Federica, Mammi, Isabella, Marino-Buslje, Cristina, Milani, Donatella, Peron, Angela, Polli, Roberta, Sartori, Stefano, Stanzial, Franco, Toldo, Irene, Turolla, Licia, Aspromonte, Maria C., Bellini, Mariagrazia, Leonardi, Emanuela, Liu, Xiaoming, Marshall, Christian, McCombie, W. Richard, Elefanti, Lisa, Menin, Chiara, Meyn, M. Stephen, Murgia, Alessandra, Nadeau, Kari C. Y., Neuhausen, Susan L., Nussbaum, Robert L., Pirooznia, Mehdi, Potash, James B., Dimster-Denk, Dago F., Rine, Jasper D., Sanford, Jeremy R., Snyder, Michael, Cote, Atina G., Sun, Song, Verby, Marta W., Weile, Jochen, Roth, Frederick P., Tewhey, Ryan, Sabeti, Pardis C., Campagna, Joan, Refaat, Marwan M., Wojciak, Julianne, Grubb, Soren, Schmitt, Nicole, Shendure, Jay, Spurdle, Amanda B., Stavropoulos, Dimitri J., Walton, Nephi A., Zandi, Peter P., Ziv, Elad, Burke, Wylie, Chen, Flavia, Carr, Lawrence R., Martinez, Selena, Paik, Jodi, Harris-Wai, Julie, Yarborough, Mark, Fullerton, Stephanie M., Koenig, Barbara A., McInnes, Gregory, Shigaki, Dustin, Chandonia, John-Marc, Furutsuki, Mabel, Kasak, Laura, Yu, Changhua, Chen, Rui, Friedberg, Iddo, Getz, Gad A., Cong, Qian, Kinch, Lisa N., Zhang, Jing, Grishin, Nick V., Voskanian, Alin, Kann, Maricel G., Tran, Elizabeth, Ioannidis, Nilah M., Hunter, Jesse M., Udani, Rupa, Cai, Binghuang, Morgan, Alexander A., Sokolov, Artem, Stuart, Joshua M., Minervini, Giovanni, Monzon, Alexander M., Batzoglou, Serafim, Butte, Atul J., Greenblatt, Marc S., Hart, Reece K., Hernandez, Ryan, Hubbard, Tim J. P., Kahn, Scott, O’Donnell-Luria, Anne, Ng, Pauline C., Shon, John, Veltman, Joris, Zook, Justin M. · 2024 · Genome Biology
Cited by 60

The DOI matches its Crossref record; 31 of the 31 references it cites were found in a registry or an index.

Abstract

Background: The Critical Assessment of Genome Interpretation (CAGI) aims to advance the state-of-the-art for computational prediction of genetic variant impact, particularly where relevant to disease. The five complete editions of the CAGI community experiment comprised 50 challenges, in which participants made blind predictions of phenotypes from genetic data, and these were evaluated by independent assessors. Results: Performance was particularly strong for clinical pathogenic variants, including some difficult-to-diagnose cases, and extends to interpretation of cancer-related variants. Missense variant interpretation methods were able to estimate biochemical effects with increasing accuracy. Assessment of methods for regulatory variants and complex trait disease risk was less definitive and indicates performance potentially suitable for auxiliary use in the clinic. Conclusions: Results show that while current methods are imperfect, they have major utility for research and clinical applications. Emerging methods and increasingly large, robust datasets for training and assessment promise further progress ahead.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1186/s13059-023-03113-6

Cites

31 of 31 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 31 references

Research works printed but not linked: 134

  1. Gibbs RA. The human genome project changed everything. Nat Rev Genet. 2020;21(10):575–6. · DOI · Paper page
  2. 1000 Genomes Project Consortium, Abecasis GR, Altshuler D, Auton A, Brooks LD, Durbin RM, Gibbs RA, Hurles ME, McVean GA. A map of human genome variation from population-scale sequencing. Nature. 2010;467(7319):1061–73. · DOI · Paper page
  3. Cutting GR. Cystic fibrosis genetics: from molecular understanding to clinical application. Nat Rev Genet. 2015;16(1):45–56. · DOI · Paper page
  4. Nielsen FC, van Overeem HT, Sorensen CS. Hereditary breast and ovarian cancer: new genes in confined pathways. Nat Rev Cancer. 2016;16(9):599–612. · DOI · Paper page
  5. Wadelius M, Pirmohamed M. Pharmacogenetics of warfarin: current status and future challenges. Pharmacogenomics J. 2007;7(2):99–111. · DOI · Paper page
  6. Katsonis P, Wilhelm K, Williams A, Lichtarge O. Genome interpretation using in silico predictors of variant impact. Hum Genet. 2022;141(10):1549–77. · DOI
  7. Sanavia T, Birolo G, Montanucci L, Turina P, Capriotti E, Fariselli P. Limitations and challenges in protein stability prediction upon genome variations: towards future applications in precision medicine. Comput Struct Biotechnol J. 2020;18:1968–79. · DOI · Paper page
  8. Backwell L, Marsh JA. Diverse molecular mechanisms underlying pathogenic protein mutations: beyond the loss-of-function paradigm. Annu Rev Genomics Hum Genet. 2022;23:475–98. · DOI · Paper page
  9. Riolo G, Cantara S, Ricci C. What’s wrong in a jump? Prediction and validation of splice site variants. Methods Protoc. 2021;4(3):62. · DOI
  10. Ibrahim DM, Mundlos S. Three-dimensional chromatin in disease: what holds us together and what drives us apart? Curr Opin Cell Biol. 2020;64:1–9. · DOI
  11. Hoskins RA, Repo S, Barsky D, Andreoletti G, Moult J, Brenner SE. Reports from CAGI: the critical assessment of genome interpretation. Hum Mutat. 2017;38(9):1039–41. · DOI · Paper page
  12. Andreoletti G, Pal LR, Moult J, Brenner SE. Reports from the fifth edition of CAGI: the Critical Assessment of Genome Interpretation. Hum Mutat. 2019;40(9):11907–1201. · DOI · Paper page
  13. Pejaver V, Babbi G, Casadio R, Folkman L, Katsonis P, Kundu K, Lichtarge O, Martelli PL, Miller M, Moult J, Pal LR, Savojardo C, Yin Y, Zhou Y, Radivojac P, Bromberg Y. Assessment of methods for predicting the effects of PTEN and TPMT protein variants. Hum Mutat. 2019;40(9):1495–506. · DOI
  14. Savojardo C, Petrosino M, Babbi G, Bovo S, Corbi-Verge C, Casadio R, Fariselli P, Folkman L, Garg A, Karimi M, Katsonis P, Kim PM, Lichtarge O, Martelli PL, Pasquo A, Pal D, Shen Y, Strokach AV, Turina P, Zhou Y, Andreoletti G, Brenner SE, Chiaraluce R, Consalvi V, Capriotti E. Evaluating the predictions of the protein stability change upon single amino acid substitutions for the FXN CAGI5 challenge. Hum Mutat. 2019;40(9):1392–9. · DOI
  15. Clark WT, Kasak L, Bakolitsa C, Hu Z, Andreoletti G, Babbi G, Bromberg Y, Casadio R, Dunbrack R, Folkman L, Ford CT, Jones D, Katsonis P, Kundu K, Lichtarge O, Martelli PL, Mooney SD, Nodzak C, Pal LR, Radivojac P, Savojardo C, Shi X, Zhou Y, Uppal A, Xu Q, Yin Y, Pejaver V, Wang M, Wei L, Moult J, Yu GK, Brenner SE, LeBowitz JH. Assessment of predicted enzymatic activity of alpha-N-acetylglucosaminidase variants of unknown significance for CAGI 2016. Hum Mutat. 2019;40(9):1519–29. · DOI
  16. Zhang J, Kinch LN, Cong Q, Weile J, Sun S, Cote AG, Roth FP, Grishin NV. Assessing predictions of fitness effects of missense mutations in SUMO-conjugating enzyme UBE2I. Hum Mutat. 2017;38(9):1051–63. · DOI
  17. Carraro M, Minervini G, Giollo M, Bromberg Y, Capriotti E, Casadio R, Dunbrack R, Elefanti L, Fariselli P, Ferrari C, Gough J, Katsonis P, Leonardi E, Lichtarge O, Menin C, Martelli PL, Niroula A, Pal LR, Repo S, Scaini MC, Vihinen M, Wei Q, Xu Q, Yang Y, Yin Y, Zaucha J, Zhao H, Zhou Y, Brenner SE, Moult J, Tosatto SCE. Performance of in silico tools for the evaluation of p16INK4a (CDKN2A) variants in CAGI. Hum Mutat. 2017;38(9):1042–50. · DOI
  18. Zhang J, Kinch LN, Cong Q, Katsonis P, Lichtarge O, Savojardo C, Babbi G, Martelli PL, Capriotti E, Casadio R, Garg A, Pal D, Weile J, Sun S, Verby M, Roth FP, Grishin NV. Assessing predictions on fitness effects of missense variants in calmodulin. Hum Mutat. 2019;40(9):1463–73. · DOI
  19. Kasak L, Hunter JM, Udani R, Bakolitsa C, Hu Z, Adhikari AN, Babbi G, Casadio R, Gough J, Guerrero RF, Jiang Y, Joseph T, Katsonis P, Kotte S, Kundu K, Lichtarge O, Martelli PL, Mooney SD, Moult J, Pal LR, Poitras J, Radivojac P, Rao A, Sivadasan N, Sunderam U, Saipradeep VG, Yin Y, Zaucha J, Brenner SE, Meyn MS. CAGI SickKids challenges: assessment of phenotype and variant predictions derived from clinical and genomic data of children with undiagnosed diseases. Hum Mutat. 2019;40(9):1373–91. · DOI
  20. Cline MS, Babbi G, Bonache S, Cao Y, Casadio R, de la Cruz X, Diez O, Gutierrez-Enriquez S, Katsonis P, Lai C, Lichtarge O, Martelli PL, Mishne G, Moles-Fernandez A, Montalban G, Mooney SD, O’Conner R, Ootes L, Ozkan S, Padilla N, Pagel KA, Pejaver V, Radivojac P, Riera C, Savojardo C, Shen Y, Sun Y, Topper S, Parsons MT, Spurdle AB, Goldgar DE, ENIGMA Consortium. Assessment of blind predictions of the clinical significance of BRCA1 and BRCA2 variants. Hum Mutat. 2019;40(9):1546–56. · DOI
  21. Carraro M, Monzon AM, Chiricosta L, Reggiani F, Aspromonte MC, Bellini M, Pagel K, Jiang Y, Radivojac P, Kundu K, Pal LR, Yin Y, Limongelli I, Andreoletti G, Moult J, Wilson SJ, Katsonis P, Lichtarge O, Chen J, Wang Y, Hu Z, Brenner SE, Ferrari C, Murgia A, Tosatto SCE, Leonardi E. Assessment of patient clinical descriptions and pathogenic variants from gene panel sequences in the CAGI-5 intellectual disability challenge. Hum Mutat. 2019;40(9):1330–45. · DOI
  22. Daneshjou R, Wang Y, Bromberg Y, Bovo S, Martelli PL, Babbi G, Lena PD, Casadio R, Edwards M, Gifford D, Jones DT, Sundaram L, Bhat RR, Li X, Pal LR, Kundu K, Yin Y, Moult J, Jiang Y, Pejaver V, Pagel KA, Li B, Mooney SD, Radivojac P, Shah S, Carraro M, Gasparini A, Leonardi E, Giollo M, Ferrari C, Tosatto SCE, Bachar E, Azaria JR, Ofran Y, Unger R, Niroula A, Vihinen M, Chang B, Wang MH, Franke A, Petersen BS, Pirooznia M, Zandi P, McCombie R, Potash JB, Altman RB, Klein TE, Hoskins RA, Repo S, Brenner SE, Morgan AA. Working toward precision medicine: predicting phenotypes from exomes in the critical assessment of genome interpretation (CAGI) challenges. Hum Mutat. 2017;38(9):1182–92. · DOI · Paper page
  23. Lek M, Karczewski KJ, Minikel EV, Samocha KE, Banks E, Fennell T, O’Donnell-Luria AH, Ware JS, Hill AJ, Cummings BB, Tukiainen T, Birnbaum DP, Kosmicki JA, Duncan LE, Estrada K, Zhao F, Zou J, Pierce-Hoffman E, Berghout J, Cooper DN, Deflaux N, DePristo M, Do R, Flannick J, Fromer M, Gauthier L, Goldstein J, Gupta N, Howrigan D, Kiezun A, Kurki MI, Moonshine AL, Natarajan P, Orozco L, Peloso GM, Poplin R, Rivas MA, Ruano-Rubio V, Rose SA, Ruderfer DM, Shakir K, Stenson PD, Stevens C, Thomas BP, Tiao G, Tusie-Luna MT, Weisburd B, Won HH, Yu D, Altshuler DM, Ardissino D, Boehnke M, Danesh J, Donnelly S, Elosua R, Florez JC, Gabriel SB, Getz G, Glatt SJ, Hultman CM, Kathiresan S, Laakso M, McCarroll S, McCarthy MI, McGovern D, McPherson R, Neale BM, Palotie A, Purcell SM, Saleheen D, Scharf JM, Sklar P, Sullivan PF, Tuomilehto J, Tsuang MT, Watkins HC, Wilson JG, Daly MJ, MacArthur DG, Exome Aggregation Consortium. Analysis of protein-coding genetic variation in 60,706 humans. Nature. 2016;536(7616):285–91. · DOI · Paper page
  24. Adzhubei IA, Schmidt S, Peshkin L, Ramensky VE, Gerasimova A, Bork P, Kondrashov AS, Sunyaev SR. A method and server for predicting damaging missense mutations. Nat Methods. 2010;7(4):248–9. · DOI · Paper page
  25. Pejaver V, Mooney SD, Radivojac P. Missense variant pathogenicity predictors generalize well across a range of function-specific prediction challenges. Hum Mutat. 2017;38(9):1092–108. · DOI
  26. Bromberg Y, Rost B. SNAP: predict effect of non-synonymous polymorphisms on function. Nucleic Acids Res. 2007;35(11):3823–35. · DOI · Paper page
  27. Carter H, Douville C, Stenson PD, Cooper DN, Karchin R. Identifying mendelian disease genes with the variant effect scoring tool. BMC Genomics. 2013;14(Suppl 3):S3. · DOI · Paper page
  28. Capriotti E, Martelli PL, Fariselli P, Casadio R. Blind prediction of deleterious amino acid variations with SNPs&GO. Hum Mutat. 2017;38(9):1064–71. · DOI
  29. Wang Z, Moult J. SNPs, protein structure, and disease. Hum Mutat. 2001;17(4):263–70. · DOI · Paper page
  30. Lugo-Martinez J, Pejaver V, Pagel KA, Jain S, Mort M, Cooper DN, Mooney SD, Radivojac P. The loss and gain of functional amino acid residues is a common mechanism causing human inherited disease. PLoS Comput Biol. 2016;12(8): e1005091. · DOI
  31. Iqbal S, Li F, Akutsu T, Ascher DB, Webb GI, Song J. Assessing the performance of computational predictors for estimating protein stability changes upon missense mutations. Brief Bioinform. 2021;22(6):bbab184. · DOI · Paper page
  32. Rost B, Radivojac P, Bromberg Y. Protein function in precision medicine: deep understanding with machine learning. FEBS Lett. 2016;590(15):2327–41. · DOI · Paper page
  33. Clark WT, Yu GK, Aoyagi-Scharber M, LeBowitz JH. Utilizing ExAC to assess the hidden contribution of variants of unknown significance to sanfilippo type B incidence. PLoS ONE. 2018;13(7): e0200008. · DOI
  34. Stenson PD, Mort M, Ball EV, Chapman M, Evans K, Azevedo L, Hayden M, Heywood S, Millar DS, Phillips AD, Cooper DN. The human gene mutation database (HGMD®): optimizing its use in a clinical diagnostic or research setting. Hum Genet. 2020;139(10):1197–207. · DOI · Paper page
  35. Liu X, Li C, Mou C, Dong Y, Tu Y. dbNSFP v4: a comprehensive database of transcript-specific functional predictions and annotations for human nonsynonymous and splice-site SNVs. Genome Med. 2020;12(1):103. · DOI · Paper page
  36. Dong C, Wei P, Jian X, Gibbs R, Boerwinkle E, Wang K, Liu X. Comparison and integration of deleteriousness prediction methods for nonsynonymous SNVs in whole exome sequencing studies. Hum Mol Genet. 2015;24(8):2125–37. · DOI · Paper page
  37. Mount SM, Avsec Z, Carmel L, Casadio R, Celik MH, Chen K, Cheng J, Cohen NE, Fairbrother WG, Fenesh T, Gagneur J, Gotea V, Holzer T, Lin CF, Martelli PL, Naito T, Nguyen TYD, Savojardo C, Unger R, Wang R, Yang Y, Zhao H. Assessing predictions of the impact of variants on splicing in CAGI5. Hum Mutat. 2019;40(9):1215–24. · DOI
  38. Kreimer A, Zeng H, Edwards MD, Guo Y, Tian K, Shin S, Welch R, Wainberg M, Mohan R, Sinnott-Armstrong NA, Li Y, Eraslan G, Amin TB, Tewhey R, Sabeti PC, Goke J, Mueller NS, Kellis M, Kundaje A, Beer MA, Keles S, Gifford DK, Yosef N. Predicting gene expression in massively parallel reporter assays: a comparative study. Hum Mutat. 2017;38(9):1240–50. · DOI
  39. Shigaki D, Adato O, Adhikari AN, Dong S, Hawkins-Hooker A, Inoue F, Juven-Gershon T, Kenlay H, Martin B, Patra A, Penzar DD, Schubach M, Xiong C, Yan Z, Boyle AP, Kreimer A, Kulakovskiy IV, Reid J, Unger R, Yosef N, Shendure J, Ahituv N, Kircher M, Beer MA. Integration of multiple epigenomic marks improves prediction of variant impact in saturation mutagenesis reporter assay. Hum Mutat. 2019;40(9):1280–91. · DOI · Paper page
  40. Chandonia JM, Adhikari A, Carraro M, Chhibber A, Cutting GR, Fu Y, Gasparini A, Jones DT, Kramer A, Kundu K, Lam HYK, Leonardi E, Moult J, Pal LR, Searls DB, Shah S, Sunyaev S, Tosatto SCE, Yin Y, Buckley BA. Lessons from the CAGI-4 Hopkins clinical panel challenge. Hum Mutat. 2017;38(9):1155–68. · DOI
  41. Testa U, Testa EP, Mavilio F, Petrini M, Sposi NM, Petti S, Samoggia P, Montesoro E, Giannella G, Bottero L, et al. Differential regulation of transferrin receptor gene expression in human hemopoietic cells: molecular and cellular aspects. J Recept Res. 1987;7(1–4):355–75. · DOI
  42. Pal LR, Yu CH, Mount SM, Moult J. Insights from GWAS: emerging landscape of mechanisms underlying complex trait disease. BMC Genomics. 2015;16(Suppl 8):S4. · DOI
  43. Wand H, Lambert SA, Tamburro C, Iacocca MA, O’Sullivan JW, Sillari C, Kullo IJ, Rowley R, Dron JS, Brockman D, Venner E, McCarthy MI, Antoniou AC, Easton DF, Hegele RA, Khera AV, Chatterjee N, Kooperberg C, Edwards K, Vlessis K, Kinnear K, Danesh JN, Parkinson H, Ramos EM, Roberts MC, Ormond KE, Khoury MJ, Janssens A, Goddard KAB, Kraft P, MacArthur JAL, Inouye M, Wojcik GL. Improving reporting standards for polygenic scores in risk prediction studies. Nature. 2021;591(7849):211–9. · DOI · Paper page
  44. Wellcome Trust Case Control Consortium. Genome-wide association study of 14,000 cases of seven common diseases and 3,000 shared controls. Nature. 2007;447(7145):661–78. · DOI · Paper page
  45. Dahlhamer JM, Zammitti EP, Ward BW, Wheaton AG, Croft JB. Prevalence of inflammatory bowel disease among adults aged ≥18 years - United States, 2015. MMWR Morb Mortal Wkly Rep. 2016;65(42):1166–9. · DOI · Paper page
  46. Chen YC, Douville C, Wang C, Niknafs N, Yeo G, Beleva-Guthrie V, Carter H, Stenson PD, Cooper DN, Li B, Mooney S, Karchin R. A probabilistic model to predict clinical phenotypic traits from genome sequencing. PLoS Comput Biol. 2014;10(9): e1003825. · DOI
  47. Sundaram L, Gao H, Padigepati SR, McRae JF, Li Y, Kosmicki JA, Fritzilas N, Hakenberg J, Dutta A, Shon J, Xu J, Batzoglou S, Li X, Farh KK. Predicting the clinical impact of human mutation with deep neural networks. Nat Genet. 2018;50(8):1161–70. · DOI · Paper page
  48. Wang Y, Miller M, Astrakhan Y, Petersen BS, Schreiber S, Franke A, Bromberg Y. Identifying crohn’s disease signal from variome analysis. Genome Med. 2019;11(1):59. · DOI
  49. Wang RJ, Radivojac P, Hahn MW. Distinct error rates for reference and nonreference genotypes estimated by pedigree analysis. Genetics. 2021;217(1):1–10. · DOI
  50. Cai B, Li B, Kiga N, Thusberg J, Bergquist T, Chen YC, Niknafs N, Carter H, Tokheim C, Beleva-Guthrie V, Douville C, Bhattacharya R, Yeo HTG, Fan J, Sengupta S, Kim D, Cline M, Turner T, Diekhans M, Zaucha J, Pal LR, Cao C, Yu CH, Yin Y, Carraro M, Giollo M, Ferrari C, Leonardi E, Tosatto SCE, Bobe J, Ball M, Hoskins RA, Repo S, Church G, Brenner SE, Moult J, Gough J, Stanke M, Karchin R, Mooney SD. Matching phenotypes to whole genomes: lessons learned from four iterations of the personal genome project community challenges. Hum Mutat. 2017;38(9):1266–76. · DOI
  51. Brenner SE, Chothia C, Hubbard TJ. Assessing sequence comparison methods with reliable structurally identified distant evolutionary relationships. Proc Natl Acad Sci U S A. 1998;95(11):6073–8. · DOI · Paper page
  52. Adams NM, Hand DJ. Comparing classifiers when the misallocation costs are uncertain. Pattern Recognit. 1999;32(7):1139–47. · DOI · Paper page
  53. Wu Y, Li R, Sun S, Weile J, Roth FP. Improved pathogenicity prediction for rare human missense variants. Am J Hum Genet. 2021;108(10):1891–906. · DOI · Paper page
  54. Bournazos AM, Riley LG, Bommireddipalli S, Ades L, Akesson LS, Al-Shinnag M, Alexander SI, Archibald AD, Balasubramaniam S, Berman Y, Beshay V, Boggs K, Bojadzieva J, Brown NJ, Bryen SJ, Buckley MF, Chong B, Davis MR, Dawes R, Delatycki M, Donaldson L, Downie L, Edwards C, Edwards M, Engel A, Ewans LJ, Faiz F, Fennell A, Field M, Freckmann ML, Gallacher L, Gear R, Goel H, Goh S, Goodwin L, Hanna B, Harraway J, Higgins M, Ho G, Hopper BK, Horton AE, Hunter MF, Huq AJ, Josephi-Taylor S, Joshi H, Kirk E, Krzesinski E, Kumar KR, Lemckert F, Leventer RJ, Lindsey-Temple SE, Lunke S, Ma A, Macaskill S, Mallawaarachchi A, Marty M, Marum JE, McCarthy HJ, Menezes MP, McLean A, Milnes D, Mohammad S, Mowat D, Niaz A, Palmer EE, Patel C, Patel SG, Phelan D, Pinner JR, Rajagopalan S, Regan M, Rodgers J, Rodrigues M, Roxburgh RH, Sachdev R, Roscioli T, Samarasekera R, Sandaradura SA, Savva E, Schindler T, Shah M, Sinnerbrink IB, Smith JM, Smith RJ, Springer A, Stark Z, Strom SP, Sue CM, Tan K, Tan TY, Tantsis E, Tchan MC, Thompson BA, Trainer AH, van Spaendonck-Zwarts K, Walsh R, Warwick L, White S, White SM, Williams MG, Wilson MJ, Wong WK, Wright DC, Yap P, Yeung A, Young H, Jones KJ, Bennetts B, Cooper ST, Australasian Consortium for RNA Diagnostics. Standardized practices for RNA diagnostics using clinically accessible specimens reclassifies 75% of putative splicing variants. Genet Med. 2022;24(1):130–45. · DOI
  55. Hastie T, Tibshirani R, Friedman JH. The elements of statistical learning: data mining, inference, and prediction. New York, NY: Springer Verlag; 2001. · DOI · Paper page
  56. Byrne S. A note on the use of empirical AUC for evaluating probabilistic forecasts. Electron J Stat. 2016;10(1):380–93. · doi:10.1214/16-ejs1109
  57. Fawcett T. An introduction to ROC analysis. Pattern Recogn Lett. 2006;27:861–74. · DOI · Paper page
  58. Efron B. Size, power and false discovery rates. Ann Stat. 2007;35(4):1351–77. · DOI · Paper page
  59. Breast Cancer Association Consortium, Dorling L, Carvalho S, Allen J, Gonzalez-Neira A, Luccarini C, Wahlstrom C, Pooley KA, Parsons MT, Fortuno C, Wang Q, Bolla MK, Dennis J, Keeman R, Alonso MR, Alvarez N, Herraez B, Fernandez V, Nunez-Torres R, Osorio A, Valcich J, Li M, Torngren T, Harrington PA, Baynes C, Conroy DM, Decker B, Fachal L, Mavaddat N, Ahearn T, Aittomaki K, Antonenkova NN, Arnold N, Arveux P, Ausems M, Auvinen P, Becher H, Beckmann MW, Behrens S, Bermisheva M, Bialkowska K, Blomqvist C, Bogdanova NV, Bogdanova-Markov N, Bojesen SE, Bonanni B, Borresen-Dale AL, Brauch H, Bremer M, Briceno I, Bruning T, Burwinkel B, Cameron DA, Camp NJ, Campbell A, Carracedo A, Castelao JE, Cessna MH, Chanock SJ, Christiansen H, Collee JM, Cordina-Duverger E, Cornelissen S, Czene K, Dork T, Ekici AB, Engel C, Eriksson M, Fasching PA, Figueroa J, Flyger H, Forsti A, Gabrielson M, Gago-Dominguez M, Georgoulias V, Gil F, Giles GG, Glendon G, Garcia EBG, Alnaes GIG, Guenel P, Hadjisavvas A, Haeberle L, Hahnen E, Hall P, Hamann U, Harkness EF, Hartikainen JM, Hartman M, He W, Heemskerk-Gerritsen BAM, Hillemanns P, Hogervorst FBL, Hollestelle A, Ho WK, Hooning MJ, Howell A, Humphreys K, Idris F, Jakubowska A, Jung A, Kapoor PM, Kerin MJ, Khusnutdinova E, Kim SW, Ko YD, Kosma VM, Kristensen VN, Kyriacou K, Lakeman IMM, Lee JW, Lee MH, Li J, Lindblom A, Lo WY, Loizidou MA, Lophatananon A, Lubinski J, MacInnis RJ, Madsen MJ, Mannermaa A, Manoochehri M, Manoukian S, Margolin S, Martinez ME, Maurer T, Mavroudis D, McLean C, Meindl A, Mensenkamp AR, Michailidou K, Miller N, Mohd Taib NA, Muir K, Mulligan AM, Nevanlinna H, Newman WG, Nordestgaard BG, Ng PS, Oosterwijk JC, Park SK, Park-Simon TW, Perez JIA, Peterlongo P, Porteous DJ, Prajzendanc K, Prokofyeva D, Radice P, Rashid MU, Rhenius V, Rookus MA, Rudiger T, Saloustros E, Sawyer EJ, Schmutzler RK, Schneeweiss A, Schurmann P, Shah M, Sohn C, Southey MC, Surowy H, Suvanto M, Thanasitthichai S, Tomlinson I, Torres D, Truong T, Tzardi M, Valova Y, van Asperen CJ, Van Dam RM, van den Ouweland AMW, van der Kolk LE, van Veen EM, Wendt C, Williams JA, Yang XR, Yoon SY, Zamora MP, Evans DG, de la Hoya M, Simard J, Antoniou AC, Borg A, Andrulis IL, Chang-Claude J, Garcia-Closas M, Chenevix-Trench G, Milne RL, Pharoah PDP, Schmidt MK, Spurdle AB, Vreeswijk MPG, Benitez J, Dunning AM, Kvist A, Teo SH, Devilee P, Easton DF. Breast cancer risk genes - association analysis in more than 113,000 women. N Engl J Med. 2021;384(5):428–39. · DOI · Paper page
  60. Jain S, White M, Radivojac P. Estimating the class prior and posterior from noisy positives and unlabeled data. Advances in Neural Information Processing Systems, 2016; pp. 2693–2701.
  61. Jain S, White M, Trosset MW, Radivojac P. Nonparametric semi-supervised learning of class proportions. arXiv:1601.01944. 2016. · arXiv
  62. Efron B, Tibshirani R. Bootstrap methods for standard errors, confidence intervals, and other measures of statistical accuracy. Stat Sci. 1986;1(1):54–77.
  63. Adhikari AN. Gene-specific features enhance interpretation of mutational impact on acid alpha-glucosidase enzyme activity. Hum Mutat. 2019;40(9):1507–18. · DOI
  64. Kraus JP, Janosik M, Kozich V, Mandell R, Shih V, Sperandeo MP, Sebastio G, de Franchis R, Andria G, Kluijtmans LA, Blom H, Boers GH, Gordon RB, Kamoun P, Tsai MY, Kruger WD, Koch HG, Ohura T, Gaustadnes M. Cystathionine beta-synthase mutations in homocystinuria. Hum Mutat. 1999;13(5):362–75. · DOI · Paper page
  65. Dimster-Denk D, Tripp KW, Marini NJ, Marqusee S, Rine J. Mono and dual cofactor dependence of human cystathionine beta-synthase enzyme variants in vivo and in vitro. G3. 2013;3(10):1619–28. · DOI
  66. Geiss-Friedlander R, Melchior F. Concepts in sumoylation: a decade on. Nat Rev Mol Cell Biol. 2007;8(12):947–56. · DOI · Paper page
  67. Sun S, Yang F, Tan G, Costanzo M, Oughtred R, Hirschman J, Theesfeld CL, Bansal P, Sahni N, Yi S, Yu A, Tyagi T, Tie C, Hill DE, Vidal M, Andrews BJ, Boone C, Dolinski K, Roth FP. An extended set of yeast-based functional assays accurately identifies human disease mutations. Genome Res. 2016;26(5):670–80. · DOI · Paper page
  68. Schulz TJ, Thierbach R, Voigt A, Drewes G, Mietzner B, Steinberg P, Pfeiffer AF, Ristow M. Induction of oxidative metabolism by mitochondrial frataxin inhibits cancer growth: Otto Warburg revisited. J Biol Chem. 2006;281(2):977–81. · DOI · Paper page
  69. Guccini I, Serio D, Condo I, Rufini A, Tomassini B, Mangiola A, Maira G, Anile C, Fina D, Pallone F, Mongiardi MP, Levi A, Ventura N, Testi R, Malisan F. Frataxin participates to the hypoxia-induced response in tumors. Cell Death Dis. 2011;2: e123. · DOI
  70. Li B, Krishnan VG, Mort ME, Xin F, Kamati KK, Cooper DN, Mooney SD, Radivojac P. Automated inference of molecular mechanisms of disease from amino acid substitutions. Bioinformatics. 2009;25(21):2744–50. · DOI · Paper page
  71. Goldgar DE, Easton DF, Byrnes GB, Spurdle AB, Iversen ES, Greenblatt MS, IARC Unclassified Genetic Variants Working Group. Genetic evidence and integration of various data sources for classifying uncertain variants into a single model. Hum Mutat. 2008;29(11):1265–72. · DOI · Paper page
  72. Plon SE, Eccles DM, Easton D, Foulkes WD, Genuardi M, Greenblatt MS, Hogervorst FB, Hoogerbrugge N, Spurdle AB, Tavtigian SV, IARC Unclassified Genetic Variants Working Group. Sequence variant classification and reporting: recommendations for improving the interpretation of cancer susceptibility genetic test results. Hum Mutat. 2008;29(11):1282–91. · DOI · Paper page
  73. Parsons MT, Tudini E, Li H, Hahnen E, Wappenschmidt B, Feliubadalo L, Aalfs CM, Agata S, Aittomaki K, Alducci E, Alonso-Cerezo MC, Arnold N, Auber B, Austin R, Azzollini J, Balmana J, Barbieri E, Bartram CR, Blanco A, Blumcke B, Bonache S, Bonanni B, Borg A, Bortesi B, Brunet J, Bruzzone C, Bucksch K, Cagnoli G, Caldes T, Caliebe A, Caligo MA, Calvello M, Capone GL, Caputo SM, Carnevali I, Carrasco E, Caux-Moncoutier V, Cavalli P, Cini G, Clarke EM, Concolino P, Cops EJ, Cortesi L, Couch FJ, Darder E, de la Hoya M, Dean M, Debatin I, Del Valle J, Delnatte C, Derive N, Diez O, Ditsch N, Domchek SM, Dutrannoy V, Eccles DM, Ehrencrona H, Enders U, Evans DG, Farra C, Faust U, Felbor U, Feroce I, Fine M, Foulkes WD, Galvao HCR, Gambino G, Gehrig A, Gensini F, Gerdes AM, Germani A, Giesecke J, Gismondi V, Gomez C, Gomez Garcia EB, Gonzalez S, Grau E, Grill S, Gross E, Guerrieri-Gonzaga A, Guillaud-Bataille M, Gutierrez-Enriquez S, Haaf T, Hackmann K, Hansen TVO, Harris M, Hauke J, Heinrich T, Hellebrand H, Herold KN, Honisch E, Horvath J, Houdayer C, Hubbel V, Iglesias S, Izquierdo A, James PA, Janssen LAM, Jeschke U, Kaulfuss S, Keupp K, Kiechle M, Kolbl A, Krieger S, Kruse TA, Kvist A, Lalloo F, Larsen M, Lattimore VL, Lautrup C, Ledig S, Leinert E, Lewis AL, Lim J, Loeffler M, Lopez-Fernandez A, Lucci-Cordisco E, Maass N, Manoukian S, Marabelli M, Matricardi L, Meindl A, Michelli RD, Moghadasi S, Moles-Fernandez A, Montagna M, Montalban G, Monteiro AN, Montes E, Mori L, Moserle L, Muller CR, Mundhenke C, Naldi N, Nathanson KL, Navarro M, Nevanlinna H, Nichols CB, Niederacher D, Nielsen HR, Ong KR, Pachter N, Palmero EI, Papi L, Pedersen IS, Peissel B, Perez-Segura P, Pfeifer K, Pineda M, Pohl-Rescigno E, Poplawski NK, Porfirio B, Quante AS, Ramser J, Reis RM, Revillion F, Rhiem K, Riboli B, Ritter J, Rivera D, Rofes P, Rump A, Salinas M, Sanchez de Abajo AM, Schmidt G, Schoenwiese U, Seggewiss J, Solanes A, Steinemann D, Stiller M, Stoppa-Lyonnet D, Sullivan KJ, Susman R, Sutter C, Tavtigian SV, Teo SH, Teule A, Thomassen M, Tibiletti MG, Tischkowitz M, Tognazzo S, Toland AE, Tornero E, Torngren T, Torres-Esquius S, Toss A, Trainer AH, Tucker KM, van Asperen CJ, van Mackelenbergh MT, Varesco L, Vargas-Parra G, Varon R, Vega A, Velasco A, Vesper AS, Viel A, Vreeswijk MPG, Wagner SA, Waha A, Walker LC, Walters RJ, Wang-Gohrke S, Weber BHF, Weichert W, Wieland K, Wiesmuller L, Witzel I, Wockel A, Woodward ER, Zachariae S, Zampiga V, Zeder-Goss C, Investigators KC, Lazaro C, De Nicolo A, Radice P, Engel C, Schmutzler RK, Goldgar DE, Spurdle AB. Large scale multifactorial likelihood quantitative analysis of BRCA1 and BRCA2 variants: an ENIGMA resource to support clinical variant classification. Hum Mutat. 2019;40(9):1557–78. · DOI
  74. Lai C, Zimmer AD, O’Connor R, Kim S, Chan R, van den Akker J, Zhou AY, Topper S, Mishne G. LEAP: using machine learning to support variant classification in a clinical setting. Hum Mutat. 2020;41(6):1079–90. · DOI
  75. Liu X, Jian X, Boerwinkle E. dbNSFP: a lightweight database of human nonsynonymous SNPs and their functional predictions. Hum Mutat. 2011;32(8):894–9. · DOI · Paper page
  76. Liu X, Jian X, Boerwinkle E. dbNSFP v2.0: a database of human non-synonymous SNVs and their functional predictions and annotations. Hum Mutat. 2013;34(9):E2393–2402. · DOI · Paper page
  77. Liu X, Wu C, Li C, Boerwinkle E. dbNSFP v30: a one-stop database of functional predictions and annotations for human nonsynonymous and splice-site SNVs. Hum Mutat. 2016;37(3):235–41. · DOI · Paper page
  78. Kircher M, Xiong C, Martin B, Schubach M, Inoue F, Bell RJA, Costello JF, Shendure J, Ahituv N. Saturation mutagenesis of twenty disease-associated regulatory elements at single base-pair resolution. Nat Commun. 2019;10(1):3583. · DOI · Paper page
  79. Halme L, Paavola-Sakki P, Turunen U, Lappalainen M, Farkkila M, Kontula K. Family and twin studies in inflammatory bowel disease. World J Gastroenterol. 2006;12(23):3668–72. · DOI · Paper page
  80. Jostins L, Ripke S, Weersma RK, Duerr RH, McGovern DP, Hui KY, Lee JC, Schumm LP, Sharma Y, Anderson CA, Essers J, Mitrovic M, Ning K, Cleynen I, Theatre E, Spain SL, et al. Host-microbe interactions have shaped the genetic architecture of inflammatory bowel disease. Nature. 2012;491(7422):119–24. · DOI · Paper page
  81. Uhlig HH, Schwerd T, Koletzko S, Shah N, Kammermeier J, Elkadri A, Ouahed J, Wilson DC, Travis SP, Turner D, Klein C, Snapper SB, Muise AM, Group CiIS, Neopics. The diagnostic approach to monogenic very early onset inflammatory bowel disease. Gastroenterology. 2014;147(5):990–1007 e1003. · DOI · Paper page
  82. Ellinghaus D, Zhang H, Zeissig S, Lipinski S, Till A, Jiang T, Stade B, Bromberg Y, Ellinghaus E, Keller A, Rivas MA, Skieceviciene J, Doncheva NT, Liu X, Liu Q, Jiang F, Forster M, Mayr G, Albrecht M, Hasler R, Boehm BO, Goodall J, Berzuini CR, Lee J, Andersen V, Vogel U, Kupcinskas L, Kayser M, Krawczak M, Nikolaus S, Weersma RK, Ponsioen CY, Sans M, Wijmenga C, Strachan DP, McArdle WL, Vermeire S, Rutgeerts P, Sanderson JD, Mathew CG, Vatn MH, Wang J, Nothen MM, Duerr RH, Buning C, Brand S, Glas J, Winkelmann J, Illig T, Latiano A, Annese V, Halfvarson J, D’Amato M, Daly MJ, Nothnagel M, Karlsen TH, Subramani S, Rosenstiel P, Schreiber S, Parkes M, Franke A. Association between variants of PRDM1 and NDP52 and crohn’s disease, based on exome sequencing and functional studies. Gastroenterology. 2013;145(2):339–47. · DOI · Paper page
  83. Voskanian A, Katsonis P, Lichtarge O, Pejaver V, Radivojac P, Mooney SD, Capriotti E, Bromberg Y, Wang Y, Miller M, Martelli PL, Savojardo C, Babbi G, Casadio R, Cao Y, Sun Y, Shen Y, Garg A, Pal D, Yu Y, Huff CD, Tavtigian SV, Young E, Neuhausen SL, Ziv E, Pal LR, Andreoletti G, Brenner SE, Kann MG. Assessing the performance of in silico methods for predicting the pathogenicity of variants in the gene CHEK2, among hispanic females with breast cancer. Hum Mutat. 2019;40(9):1612–22. · DOI
  84. Zakai NA, McClure LA. Racial differences in venous thromboembolism. J Thromb Haemost. 2011;9(10):1877–82. · DOI · Paper page
  85. Feero WG. Genetic thrombophilia. Prim Care. 2004;31(3):685–709. · DOI
  86. McInnes G, Daneshjou R, Katsonis P, Lichtarge O, Srinivasan R, Rana S, Radivojac P, Mooney SD, Pagel KA, Stamboulian M, Jiang Y, Capriotti E, Wang Y, Bromberg Y, Bovo S, Savojardo C, Martelli PL, Casadio R, Pal LR, Moult J, Brenner SE, Altman R. Predicting venous thromboembolism risk from exomes in the critical assessment of genome interpretation (CAGI) challenges. Hum Mutat. 2019;40(9):1314–20. · DOI
  87. Canela-Xandri O, Rawlik K, Tenesa A. An atlas of genetic associations in UK biobank. Nat Genet. 2018;50(11):1593–9. · DOI · Paper page
  88. Soria JM, Morange PE, Vila J, Souto JC, Moyano M, Tregouet DA, Mateo J, Saut N, Salas E, Elosua R. Multilocus genetic risk scores for venous thromboembolism risk assessment. J Am Heart Assoc. 2014;3(5): e001060. · DOI
  89. Fairfield H, Gilbert GJ, Barter M, Corrigan RR, Curtain M, Ding Y, D’Ascenzo M, Gerhardt DJ, He C, Huang W, Richmond T, Rowe L, Probst FJ, Bergstrom DE, Murray SA, Bult C, Richardson J, Kile BT, Gut I, Hager J, Sigurdsson S, Mauceli E, Di Palma F, Lindblad-Toh K, Cunningham ML, Cox TC, Justice MJ, Spector MS, Lowe SW, Albert T, Donahue LR, Jeddeloh J, Shendure J, Reinholdt LG. Mutation discovery in mice by whole exome sequencing. Genome Biol. 2011;12(9):R86. · DOI · Paper page
  90. Calabrese R, Capriotti E, Fariselli P, Martelli PL, Casadio R. Functional annotations improve the predictive score of human disease-related mutations in proteins. Hum Mutat. 2009;30(8):1237–44. · DOI · Paper page
  91. Deutschbauer A, Price MN, Wetmore KM, Shao W, Baumohl JK, Xu Z, Nguyen M, Tamse R, Davis RW, Arkin AP. Evidence-based annotation of gene function in shewanella oneidensis MR-1 using genome-wide fitness profiling across 121 conditions. PLoS Genet. 2011;7(11): e1002385. · DOI · Paper page
  92. Lai R, Ingham RJ. The pathobiology of the oncogenic tyrosine kinase NPM-ALK: a brief update. Ther Adv Hematol. 2013;4(2):119–31. · DOI
  93. Lu L, Ghose AK, Quail MR, Albom MS, Durkin JT, Holskin BP, Angeles TS, Meyer SL, Ruggeri BA, Cheng M. ALK mutants in the kinase domain exhibit altered kinase activity and differential sensitivity to small molecule ALK inhibitors. Biochemistry. 2009;48(16):3600–9. · DOI
  94. Larsen CC, Karaviti LP, Seghers V, Weiss RE, Refetoff S, Dumitrescu AM. A new family with an activating mutation (G431S) in the TSH receptor gene: a phenotype discussion and review of the literature. Int J Pediatr Endocrinol. 2014;2014(1):23. · DOI
  95. Robinson PN, Mundlos S. The human phenotype ontology. Clin Genet. 2010;77(6):525–34. · DOI · Paper page
  96. Pal LR, Kundu K, Yin Y, Moult J. CAGI4 SickKids clinical genomes challenge: a pipeline for identifying pathogenic variants. Hum Mutat. 2017;38(9):1169–81. · DOI
  97. Budnitz DS, Lovegrove MC, Shehab N, Richards CL. Emergency hospitalizations for adverse drug events in older americans. N Engl J Med. 2011;365(21):2002–12. · DOI · Paper page
  98. International Warfarin Pharmacogenetics Consortium, Klein TE, Altman RB, Eriksson N, Gage BF, Kimmel SE, Lee MT, Limdi NA, Page D, Roden DM, Wagner MJ, Caldwell MD, Johnson JA. Estimation of the warfarin dose with clinical and pharmacogenetic data. N Engl J Med. 2009;360(8):753–64. · DOI · Paper page
  99. Daneshjou R, Klein TE, Altman RB. Genotype-guided dosing of vitamin K antagonists. N Engl J Med. 2014;370(18):1762–3.
  100. Sundaram L, Bhat RR, Viswanath V, Li X. DeepBipolar: identifying genomic mutations for bipolar disorder via deep learning. Hum Mutat. 2017;38(9):1217–24. · DOI
  101. Wang MH, Chang B, Sun R, Hu I, Xia X, Wu WKK, Chong KC, Zee BC. Stratified polygenic risk prediction model with application to CAGI bipolar disorder sequencing data. Hum Mutat. 2017;38(9):1235–9. · doi:10.1002/humu.23229
  102. Niroula A, Vihinen M. PON-P and PON-P2 predictor performance in CAGI challenges: lessons learned. Hum Mutat. 2017;38(9):1085–91. · DOI
  103. Katsonis P, Lichtarge O. CAGI5: objective performance assessments of predictions based on the evolutionary action equation. Hum Mutat. 2019;40(9):1436–54. · DOI
  104. Garg A, Pal D. Exploring the use of molecular dynamics in assessing protein variants for phenotypic alterations. Hum Mutat. 2019;40(9):1424–35. · DOI
  105. Kasak L, Bakolitsa C, Hu Z, Yu C, Rine J, Dimster-Denk DF, Pandey G, De Baets G, Bromberg Y, Cao C, Capriotti E, Casadio R, Van Durme J, Giollo M, Karchin R, Katsonis P, Leonardi E, Lichtarge O, Martelli PL, Masica D, Mooney SD, Olatubosun A, Radivojac P, Rousseau F, Pal LR, Savojardo C, Schymkowitz J, Thusberg J, Tosatto SCE, Vihinen M, Valiaho J, Repo S, Moult J, Brenner SE, Friedberg I. Assessing computational predictions of the phenotypic effect of cystathionine-beta-synthase variants. Hum Mutat. 2019;40(9):1530–45. · DOI
  106. Katsonis P, Lichtarge O. Objective assessment of the evolutionary action equation for the fitness effect of missense mutations across CAGI-blinded contests. Hum Mutat. 2017;38(9):1072–84. · DOI
  107. Savojardo C, Babbi G, Bovo S, Capriotti E, Martelli PL, Casadio R. Are machine learning based methods suited to address complex biological problems? lessons from CAGI-5 challenges. Hum Mutat. 2019;40(9):1455–62. · DOI
  108. Wang Y, Bromberg Y. Identifying mutation-driven changes in gene functionality that lead to venous thromboembolism. Hum Mutat. 2019;40(9):1321–9. · DOI
  109. Giollo M, Jones DT, Carraro M, Leonardi E, Ferrari C, Tosatto SCE. Crohn disease risk prediction-best practices and pitfalls with exome data. Hum Mutat. 2017;38(9):1193–200. · DOI
  110. Pal LR, Kundu K, Yin Y, Moult J. CAGI4 crohn’s exome challenge: marker SNP versus exome variant models for assigning risk of crohn disease. Hum Mutat. 2017;38(9):1225–34. · DOI
  111. Cao Y, Sun Y, Karimi M, Chen H, Moronfoye O, Shen Y. Predicting pathogenicity of missense variants with weakly supervised regression. Hum Mutat. 2019;40(9):1579–92. · DOI
  112. Padilla N, Moles-Fernandez A, Riera C, Montalban G, Ozkan S, Ootes L, Bonache S, Diez O, Gutierrez-Enriquez S, de la Cruz X. BRCA1- and BRCA2-specific in silico tools for variant interpretation in the CAGI 5 ENIGMA challenge. Hum Mutat. 2019;40(9):1593–611. · DOI
  113. Zeng H, Edwards MD, Guo Y, Gifford DK. Accurate eQTL prioritization with an ensemble-based framework. Hum Mutat. 2017;38(9):1259–65. · DOI
  114. Beer MA. Predicting enhancer activity and variant impact using gkm-SVM. Hum Mutat. 2017;38(9):1251–8. · DOI
  115. Strokach A, Corbi-Verge C, Kim PM. Predicting changes in protein stability caused by mutation using sequence-and structure-based methods in a CAGI5 blind challenge. Hum Mutat. 2019;40(9):1414–23. · DOI
  116. Petrosino M, Pasquo A, Novak L, Toto A, Gianni S, Mantuano E, Veneziano L, Minicozzi V, Pastore A, Puglisi R, Capriotti E, Chiaraluce R, Consalvi V. Characterization of human frataxin missense variants in cancer tissues. Hum Mutat. 2019;40(9):1400–13. · DOI
  117. Kundu K, Pal LR, Yin Y, Moult J. Determination of disease phenotypes and pathogenic variants from exome sequence data in the CAGI 4 gene panel challenge. Hum Mutat. 2017;38(9):1201–16. · DOI
  118. Aspromonte MC, Bellini M, Gasparini A, Carraro M, Bettella E, Polli R, Cesca F, Bigoni S, Boni S, Carlet O, Negrin S, Mammi I, Milani D, Peron A, Sartori S, Toldo I, Soli F, Turolla L, Stanzial F, Benedicenti F, Marino-Buslje C, Tosatto SCE, Murgia A, Leonardi E. Characterization of intellectual disability and autism comorbidity through gene panel sequencing. Hum Mutat. 2019;40(9):1346–63. · DOI
  119. Chen J. A fully-automated event-based variant prioritizing solution to the CAGI5 intellectual disability gene panel challenge. Hum Mutat. 2019;40(9):1364–72. · DOI
  120. Rhine CL, Neil C, Glidden DT, Cygan KJ, Fredericks AM, Wang J, Walton NA, Fairbrother WG. Future directions for high-throughput splicing assays in precision medicine. Hum Mutat. 2019;40(9):1225–34. · DOI
  121. Cheng J, Celik MH, Nguyen TYD, Avsec Z, Gagneur J. CAGI 5 splicing challenge: improved exon skipping and intron retention predictions with MMSplice. Hum Mutat. 2019;40(9):1243–51. · DOI
  122. Naito T. Predicting the impact of single nucleotide variants on splicing via sequence-based deep neural networks and genomic features. Hum Mutat. 2019;40(9):1261–9. · DOI
  123. Yin Y, Kundu K, Pal LR, Moult J. Ensemble variant interpretation methods to predict enzyme activity and assign pathogenicity in the CAGI4 NAGLU (human N-acetyl-glucosaminidase) and UBE2I (human SUMO-ligase) challenges. Hum Mutat. 2017;38(9):1109–22. · DOI
  124. Monzon AM, Carraro M, Chiricosta L, Reggiani F, Han J, Ozturk K, Wang Y, Miller M, Bromberg Y, Capriotti E, Savojardo C, Babbi G, Martelli PL, Casadio R, Katsonis P, Lichtarge O, Carter H, Kousi M, Katsanis N, Andreoletti G, Moult J, Brenner SE, Ferrari C, Leonardi E, Tosatto SCE. Performance of computational methods for the evaluation of pericentriolar material 1 missense variants in CAGI-5. Hum Mutat. 2019;40(9):1474–85. · DOI
  125. Miller M, Wang Y, Bromberg Y. What went wrong with variant effect predictor performance for the PCM1 challenge. Hum Mutat. 2019;40(9):1486–94. · DOI
  126. Tang Q, Fenton AW. Whole-protein alanine-scanning mutagenesis of allostery: a large percentage of a protein can contribute to mechanism. Hum Mutat. 2017;38(9):1132–43. · DOI
  127. Tang Q, Alontaga AY, Holyoak T, Fenton AW. Exploring the limits of the usefulness of mutagenesis in studies of allosteric mechanisms. Hum Mutat. 2017;38(9):1144–54. · DOI
  128. Xu Q, Tang Q, Katsonis P, Lichtarge O, Jones D, Bovo S, Babbi G, Martelli PL, Casadio R, Lee GR, Seok C, Fenton AW, Dunbrack RL Jr. Benchmarking predictions of allostery in liver pyruvate kinase in CAGI4. Hum Mutat. 2017;38(9):1123–31. · DOI
  129. Dong S, Boyle AP. Predicting functional variants in enhancer and promoter elements using RegulomeDB. Hum Mutat. 2019;40(9):1292–8. · DOI
  130. Kreimer A, Yan Z, Ahituv N, Yosef N. Meta-analysis of massively parallel reporter assays enables prediction of regulatory function across cell types. Hum Mutat. 2019;40(9):1299–313. · DOI
  131. Pal LR, Kundu K, Yin Y, Moult J. Matching whole genomes to rare genetic disorders: identification of potential causative variants using phenotype-weighted knowledge in the CAGI SickKids5 clinical genomes challenge. Hum Mutat. 2020;41(2):347–62. · DOI
  132. Gotea V, Margolin G, Elnitski L. CAGI experiments: modeling sequence variant impact on gene splicing using predictions from computational tools. Hum Mutat. 2019;40(9):1252–60. · DOI
  133. Wang R, Wang Y, Hu Z. Using secondary structure to predict the effects of genetic variants on alternative splicing. Hum Mutat. 2019;40(9):1270–9. · DOI
  134. Chen K, Lu Y, Zhao H, Yang Y. Predicting the change of exon splicing caused by genetic variant using support vector regression. Hum Mutat. 2019;40(9):1235–42. · DOI

Cited by

See all 60

Metadata

Authors
The Critical Assessment of Genome Interpretation Consortium, Jain, S., Bakolitsa, C., Brenner, S. E., Radivojac, P., Moult, J., Repo, S., Hoskins, R. A., Andreoletti, G., Barsky, D., Chellapan, A., Chu, H., Dabbiru, N., Kollipara, N. K., Ly, M., Neumann, A. J., Pal, L. R., Odell, E., Pandey, G., Peters-Petrulewicz, R. C., Srinivasan, R., Yee, S. F., Yeleswarapu, S. J., Zuhl, M., Adebali, O., Patra, A., Beer, M. A., Hosur, R., Peng, J., Bernard, B. M., Berry, M., Dong, S., Boyle, A. P., Adhikari, A., Chen, J., Hu, Z., Wang, R., Wang, Y., Miller, M., Wang, Y., Bromberg, Y., Turina, P., Capriotti, E., Han, J. J., Ozturk, K., Carter, H., Babbi, G., Bovo, S., Di Lena, P., Martelli, P. L., Savojardo, C., Casadio, R., Cline, M. S., De Baets, G., Bonache, S., Díez, O., Gutiérrez-Enríquez, S., Fernández, A., Montalban, G., Ootes, L., Özkan, S., Padilla, N., Riera, C., De la Cruz, X., Diekhans, M., Huwe, P. J., Wei, Q., Xu, Q., Dunbrack, R. L., Gotea, V., Elnitski, L., Margolin, G., Fariselli, P., Kulakovskiy, I. V., Makeev, V. J., Penzar, D. D., Vorontsov, I. E., Favorov, A. V., Forman, J. R., Hasenahuer, M., Fornasari, M. S., Parisi, G., Avsec, Z., Çelik, M. H., Nguyen, T. Y. D., Gagneur, J., Shi, F.-Y., Edwards, M. D., Guo, Y., Tian, K., Zeng, H., Gifford, D. K., Göke, J., Zaucha, J., Gough, J., Ritchie, G. R. S., Frankish, A., Mudge, J. M., Harrow, J., Young, E. L., Yu, Y., Huff, C. D., Murakami, K., Nagai, Y., Imanishi, T., Mungall, C. J., Jacobsen, J. O. B., Kim, D., Jeong, C.-S., Jones, D. T., Li, M. J., Guthrie, V. B., Bhattacharya, R., Chen, Y.-C., Douville, C., Fan, J., Kim, D., Masica, D., Niknafs, N., Sengupta, S., Tokheim, C., Turner, T. N., Yeo, H. T. G., Karchin, R., Shin, S., Welch, R., Keles, S., Li, Y., Kellis, M., Corbi-Verge, C., Strokach, A. V., Kim, P. M., Klein, T. E., Mohan, R., Sinnott-Armstrong, N. A., Wainberg, M., Kundaje, A., Gonzaludo, N., Mak, A. C. Y., Chhibber, A., Lam, H. Y. K., Dahary, D., Fishilevich, S., Lancet, D., Lee, I., Bachman, B., Katsonis, P., Lua, R. C., Wilson, S. J., Lichtarge, O., Bhat, R. R., Sundaram, L., Viswanath, V., Bellazzi, R., Nicora, G., Rizzo, E., Limongelli, I., Mezlini, A. M., Chang, R., Kim, S., Lai, C., O’Connor, R., Topper, S., van den Akker, J., Zhou, A. Y., Zimmer, A. D., Mishne, G., Bergquist, T. R., Breese, M. R., Guerrero, R. F., Jiang, Y., Kiga, N., Li, B., Mort, M., Pagel, K. A., Pejaver, V., Stamboulian, M. H., Thusberg, J., Mooney, S. D., Teerakulkittipong, N., Cao, C., Kundu, K., Yin, Y., Yu, C.-H., Kleyman, M., Lin, C.-F., Stackpole, M., Mount, S. M., Eraslan, G., Mueller, N. S., Naito, T., Rao, A. R., Azaria, J. R., Brodie, A., Ofran, Y., Garg, A., Pal, D., Hawkins-Hooker, A., Kenlay, H., Reid, J., Mucaki, E. J., Rogan, P. K., Schwarz, J. M., Searls, D. B., Lee, G. R., Seok, C., Krämer, A., Shah, S., Huang, C. V., Kirsch, J. F., Shatsky, M., Cao, Y., Chen, H., Karimi, M., Moronfoye, O., Sun, Y., Shen, Y., Shigeta, R., Ford, C. T., Nodzak, C., Uppal, A., Shi, X., Joseph, T., Kotte, S., Rana, S., Rao, A., Saipradeep, V. G., Sivadasan, N., Sunderam, U., Stanke, M., Su, A., Adzhubey, I., Jordan, D. M., Sunyaev, S., Rousseau, F., Schymkowitz, J., Van Durme, J., Tavtigian, S. V., Carraro, M., Giollo, M., Tosatto, S. C. E., Adato, O., Carmel, L., Cohen, N. E., Fenesh, T., Holtzer, T., Juven-Gershon, T., Unger, R., Niroula, A., Olatubosun, A., Väliaho, J., Yang, Y., Vihinen, M., Wahl, M. E., Chang, B., Chong, K. C., Hu, I., Sun, R., Wu, W. K. K., Xia, X., Zee, B. C., Wang, M. H., Wang, M., Wu, C., Lu, Y., Chen, K., Yang, Y., Yates, C. M., Kreimer, A., Yan, Z., Yosef, N., Zhao, H., Wei, Z., Yao, Z., Zhou, F., Folkman, L., Zhou, Y., Daneshjou, R., Altman, R. B., Inoue, F., Ahituv, N., Arkin, A. P., Lovisa, F., Bonvini, P., Bowdin, S., Gianni, S., Mantuano, E., Minicozzi, V., Novak, L., Pasquo, A., Pastore, A., Petrosino, M., Puglisi, R., Toto, A., Veneziano, L., Chiaraluce, R., Ball, M. P., Bobe, J. R., Church, G. M., Consalvi, V., Cooper, D. N., Buckley, B. A., Sheridan, M. B., Cutting, G. R., Scaini, M. C., Cygan, K. J., Fredericks, A. M., Glidden, D. T., Neil, C., Rhine, C. L., Fairbrother, W. G., Alontaga, A. Y., Fenton, A. W., Matreyek, K. A., Starita, L. M., Fowler, D. M., Löscher, B.-S., Franke, A., Adamson, S. I., Graveley, B. R., Gray, J. W., Malloy, M. J., Kane, J. P., Kousi, M., Katsanis, N., Schubach, M., Kircher, M., Mak, A. C. Y., Tang, P. L. F., Kwok, P.-Y., Lathrop, R. H., Clark, W. T., Yu, G. K., LeBowitz, J. H., Benedicenti, F., Bettella, E., Bigoni, S., Cesca, F., Mammi, I., Marino-Buslje, C., Milani, D., Peron, A., Polli, R., Sartori, S., Stanzial, F., Toldo, I., Turolla, L., Aspromonte, M. C., Bellini, M., Leonardi, E., Liu, X., Marshall, C., McCombie, W. R., Elefanti, L., Menin, C., Meyn, M. S., Murgia, A., Nadeau, K. C. Y., Neuhausen, S. L., Nussbaum, R. L., Pirooznia, M., Potash, J. B., Dimster-Denk, D. F., Rine, J. D., Sanford, J. R., Snyder, M., Cote, A. G., Sun, S., Verby, M. W., Weile, J., Roth, F. P., Tewhey, R., Sabeti, P. C., Campagna, J., Refaat, M. M., Wojciak, J., Grubb, S., Schmitt, N., Shendure, J., Spurdle, A. B., Stavropoulos, D. J., Walton, N. A., Zandi, P. P., Ziv, E., Burke, W., Chen, F., Carr, L. R., Martinez, S., Paik, J., Harris-Wai, J., Yarborough, M., Fullerton, S. M., Koenig, B. A., McInnes, G., Shigaki, D., Chandonia, J.-M., Furutsuki, M., Kasak, L., Yu, C., Chen, R., Friedberg, I., Getz, G. A., Cong, Q., Kinch, L. N., Zhang, J., Grishin, N. V., Voskanian, A., Kann, M. G., Tran, E., Ioannidis, N. M., Hunter, J. M., Udani, R., Cai, B., Morgan, A. A., Sokolov, A., Stuart, J. M., Minervini, G., Monzon, A. M., Batzoglou, S., Butte, A. J., Greenblatt, M. S., Hart, R. K., Hernandez, R., Hubbard, T. J. P., Kahn, S., O’Donnell-Luria, A., Ng, P. C., Shon, J., Veltman, J., & Zook, J. M.
Venue
Genome Biology
Published
2024-02-22
Volume
25
Issue
1
Pages
53-53
ISSN
  • 1465-6906 (ISSN-L)
  • 1474-760X
OpenAlex
W4392044215
Cited by
60
References
31
FWCI
14.03
Type
journal article
Language
en
Open access
gold
Licence
Funders
NHGRI (U24 HG007346)

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: CAGI, the Critical Assessment of Genome Interpretation, establishes progress and prospects for computational genetic variant interpretation methods. https://app.cochat.ai/papers/10.1186/s13059-023-03113-6
The paper
The Critical Assessment of Genome Interpretation Consortium, Jain, S., Bakolitsa, C., Brenner, S. E., Radivojac, P., Moult, J., Repo, S., Hoskins, R. A., Andreoletti, G., Barsky, D., Chellapan, A., Chu, H., Dabbiru, N., Kollipara, N. K., Ly, M., Neumann, A. J., Pal, L. R., Odell, E., Pandey, G., . . . Zook, J. M. (2024). CAGI, the Critical Assessment of Genome Interpretation, establishes progress and prospects for computational genetic variant interpretation methods. Genome Biology. https://doi.org/10.1186/s13059-023-03113-6

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.