CoChat

Reports from CAGI: The Critical Assessment of Genome Interpretation

Open access
Hoskins, Roger A, Repo, Susanna, Barsky, Daniel, Andreoletti, Gaia, Moult, John, Brenner, Steven E. · 2017 · Human Mutation
Cited by 36

The DOI matches its Crossref record.

Abstract

New genomic and large-scale data hold the promise of revolutionizing our understanding and treatment of human disease, and are already influencing clinical practice. Multiple barriers stand between the acquisition of the data and fully realizing these and other benefits. In particular, we need powerful and well-characterized computational methods for deducing the phenotypic impact of genomic and system-level perturbations. Many such methods have been developed, but currently, even though some are already deployed in clinical settings, we often remain ignorant of how they actually perform, as well as how and when they should be applied. Further, it is already clear that new and more sophisticated approaches must be developed to fully meet these challenges. The Critical Assessment of Genome Interpretation (CAGI, \ˈkā-jē\) conducts community experiments to objectively assess computational methods for determining the phenotypic impacts of genomic variation. The primary goals are to establish the state of the art, to show where future progress may best be made, to highlight innovations and progress, and to build a strong collaborative community. In the CAGI experiments, participants are typically provided genetic variants and make blind predictions of resulting phenotypes. These predictions are evaluated against gold-standard experimental or clinical data by independent assessors. Four CAGI experiments have been conducted to date—a pilot in 2010, and three full-scale events in 2011, 2013, and 2016. Each edition of CAGI involves about 10 challenges. The experiment is conducted over a period of about a year, starting with the identification and development of suitable challenges, followed by a period during which participants are invited to submit their predictions, then a term in which the independent assessors evaluate the results, and concluding with a meeting to discuss the outcomes. CAGI challenges span a wide range of relationships between genetic variation and disease. For single base variants, there are challenges that address the problem of interpreting the impact of missense mutations on protein activity using a variety of molecular and cellular phenotypes, challenges that test the ability to predict the effect of mutations in cancer driver genes on cell growth, and challenges on the effect of single-base variants on RNA expression levels and splicing (including Beer, 2017; Capriotti, Martelli, Fariselli, & Casadio, 2017; Carraro et al., 2017; Katsonis & Lichtarge, 2017; Kreimer et al., 2017; Niroula & Vihinen 2017; Pejaver et al., 2017; Tang et al., 2017; Tang & Fenton, 2017; Xu et al., 2017; Yin et al., 2017; Zeng, Edwards, Guo, & Gifford, 2017; Zhang et al., 2017). At the level of full exome and genome sequence, there are challenges that assess methods for assigning complex traits phenotypes and that evaluate the ability to associate genome sequence and an extensive profile of phenotypic traits (including Cai et al., 2017; Daneshjou et al., 2014; Daneshjou et al., 2017; Giollo et al., 2017; Laksshman, Bhat, Viswanath, & Li, 2017; Pal, Kundu, Yin, & Moult, 2017a; Wang et al., 2017). CAGI has also included challenges in which participants were asked to identify causative variants for rare diseases in gene panel, exome, and whole-genome sequence data (including Chandonia et al., 2017; Kundu, Pal, Yin, & Moult, 2017; Pal, Kundu, Yin, & Moult, 2017b). Many challenges have focused on cancer, given its prevalence and the impact of genetics. This special issue of Human Mutation contains a selection papers reporting the assessments of challenge results, as well as papers from some individual participating teams, describing their methods, and the results obtained. Most papers report on the recent challenges, from CAGI4, held in 2016. As CAGI best helps further development when challenges reoccur year after year, some manuscripts discuss results from the earlier editions of CAGI and their development over time. Together, these results from CAGI offer powerful insights into the appropriate level of confidence to place in variant annotations and interpretation methods, and which classes of approaches are most suitable for a particular application. They reveal limitations of current data collection and analysis approaches and point to areas for future research and new approaches. The fifth CAGI edition is presently underway. Full information about this and the previous CAGI editions is available at http://www.genomeinterpretation.org. We are most grateful to all CAGI participants. The primary contributors whose work is assessed in CAGI is that of the predictors: Allison Abad, Ogun Adebali, Ivan Adzhubey, Talal Amin, Johnathan R. Azaria, Giulia Babbi, Eraan Bachar, Benjamin Bachman, Minkyung Baek, Greet De Baets, Michael Beer, Violeta Beleva-Guthrie, Bonnie Berger, Brady Bernard, Rajendra Bhat, Rohit Bhattacharya, Samuele Bovo, Marcus Breese, Aharon S. Brodie, Yana Bromberg, Binghuang Cai, Colin Campbell, Chen Cao,

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1002/humu.23290

Cites

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cited by

See all 36

Metadata

Authors
Hoskins, R. A., Repo, S., Barsky, D., Andreoletti, G., Moult, J., & Brenner, S. E.
Venue
Human Mutation
Publisher
Wiley
Published
2017-08-17
Volume
38
Issue
9
Pages
1039-1041
ISSN
  • 1059-7794 (ISSN-L)
  • 1098-1004
OpenAlex
W2735037223
Cited by
36
References
0
FWCI
3.60
Type
journal article
Language
en
Open access
bronze

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Reports from CAGI: The Critical Assessment of Genome Interpretation. https://app.cochat.ai/papers/10.1002/humu.23290
The paper
Hoskins, R. A., Repo, S., Barsky, D., Andreoletti, G., Moult, J., & Brenner, S. E. (2017). Reports from CAGI: The Critical Assessment of Genome Interpretation. Human Mutation. https://doi.org/10.1002/humu.23290

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.