The DOI matches its Crossref record; 123 of the 123 references it cites were found in a registry or an index.
Cloud computing for genomic data analysis and collaboration
Open accessAbstract
Next-generation sequencing has made major strides in the past decade. Studies based on large sequencing data sets are growing in number, and public archives for raw sequencing data have been doubling in size every 18 months. Leveraging these data requires researchers to use large-scale computational resources. Cloud computing, a model whereby users rent computers and storage from large data centres, is a solution that is gaining traction in genomics research. Here, we describe how cloud computing is used in genomics for research and large-scale collaborations, and argue that its elasticity, reproducibility and privacy features make it ideally suited for the large-scale reanalysis of publicly available archived data, including privacy-protected data.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“As the cost of genome sequencing continues to decrease, genomic datasets grow in both size and availability [1].”
From Accelerating genomic workflows using NVIDIA Parabricks (2023) · cites this paper as [1]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
STAR: ultrafast universal RNA-seq aligner
Dobin, A., Davis, C. A., Schlesinger, F., Drenkow, J., Zaleski, C., Jha, S., Batut, P., Chaisson, M., & Gingeras, T. R. (2013) · Bioinformatics · doi:10.1093/bioinformatics/bts635 · Cited by 51,709
The Genome Analysis Toolkit: A MapReduce framework for analyzing next-generation DNA sequencing data
McKenna, A., Hanna, M., Banks, E., Sivachenko, A., Cibulskis, K., Kernytsky, A., Garimella, K., Altshuler, D., Gabriel, S., Daly, M., et al. (2010) · Genome Research · doi:10.1101/gr.107524.110 · Cited by 27,895 · cited 1×
“Notably, the popular Genome Analysis Toolkit (GATK) variant caller 90 uses MapReduce to parallelize population-scale variant calling.”
MethodRSEM: accurate transcript quantification from RNA-Seq data with or without a reference genome
Li, B., & Dewey, C. N. (2011) · BMC Bioinformatics · doi:10.1186/1471-2105-12-323 · Cited by 22,179 · cited 2×
“After a total of approximately 1.3 million core hours of ana lysis using Spliced Transcripts Alignment to a Reference (STAR) 64 , RNA-seq by Expectation Maximization (RSEM) 65 and Kallisto 66 , the estimated cost for using discounted ‘pre-emptible’ (that is, excess) instances was about US$1.30 per sample, but a much l…”
MethodInfluentialMapReduce: simplified data processing on large clusters
Dean, J., & Ghemawat, S. (2008) · Communications of the ACM · doi:10.1145/1327452.1327492 · Cited by 18,476
A global reference for human genetic variation
The 1000 Genomes Project Consortium, authors, C., Auton, A., Abecasis, G. R., Steering committee, Altshuler, D. M., Durbin, R. M., Abecasis, G. R., Bentley, D. R., Chakravarti, A., et al. (2015) · Nature · doi:10.1038/nature15393 · Cited by 18,247
An integrated encyclopedia of DNA elements in the human genome
The ENCODE Project Consortium (2012) · Nature · doi:10.1038/nature11247 · Cited by 18,072
The NIST definition of cloud computing
Mell, P., & Grance, T. (2011) · doi:10.6028/nist.sp.800-145 · Cited by 12,637 · cited 2×
“The NIST definition of cloud computing describes a range of ‘deployment models’ 27 , including private clouds, where resources are made available only to users in an organization; community clouds, where several organizations share access to an otherwise private cloud; public clouds, where…”
BackgroundNear-optimal probabilistic RNA-seq quantification
Bray, N. L., Pimentel, H., Melsted, P., & Pachter, L. (2016) · Nature Biotechnology · doi:10.1038/nbt.3519 · Cited by 9,723 · cited 3×
“One of the pipelines proposed in this work, again based on Kallisto, was shown to achieve a cost of US$0.09 per sample, the lowest of any effort to date and a small fraction of the cost of sequencing.”
MethodInfluentialAnalysis of protein-coding genetic variation in 60,706 humans
Exome Aggregation Consortium, Lek, M., Karczewski, K. J., Minikel, E. V., Samocha, K. E., Banks, E., Fennell, T., O’Donnell-Luria, A. H., Ware, J. S., Hill, A. J., et al. (2016) · Nature · doi:10.1038/nature19057 · Cited by 9,587
The cancer genome atlas pan-cancer analysis project
The Cancer Genome Atlas Research Network, Weinstein, J. N., Collisson, E. A., Mills, G. B., Shaw, K. R. M., Ozenberger, B. A., Ellrott, K., Shmulevich, I., Sander, C., & Stuart, J. M. (2013) · Nature Genetics · doi:10.1038/ng.2764 · Cited by 9,092
The Cancer Cell Line Encyclopedia enables predictive modelling of anticancer drug sensitivity
Barretina, J., Caponigro, G., Stransky, N., Venkatesan, K., Margolin, A. A., Kim, S., Wilson, C. J., Lehár, J., Kryukov, G. V., Sonkin, D., et al. (2012) · Nature · doi:10.1038/nature11003 · Cited by 7,950
The Grid 2: Blueprint for a New Computing Infrastructure
Foster, I. T., & Kesselman, C. (1998) · Cited by 7,599
A New Initiative on Precision Medicine
Collins, F. S., & Varmus, H. (2015) · New England Journal of Medicine · doi:10.1056/nejmp1500523 · Cited by 5,202
Coming of age: ten years of next-generation sequencing technologies
Goodwin, S., McPherson, J. D., & McCombie, W. R. (2016) · Nature Reviews Genetics · doi:10.1038/nrg.2016.49 · Cited by 4,352
Genetic effects on gene expression across human tissues
GTEx Consortium (2017) · Nature · doi:10.1038/nature24277 · Cited by 4,036
Nextflow enables reproducible computational workflows
Di Tommaso, P., Chatzou, M., Floden, E. W., Barja, P. P., Palumbo, E., & Notredame, C. (2017) · Nature Biotechnology · doi:10.1038/nbt.3820 · Cited by 4,001
Next-generation genotype imputation service and methods
Das, S., Forer, L., Schönherr, S., Sidore, C., Locke, A. E., Kwong, A., Vrieze, S. I., Chew, E. Y., Levy, S., McGue, M., et al. (2016) · Nature Genetics · doi:10.1038/ng.3656 · Cited by 3,808 · cited 1×
“For services available to a wide user community (for example, a genetic imputation server 32 ), elasticity allows the computational burden of answering user queries to be spread across many computers, avoiding bottlenecks and delays (FIG.”
BackgroundDocker: lightweight Linux containers for consistent development and deployment
Merkel, D. (2014) · Linux journal · Cited by 3,191
XSEDE: Accelerating Scientific Discovery
Towns, J. W., Cockerill, T. M., Dahan, M., Foster, I. T., Gaither, K., Grimshaw, A. S., Hazlewood, V., Lathrop, S. A., Lifka, D. A., Peterson, G. D., et al. (2014) · Computing in Science & Engineering · doi:10.1109/mcse.2014.80 · Cited by 3,114 · cited 2×
“For example, although smaller than the commercial clouds, the NSF-funded Jetstream can handle cloud workloads free of charge through XSEDE programme grants 75 .”
BackgroundMethodLeinonen, R., Sugawara, H., & Shumway, M. F. (2010) · Nucleic Acids Research · doi:10.1093/nar/gkq1019 · Cited by 2,872
Singularity: Scientific containers for mobility of compute
Kurtzer, G. M., Sochat, V., & Bauer, M. W. (2017) · PLOS ONE · doi:10.1371/journal.pone.0177459 · Cited by 2,740 · cited 2×
“Similar reproducibility advantages are possible on non-cloud computers using virtual machines 43,44 and Docker 45 or Singularity 46 containers , tools that package software with all the necessary components to enable reproducible deployment in different computing environments (TABLE 4) .”
MethodThe landscape of long noncoding RNAs in the human transcriptome
Iyer, M. K., Niknafs, Y. S., Malik, R., Singhal, U., Sahu, A., Hosono, Y., Barrette, T. R., Prensner, J. R., Evans, J. R., Zhao, S., et al. (2015) · Nature Genetics · doi:10.1038/ng.3192 · Cited by 2,738 · cited 1×
“For example, one recent study gathered human RNA sequencing (RNA-seq) samples from various projects, including The Cancer Genome Atlas (TCGA) and ENCODE, to create a catalogue of long non-coding RNAs 51 .”
BackgroundFranke, A., McGovern, D. P. B., Barrett, J. C., Wang, K., Radford-Smith, G. L., Ahmad, T., Lees, C. W., Balschun, T., Lee, J., Roberts, R., et al. (2010) · Nature Genetics · doi:10.1038/ng.717 · Cited by 2,520 · cited 1×
“That is, researchers can use public data to boost the power available to analyse a locally generated data set, a paradigm that already prevails in micro-array data analysis 8,9,12,92 .”
BackgroundInternational network of cancer genome projects
The International Cancer Genome Consortium (2010) · Nature · doi:10.1038/nature08987 · Cited by 2,299
Tackling the widespread and critical impact of batch effects in high-throughput data
Leek, J. T., Scharpf, R. B., Bravo, H. C., Simcha, D., Langmead, B., Johnson, W. E., Geman, D., Baggerly, K., & Irizarry, R. A. (2010) · Nature Reviews Genetics · doi:10.1038/nrg2825 · Cited by 2,196 · cited 1×
“That is, researchers can use public data to boost the power available to analyse a locally generated data set, a paradigm that already prevails in micro-array data analysis 8,9,12,92 .”
BackgroundSelf-citationThe Galaxy platform for accessible, reproducible and collaborative biomedical analyses: 2016 update
Afgan, E., Baker, D., van den Beek, M., Blankenberg, D. J., Bouvier, D., Čech, M., Chilton, J. M., Clements, D., Coraor, N., Eberhard, C., et al. (2016) · Nucleic Acids Research · doi:10.1093/nar/gkw343 · Cited by 2,150
Wellcome Trust Case Control Consortium, Zeggini, E., Scott, L. J., Saxena, R., Voight, B. F., Marchini, J. L., Hu, T., de Bakker, P. I., Abecasis, G. R., Almgren, P., et al. (2008) · Nature Genetics · doi:10.1038/ng.120 · Cited by 1,864
Evolving gene/transcript definitions significantly alter the interpretation of GeneChip data
Dai, M. (2005) · Nucleic Acids Research · doi:10.1093/nar/gni179 · Cited by 1,807
Toil enables reproducible, open source, big biomedical data analyses
Vivian, J., Rao, A. A., Nothaft, F. A., Ketchum, C., Armstrong, J., Novak, A., Pfeil, J., Narkizian, J., Deran, A. D., Musselman-Brown, A., et al. (2017) · Nature Biotechnology · doi:10.1038/nbt.3772 · Cited by 1,365 · cited 2×
“The Toil effort 63 used a standardized computational pipeline and AWS to analyse nearly 20,000 samples spanning four major studies, including TCGA and GTEx, in under 4 days.”
MethodBig Data: Astronomical or Genomical?
Stephens, Z. D., Lee, S. Y., Faghri, F., Campbell, R. H., Zhai, C., Efron, M. J., Iyer, R., Schatz, M. C., Sinha, S., & Robinson, G. E. (2015) · PLOS Biology · doi:10.1371/journal.pbio.1002195 · Cited by 1,340
The human transcriptome across tissues and individuals
Melé, M., Ferreira, P. G., Reverter, F., DeLuca, D. S., Monlong, J., Sammeth, M., Young, T. R., Goldmann, J. M., Pervouchine, D. D., Sullivan, T. J., et al. (2015) · Science · doi:10.1126/science.aaa0355 · Cited by 1,304
An introduction to Docker for reproducible research
Boettiger, C. (2015) · ACM SIGOPS Operating Systems Review · doi:10.1145/2723872.2723882 · Cited by 1,059
Million Veteran Program: A mega-biobank to study genetic influences on health and disease
Gaziano, J. M., Concato, J., Brophy, M., Fiore, L., Pyarajan, S., Breeling, J., Whitbourne, S., Deen, J., Shannon, C., Humphries, D., et al. (2016) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2015.09.016 · Cited by 1,055
Rhodes, D. R., Yu, J., Sundaram, S. K., Deshpande, N., Varambally, R., Ghosh, D., Barrette, T. R., Pandey, A., & Chinnaiyan, A. M. (2004) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0401994101 · Cited by 1,018
The sequence read archive: explosive growth of sequencing data
Kodama, Y., Shumway, M., Leinonen, R., & on behalf of the International Nucleotide Sequence Database Collaboration (2012) · Nucleic Acids Research · doi:10.1093/nar/gkr854 · Cited by 894
Unlocking the secrets of the genome
Celniker, S., Dillon, L. A. L., Gerstein, M., Gunsalus, K. C., Henikoff, S., Karpen, G. H., Kellis, M., Lai, E. C., Lieb, J. D., MacAlpine, D. M., et al. (2009) · Nature · doi:10.1038/459927a · Cited by 781 · cited 1×
“…most visibly in projects such as the ICGC and the related Pan-Cancer Analysis of Whole Genomes (PCAWG) effort 21 , the National Cancer Institute (NCI) Cancer Genomics Cloud (CGC) Pilots 22–24 and the Encyclopedia of DNA Elements (ENCODE and Model Organism ENCODE (modENCODE)) projects 25,26 .”
BackgroundDiversity and dynamics of the Drosophila transcriptome
Brown, J., Boley, N. P., Eisman, R. C., May, G. E., Stoiber, M. H., Duff, M. O., Booth, B. W., Wen, J., Park, S. H., Suzuki, A. M., et al. (2014) · Nature · doi:10.1038/nature12962 · Cited by 737 · cited 1×
“Another effort reanalysed RNA-seq data from modENCODE 52,53 that detailed transcription in Drosophila melanogaster over developmental time.”
BackgroundArrayExpress update—simplifying data submissions
Kolesnikov, N. N., Hastings, E., Keays, M., Melnichuk, O., Tang, A., Williams, E. M., Dylag, M., Kurbatova, N., Brandizi, M., Burdett, T., et al. (2014) · Nucleic Acids Research · doi:10.1093/nar/gku1057 · Cited by 700
CloudBurst: highly sensitive read mapping with MapReduce
Schatz, M. C. (2009) · Bioinformatics · doi:10.1093/bioinformatics/btp236 · Cited by 630
The Globus Striped GridFTP Framework and Server
Allcock, W. E., Bresnahan, J., Kettimuthu, R., & Link, M. (2005) · ACM/IEEE Conference on Supercomputing (SC) · doi:10.1109/sc.2005.72 · Cited by 607
Frozen robust multiarray analysis (fRMA)
McCall, M. N., Bolstad, B. M., & Irizarry, R. A. (2010) · Biostatistics · doi:10.1093/biostatistics/kxp059 · Cited by 594 · cited 1×
“That is, researchers can use public data to boost the power available to analyse a locally generated data set, a paradigm that already prevails in micro-array data analysis 8,9,12,92 .”
BackgroundOpenStack: Toward an Open-source Solution for Cloud Computing
Sefraoui, O., Aissaoui, M., & Eleuldj, M. (2012) · International Journal of Computer Applications · doi:10.5120/8738-2991 · Cited by 557 · cited 4×
“Users can also install Galaxy on cloud clusters using CloudMan 77 , which not only supports AWS for pay-as-you-go storage and computing but also private and public clouds that leverage OpenStack 30 or OpenNebula 31 .”
BackgroundMethodInfluentialRepeatability of published microarray gene expression analyses
Ioannidis, J. P. A., Allison, D. B., Ball, C. A., Coulibaly, I., Cui, X., Culhane, A. C., Falchi, M., Furlanello, C., Game, L., Jurman, G., et al. (2009) · Nature Genetics · doi:10.1038/ng.295 · Cited by 551 · cited 1×
“This is crucial in a field that has seen reproducibility failures 38–40 and where many are calling for more rigorous accounting of factors, such as underlying software versions, that directly or indirectly affect computational analyses 41,42 .”
BackgroundNCBI’s Database of Genotypes and Phenotypes: dbGaP
Tryka, K. A., Hao, L., Sturcke, A., Jin, Y., Wang, Z. Y., Ziyabari, L., Lee, M., Popova, N. V., Sharopova, N., Kimura, M., et al. (2013) · Nucleic Acids Research · doi:10.1093/nar/gkt1211 · Cited by 531 · cited 2×
“Such ‘dbGaP-protected’ data 50 have been de-identified, and potential users must be approved by a data access committee and conduct their analysis in a way that maintains donor anonymity. dbGaP and the related European Genome–Phenome Archive (EGA) 103 serve as portals for protected data sets,…”
BackgroundThe case for cloud computing in genome informatics
Stein, L. D. (2010) · Genome biology · doi:10.1186/gb-2010-11-5-207 · Cited by 531 · cited 1×
“The larger and the more decentralized the project, the more copies must be made 72 (FIG.”
BackgroundSearching for SNPs with cloud computing
Langmead, B., Schatz, M. C., Lin, J., Pop, M., & Salzberg, S. L. (2009) · Genome biology · doi:10.1186/gb-2009-10-11-r134 · Cited by 490 · cited 1×
“MapReduce and similar frameworks have been used in variant calling 87,88 , RNA-seq analysis 59,62,63 and chromatin immunoprecipitation followed by sequencing (ChIP– seq) analysis 89 .”
MethodSelf-citationAnalysis of matched mRNA measurements from two different microarray technologies
Kuo, W. P., Jenssen, T.-K., Butte, A. J., Lucila Ohno-machado, A., & Kohane, I. S. (2002) · Bioinformatics · doi:10.1093/bioinformatics/18.3.405 · Cited by 473
‘Big data’, Hadoop and cloud computing in genomics
O’Driscoll, A., Daugelaite, J., & Sleator, R. D. (2013) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2013.07.001 · Cited by 448
Reproducible RNA-seq analysis using recount2
Collado‐Torres, L., Nellore, A., Kammers, K., Ellis, S., Taub, M. A., Hansen, K. D., Jaffe, A. E., Langmead, B., & Leek, J. T. (2017) · Nature Biotechnology · doi:10.1038/nbt.3838 · Cited by 448 · cited 1×
“61 ) efforts used AWS and a custom cloud-enabled RNA-seq aligner 62 to reanalyse over 70,000 human samples spanning TCGA, GTEx and the SRA, releasing expres-sion-level summaries at the level of splice junctions, exons, genes and individual genomic bases.”
MethodSelf-citationAn overview of the Hadoop/MapReduce/HBase framework and its current applications in bioinformatics
Taylor, R. C. (2010) · BMC Bioinformatics · doi:10.1186/1471-2105-11-s12-s1 · Cited by 432
The European Genome-phenome Archive of human data consented for biomedical research
Lappalainen, I., Almeida, J. P., Kumanduri, V., Senf, A., Spalding, D., Ur-Rehman, S., Saunders, G. I., Kandasamy, J., Cáccamo, M., Leinonen, R., et al. (2015) · Nature Genetics · doi:10.1038/ng.3312 · Cited by 366 · cited 2×
“Such ‘dbGaP-protected’ data 50 have been de-identified, and potential users must be approved by a data access committee and conduct their analysis in a way that maintains donor anonymity. dbGaP and the related European Genome–Phenome Archive (EGA) 103 serve as portals for protected data sets, providing a way to apply…”
BackgroundGlobus Online: Accelerating and Democratizing Science through Cloud-Based Services
Foster, I. T. (2011) · IEEE Internet Computing · doi:10.1109/mic.2011.64 · Cited by 348 · cited 1×
“An initiative by the Computation Institute at the University of Chicago, Globus Genomics not only combines an enhanced Galaxy instance for workflow management with AWS for powering computational analyses 78 but also uses Globus Online 79 to speed up data transfer.”
MethodPetryszak, R., Burdett, T., Fiorelli, B., Fonseca, N. A., Gonzàlez-Porta, M., Hastings, E., Huber, W. K., Jupp, S., Keays, M., Kryvych, N., et al. (2013) · Nucleic Acids Research · doi:10.1093/nar/gkt1270 · Cited by 343
Next-generation sequencing data interpretation: enhancing reproducibility and accessibility
Nekrutenko, A., & Taylor, J. (2012) · Nature Reviews Genetics · doi:10.1038/nrg3305 · Cited by 328 · cited 1×
“This is crucial in a field that has seen reproducibility failures 38–40 and where many are calling for more rigorous accounting of factors, such as underlying software versions, that directly or indirectly affect computational analyses 41,42 .”
BackgroundCloud-scale RNA-sequencing differential expression analysis with Myrna
Langmead, B., Hansen, K. D., & Leek, J. T. (2010) · Genome biology · doi:10.1186/gb-2010-11-8-r83 · Cited by 320 · cited 2×
“ReCount 58 used Myrna 59 , a cloud-enabled tool for calculating differential gene expression in large RNA-seq data sets, and computing resources rented from AWS to reanalyse 475 RNA-seq samples from 18 different experiments.”
MethodSelf-citationBaggerly, K., & Coombes, K. R. (2009) · The Annals of Applied Statistics · doi:10.1214/09-aoas291 · Cited by 311
Cloud computing and the DNA data race
Schatz, M. C., Langmead, B., & Salzberg, S. L. (2010) · Nature Biotechnology · doi:10.1038/nbt0710-691 · Cited by 284
Self-citationAngiuoli, S. V., Matalka, M. N., Gussman, A., Galens, K., Vangala, M., Riley, D. R. J., Arze, C., White, J. R., White, O. R., & Fricke, W. F. (2011) · BMC Bioinformatics · doi:10.1186/1471-2105-12-356 · Cited by 282 · cited 1×
“Similar reproducibility advantages are possible on non-cloud computers using virtual machines 43,44 and Docker 45 or Singularity 46 containers , tools that package software with all the necessary components to enable reproducible deployment in different computing environments (TABLE 4) .”
MethodIaaS Cloud Architecture: From Virtualized Datacenters to Federated Cloud Infrastructures
Moreno‐Vozmediano, R., Montero, R. S., & Llorente, I. M. (2012) · Computer · doi:10.1109/mc.2012.76 · Cited by 255 · cited 2×
“Users can also install Galaxy on cloud clusters using CloudMan 77 , which not only supports AWS for pay-as-you-go storage and computing but also private and public clouds that leverage OpenStack 30 or OpenNebula 31 .”
BackgroundMethodEvaluation of TagSeq, a reliable low‐cost alternative for RNA seq
Lohman, B. K., Weber, J. N., & Bolnick, D. I. (2016) · Molecular Ecology Resources · doi:10.1111/1755-0998.12529 · Cited by 224
Roosan, M. R., Jackson, L. K., Johnson, W. E., Li, D. Y., Bild, A. H., & Piccolo, S. (2015) · Bioinformatics · doi:10.1093/bioinformatics/btv377 · Cited by 212
Common Workflow Language, v1.0
Amstutz, P., Crusoe, M. R., Tijanić, N., et al. (2016) · eScholarship (California Digital Library) · doi:10.6084/m9.figshare.3115156.v2 · Cited by 210
A federated ecosystem for sharing genomic, clinical data
The Global Alliance for Genomics and Health* (2016) · Science · doi:10.1126/science.aaf6162 · Cited by 204
Common Workflow Language, draft 3
Amstutz, P., Andeer, R., Chapman, B., Chilton, J., Crusoe, M. R., Guimerà, R. V., Hernandez, G. C., Ivkovic, S., Kartashov, A., Kern, J., et al. (2016) · eScholarship (California Digital Library) · doi:10.6084/m9.figshare.3115156 · Cited by 204
Galaxy CloudMan: delivering cloud compute clusters
Afgan, E., Baker, D., Coraor, N., Chapman, B., Nekrutenko, A., & Taylor, J. (2010) · BMC Bioinformatics · doi:10.1186/1471-2105-11-s12-s4 · Cited by 179 · cited 1×
“Users can also install Galaxy on cloud clusters using CloudMan 77 , which not only supports AWS for pay-as-you-go storage and computing but also private and public clouds that leverage OpenStack 30 or OpenNebula 31 .”
MethodTools and techniques for computational reproducibility
Piccolo, S., & Frampton, M. (2016) · GigaScience · doi:10.1186/s13742-016-0135-4 · Cited by 179 · cited 1×
“This is crucial in a field that has seen reproducibility failures 38–40 and where many are calling for more rigorous accounting of factors, such as underlying software versions, that directly or indirectly affect computational analyses 41,42 .”
BackgroundLau, J. W., Lehnert, E. M., Sethi, A., Malhotra, R., Kaushik, G. N., Onder, Z., Groves-Kirkby, N., Mihajlović, A., DiGiovanna, J., Srdic, M., et al. (2017) · Cancer Research · doi:10.1158/0008-5472.can-17-0387 · Cited by 171
PeakRanger: A cloud-enabled peak caller for ChIP-seq data
Feng, X., Grossman, R. L., & Stein, L. D. (2011) · BMC Bioinformatics · doi:10.1186/1471-2105-12-139 · Cited by 158 · cited 1×
“MapReduce and similar frameworks have been used in variant calling 87,88 , RNA-seq analysis 59,62,63 and chromatin immunoprecipitation followed by sequencing (ChIP– seq) analysis 89 .”
MethodCloud BioLinux: pre-configured and on-demand bioinformatics computing for the genomics community
Krampis, K., Booth, T., Chapman, B., Tiwari, B., Bicak, M., Field, D., & Nelson, K. E. (2012) · BMC Bioinformatics · doi:10.1186/1471-2105-13-42 · Cited by 158 · cited 1×
“Similar reproducibility advantages are possible on non-cloud computers using virtual machines 43,44 and Docker 45 or Singularity 46 containers , tools that package software with all the necessary components to enable reproducible deployment in different computing environments (TABLE 4) .”
MethodReCount: A multi-experiment resource of analysis-ready RNA-seq gene count datasets
Frazee, A. C., Langmead, B., & Leek, J. T. (2011) · BMC Bioinformatics · doi:10.1186/1471-2105-12-449 · Cited by 157 · cited 1×
“ReCount 58 used Myrna 59 , a cloud-enabled tool for calculating differential gene expression in large RNA-seq data sets, and computing resources rented from AWS to reanalyse 475 RNA-seq samples from 18 different experiments.”
MethodSelf-citationReproducibility of computational workflows is automated using continuous analysis
Beaulieu‐Jones, B. K., & Greene, C. S. (2017) · Nature Biotechnology · doi:10.1038/nbt.3780 · Cited by 151
Kelly, B., Fitch, J., Hu, Y. P., Corsmeier, D. J., Zhong, H., Wetzel, A. N., Nordquist, R. D., Newsom, D. L., & White, P. (2015) · Genome biology · doi:10.1186/s13059-014-0577-x · Cited by 137
In silico research in the era of cloud computing
Dudley, J. T., & Butte, A. J. (2010) · Nature Biotechnology · doi:10.1038/nbt1110-1181 · Cited by 125
Perspective: Sustaining the big-data ecosystem
Bourne, P. E., Lorsch, J. R., & Green, E. D. (2015) · Nature · doi:10.1038/527s16a · Cited by 116 · cited 1×
“For example, the cloud is the home of the National Institutes of Health (NIH) Data Commons Pilot, an effort to increase avail ability and utility of data and software from NIH-funded efforts 48,49 .”
BackgroundData analysis: Create a cloud commons
Stein, L. D., Knoppers, B. M., Campbell, P. J., Getz, G. A., & Korbel, J. O. (2015) · Nature · doi:10.1038/523149a · Cited by 110 · cited 1×
“105 ) .”
Pseudoalignment for metagenomic read assignment
Schaeffer, L., Pimentel, H., Bray, N., Melsted, P., & Pachter, L. S. (2017) · Bioinformatics · doi:10.1093/bioinformatics/btx106 · Cited by 103
Nellore, A., Jaffe, A. E., Fortin, J., Alquicira-Hernández, J., Collado‐Torres, L., Wang, S., Phillips, R. A., Karbhari, N., Hansen, K. D., Langmead, B., et al. (2016) · Genome biology · doi:10.1186/s13059-016-1118-6 · Cited by 103 · cited 2×
“If data are protected, for example, by dbGaP, existing protocols make it possible to craft a compliant cloud-based computational setup 57 .”
BackgroundMethodSelf-citationA cloud-based workflow to quantify transcript-expression levels in public cancer compendia
Tatlow, P., & Piccolo, S. (2016) · Scientific Reports · doi:10.1038/srep39259 · Cited by 101 · cited 1×
“Another effort used Google Cloud Platform to quantify transcript expression levels for over 12,000 RNA-seq samples from large cancer projects 68 .”
MethodBioBlend: automating pipeline analyses within Galaxy and CloudMan
Sloggett, C., Goonasekera, N., & Afgan, E. (2013) · Bioinformatics · doi:10.1093/bioinformatics/btt199 · Cited by 97
The developmental transcriptome of Drosophila melanogaster
Graveley, B. R. (2010) · Genome biology · doi:10.1186/gb-2010-11-s1-i11 · Cited by 93
Genomic cloud computing: legal and ethical points to consider
Dove, E. S., Joly, Y., Tassé, A.-M., & Knoppers, B. M. (2014) · European Journal of Human Genetics · doi:10.1038/ejhg.2014.196 · Cited by 92
MetaSRA: normalized human sample-specific metadata for the Sequence Read Archive
Bernstein, M. N., Doan, A., & Dewey, C. N. (2017) · Bioinformatics · doi:10.1093/bioinformatics/btx334 · Cited by 87
Androgen-Dependent Repression of ERRγ Reprograms Metabolism in Prostate Cancer
Audet‐Walsh, É., Yee, T., McGuirk, S., Vernier, M., Ouellet, C., St‐Pierre, J., & Giguère, V. (2016) · Cancer Research · doi:10.1158/0008-5472.can-16-1204 · Cited by 86
Cloud-based bioinformatics workflow platform for large-scale next-generation sequencing analyses
Liu, B., Madduri, R. K., Sotomayor, B., Chard, K., Lacinski, L., Dave, U. J., Li, J., Liu, C., & Foster, I. T. (2014) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2014.01.005 · Cited by 81 · cited 1×
“An initiative by the Computation Institute at the University of Chicago, Globus Genomics not only combines an enhanced Galaxy instance for workflow management with AWS for powering computational analyses 78 but also uses Globus Online 79 to speed up data transfer.”
MethodOmics Pipe: a community-based framework for reproducible multi-omics data analysis
Fisch, K. M., Meißner, T., Gioia, L., Ducom, J.-C., Carland, T. M., Loguercio, S., & Su, A. I. (2015) · Bioinformatics · doi:10.1093/bioinformatics/btv061 · Cited by 79
European Nucleotide Archive in 2016
Toribio, A. L., Alako, B., Amid, C., Cerdeño-Tárraga, A. M., Clarke, L., Cleland, I., Fairley, S., Gibson, R., Goodgame, N., ten Hoopen, P., et al. (2016) · Nucleic Acids Research · doi:10.1093/nar/gkw1106 · Cited by 78
Bionimbus: a cloud for managing, analyzing and sharing large genomics datasets
Heath, A. P., Greenway, M. J., Powell, R., Spring, J., Suarez, R. D., Hanley, D., Bandlamudi, C., McNerney, M. E., White, K. P., & Grossman, R. L. (2014) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2013-002155 · Cited by 74
Madduri, R. K., Sulakhe, D., Lacinski, L., Liu, B., Rodriguez, A., Chard, K., Dave, U. J., & Foster, I. T. (2014) · Concurrency and Computation: Practice and Experience · doi:10.1002/cpe.3274 · Cited by 73 · cited 6×
“A federated cloud approach is also being explored by the Global Alliance for Genomics and Health (GA4GH) 82 .”
MethodInfluentialCloudNeo: a cloud pipeline for identifying patient-specific tumor neoantigens
Bais, P., Namburi, S., Gatti, D. M., Zhang, X., & Chuang, J. H. (2017) · Bioinformatics · doi:10.1093/bioinformatics/btx375 · Cited by 69 · cited 1×
“One recently published workflow used the CGC platform to detect patient-specific tumour neoantigens from sequencing data 73 .”
MethodNational Institute of Standards and Technology
(2012) · Choice Reviews Online · doi:10.5860/choice.50-2030 · Cited by 67
Dynamics of Wolbachia pipientis Gene Expression Across the Drosophila melanogaster Life Cycle
Gutzwiller, F., Carmo, C. R., Miller, D. E., Rice, D. W., Newton, I. L. G., Hawley, R. S., Teixeira, L., & Bergman, C. (2015) · G3 Genes Genomes Genetics · doi:10.1534/g3.115.021931 · Cited by 65 · cited 1×
“The reanalysis focused on gene expression in the endo-symbiont Wolbachia pipientis and yielded new insights into gene expression patterns related to symbiosis 54 .”
BackgroundThe ISB Cancer Genomics Cloud: A Flexible Cloud-Based Platform for Cancer Genomics Research
Reynolds, S. M., Miller, M., Lee, P., Leinonen, K., Paquette, S. M., Rodebaugh, Z., Hahn, A., Gibbs, D. L., Slagel, J., Longabaugh, W. J., et al. (2017) · Cancer Research · doi:10.1158/0008-5472.can-17-0617 · Cited by 63 · cited 2×
“…most visibly in projects such as the ICGC and the related Pan-Cancer Analysis of Whole Genomes (PCAWG) effort 21 , the National Cancer Institute (NCI) Cancer Genomics Cloud (CGC) Pilots 22–24 and the Encyclopedia of DNA Elements (ENCODE and Model Organism ENCODE (modENCODE)) projects 25,26 .”
BackgroundBateman, A. G., & Wood, M. (2009) · Bioinformatics · doi:10.1093/bioinformatics/btp274 · Cited by 62
Globus platform‐as‐a‐service for collaborative science applications
Ananthakrishnan, R., Chard, K., Foster, I. T., & Tuecke, S. J. (2014) · Concurrency and Computation Practice and Experience · doi:10.1002/cpe.3262 · Cited by 59 · cited 2×
“…data continue to evolve, we expect readers will increasingly encounter secure application programming interfaces (APIs), such as the Seven Bridges API 110 , and the related ideas of authentication, authorization and identity federation (for overviews of these concepts, see REFS 111,112 ).”
BackgroundDynamics of Wolbachia pipientis gene expression across the Drosophila melanogaster life cycle
Gutzwiller, F., do Carmo, C. R., Miller, D. E., et al. (2015) · Portuguese National Funding Agency for Science, Research and Technology (RCAAP Project by FCT) · doi:10.48550/arxiv.1505.05782 · Cited by 58 · cited 1×
“The reanalysis focused on gene expression in the endo-symbiont Wolbachia pipientis and yielded new insights into gene expression patterns related to symbiosis 54 .”
BackgroundRail-RNA: scalable analysis of RNA-seq splicing and coverage
Nellore, A., Collado‐Torres, L., Jaffe, A. E., Alquicira-Hernández, J., Wilks, C., Pritt, J., Morton, J. T., Leek, J. T., & Langmead, B. (2016) · Bioinformatics · doi:10.1093/bioinformatics/btw575 · Cited by 54
Self-citationSharing Clinical and Genomic Data on Cancer — The Need for Global Solutions
The Clinical Cancer Genome Task Team of the Global Alliance for Genomics and Health (2017) · New England Journal of Medicine · doi:10.1056/nejmp1612254 · Cited by 53
A simple and reproducible breast cancer prognostic test
Marchionni, L., Afsari, B., Geman, D., & Leek, J. T. (2013) · BMC Genomics · doi:10.1186/1471-2164-14-336 · Cited by 52 · cited 1×
“That is, researchers can use public data to boost the power available to analyse a locally generated data set, a paradigm that already prevails in micro-array data analysis 8,9,12,92 .”
BackgroundAbstract 2584: The UCSC Xena system for cancer genomics data visualization and interpretation
Goldman, M. J., Craft, B., Zhu, J., & Haussler, D. H. (2017) · Cancer Research · doi:10.1158/1538-7445.am2017-2584 · Cited by 51
Genomic knowledge sharing: A review of the ethical and legal issues
Francis, L. P. (2014) · Applied & Translational Genomics · doi:10.1016/j.atg.2014.09.003 · Cited by 47 · cited 1×
“Issues of privacy and data sovereignty go deeper than we explore here, and we refer the reader to other reviews on this topic 107–109 .”
BackgroundThe Proteome Folding Project: Proteome-scale prediction of structure and functionPreprint
Drew, K., Winters, P., Butterfoss, G. L., Berstis, V., Uplinger, K., Armstrong, J., Riffle, M. E., Schweighofer, E., Bovermann, B., Goodlett, D. R., et al. (2011) · Genome Research · doi:10.1101/gr.121475.111 · Cited by 46 · cited 1×
“The vast computational needs, together with issues of data sovereignty, motivated the use of a distributed analysis strategy reminiscent of the ‘grid comput-ing’ paradigm 69 , which predates cloud computing and has its own history of being applied to large-scale life science data sets 70 .”
MethodChard, R., Chard, K., Bubendorfer, K., Łaciński, Ł., Madduri, R., & Foster, I. T. (2015) · IEEE International Conference on eScience · doi:10.1109/escience.2015.67 · Cited by 45
Birger, C., Hanna, M., Salinas, E. A., Neff, J., Saksena, G., Livitz, D. G., Rosebrock, D., Stewart, C., Leshchiner, I., Baumann, A., et al. (2017) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/209494 · Cited by 45 · cited 4×
“…most visibly in projects such as the ICGC and the related Pan-Cancer Analysis of Whole Genomes (PCAWG) effort 21 , the National Cancer Institute (NCI) Cancer Genomics Cloud (CGC) Pilots 22–24 and the Encyclopedia of DNA Elements (ENCODE and Model Organism ENCODE (modENCODE)) projects 25,26 .”
BackgroundInfluentialThe RNASeq-er API—a gateway to systematically updated analysis of public RNA-seq data
Petryszak, R., Fonseca, N. A., Füllgrabe, A., Huerta, L., Keays, M., Tang, A., & Brāzma, A. (2017) · Bioinformatics · doi:10.1093/bioinformatics/btx143 · Cited by 31
Secure cloud computing for genomic data
Datta, S., Bettinger, K., & Snyder, M. (2016) · Nature Biotechnology · doi:10.1038/nbt.3496 · Cited by 30 · cited 1×
“Issues of privacy and data sovereignty go deeper than we explore here, and we refer the reader to other reviews on this topic 107–109 .”
BackgroundFederation in genomics pipelines: techniques and challenges
Chaterji, S., Koo, J., Li, N., Meyer, F., Grama, A. Y., & Bagchi, S. (2017) · Briefings in Bioinformatics · doi:10.1093/bib/bbx102 · Cited by 29 · cited 1×
“…data continue to evolve, we expect readers will increasingly encounter secure application programming interfaces (APIs), such as the Seven Bridges API 110 , and the related ideas of authentication, authorization and identity federation (for overviews of these concepts, see REFS 111,112 ).”
BackgroundLarge-Scale Uniform Analysis of Cancer Whole Genomes in Multiple Computing EnvironmentsPreprint
Yung, C. K., O’Connor, B. D., Yakneen, S., Zhang, J., Ellrott, K., Kleinheinz, K., Miyoshi, N., Raine, K. M., Royo, R., Saksena, G., et al. (2017) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/161638 · Cited by 28 · cited 2×
“Third parties have relevant SaaS offerings as well; for instance, preliminary PCAWG studies used a SaaS product called Elasticsearch to manage and index meta-data 36,37 and other products called Logstash and Kibana to index and analyse logging data 36 .”
BackgroundMethodLow-cost, low-input RNA-seq protocols perform nearly as well as high-input protocols
Combs, P. A., & Eisen, M. B. (2015) · PeerJ · doi:10.7717/peerj.869 · Cited by 25
Rapid and efficient analysis of 20,000 RNA-seq samples with ToilPreprint
Vivian, J., Rao, A. A., Nothaft, F. A., et al. (2016) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/062497 · Cited by 24
Ontology-based annotations and semantic relations in large-scale (epi)genomics data
Galeota, E., & Pelizzola, M. (2016) · Briefings in Bioinformatics · doi:10.1093/bib/bbw036 · Cited by 23
Marx, V. (2013) · Nature Methods · doi:10.1038/nmeth.2654 · Cited by 20
Don’t let useful data go to waste
Denk, F. (2017) · Nature · doi:10.1038/543007a · Cited by 20 · cited 1×
“Downloading, storing and analysing the data are resource-intensive and subject to bottlenecks, such as internet uplink speeds; many labs are not equipped for this, so valuable data go unused 6 .”
BackgroundProceedings of the 2005 ACM/IEEE conference on Supercomputing
Kramer, W. (2005) · Cited by 14
(2016) · Nature Genetics · doi:10.1038/ng.3642 · Cited by 14
Campbell, S. T. (2016) · Computer · doi:10.1109/mc.2016.286 · Cited by 11 · cited 1×
“It has been argued that this is in part because the educator cannot simply teach programming; to teach cloud computing requires us “to venture into operations, [operating systems], networking, and other applied areas” 113 .”
BackgroundHaq, A. I., Sarkar, M., Roy, S., & Alam, M. F. (2017) · Bangladesh Journal of Psychiatry · doi:10.3329/bjpsy.v29i1.32744 · Cited by 9
Rail-dbGaP: analyzing dbGaP-protected data in the cloud with Amazon Elastic MapReduce
Nellore, A., Wilks, C., Hansen, K. D., Leek, J. T., & Langmead, B. (2016) · Bioinformatics · doi:10.1093/bioinformatics/btw177 · Cited by 8 · cited 1×
“The Intropolis 60 and recount2 (REF.”
Self-citationCloud-based uniform ChIP-Seq processing tools for modENCODE and ENCODE
Trinh, Q. M., Jen, F.-Y. A., Zhou, Z., Chu, K. M., Perry, M. D., Kephart, E. T., Contrino, S., Ruzanov, P., & Stein, L. D. (2013) · BMC Genomics · doi:10.1186/1471-2164-14-494 · Cited by 6
Should biomedical research be like Airbnb?
Bonazzi, V. R., & Bourne, P. E. (2017) · PLoS Biology · doi:10.1371/journal.pbio.2001818 · Cited by 4 · cited 1×
“For example, the cloud is the home of the National Institutes of Health (NIH) Data Commons Pilot, an effort to increase avail ability and utility of data and software from NIH-funded efforts 48,49 .”
BackgroundAbstract 378: The Cancer Genome Collaboratory
Yung, C. K., Mihaiescu, G. L., Tiernay, B., Zhang, J., Gerthoffert, F., Yang, A., Baker, J., Bourque, G., Boutros, P. C., Knoppers, B. M., et al. (2017) · Cancer Research · doi:10.1158/1538-7445.am2017-378 · Cited by 3
Enabling rapid cloud-based analysis of thousands of human genomes via ButlerPreprint
Yakneen, S., Waszak, S. M., Gertz, M., & Korbel, J. O. (2017) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/185736 · Cited by 3 · cited 2×
“Third parties have relevant SaaS offerings as well; for instance, preliminary PCAWG studies used a SaaS product called Elasticsearch to manage and index meta-data 36,37 and other products called Logstash and Kibana to index and analyse logging data 36 .”
BackgroundMethodPetabyte-Scale Cancer Genomics in the Cloud
Davis‐Dusenbery, B. N. (2015) · Cancer Genetics · doi:10.1016/j.cancergen.2015.05.012 · Cited by 2
Abstract 3605: ICGC in the cloud
Yung, C. K., Bourque, G., Boutros, P. C., et al. (2016) · Cancer Research · doi:10.1158/1538-7445.am2016-3605
Research works printed but not linked: 2
- Mathews, L. Just how big is Amazon's AWS business? (hint: it's absolutely massive). Geek.com https://www.geek.com/chips/just-how-big-is-amazons-aws-business-hint-its-absolutely-massive-1610221/ (2014).
- Stewart, C. A. et al. in Proc. 2015 XSEDE Conference: Scientific Advancements Enabled by Enhanced Cyberinfrastructure https://dl.acm.org/citation.cfm?id=2792745 (2015).
Cited by
Sorted by citations
Visualizing and interpreting cancer genomics data via the Xena platform
Goldman, M. J., Craft, B., Hastie, M., Repečka, K., McDade, F., Kamath, A., Banerjee, A., Luo, Y., Rogers, D., Brooks, A. N., et al. (2020) · Nature Biotechnology · doi:10.1038/s41587-020-0546-8 · Cited by 4,481
Database resources of the National Center for Biotechnology Information
Sayers, E. W., Beck, J. D., Bolton, E., Bourexis, D., Brister, J. R., Canese, K., Comeau, D. C., Funk, K., Kim, S., Klimke, W. A., et al. (2020) · Nucleic Acids Research · doi:10.1093/nar/gkaa892 · Cited by 980
“The cloud environment offers many advantages for the transfer and analysis of data (12), and the availability of SRA on cloud platforms should facilitate large-scale computational operations and collaborations.”
BackgroundHyPhy 2.5—A Customizable Platform for Evolutionary Hypothesis Testing Using Phylogenies
Pond, S. L. K., Poon, A. F. Y., Velazquez, R., Weaver, S., Hepler, N. L., Murrell, B., Shank, S. D., Magalis, B. R., Bouvier, D., Nekrutenko, A., et al. (2019) · Molecular Biology and Evolution · doi:10.1093/molbev/msz197 · Cited by 767
“…computing infrastructure (Weaver et al. 2018) free of charge to the global Kosakovsky Pond et al. . doi:10.1093/molbev/msz197 MBE community, which can be accessed via an internet browser, with the ultimate goal of democratizing access to scientific computing resources (Langmead and Nellore 2018).”
BackgroundInfluentialThe UCSC Xena platform for public and private cancer genomics data visualization and interpretationPreprint
Goldman, M., Craft, B., Hastie, M., Repečka, K., Kamath, A., McDade, F., Rogers, D., Brooks, A. N., Zhu, J., & Haussler, D. H. (2018) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/326470 · Cited by 610
Methods for ChIP-seq analysis: A practical workflow and advanced applications
Nakato, R., & Sakata, T. (2021) · Methods · doi:10.1016/j.ymeth.2020.03.005 · Cited by 267
Metadata
- Authors
- Langmead, B., & Nellore, A.
- Affiliations
- Johns Hopkins University (US)
- Oregon Health & Science University (US)
- Corresponding author
- Langmead, Ben
- Nellore, Abhinav
- Venue
- Nature Reviews Genetics
- Published
- 2018-01-30
- Volume
- 19
- Issue
- 4
- Pages
- 208-219
- ISSN
- 1471-0056 (ISSN-L)
- 1471-0064
- PMC
- PMC6452449
- OpenAlex
- W2785694006
- Cited by
- 310
- References
- 118
- Citations by year
- 2026: 15 · 2025: 34 · 2024: 37 · 2023: 32 · 2022: 43 · 2021: 57 · 2020: 42 · 2019: 38 · 2018: 11
- FWCI
- 6.89
- Type
- journal article
- Language
- en
- Update notices
- erratum (2018-02-12)
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- Nature Reviews Genetics (publishedVersion)
- PubMed (publishedVersion)
- Europe PMC (PubMed Central) (acceptedVersion)
- PubMed Central (acceptedVersion)
- Funders
- University of California, Santa Cruz
- University of Chicago
- European Bioinformatics Institute
- National Cancer Institute
- National Institute of General Medical Sciences (R01GM118568)
- Topics
- Physical Sciences › Computer Science › Information Systems › Cloud Computing and Resource Management
- Social Sciences › Decision Sciences › Information Systems and Management › Scientific Computing and Data Management
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Genetics, Bioinformatics, and Biomedical Research
- Keywords
- cloud computing, genomic data analysis, next-generation sequencing, data sharing, international research collaboration, computational resources, data privacy, reproducibility
- MeSH terms
- Cloud Computing, Humans, Computational Biology, Internet, Genomics, High-Throughput Nucleotide Sequencing
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
