Bootstrapping cluster analysis: Assessing the reliability of conclusions from microarray experiments
Open accessAbstract
We introduce a general technique for making statistical inference from clustering tools applied to gene expression microarray data. The approach utilizes an analysis of variance model to achieve normalization and estimate differential expression of genes across multiple conditions. Statistical inference is based on the application of a randomization technique, bootstrapping. Bootstrapping has previously been used to obtain confidence intervals for estimates of differential expression for individual genes. Here we apply bootstrapping to assess the stability of results from a cluster analysis. We illustrate the technique with a publicly available data set and draw conclusions about the reliability of clustering results in light of variation in the data. The bootstrapping procedure relies on experimental replication. We discuss the implications of replication and good design in microarray experiments.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
After normalization, gene shaving, bootstrapping, and cluster analysis were performed, and slide effect, dye effect, variety effect, and duplicate design were taken into account in an ANOVA model.
“After normalization, gene shaving (22), bootstrapping (23), and cluster analysis were performed (24), and the slide effect, dye effect, variety effect, and duplicate design were taken into account in an ANOVA model (25).”
From A High-Fat Diet Coordinately Downregulates Genes Required for Mitochondrial Oxidative Phosphorylation in Skeletal Muscle (2005) · cites this paper as [23]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
CONFIDENCE LIMITS ON PHYLOGENIES: AN APPROACH USING THE BOOTSTRAP
Felsenstein, J. (1985) · Evolution · doi:10.1111/j.1558-5646.1985.tb00420.x · Cited by 44,735 · see citing works
An Introduction to the Bootstrap
Efron, B., & Tibshirani, R. J. (1994) · doi:10.1201/9780429246593 · Cited by 40,500 · see citing works
Draper, N. R., & Smith, H. (2005) · Technometrics · doi:10.1198/tech.2005.s303 · Cited by 17,993 · see citing works
Cluster analysis and display of genome-wide expression patterns
Eisen, M. B., Spellman, P. T., Brown, P. O., & Botstein, D. (1998) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.95.25.14863 · Cited by 12,617 · see citing works
Bootstrap Methods and their Application
Davison, A. C., & Hinkley, D. V. (1997) · doi:10.1017/cbo9780511802843 · Cited by 7,350 · see citing works
Bootstrap Methods and Their Application
Loughin, T. M., Davison, A. C., & HINKLEY, D. V. (1999) · Journal of the American Statistical Association · doi:10.2307/2669707 · Cited by 5,478 · see citing works
Draper, N. R., & Smith, H. (1998) · Wiley series in probability and statistics · doi:10.1002/9781118625590 · Cited by 5,086 · see citing works
Fisher, R. A. (1936) · BMJ · doi:10.1136/bmj.1.3923.554-a · Cited by 4,980 · see citing works
Gunst, R. F., Draper, N. R., & Smith, H. C. (1999) · Technometrics · doi:10.2307/1270576 · Cited by 3,795 · see citing works
An Introduction to the Bootstrap.
Buckland, S. T., Efron, B., & Tibshirani, R. J. (1994) · Biometrics · doi:10.2307/2532810 · Cited by 3,236 · see citing works
Tamayo, P., Slonim, D. K., Mesirov, J. P., Zhu, Q., Kitareewan, S., Dmitrovsky, E., Lander, E. S., & Golub, T. R. (1999) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.96.6.2907 · Cited by 2,924 · see citing works
Exploring the new world of the genome with DNA microarrays
Brown, P. O., & Botstein, D. (1999) · Nature Genetics · doi:10.1038/4462 · Cited by 2,430 · see citing works
Molecular classification of cutaneous malignant melanoma by gene expression profiling
Bittner, M. L., Meltzer, P. S., Chen, Y., Jiang, Y. Y., Seftor, E. A., Hendrix, M. J. C., Radmacher, M. D., Simon, R., Zohar Yakhini, B., Ben‐Dor, A., et al. (2000) · Nature · doi:10.1038/35020115 · Cited by 1,989 · see citing works
The Transcriptional Program of Sporulation in Budding Yeast
Chu, S., DeRisi, J. L., Eisen, M. B., Mulholland, J., Botstein, D., Brown, P. O., & Herskowitz, I. (1998) · Science · doi:10.1126/science.282.5389.699 · Cited by 1,860 · see citing works
Analysis of Variance for Gene Expression Microarray Data
Kerr, K. F., Martin, M., & Churchill, G. A. (2000) · Journal of Computational Biology · doi:10.1089/10665270050514954 · Cited by 1,310 · see citing works
Self-citationLee, M. T., Kuo, F. C., Whitmore, G. A., & Sklar, J. (2000) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.97.18.9834 · Cited by 861 · see citing works
Bootstrap confidence levels for phylogenetic trees.
Efron, B., Halloran, E., & Holmes, S. (1996) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.93.14.7085 · Cited by 713 · see citing works
Experimental design for gene expression microarrays
Kerr, K. F. (2001) · Biostatistics · doi:10.1093/biostatistics/2.2.183 · Cited by 709 · see citing works
Bootstrap Methods and Their Application
Buckland, S. T., Davison, A. C., & HINKLEY, D. V. (1998) · Biometrics · doi:10.2307/3109789 · Cited by 581 · see citing works
Statistical design and the analysis of gene expression microarray data
Kerr, K. F., & Churchill, G. A. (2007) · Genetics Research · doi:10.1017/s0016672308009713 · Cited by 561 · see citing works
Self-citationBootstrap confidence levels for phylogenetic trees
Efron, B., Halloran, E., & Holmes, S. (1996) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.93.23.13429 · Cited by 560 · see citing works
'Gene shaving' as a method for identifying distinct sets of genes with similar expression patterns
Hastie, T., Tibshirani, R., Eisen, M. B., Alizadeh, A., Levy, R., Staudt, L., Chan, W. C., Botstein, D., & Brown, P. (2000) · Genome Biology · doi:10.1186/gb-2000-1-2-research0003 · Cited by 547 · see citing works
Statistical design and the analysis of gene expression microarray data
Kerr, K. F., & Churchill, G. A. (2001) · Genetics Research · doi:10.1017/s0016672301005055 · Cited by 501 · see citing works
Self-citation
Flagged first, then by citations
Research works printed but not linked: 2
- [5]Hastie, T., Tibshirani, R., Eisen, M. B., Alizadeh, A., Levy, R., Staudt, L., Chan, W. C., Botstein, D. & Brown, P. (August 4, 2000) Genome Biol., http://genomebiology.com/2000/1/2/research/0003/. [DOI] [PMC free article] [PubMed]
- [6]Lazzeroni L, Owen A. Stanford Biostatistics Series: Technical Report 211. Stanford, CA: Stanford Univ.; 2000. [Google Scholar]
Cited by
Sorted by citations
Empirical Bayes analysis of a microarray experiment
Efron, B., Tibshirani, R., Storey, J. D., & Tusher, V. (2001) · Journal of the American Statistical Association · doi:10.1198/016214501753382129 · Cited by 1,733 · see citing works
Microarray data analysis: from disarray to consolidation and consensus
Allison, D. B., Cui, X., Page, G. P., & Sabripour, M. (2006) · Nature Reviews Genetics · doi:10.1038/nrg1749 · Cited by 1,339 · see citing works
Discovering regulatory and signalling circuits in molecular interaction networks
Ideker, T., Ozier, O., Schwikowski, B., & Siegel, A. F. (2002) · Bioinformatics · doi:10.1093/bioinformatics/18.suppl_1.s233 · Cited by 1,197 · see citing works
Is My Network Module Preserved and Reproducible?
Langfelder, P., Luo, R., Oldham, M. C., & Horvath, S. (2011) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1001057 · Cited by 1,012 · see citing works
Sparks, L. M., Xie, H., Koza, R. A., Mynatt, R., Hulver, M. W., Bray, G. A., & Smith, S. R. (2005) · Diabetes · doi:10.2337/diabetes.54.7.1926 · Cited by 629 · see citing works
Greenfield, A., Madar, A., Ostrer, H., & Bonneau, R. (2010) · PLoS ONE · doi:10.1371/journal.pone.0013397 · Cited by 255 · see citing works
Progress in the use of microarray technology to study the neurobiology of disease
Mirnics, K., & Pevsner, J. (2004) · Nature Neuroscience · doi:10.1038/nn1230 · Cited by 150 · see citing works
- Authors
- Kerr, M. K., & Churchill, G. A.
- Affiliations
- The Jackson Laboratory, Bar Harbor, ME 04609
- Jackson Laboratory (US)
- Corresponding author
- Churchill, Gary A.
- ORCID
- Churchill, Gary A.
- Venue
- Proceedings of the National Academy of Sciences
- Publisher
- National Academy of Sciences
- Published
- 2001-07-24
- Volume
- 98
- Issue
- 16
- Pages
- 8961-8965
- ISSN
- 0027-8424 (ISSN-L)
- 1091-6490
- PMC
- PMC55356
- OpenAlex
- W2137784044
- Cited by
- 474
- References
- 23
- Citations by year
- 2026: 4 · 2025: 4 · 2024: 2 · 2023: 4 · 2022: 4 · 2021: 4 · 2020: 7 · 2019: 8 · 2018: 7 · 2017: 10 · 2016: 8 · 2015: 5 · 2014: 13 · 2013: 14 · 2012: 21
- FWCI
- 16.52
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- Proceedings of the National Academy of Sciences (publishedVersion)
- Europe PMC (PubMed Central) (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- The Mouseion at the JAXlibrary (Jackson Laboratory) (submittedVersion)
- Funders
- Wellcome Trust
- National Institutes of Health (5u01hl066620-04)
- National Heart, Lung, and Blood Institute (HL66620)
- National Cancer Institute (CA88327, R21CA088327, R33CA088327)
- Topics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Gene expression and cancer classification
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Bioinformatics and Genomic Networks
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Cancer Research › Breast Cancer Treatment Studies
- Keywords
- bootstrapping, clustering, gene expression microarrays, differential gene expression, randomization, experimental replication
- MeSH terms
- Analysis of Variance, Models, Statistical, Cluster Analysis, Oligonucleotide Array Sequence Analysis
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
