The DOI matches its Crossref record; 63 of the 63 references it cites were found in a registry or an index.
The MicroArray Quality Control (MAQC)-II study of common practices for the development and validation of microarray-based predictive models
Open accessAbstract
Gene expression data from microarrays are being applied to predict preclinical and clinical endpoints, but the reliability of these predictions has not been established. In the MAQC-II project, 36 independent teams analyzed six microarray data sets to generate predictive models for classifying a sample with respect to one of 13 endpoints indicative of lung or liver toxicity in rodents, or of breast cancer, multiple myeloma or neuroblastoma in humans. In total, >30,000 models were built using many combinations of analytical methods. The teams generated predictive models without knowing the biological meaning of some of the endpoints and, to mimic clinical reality, tested the models on data that had not been used for training. We found that model performance depended largely on the endpoint and team proficiency and that different approaches generated models of similar performance. The conclusions and recommendations from MAQC-II should be useful for regulatory agencies, study committees and independent investigators that evaluate methods for global gene expression analysis.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Using the common intra-site DEGs to validate the called inter-site DEGs, the MCC41, 42 showed that PEER also yields a higher accuracy than the original limmavoom method (Supplementary Fig. 22d,e), indicating that global data normalization analysis methods such as PEER can also be used to improve RNA-seq data across both Illumina and the PGM platforms.”
From Detecting and correcting systematic variation in large-scale RNA sequencing data (2014) · cites this paper as [42]
In the citing paper
“for instance, the Food and Drug Administration (FDA) agency of the USA employed the MCC as the main evaluation measure in the MicroArray II / Sequencing Quality Control (MAQC/SEQC) projects [41, 42].”
From The advantages of the Matthews correlation coefficient (MCC) over F1 score and accuracy in binary classification evaluation (2020) · cites this paper as [41]
In the citing paper
“Within the MicroArray Quality Control (MAQC)-II study [7], we observed, however, that the performance of gene expression models in predicting clinical outcome was limited and largely dependent on the respective clinical endpoint.”
From Comparison of RNA-seq and microarray-based models for clinical endpoint prediction (2015) · cites this paper as [7]
In the citing paper
“Within the MicroArray Quality Control (MAQC)-II study [7], we observed, however, that the performance of gene expression models in predicting clinical outcome was limited and largely dependent on the respective clinical endpoint.”
From Comparison of RNA-seq and microarray-based models for clinical endpoint prediction (2015) · cites this paper as [7]
In the citing paper
“Considering the fact that the nature of the clinical endpoint strongly affects a classifier’s performance [7, 18], we included endpoints that are known to cover a broad range of predictive difficulties (low difficulty, SEX, and CLASS LABEL; intermediate difficulty, EFS ALL, and OS ALL; high difficulty, EFS HR, and OS HR).”
From Comparison of RNA-seq and microarray-based models for clinical endpoint prediction (2015) · cites this paper as [7]
In the citing paper
“Evaluation of the prediction performances in the validation set revealed that the endpoint represents the most relevant factor affecting model performances, which is well in line with the findings of the MAQC-II study [7].”
From Comparison of RNA-seq and microarray-based models for clinical endpoint prediction (2015) · cites this paper as [7]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Gene expression profiling predicts clinical outcome of breast cancer
van 't Veer, L. J., Dai, H., van de Vijver, M. J., He, Y. D., Hart, A. A. M., Mao, M., Peterse, H. L., van der Kooy, K., Marton, M. J., Witteveen, A. T., et al. (2002) · Nature · doi:10.1038/415530a · Cited by 9,536
Unbiased Recursive Partitioning: A Conditional Inference Framework
Hothorn, T., Hornik, K., & Zeileis, A. (2006) · Journal of Computational and Graphical Statistics · doi:10.1198/106186006x133933 · Cited by 4,021
Comparison of Discrimination Methods for the Classification of Tumors Using Gene Expression Data
Dudoit, S., Fridlyand, J., & Speed, T. P. (2002) · Journal of the American Statistical Association · doi:10.1198/016214502753479248 · Cited by 2,671 · cited 1×
“Although previous studies have compared and benchmarked individual steps in the model development proces”
ResultMAQC Consortium, Shi, L., Shi, L., Reid, L. H., Jones, W. D., Shippy, R., Warrington, J. A., Baker, S. C., Collins, P. J., de Longueville, F., et al. (2006) · Nature Biotechnology · doi:10.1038/nbt1239 · Cited by 2,126
Self-citationThat BLUP is a Good Thing: The Estimation of Random Effects
Robinson, G. K. (1991) · Statistical Science · doi:10.1214/ss/1177011926 · Cited by 1,694
Selection bias in gene extraction on the basis of microarray gene-expression data
Ambroise, C., & McLachlan, G. J. (2002) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.102102699 · Cited by 1,537 · cited 1×
“, poor data quality and/or insufficient cross-validation of all model development step”
BackgroundDavé, S. S., Wright, G. W., Tan, B. K., Rosenwald, A., Gascoyne, R. D., Chan, W. C., Fisher, R. I., Braziel, R. M., Rimsza, L. M., Grogan, T. M., et al. (2004) · New England Journal of Medicine · doi:10.1056/nejmoa041869 · Cited by 1,417 · cited 1×
“Previously, reanalysis of a widely cited single stud”
BackgroundMicroarray data analysis: from disarray to consolidation and consensus
Allison, D. B., Cui, X., Page, G. P., & Sabripour, M. (2006) · Nature Reviews Genetics · doi:10.1038/nrg1749 · Cited by 1,339
Buyse, M. E., Loi, S., Veer, L. V., Viale, G., Delorenzi, M., Glas, A. M., Saghatchian, M., Bergh, J. C. S., Lidereau, R., Ellis, P., et al. (2006) · JNCI Journal of the National Cancer Institute · doi:10.1093/jnci/djj329 · Cited by 1,238
Shedden, K., Taylor, J. M. G., Misek, D. E., Chang, A. C., Hanash, S., Kuick, R. D., Beer, D. G., Enkemann, S. A., Yeatman, T. J., Eschrich, S. A., et al. (2008) · Nature Medicine · doi:10.1038/nm.1790 · Cited by 1,176
Flagged first, then by citations
Cited by
Sorted by citations
Chicco, D., & Jurman, G. (2020) · BMC Genomics · doi:10.1186/s12864-019-6413-7 · Cited by 6,143
Applications of machine learning in drug discovery and development
Vamathevan, J., Clark, D., Czodrowski, P., Dunham, I., Ferran, E., Lee, G., Li, B., Madabhushi, A., Shah, P., Spitzer, M., et al. (2019) · Nature Reviews Drug Discovery · doi:10.1038/s41573-019-0024-5 · Cited by 3,228
Moffitt, R. A., Marayati, R., Flate, E. L., Volmar, K. E., Loeza, S. G. H., Hoadley, K. A., Rashid, N. U., Williams, L. A., Eaton, S. C., Chung, A. H., et al. (2015) · Nature Genetics · doi:10.1038/ng.3398 · Cited by 2,194
Self-citationOptimal classifier for imbalanced data using Matthews Correlation Coefficient metric
Boughorbel, S., Jarray, F., & El-Anbari, M. (2017) · PLOS ONE · doi:10.1371/journal.pone.0177678 · Cited by 1,222
The genetic landscape of high-risk neuroblastoma
Pugh, T. J., Morozova, O., Attiyeh, E. F., Asgharzadeh, S., Wei, J. S., Auclair, D., Carter, S. L., Cibulskis, K., Hanna, M., Kieżun, A., et al. (2013) · Nature Genetics · doi:10.1038/ng.2529 · Cited by 1,198
Metadata
- Authors
- MAQC Consortium
- Venue
- Nature Biotechnology
- Published
- 2010-07-30
- Volume
- 28
- Issue
- 8
- Pages
- 827-838
- ISSN
- 0733-222X (ISSN-L)
- 1087-0156
- 1546-1696
- PMC
- PMC3315840
- OpenAlex
- W2021560851
- Cited by
- 889
- References
- 63
- Citations by year
- 2026: 13 · 2025: 33 · 2024: 39 · 2023: 35 · 2022: 40 · 2021: 58 · 2020: 61 · 2019: 58 · 2018: 46 · 2017: 35 · 2016: 57 · 2015: 54 · 2014: 87 · 2013: 85 · 2012: 79
- FWCI
- 34.94
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- Nature Biotechnology (publishedVersion)
- PubMed (publishedVersion)
- Europe PMC (PubMed Central) (acceptedVersion)
- PubMed Central (acceptedVersion)
- Dépôt institutionnel de l'Université libre de Bruxelles (Université Libre de Bruxelles) (submittedVersion)
- Infoscience (Ecole Polytechnique Fédérale de Lausanne) (submittedVersion)
- Infoscience (Ecole Polytechnique Fédérale de Lausanne) (submittedVersion, cc-by-nc-nd)
- Institute of Process Engineering, CAS Institutional Repository (submittedVersion, cc-by-nc-sa)
- IRIS (submittedVersion)
- Repository of Futwangen University of Applied Sciences (Furtwangen University) (submittedVersion)
- SERVAL (Université de Lausanne) (submittedVersion)
- Funders
- National Institutes of Health (Z99 LM999999, Z99 HL999999, Z99 MH999999, Z99CL999999)
- National Cancer Institute (CA97513-01, CA55819-01A1, R00CA126184, P01CA055819, R33CA097513)
- National Institute of General Medical Sciences (T32GM074906, R01GM083084, R01GM083226, R01GM072611)
- National Center for Research Resources (R01RR021967)
- Topics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Gene expression and cancer classification
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Cancer Research › Breast Cancer Treatment Studies
- Life Sciences › Pharmacology, Toxicology and Pharmaceutics › Drug Discovery › Computational Drug Discovery Methods
- Keywords
- microarray quality control, gene expression profiling, prediction model, genomic classifier, model validation, pulmonary toxicity, hepatotoxicity, breast cancer, multiple myeloma, neuroblastoma, rodent toxicity, method comparison, reproducibility
- MeSH terms
- Animals, Breast Neoplasms, Disease Models, Animal, Female, Humans, Liver Diseases, Lung Diseases, Multiple Myeloma, Neoplasms, Neuroblastoma, Predictive Value of Tests, Quality Control, Survival Analysis, Guidelines as Topic, Oligonucleotide Array Sequence Analysis, Gene Expression Profiling, Rats
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
