The MicroArray Quality Control (MAQC) project shows inter- and intraplatform reproducibility of gene expression measurements
Open accessThe DOI matches its Crossref record; 43 of the 43 references it cites were found in a registry or an index.
Abstract
Over the last decade, the introduction of microarray technology has had a profound impact on gene expression research. The publication of studies with dissimilar or altogether contradictory results, obtained using different microarray platforms to analyze identical RNA samples, has raised concerns about the reliability of this technology. The MicroArray Quality Control (MAQC) project was initiated to address these concerns, as well as other performance and data analysis issues. Expression data on four titration pools from two distinct reference RNA samples were generated at multiple test sites using a variety of microarray-based and alternative technology platforms. Here we describe the experimental design and probe mapping efforts behind the MAQC project. We show intraplatform consistency across test sites as well as a high level of interplatform concordance in terms of genes identified as differentially expressed. This study provides a resource that represents an important first step toward establishing a framework for the use of microarrays in clinical and regulatory settings.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Next, we applied the |log 2 fold-change|>1 filter advocated in the MAQC study on microarrays, 10 and for microarrays observed a reduction in the eFDR to below 1.5%, save for one outlier site (Figs 3c,d).”
From A comprehensive assessment of RNA-seq accuracy, reproducibility and information content by the Sequencing Quality Control Consortium (2014) · cites this paper as [10]
In the citing paper
“Studies on microarrays have shown that results of typical statistical differential expression tests thresholded by p -value need to be filtered and sorted by effect strength (fold-change) in order to attain robust comparisons across platforms and sites. 10”
From A comprehensive assessment of RNA-seq accuracy, reproducibility and information content by the Sequencing Quality Control Consortium (2014) · cites this paper as [10]
In the citing paper
“The Microarray Quality Control (MAQC) Consortium landmark study10 examined the degree of variance within and across many different microarray platforms and found similar coefficients of variation between platforms.”
From Multi-platform assessment of transcriptome profiling using RNA-seq in the ABRF next-generation sequencing study (2014) · cites this paper as [10]
In the citing paper
“prior work comparing microarray platforms and methods showed high levels of inter-platform concordance for the ability to detect differentially expressed genes.”
From Multi-platform assessment of transcriptome profiling using RNA-seq in the ABRF next-generation sequencing study (2014) · cites this paper as [10]
Citations
Cites
In silico prediction of protein-protein interactions in human macrophages
Souiai, O., Guerfali, F., Ben Miled, S., Brun, C., & Benkahla, A. (2014) · BMC Research Notes · doi:10.1186/1756-0500-7-157 · Cited by 10,803
Significance analysis of microarrays applied to the ionizing radiation response
Tusher, V. G., Tibshirani, R., & Chu, G. (2001) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.091062498 · Cited by 10,568
Pruitt, K. D., Tatusova, T., & Maglott, D. R. (2007) · Nucleic Acids Research · doi:10.1093/nar/gkl842 · Cited by 4,481 · cited 1×
“We mapped the probes (see Supplementary Methods online and Supplementary Notes online) to the RefSeq human mRNA databas”
Method"Stemness": Transcriptional Profiling of Embryonic and Adult Stem Cells
Ramalho-Santos, M., Yoon, S., Matsuzaki, Y., Mulligan, R. C., & Melton, D. A. (2002) · Science · doi:10.1126/science.1072530 · Cited by 1,709
Pruitt, K. D. (2004) · Nucleic Acids Research · doi:10.1093/nar/gki025 · Cited by 1,660 · cited 1×
“We mapped the probes (see Supplementary Methods online and Supplementary Notes online) to the RefSeq human mRNA database 31 (http://www.ncbi.nlm.nih.gov/RefSeq) and to the AceView database 32 (http://www.ncbi.nlm.nih.gov/IEB/Research/Acembly), a less curated but more comprehensive database, which…”
MethodA Stem Cell Molecular Signature
Ivanova, N. B., Dimos, J. T., Schaniel, C., Hackney, J. A., Moore, K. A., & Lemischka, I. R. (2002) · Science · doi:10.1126/science.1073823 · Cited by 1,542
Prediction of cancer outcome with microarrays: a multiple random validation strategy
Michiels, S., Koscielny, S., & Hill, C. (2005) · The Lancet · doi:10.1016/s0140-6736(05)17866-0 · Cited by 1,025 · cited 1×
“, raising doubts about the repeatability, reproducibility and comparability of microarray technolog”
BackgroundMultiple-laboratory comparison of microarray platforms
Irizarry, R. A., Warren, D., Spencer, F., Kim, I. F., Biswal, S., Frank, B. C., Gabrielson, E., Garcia, J. G. N., Geoghegan, J., Germino, G., et al. (2005) · Nature Methods · doi:10.1038/nmeth756 · Cited by 862
Self-citationEvaluation of gene expression measurements from commercial microarray platforms
Tan, P. K. (2003) · Nucleic Acids Research · doi:10.1093/nar/gkg763 · Cited by 706
Thousands of samples are needed to generate a robust gene list for predicting outcome in cancer
Ein-Dor, L., Zuk, O., & Domany, E. (2006) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0601231103 · Cited by 689 · cited 1×
“…to appear in peer-reviewed journals 9,10 , raising doubts about the repeatability, reproducibility and comparability of microarray technology 11–13 , several studies have also been recently published showing increased reproducibility of microarray data generated at different test sites and/or…”
Background
Research works printed but not linked: 1
- Reid, L.H. The value of a proficiency testing program to monitor performance in microarray laboratories. Pharm. Discov. 5, 20–25 (2005).
Cited by
Items 201–300 of 1,004
Hoyles, R. K., Derrett-Smith, E. C., Khan, K., Shiwen, X., Howat, S. L., Wells, A. U., Abraham, D. J., & Denton, C. P. (2011) · American Journal of Respiratory and Critical Care Medicine · doi:10.1164/rccm.201002-0279oc · Cited by 126
A protocol to evaluate RNA sequencing normalization methods
Abrams, Z. B., Johnson, T. S., Huang, K., Payne, P., & Coombes, K. R. (2019) · BMC Bioinformatics · doi:10.1186/s12859-019-3247-x · Cited by 141
Voisin, D., Nawrath, C., Kurdyukov, S., Franke, R. B., Reina-Pinto, J. J., Efremova, N., Will, I., Schreiber, L., & Yephremov, A. (2009) · PLoS Genetics · doi:10.1371/journal.pgen.1000703 · Cited by 129
Cahan, P., Rovegno, F., Mooney, D. L., Newman, J. C., Laurent, G. S., & McCaffrey, T. A. (2007) · Gene · doi:10.1016/j.gene.2007.06.016 · Cited by 129
Systematic evaluation of variability in ChIP-chip experiments using predefined DNA targets
Johnson, D. S., Li, W., Gordon, D. B., Bhattacharjee, A., Curry, B., Ghosh, J., Brizuela, L., Carroll, J. S., Brown, M., Flicek, P., et al. (2008) · Genome Research · doi:10.1101/gr.7080508 · Cited by 130
Implementing prognostic and predictive biomarkers in CRC clinical trials
Schaeybroeck, S. V., Allen, W. L., Turkington, R. C., & Johnston, P. G. (2011) · Nature Reviews Clinical Oncology · doi:10.1038/nrclinonc.2011.15 · Cited by 141
“The phase I component of this study has demonstrated that microarray experiments can be reproducible and comparable if careful experimental design and data analysis is used”
MethodMunro, S. A., Lund, S. P., Pine, P. S., Binder, H., Clevert, D.-A., Conesa, A., Dopazo, J., Fasold, M., Hochreiter, S., Hong, H., et al. (2014) · Nature Communications · doi:10.1038/ncomms6125 · Cited by 124
Self-citationElectrophoretic and field-effect graphene for all-electrical DNA array technology
Xu, G., Abbott, J., Qin, L., Yeung, K. Y. M., Song, Y., Yoon, H., Kong, J., & Ham, D. (2014) · Nature Communications · doi:10.1038/ncomms5866 · Cited by 126
New Strategies in Hepatocellular Carcinoma: Genomic Prognostic Markers
Villanueva, A., Hoshida, Y., Toffanin, S., Lachenmayer, A., Alsinet, C., Savić, R., Cornellà, H., & Llovet, J. M. (2010) · Clinical Cancer Research · doi:10.1158/1078-0432.ccr-09-1811 · Cited by 129
Effects of Tissue Handling on RNA Integrity and Microarray Measurements From Resected Breast Cancers
Hatzis, C., Sun, H., Yao, H., Hubbard, R. E., Meric-Bernstam, F., Babiera, G. V., Wu, Y., Pusztai, L., & Symmans, W. F. (2011) · JNCI: Journal of the National Cancer Institute · doi:10.1093/jnci/djr438 · Cited by 122
Self-citationWang, M., Gong, Q., Zhang, J., Chen, L., Zhang, Z., Lu, L., Yu, D., Han, Y., Zhang, D., Chen, P., et al. (2017) · Scientific Reports · doi:10.1038/srep43446 · Cited by 126
Pathway discovery using transcriptomic profiles in adult-onset severe asthma
Hekking, P.-P. W., Loza, M. J., Pavlidis, S. P., Meulder, B. D., Lefaudeux, D., Baribaud, F., Auffray, C., Wagener, A. H., Brinkman, P., Lutter, R., et al. (2017) · Journal of Allergy and Clinical Immunology · doi:10.1016/j.jaci.2017.06.037 · Cited by 134
“This procedure follows the Microarray Consortium for Quality Control recommendations for the importance of applying group-difference thresholds.(21) RESULTS Of all 421 adult patients with severe asthma in theU-BIOPRED cohort, 253 (60.”
MethodChallenges translating breast cancer gene signatures into the clinic
Weigelt, B., Pusztai, L., Ashworth, A., & Reis-Filho, J. S. (2011) · Nature Reviews Clinical Oncology · doi:10.1038/nrclinonc.2011.125 · Cited by 133
“Notably, microarray results, particularly if performed in different laboratories, also show variability and discordance”
Self-citationApparently low reproducibility of true differential expression discoveries in microarray studies
Zhang, M., Yao, C., Guo, Z., Zou, J., Zhang, L., Xiao, H., Wang, D., Yang, D., Gong, X., Zhu, J., et al. (2008) · Bioinformatics · doi:10.1093/bioinformatics/btn365 · Cited by 131
Self-citationPadler-Karavani, V., Song, X., Yu, H., Hurtado-Ziola, N., Huang, S., Muthana, S., Chokhawala, H. A., Cheng, J., Verhagen, A., Langereis, M. A., et al. (2012) · Journal of Biological Chemistry · doi:10.1074/jbc.m112.359323 · Cited by 120
Self-citationWang, L., Brugge, J. S., & Janes, K. A. (2011) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1103423108 · Cited by 132
“Although indispensable for global analyses, microarrays tend to dampen expression differences compared with when the same samples are analyzed by qPCR (40).”
BackgroundMolecular intrinsic versus clinical subtyping in breast cancer: A comprehensive review
Szymiczek, A., Lone, A., & Akbari, M. R. (2020) · Clinical Genetics · doi:10.1111/cge.13900 · Cited by 135
Yu, Y., Zhang, N., Mai, Y., Ren, L., Chen, Q., Cao, Z., Chen, Q., Liu, Y., Hou, W., Yang, J., et al. (2023) · Genome Biology · doi:10.1186/s13059-023-03047-z · Cited by 109
“According to recommendations from MAQC/SEQC Consortia [5, 60], a nonstringent t-test p value cutoff with a sufficient FC could be used to identify differentially expressed genes.”
BackgroundMethodSelf-citationDiaz‐Romero, J., Nešić, D., Grogan, S. P., Heini, P., & Mainil‐Varlet, P. (2007) · Journal of Cellular Physiology · doi:10.1002/jcp.21161 · Cited by 133
“…using a combination of the relative fold-change of the expression level at different time points compared to the beginning of the culture and a nonstringent P cutoff, an approach used for exploring differentially expressed genes by microarray technology (Zhao et al., 2004; Shi et al., 2006).”
MethodPavlopoulos, G. A., Malliarakis, D., Papanikolaou, N., Θεοδοσίου, Θ., Enright, A. J., & Iliopoulos, I. (2015) · GigaScience · doi:10.1186/s13742-015-0077-2 · Cited by 129
“Although microarrays and RNAseq technologies are comparable to each other [113], the latter tends to dominate, especially as sequencing technologies have improved, and there now are robust statistics to model the particular noise characteristics of RNAseq, particularly for low expression [114].”
BackgroundBaaske, M. D., & Vollmer, F. (2011) · ChemPhysChem · doi:10.1002/cphc.201100757 · Cited by 134
Enciso, M., Carrascosa, J. P., Sarasa, J., Martínez-Ortiz, P. A., Munné, S., Horcajadas, J. A., & Aizpurúa, J. (2018) · Human Reproduction · doi:10.1093/humrep/dex370 · Cited by 133
“(MAQC, 2006; Wang et al., 2009).”
Adjusting for Batch Effects in DNA Methylation Microarray Data, a Lesson Learned
Price, E. M., & Robinson, W. P. (2018) · Frontiers in Genetics · doi:10.3389/fgene.2018.00083 · Cited by 129
“In 2006, a consortium of scientists and organizations, ran the MicroArray Quality Control or MAQC project to systematically test for batch effects in gene expression microarray data (Shi et al., 2006).”
BackgroundWeber, G. J., Sepúlveda, M. S., Peterson, S. M., Lewis, S. S., & Freeman, J. L. (2013) · Toxicological Sciences · doi:10.1093/toxsci/kft017 · Cited by 117
“…list of differentially expressed genes for each atrazine treatment using recommendations from the Microarray Quality Consortium (Guo et al., 2006; Shi et al., 2006) was first determined by genes consistently DEVELOPMENTAL ATRAZINE TOXICITY expressed using an ANOVA in SAS software (p < 0.1) and…”
MethodInfluentialThe influence of genetic variation on gene expression
Williams, R. B. H., Chan, E. K. F., Cowley, M. J., & Little, P. (2007) · Genome Research · doi:10.1101/gr.6981507 · Cited by 135
Culture, Immortalization, and Characterization of Human Meibomian Gland Epithelial Cells
Liu, S., Hatton, M. P., Khandelwal, P., & Sullivan, D. A. (2010) · Investigative Ophthalmology & Visual Science · doi:10.1167/iovs.09-5108 · Cited by 117
“Normalized hybridization intensity values were adjusted by adding a constant such that the lowest intensity value for any sample equaled 16.(35) Normalized data were evaluated with a comprehensive program (GeneSifter.”
MethodBustin, S., & Nolan, T. (2017) · European Journal of Clinical Investigation · doi:10.1111/eci.12801 · Cited by 113
Szafranska, A. E., Davison, T. S., Shingara, J., Doleshal, M., Riggenbach, J. A., Morrison, C. D., Jewell, S. D., & Labourier, E. (2008) · Journal of Molecular Diagnostics · doi:10.2353/jmoldx.2008.080018 · Cited by 123
Self-citationHuang, R., Jaritz, M., Guenzl, P., Vlatkovic, I., Sommer, A., Tamir, I. M., Marks, H., Klampfl, T., Kralovics, R., Stunnenberg, H. G., et al. (2011) · PLoS ONE · doi:10.1371/journal.pone.0027288 · Cited by 115
“However compared to the rapid development of bioinformatic analysis tools, little progress has been made in standardizing template preparation protocols in a similar manner as proposed for microarray expression platforms [18].”
BackgroundRohart, F., Eslami, A., Matigian, N., Bougeard, S., & Lê Cao, K.-A. (2017) · BMC Bioinformatics · doi:10.1186/s12859-017-1553-8 · Cited by 108
“The extensive MAQC project focused on assessing microarray technologies reproducibility in a controlled environment [5].”
BackgroundInfluentialZhang, M., Zhang, L., Zou, J., Yao, C., Xiao, H., Liu, Q., Wang, J., Wang, D., Wang, C., & Guo, Z. (2009) · Bioinformatics · doi:10.1093/bioinformatics/btp295 · Cited by 116
“Results: We proposed metrics percentage of overlapping genesrelated (POGR) and normalized POGR (nPOGR) to evaluate the consistency between two DEG lists for a complex disease, considering correlated molecular changes rather than only counting gene overlaps between the lists.”
MethodSelf-citationImproved microRNA quantification in total RNA from clinical samples
Andreasen, D., Fog, J. U., Biggs, W., Salomon, J., Dahslveen, I. K., Baker, A., & Mouritzen, P. (2010) · Methods · doi:10.1016/j.ymeth.2010.01.006 · Cited by 124
Barry, W. T., Kernagis, D., Dressman, H. K., Griffis, R. J., Hunter, J. D., Olson, J. A., Marks, J., Ginsburg, G. S., Marcom, P. K., Nevins, J. R., et al. (2010) · Journal of Clinical Oncology · doi:10.1200/jco.2009.26.7245 · Cited by 110
“project.(8-12) Variance intrinsic to a sample is more difficult to control, particularly for solid tumor specimens where intratumor heterogeneity could result in significant sampling bias.”
BackgroundBrucklacher, R. M., Patel, K. M., VanGuilder, H. D., Bixler, G. V., Barber, A. J., Antonetti, D. A., Lin, C.-M., LaNoue, K. F., Gardner, T. W., Bronson, S. K., et al. (2008) · BMC Medical Genomics · doi:10.1186/1755-8794-1-26 · Cited by 107
Disease signatures are robust across tissues and experiments
Dudley, J. T., Tibshirani, R., Deshpande, T., & Butte, A. J. (2009) · Molecular Systems Biology · doi:10.1038/msb.2009.66 · Cited by 110
“One might have expected a relatively strong degree of concordance between diseases experiments sampled from the same tissue, given the number of genes likely to be involved in tissue-specific biology (Kilpinen et al, 2008; Shyamsundar et al, 2005).”
BackgroundJung, K., Dardick, C. D., Bartley, L., Cao, P., Phetsom, J., Canlas, P. E., Seo, Y.-S., Shultz, M. A., Ouyang, S., Yuan, Q., et al. (2008) · PLoS ONE · doi:10.1371/journal.pone.0003337 · Cited by 116
“Recently, the MicroArray Quality Control (MAQC) project determined that there was a high level of consistency in data across different array platforms [59].”
BackgroundEvaluation of external RNA controls for the assessment of microarray performance
Tong, W., Lucas, A. B., Shippy, R., Fan, X., Fang, H., Hong, H., Orr, M. S., Chu, T., Guo, X., Collins, P. J., et al. (2006) · Nature Biotechnology · doi:10.1038/nbt1237 · Cited by 109
Self-citationCharacterizing the mouse ES cell transcriptome with Illumina sequencing
Rosenkranz, R., Borodina, T., Lehrach, H., & Himmelbauer, H. (2008) · Genomics · doi:10.1016/j.ygeno.2008.05.011 · Cited by 106
Brancato, V., Esposito, G., Coppola, L., Cavaliere, C., Mirabelli, P., Scapicchio, C., Borgheresi, R., Neri, E., Salvatore, M., & Aiello, M. (2024) · Journal of Translational Medicine · doi:10.1186/s12967-024-04891-8 · Cited by 106
Biophysical properties of nucleic acids at surfaces relevant to microarray performance
Rao, A. N., & Grainger, D. W. (2014) · Biomaterials Science · doi:10.1039/c3bm60181a · Cited by 120
Lahens, N. F., Ricciotti, E., Smirnova, O., Toorens, E., Kim, E. J., Baruzzo, G., Hayer, K. E., Ganguly, T., Schug, J., & Grant, G. R. (2017) · BMC Genomics · doi:10.1186/s12864-017-4011-0 · Cited by 115
“The UHRR is a mixture of ten human cell lines derived from various tissues [10] and has served as a standard for many studies benchmarking microarray and RNASeq performance [11, 12].”
BackgroundRevisiting inconsistency in large pharmacogenomic studies
Safikhani, Z., Smirnov, P., Freeman, M., El-Hachem, N., She, A., Rene, Q., Goldenberg, A., Birkbak, N. J., Hatzis, C., Shi, L., et al. (2017) · F1000Research · doi:10.12688/f1000research.9611.3 · Cited by 76
Self-citationDeMix: deconvolution for mixed cancer transcriptomes using raw measured data
Ahn, J., Yuan, Y., Parmigiani, G., Suraokar, M. B., Diao, L., Wistuba, I. I., & Wang, W. (2013) · Bioinformatics · doi:10.1093/bioinformatics/btt301 · Cited by 106
Highly sensitive and specific microRNA expression profiling using BeadArray technology
Chen, J., Lozach, J., Garcia, E. W., Barnes, B., Luo, S., Mikoulitch, I., Zhou, L., Schroth, G., & Fan, J.-B. (2008) · Nucleic Acids Research · doi:10.1093/nar/gkn387 · Cited by 105
“However, this correlation range is quite similar to the cross-microarray platform comparisons done in the MAQC study (35).”
ResultSelf-citationWimmer, I., Tröscher, A. R., Brunner, F., Rubino, S. J., Bien, C. G., Weiner, H. L., Lassmann, H., & Bauer, J. (2018) · Scientific Reports · doi:10.1038/s41598-018-24781-6 · Cited by 111
Transcriptomes of the B and T Lineages Compared by Multiplatform Microarray Profiling
Painter, M. W., Davis, S., Hardy, R. R., Mathis, D., Benoist, C., The Immunological Genome Project Consortium, Zhou, Y., Shinton, S., Hardy, R., Asinovski, N., et al. (2011) · The Journal of Immunology · doi:10.4049/jimmunol.1002695 · Cited by 102
Identification of Common Differentially Expressed Genes in Urinary Bladder Cancer
Zaravinos, A., Lambrou, G. Ι., Boulalas, I., Delakas, D., & Spandidos, D. A. (2011) · PLoS ONE · doi:10.1371/journal.pone.0018135 · Cited by 110
“Microarray Experimentation and Inclusion of Publicly available Microarray Datasets Oligos microarray chips (,57 k genes) were obtained from GE HealthCare (IL) and AppliedMicroarrays (MA) (former Amersham Biosciences) (CodeLink 57 k Human Whole Genome) [11,12,13].”
MethodEcotoxicogenomics: Emerging Technologies for Emerging Contaminants1
Poynton, H. C., & Vulpe, C. D. (2009) · JAWRA Journal of the American Water Resources Association · doi:10.1111/j.1752-1688.2008.00291.x · Cited by 135
Medeiros, F., Rigl, C. T., Anderson, G. G., Becker, S. H., & Halling, K. C. (2007) · Archives of Pathology & Laboratory Medicine · doi:10.5858/2007-131-1805-thfgea · Cited by 114
Nehmé, A., Lobenhofer, E. K., Stamer, W. D., & Edelman, J. L. (2009) · BMC Medical Genomics · doi:10.1186/1755-8794-2-58 · Cited by 104
“RNA isolation and gene expression profiling RNA isolation and gene expression profiling were performed at CogenicsTM, A Division of Clinical Data® (Morrisville, NC), using methods that have been previously described [28-31].”
Self-citationOvercoming bias and systematic errors in next generation sequencing data
Taub, M. A., Bravo, H. C., & Irizarry, R. A. (2010) · Genome Medicine · doi:10.1186/gm208 · Cited by 120
“Some of this work was done under the auspices of the Food and Drug Administration (FDA), most notably the Microarray Quality Control (MAQC) studies, which were developed specifically in order to determine the utility of microarray technologies in a clinical setting [ 5 ,6].”
MethodRNA-seq of human reference RNA samples using a thermostable group II intron reverse transcriptase
Nottingham, R. M., Wu, D. C., Qin, Y., Yao, J., Hunicke‐Smith, S., & Lambowitz, A. M. (2016) · RNA · doi:10.1261/rna.055558.115 · Cited by 114
Data Standards for Omics Data: The Basis of Data Sharing and Reuse
Chervitz, S. A., Deutsch, E. W., Field, D., Parkinson, H., Quackenbush, J., Rocca-Serra, P., Sansone, S.-A., Stoeckert, C. J., Taylor, C. F., Taylor, R., et al. (2011) · Methods in Molecular Biology · doi:10.1007/978-1-61779-027-0_2 · Cited by 103
“…diagnostic arena, organizations such as the US Food and Drug Administration (FDA) and technology manufacturers are becoming more involved in a range of standards efforts, for example the MicroArray Quality Control (MAQC) consortium brings together representatives of many such organizations (20).”
BackgroundIdentification of Alzheimer's disease–associated long noncoding RNAs
Zhou, X., & Xu, J. (2015) · Neurobiology of Aging · doi:10.1016/j.neurobiolaging.2015.07.015 · Cited by 102
Self-citationHornett, E. A., & Wheat, C. W. (2012) · BMC Genomics · doi:10.1186/1471-2164-13-361 · Cited by 115
“RNA-Seq data for all analyses were generated from the same RNA reference sample (Human Brain Reference RNA) of the MicroArray Quality Control (MAQC) project (MAQC Consortium 2006; [43]).”
MethodHu, Y., Chopra, V., Chopra, R., Locascio, J. J., Liao, Z., Ding, H., Zheng, B., Matson, W. R., Ferrante, R. J., Rosas, H. D., et al. (2011) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1104409108 · Cited by 102
Spivey, T., Uccellini, L., Ascierto, M. L., Zoppoli, G. M., Giorgi, V. D., Delogu, L. G., Engle, A. M., Thomas, J. M., Wang, E., Marincola, F. M., et al. (2011) · Journal of Translational Medicine · doi:10.1186/1479-5876-9-174 · Cited by 108
“response in vivo in humans [11,12].”
Gyorffy, B., Molnar, B., Lage, H., Szallasi, Z., & Eklund, A. C. (2009) · PLoS ONE · doi:10.1371/journal.pone.0005645 · Cited by 102
“As an exception, alternative parameter values for PLIER were chosen by the manufacturer for use in a large evaluative study, so we evaluated these settings (PLIER+16) in addition to the defaults [3].”
BackgroundMethodSelf-citationZimmermann, P., Laule, O., Schmitz, J., Hrúz, T., Bleuler, S., & Gruissem, W. (2008) · Molecular Plant · doi:10.1093/mp/ssn048 · Cited by 101
Harrill, J., Shah, I., Setzer, R. W., Haggard, D., Auerbach, S., Judson, R., & Thomas, R. S. (2019) · Current Opinion in Toxicology · doi:10.1016/j.cotox.2019.05.004 · Cited by 101
Gene-expression profiling in epithelial ovarian cancer
Konstantinopoulos, P. A., Spentzos, D., & Cannistra, S. A. (2008) · Nature Clinical Practice Oncology · doi:10.1038/ncponc1178 · Cited by 115
“The MicroArray Quality Control project demonstrated that microarray experiments, if performed using high-quality standardized criteria, can produce reliable and reproducible results with acceptable intraplatform and interplatform consistency”
MethodHow high is the level of technical noise in microarray data?
Klebanov, L. B., & Yakovlev, A. (2007) · Biology Direct · doi:10.1186/1745-6150-2-9 · Cited by 101
“rather than its variation coefficient [3], is all that is impor-”
BackgroundInfluentialMicro-RNA expression in cisplatin resistant germ cell tumor cell lines
Port, M., Glaesener, S., Ruf, C. G., Riecke, A., Bokemeyer, C., Meineke, V., Honecker, F. U., & Abend, M. (2011) · Molecular Cancer · doi:10.1186/1476-4598-10-52 · Cited by 113
Romanov, S., Medvedev, A. V., Gambarian, M., Poltoratskaya, N., Moeser, M., Medvedeva, L., Gambarian, M., Diatchenko, L. B., & Makarov, S. S. (2008) · Nature Methods · doi:10.1038/nmeth.1186 · Cited by 106
Dual-color Proteomic Profiling of Complex Samples with a Microarray of 810 Cancer-related Antibodies
Schröder, C., Jacob, A., Tonack, S., Radon, T. P., Sill, M., Zucknick, M., Rüffer, S., Costello, E., Neoptolemos, J. P., Crnogorac‐Jurcevic, T., et al. (2010) · Molecular & Cellular Proteomics · doi:10.1074/mcp.m900419-mcp200 · Cited by 103
Győrffy, B., Benke, Z., Lánczky, A., Balázs, B., Szállási, Z., Tı́már, J., & Schäfer, R. (2011) · Breast Cancer Research and Treatment · doi:10.1007/s10549-011-1676-y · Cited by 98
DNA Microarrays for Identifying Fishes
Kochzius, M., Nölte, M., Weber, H., Silkenbeumer, N., Hjörleifsdóttir, S., Hreggviðsson, G. Ó., Marteinsson, V., van Kappel, K., Planes, S., Tinti, F., et al. (2008) · Marine Biotechnology · doi:10.1007/s10126-007-9068-3 · Cited by 111
Empirical comparison of cross-platform normalization methods for gene expression data
Rudy, J., & Valafar, F. (2011) · BMC Bioinformatics · doi:10.1186/1471-2105-12-467 · Cited by 108
“In this paper we provide a comparison of available methods based on the MicroArray Quality Control (MAQC) project [17] data set and a human sperm data set [44] containing data from multiple platforms (see Methods for details).”
MethodResultInfluentialTranscriptome assembly and isoform expression level estimation from biased RNA-Seq reads
Li, W., & Jiang, T. (2012) · Bioinformatics · doi:10.1093/bioinformatics/bts559 · Cited by 97
Stolfa, G., Smonskey, M. T., Boniface, R., Hachmann, A., Gulde, P., Joshi, A. D., Pierce, A. P., Jacobia, S. J., & Campbell, A. (2018) · Biotechnology Journal · doi:10.1002/biot.201700227 · Cited by 97
Multiplatform single-sample estimates of transcriptional activation
Piccolo, S. R., Withers, M. R., Francis, O. E., Bild, A. H., & Johnson, W. E. (2013) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1305823110 · Cited by 97
“To further evaluate our method on RNA-Seq data, we used TaqMan qPCR expression levels for brain tissue from the Microarray Quality Control (MAQC) project (13) as a reference standard.”
MethodYang, F., Shi, Z., Bai, S., Ward, R. D., & Zhang, A. (2013) · Molecular Ecology Resources · doi:10.1111/1755-0998.12165 · Cited by 97
“© 2013 John Wiley & Sons Ltd microarrays used in previous studies (Peplies et al. 2003; Warsen et al. 2004; Rønning et al. 2005; Shi et al. 2006; Tobler et al. 2006), were compromised by the fluorescence intensities of true-positive hybridization being heterogeneous.”
MethodHArtung, T. (2012) · ALTEX · doi:10.14573/altex.2012.2.119 · Cited by 87
Enhancing Reproducibility in Cancer Drug Screening: How Do We Move Forward?
Hatzis, C., Bedard, P. L., Birkbak, N. J., Beck, A. H., Aerts, H. J. W. L., Stern, D. F., Shi, L., Clarke, R., Quackenbush, J., & Haibe-Kains, B. (2014) · Cancer Research · doi:10.1158/0008-5472.can-14-0725 · Cited by 99
“For the objective assessment of quality control of cell-based HTS assays, a community-wide consortium effort ("HTSQC") similar to the MAQC-I (37) could be invaluable.”
MethodSelf-citationClark-Langone, K. M., Sangli, C., Krishnakumar, J., & Watson, D. (2010) · BMC Cancer · doi:10.1186/1471-2407-10-691 · Cited by 95
R-ODAF: Omics data analysis framework for regulatory application
Verheijen, M. C. T., Meier, M. J., Asensio, J. O., Gant, T. W., Tong, W., Yauk, C. L., & Caiment, F. (2022) · Regulatory Toxicology and Pharmacology · doi:10.1016/j.yrtph.2022.105143 · Cited by 100
Self-citationTEIXEIRA, V. H., Olaso, R., Martin‐Magniette, M., Lasbleiz, S., Jacq, L., Oliveira, C. R., Hilliquin, P., Gut, I. G., Cornélis, F., & Petit, É. (2009) · PLoS ONE · doi:10.1371/journal.pone.0006803 · Cited by 105
Gene Expression Profiling of Breast Cancer
Cheang, M. C. U., van de Rijn, M., & Nielsen, T. O. (2007) · Annual Review of Pathology Mechanisms of Disease · doi:10.1146/annurev.pathmechdis.3.121806.151505 · Cited by 109
“However, recent large-scale studies have supported technical consistency across platforms and laboratories (24).”
Huang, S. K., Scruggs, A. M., McEachin, R. C., White, E. S., & Peters-Golden, M. (2014) · PLoS ONE · doi:10.1371/journal.pone.0107055 · Cited by 93
Dumur, C. I., Lyons‐Weiler, M. A., Sciulli, C., Garrett, C. T., Schrijver, I., Holley, T., Rodriguez‐Paris, J., Pollack, J. R., Zehnder, J. L., Price, M., et al. (2007) · Journal of Molecular Diagnostics · doi:10.2353/jmoldx.2008.070099 · Cited by 101
“It is important to note that analytical variations found in everyday clinical practice were part of this study; thus, the study reflects the environment where this test would be performed clinically and therefore markedly differs from other studies of reproducibility of microarrays in which total RNA samples, not tiss…”
ResultMolecular diagnostics in transplantation
Naesens, M., & Sarwal, M. (2010) · Nature Reviews Nephrology · doi:10.1038/nrneph.2010.113 · Cited by 102
“Each of these platforms have strengths and weaknesses, and researchers must choose the appropriate platform based on their priorities”
BackgroundSutherland, J. J., Webster, Y., Willy, J. A., Searfoss, G. H., Goldstein, K. M., Irizarry, A. R., Hall, D. G., & Stevens, J. L. (2017) · The Pharmacogenomics Journal · doi:10.1038/tpj.2017.17 · Cited by 97
Liang, L., Ai, L., Qian, J., Fang, J.-Y., & Xu, J. (2015) · Scientific Reports · doi:10.1038/srep11763 · Cited by 92
Self-citationConvergence of biomarkers, bioinformatics and nanotechnology for individualized cancer treatment
Phan, J. H., Moffitt, A. B., Stokes, T. H., Liu, J., Young, A. N., Nie, S., & Wang, M. D. (2009) · Trends in biotechnology · doi:10.1016/j.tibtech.2009.02.010 · Cited by 94
“Although gene expression assays are generally reproducible (78), statistical artifacts in smaller datasets need to be identified and either corrected or removed prior to further data analysis.”
BackgroundMcAninch, E. A., Jo, S., Preite, N. Z., Farkas, E., Mohácsik, P., Fekete, C., Egri, P., Gereben, B., Li, Y., Deng, Y., et al. (2015) · The Journal of Clinical Endocrinology & Metabolism · doi:10.1210/jc.2014-4092 · Cited by 93
Sparfel, L., Pinel‐Marie, M., Boize, M., Koscielny, S., Desmots, S., Péry, A. R. R., & Fardel, O. (2010) · Toxicological Sciences · doi:10.1093/toxsci/kfq007 · Cited by 94
“Genes with more than or equal to twofold upregulation or downregulation were considered differentially expressed, in agreement with criteria retained by the microarray quality control consortium (Shi et al., 2006).”
ResultMeasuring the Stability of Feature Selection
Nogueira, S., & Brown, G. (2016) · ECML/PKDD · doi:10.1007/978-3-319-46227-1_28 · Cited by 88
“These measures are generalizations of POG, nPOG (called POGR and nPOGR [21]) and of the Dice coefficient [19] and reduce to these when there is no redundancy between the features.”
BackgroundInfluentialRoss, J. W., Ashworth, M. D., Stein, D. R., Couture, O. P., Tuggle, C. K., & Geisert, R. D. (2008) · Physiological Genomics · doi:10.1152/physiolgenomics.00022.2008 · Cited by 100
“While it should not be assumed that genes whose expression changes are less than twofold do not have a biological effect, production of a differentially expressed gene list based solely on statistical analysis can be irreproducible and this can be overcome by utilizing additional restrictions such as a minimum of twof…”
BackgroundSecco, M., Moreira, Y. B., Zucconi, E., Vieira, N. M., Jazedje, T., Muotri, A. R., Okamoto, O. K., Verjovski‐Almeida, S., & Zatz, M. (2009) · Stem Cell Reviews and Reports · doi:10.1007/s12015-009-9098-5 · Cited by 97
“In fact, recent microarray validation studies [44] have shown that the Agilent oligoarray technology, used in the present study, has been optimized to attain reproducible results.”
Cluzeau, C. V. M., Watkins-Chow, D. E., Fu, R., Borate, B., Yanjanin, N., Dail, M. K., Davidson, C. D., Walkley, S. U., Ory, D. S., Wassif, C. A., et al. (2012) · Human Molecular Genetics · doi:10.1093/hmg/dds193 · Cited by 93
Li, W., Turner, A., Aggarwal, P., Matter, A., Storvick, E., Arnett, D. K., & Broeckel, U. (2015) · BMC Genomics · doi:10.1186/s12864-015-2270-1 · Cited by 92
“One dataset is comprised of TaqMan assays for 843 genes from the MAQC-I study (MAQC) (GSE5350) [20].”
BackgroundMethodInfluentialTechnical variables in high-throughput miRNA expression profiling: much work remains to be done
NELSON, P., WANG, W., WILFRED, B., & TANG, G. (2008) · Biochimica et Biophysica Acta (BBA) - Gene Regulatory Mechanisms · doi:10.1016/j.bbagrm.2008.03.012 · Cited by 92
“This focus has been reflected by a great deal of work, as well as large-scale international consortiums dedicated to quality control [11,24].”
BackgroundTang, X., Zhang, Z., Zhang, W., Zhao, X., Li, X., Zhang, D., Liu, Q., & Tang, W. (2010) · PLANT PHYSIOLOGY · doi:10.1104/pp.110.161661 · Cited by 101
“For 60-mer oligomicroarray hybridization, the mRNA concentration (which represents expression level) linearly correlates with signal intensity spanning more than 5 orders of magnitude until intensity values approach saturation (Hughes et al., 2001; Shi et al., 2006).”
BackgroundGlobal Regulation by (p)ppGpp and CodY in Streptococcus mutans
Lemos, J. A., Nascimento, M. M., Lin, V. K., Abranches, J., & Burne, R. A. (2008) · Journal of Bacteriology · doi:10.1128/jb.00288-08 · Cited by 87
“The benefits and analytical methods associated with the utilization of a reference RNA as a normalization tool in microarray experiments are discussed elsewhere ( 36 ).”
MethodGyőrffy, B., & Schäfer, R. (2008) · Breast Cancer Research and Treatment · doi:10.1007/s10549-008-0242-8 · Cited by 91
“This mapping relies on the proven fundamental assumption that different microarrays are capable to reproducibly measure gene expression [25].”
BackgroundOsborne, J. R., Port, E. R., Gönen, M., Doane, A. S., YEUNG, H. W. D., Gerald, W. L., Cook, J., & Larson, S. M. (2010) · Journal of Nuclear Medicine · doi:10.2967/jnumed.108.060459 · Cited by 92
“Microarray analysis of breast cancer contributes to patient prognosis and will continue to evolve with more years of clinical validation (16,30).”
BackgroundSchneider, M., Huber, J., Hadaschik, B., Siegers, G. M., Fiebig, H.-H., & Schüler, J. (2012) · BMC Cancer · doi:10.1186/1471-2407-12-96 · Cited by 75
“0 mRNA expression arrays were used to determine the expression of 47,400 transcripts, corresponding to 38,500 human genes [22-27].”
Boutros, P. C., Yan, R., Moffat, I. D., Pohjanvirta, R., & Okey, A. B. (2008) · BMC Genomics · doi:10.1186/1471-2164-9-419 · Cited by 95
“This provides a cautionary note to the use of toxicogenomics for high-throughput testing, and stands in contrast to the optimistic findings of the MAQC study [54,55].”
mRNA stability changes precede changes in steady-state mRNA amounts during hyperosmotic stress
Molin, C., Jauhiainen, A., Warringer, J., Nerman, O., & Sunnerhagen, P. (2009) · RNA · doi:10.1261/rna.1403509 · Cited by 104
Kemiläinen, H., Adam, M., Mäki-Jouppila, J. H., Damdimopoulou, P., Damdimopoulos, A. E., Kere, J., Hovatta, O., Laajala, T. D., Aittokallio, T. A., Adamski, J., et al. (2016) · Endocrinology · doi:10.1210/en.2016-1252 · Cited by 89
Metadata
- Authors
- MAQC Consortium, Shi, L., Shi, L., Reid, L. H., Jones, W. D., Shippy, R., Warrington, J. A., Baker, S. C., Collins, P. J., de Longueville, F., Kawasaki, E. S., Lee, K. Y., Luo, Y., Sun, Y. A., Willey, J. C., Setterquist, R. A., Fischer, G. M., Tong, W., Dragan, Y. P., Dix, D. J., Frueh, F. W., Goodsaid, F. M., Herman, D., Jensen, R. V., Johnson, C. D., Lobenhofer, E. K., Puri, R. K., Scherf, U., Thierry-Mieg, J., Wang, C., Wilson, M., Wolber, P. K., Zhang, L., Amur, S., Bao, W., Barbacioru, C. C., Lucas, A. B., Bertholet, V., Boysen, C., Bromley, B., Brown, D., Brunner, A., Canales, R., Cao, X. M., Cebula, T. A., Chen, J. J., Cheng, J., Chu, T.-M., Chudin, E., Corson, J., Corton, J. C., Croner, L. J., Davies, C., Davison, T. S., Delenstarr, G., Deng, X., Dorris, D., Eklund, A. C., Fan, X.-H., Fang, H., Fulmer-Smentek, S., Fuscoe, J. C., Gallagher, K., Ge, W., Guo, L., Guo, X., Hager, J., Haje, P. K., Han, J., Han, T., Harbottle, H. C., Harris, S. C., Hatchwell, E., Hauser, C. A., Hester, S., Hong, H., Hurban, P., Jackson, S. A., Ji, H., Knight, C. R., Kuo, W. P., LeClerc, J. E., Levy, S., Li, Q.-Z., Liu, C., Liu, Y., Lombardi, M. J., Ma, Y., Magnuson, S. R., Maqsodi, B., McDaniel, T., Mei, N., Myklebost, O., Ning, B., Novoradovskaya, N., Orr, M. S., Osborn, T. W., Papallo, A., Patterson, T. A., Perkins, R. G., Peters, E. H., Peterson, R., Philips, K. L., Pine, P. S., Pusztai, L., Qian, F., Ren, H., Rosen, M., Rosenzweig, B. A., Samaha, R. R., Schena, M., Schroth, G. P., Shchegrova, S., Smith, D. D., Staedtler, F., Su, Z., Sun, H., Szallasi, Z., Tezak, Z., Thierry-Mieg, D., Thompson, K. L., Tikhonova, I., Turpaz, Y., Vallanat, B., Van, C., Walker, S. J., Wang, S. J., Wang, Y., Wolfinger, R., Wong, A., Wu, J., Xiao, C., Xie, Q., Xu, J., Yang, W., Zhang, L., Zhong, S., Zong, Y., & Slikker, W.
- Affiliations
- Affymetrix (United States) (US)
- Illumina (United States) (US)
- Agilent Technologies (United States) (US)
- National Cancer Institute (US)
- National Center for Toxicological Research (US)
- Environmental Protection Agency (US)
- United States Food and Drug Administration (US)
- National Institutes of Health (US)
- University of Massachusetts Boston (US)
- Asuragen (United States) (US)
- Center for Biologics Evaluation and Research (US)
- Cedars-Sinai Medical Center (US)
- SAS Institute (United States) (US)
- Center for Food Safety and Applied Nutrition (US)
- Biogen (United States) (US)
- Yale University (US)
- Center for Veterinary Medicine (US)
- Cold Spring Harbor Laboratory (US)
- Sanford Burnham Prebys Medical Discovery Institute (US)
- Stanford Medicine (US)
- Harvard University (US)
- Vanderbilt University (US)
- The University of Texas Southwestern Medical Center (US)
- The University of Texas at Dallas (US)
- GenUs BioSystems (United States) (US)
- Oslo University Hospital (NO)
- The University of Texas MD Anderson Cancer Center (US)
- Luminex (United States) (US)
- Wake Forest University Health Sciences (US)
- University of Illinois Urbana-Champaign (US)
- ORCID
- Jones, Wendell D
- Collins, Patrick J
- Luo, Yuling
- Sun, Yongming Andrew
- Willey, James C
- Tong, Weida
- Frueh, Felix W
- Goodsaid, Federico M
- Johnson, Charles D
- Puri, Raj K
- Scherf, Uwe
- Wang, Charles
- Bao, Wenjun
- Brown, Donna
- Chen, James J
- Cheng, Jing
- Corton, J Christopher
- Croner, Lisa J
- Deng, Xutao
- Eklund, Aron C
- Fang, Hong
- Guo, Lei
- Guo, Xu
- Han, Jing
- Han, Tao
- Harbottle, Heather C
- Harris, Stephen C
- Hatchwell, Eli
- Hester, Susan
- Hong, Huixiao
- Jackson, Scott A
- Ji, Hanlee
- Levy, Shawn
- Li, Quan-Zhen
- Liu, Chunmei
- Liu, Ying
- Ma, Yunqing
- Mei, Nan
- Myklebost, Ola
- Ning, Baitang
- Patterson, Tucker A
- Pusztai, Lajos
- Qian, Feng
- Schroth, Gary P
- Su, Zhenqiang
- Sun, Hongmei
- Tikhonova, Irina
- Turpaz, Yaron
- Walker, Stephen J
- Wang, Sue Jane
- Wang, Yonghong
- Wong, Alex
- Wu, Jie
- Xiao, Chunlin
- Xie, Qian
- Xu, Jun
- Yang, Wen
- Zhang, Liang
- Zhong, Sheng
- Slikker, William
- Venue
- Nature Biotechnology
- Published
- 2006-09-01
- Volume
- 24
- Issue
- 9
- Pages
- 1151-1161
- ISSN
- 0733-222X (ISSN-L)
- 1087-0156
- 1546-1696
- PMC
- PMC3272078
- OpenAlex
- W2010637767
- Cited by
- 2,126
- References
- 43
- Citations by year
- 2026: 21 · 2025: 25 · 2024: 23 · 2023: 39 · 2022: 37 · 2021: 60 · 2020: 43 · 2019: 59 · 2018: 68 · 2017: 87 · 2016: 87 · 2015: 98 · 2014: 122 · 2013: 155 · 2012: 160
- FWCI
- 87.96
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Licence
- www.springer.com/tdm (tdm)
- www.springer.com/tdm (vor)
- Locations
- Nature Biotechnology (publishedVersion)
- PubMed (publishedVersion)
- Europe PMC (PubMed Central) (acceptedVersion)
- PubMed Central (acceptedVersion)
- Digital Access to Libraries (Université catholique de Louvain (UCL), l'Université de Namur (UNamur) and the Université Saint-Louis (USL-B)) (submittedVersion)
- Funders
- U.S. Environmental Protection Agency
- National Institutes of Health (Z99 LM999999, Z99 HL999999, Z99 MH999999, Z99CL999999)
- Topics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Gene expression and cancer classification
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Molecular Biology Techniques and Applications
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Genomics and Phylogenetic Studies
- Keywords
- probe mapping, differentially expressed genes, microarray data analysis
- MeSH terms
- Equipment Design, Quality Assurance, Health Care, Quality Control, Sensitivity and Specificity, United States, Reproducibility of Results, Equipment Failure Analysis, Oligonucleotide Array Sequence Analysis, Gene Expression Profiling
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.