The MicroArray Quality Control (MAQC) project shows inter- and intraplatform reproducibility of gene expression measurements
Open accessThe DOI matches its Crossref record; 43 of the 43 references it cites were found in a registry or an index.
Abstract
Over the last decade, the introduction of microarray technology has had a profound impact on gene expression research. The publication of studies with dissimilar or altogether contradictory results, obtained using different microarray platforms to analyze identical RNA samples, has raised concerns about the reliability of this technology. The MicroArray Quality Control (MAQC) project was initiated to address these concerns, as well as other performance and data analysis issues. Expression data on four titration pools from two distinct reference RNA samples were generated at multiple test sites using a variety of microarray-based and alternative technology platforms. Here we describe the experimental design and probe mapping efforts behind the MAQC project. We show intraplatform consistency across test sites as well as a high level of interplatform concordance in terms of genes identified as differentially expressed. This study provides a resource that represents an important first step toward establishing a framework for the use of microarrays in clinical and regulatory settings.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Next, we applied the |log 2 fold-change|>1 filter advocated in the MAQC study on microarrays, 10 and for microarrays observed a reduction in the eFDR to below 1.5%, save for one outlier site (Figs 3c,d).”
From A comprehensive assessment of RNA-seq accuracy, reproducibility and information content by the Sequencing Quality Control Consortium (2014) · cites this paper as [10]
In the citing paper
“Studies on microarrays have shown that results of typical statistical differential expression tests thresholded by p -value need to be filtered and sorted by effect strength (fold-change) in order to attain robust comparisons across platforms and sites. 10”
From A comprehensive assessment of RNA-seq accuracy, reproducibility and information content by the Sequencing Quality Control Consortium (2014) · cites this paper as [10]
In the citing paper
“The Microarray Quality Control (MAQC) Consortium landmark study10 examined the degree of variance within and across many different microarray platforms and found similar coefficients of variation between platforms.”
From Multi-platform assessment of transcriptome profiling using RNA-seq in the ABRF next-generation sequencing study (2014) · cites this paper as [10]
In the citing paper
“prior work comparing microarray platforms and methods showed high levels of inter-platform concordance for the ability to detect differentially expressed genes.”
From Multi-platform assessment of transcriptome profiling using RNA-seq in the ABRF next-generation sequencing study (2014) · cites this paper as [10]
Citations
Cites
In silico prediction of protein-protein interactions in human macrophages
Souiai, O., Guerfali, F., Ben Miled, S., Brun, C., & Benkahla, A. (2014) · BMC Research Notes · doi:10.1186/1756-0500-7-157 · Cited by 10,803
Significance analysis of microarrays applied to the ionizing radiation response
Tusher, V. G., Tibshirani, R., & Chu, G. (2001) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.091062498 · Cited by 10,568
Pruitt, K. D., Tatusova, T., & Maglott, D. R. (2007) · Nucleic Acids Research · doi:10.1093/nar/gkl842 · Cited by 4,481 · cited 1×
“We mapped the probes (see Supplementary Methods online and Supplementary Notes online) to the RefSeq human mRNA databas”
Method"Stemness": Transcriptional Profiling of Embryonic and Adult Stem Cells
Ramalho-Santos, M., Yoon, S., Matsuzaki, Y., Mulligan, R. C., & Melton, D. A. (2002) · Science · doi:10.1126/science.1072530 · Cited by 1,709
Pruitt, K. D. (2004) · Nucleic Acids Research · doi:10.1093/nar/gki025 · Cited by 1,660 · cited 1×
“We mapped the probes (see Supplementary Methods online and Supplementary Notes online) to the RefSeq human mRNA database 31 (http://www.ncbi.nlm.nih.gov/RefSeq) and to the AceView database 32 (http://www.ncbi.nlm.nih.gov/IEB/Research/Acembly), a less curated but more comprehensive database, which…”
MethodA Stem Cell Molecular Signature
Ivanova, N. B., Dimos, J. T., Schaniel, C., Hackney, J. A., Moore, K. A., & Lemischka, I. R. (2002) · Science · doi:10.1126/science.1073823 · Cited by 1,542
Prediction of cancer outcome with microarrays: a multiple random validation strategy
Michiels, S., Koscielny, S., & Hill, C. (2005) · The Lancet · doi:10.1016/s0140-6736(05)17866-0 · Cited by 1,025 · cited 1×
“, raising doubts about the repeatability, reproducibility and comparability of microarray technolog”
BackgroundMultiple-laboratory comparison of microarray platforms
Irizarry, R. A., Warren, D., Spencer, F., Kim, I. F., Biswal, S., Frank, B. C., Gabrielson, E., Garcia, J. G. N., Geoghegan, J., Germino, G., et al. (2005) · Nature Methods · doi:10.1038/nmeth756 · Cited by 862
Self-citationEvaluation of gene expression measurements from commercial microarray platforms
Tan, P. K. (2003) · Nucleic Acids Research · doi:10.1093/nar/gkg763 · Cited by 706
Thousands of samples are needed to generate a robust gene list for predicting outcome in cancer
Ein-Dor, L., Zuk, O., & Domany, E. (2006) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0601231103 · Cited by 689 · cited 1×
“…to appear in peer-reviewed journals 9,10 , raising doubts about the repeatability, reproducibility and comparability of microarray technology 11–13 , several studies have also been recently published showing increased reproducibility of microarray data generated at different test sites and/or…”
Background
Research works printed but not linked: 1
- Reid, L.H. The value of a proficiency testing program to monitor performance in microarray laboratories. Pharm. Discov. 5, 20–25 (2005).
Cited by
Items 301–400 of 1,004
Li, X., Brock, G. N., Rouchka, E. C., Cooper, N. G. F., Wu, D., O’Toole, T. E., Gill, R. S., Eteleeb, A. M., O’Brien, L., & Rai, S. N. (2017) · PLoS ONE · doi:10.1371/journal.pone.0176185 · Cited by 97
Changes in Gene Expression in Human Meibomian Gland Dysfunction
Liu, S., Richards, S. M., Lo, K., Hatton, M. P., Fay, A. M., & Sullivan, D. A. (2011) · Investigative Ophthalmology & Visual Science · doi:10.1167/iovs.10-6482 · Cited by 89
“intensity values were adjusted by adding a constant, such that the lowest intensity value for any sample was 16.(51) Hybridization intensity”
BackgroundMethodGeng, N., Ren, X., Gong, Y., Zhang, H., Wang, F., Xing, L., Cao, R., Xu, J., Gao, Y., Giesy, J. P., et al. (2019) · Environment International · doi:10.1016/j.envint.2019.105231 · Cited by 88
“Transcriptomics identi fi es broad-scale changes in expression levels of genes, which might help researchers ascertain speci fi c, molecular responses as endpoints of low-dose e ff ects and lead to the generation of hypotheses about the mechanisms of action (Shi et al., 2006).”
Self-citationRootstock-regulated gene expression patterns in apple tree scions
Jensen, P. J., Makałowska, I., Altman, N., Fazio, G., Praul, C. A., Maximova, S. N., Crassweller, R. M., Travis, J. W., & McNellis, T. W. (2009) · Tree Genetics & Genomes · doi:10.1007/s11295-009-0228-7 · Cited by 97
“This level of agreement is on par with studies of microarray reproducibility (Shi et al. 2006) and correlation between microarrays and quantitative, real-time PCR (Morey et al. 2006).”
ResultAssessing affymetrix GeneChip microarray quality
McCall, M. N., Murakami, P. N., Lukk, M., Huber, W. K., & Irizarry, R. A. (2011) · BMC Bioinformatics · doi:10.1186/1471-2105-12-137 · Cited by 97
“Various initiatives such as the External RNA Control Consortium (ERCC) [1] and the MicroArray Quality Control (MAQC) projects [2,3] have explored ways to provide standards for the technology.”
Spiess, A., Feig, C., Schulze, W., Chalmel, F., Cappallo‐Obermann, H., Primig, M., & Kirchhoff, C. (2007) · Human Reproduction · doi:10.1093/humrep/dem292 · Cited by 93
“Products were gel separated and their identity verified by sequence analysis.”
MethodTranscriptomic Biomarkers for Individual Risk Assessment in New-Onset Heart Failure
Heidecker, B., Kasper, E. K., Wittstein, I. S., Champion, H. C., Breton, E., Russell, S. D., Kittleson, M. M., Baughman, K. L., & Hare, J. M. (2008) · Circulation · doi:10.1161/circulationaha.107.756544 · Cited by 87
Su, Z., Li, Z., Chen, T., Li, Q.-Z., Fang, H., Ding, D., Ge, W., Ning, B., Hong, H., Perkins, R. G., et al. (2011) · Chemical Research in Toxicology · doi:10.1021/tx200103b · Cited by 85
Self-citationKadota, K., Nakai, Y., & Shimizu, K. (2009) · Algorithms for Molecular Biology · doi:10.1186/1748-7188-4-7 · Cited by 88
“This is despite the fact that only using the AD method can increase the reproducibility [21].”
InfluentialMolecular markers associated with outcome and metastasis in human pancreatic cancer
den broeck, A. V., Vankelecom, H. E. J., van Eijsden, R. G. E., Govaere, O., & Topal, B. (2012) · Journal of Experimental & Clinical Cancer Research · doi:10.1186/1756-9966-31-68 · Cited by 86
“001 was used as an alternative, pragmatic approach to balance the number of false positives and false negatives [17].”
MethodLiu, F., Rainosek, S. W., Frisch‐Daiello, J. L., Patterson, T. A., Paule, M. G., Slikker, W., Wang, C., & Han, X. (2015) · Toxicological Sciences · doi:10.1093/toxsci/kfv150 · Cited by 87
Self-citationGingival Transcriptome Patterns During Induction and Resolution of Experimental Gingivitis in Humans
Offenbacher, S., Barros, S. P., Paquette, D. W., Winston, J. L., Biesbrock, A. R., Thomason, R. G., Gibb, R., Fulmer, A. W., Tiesman, J. P., Juhlin, K. D., et al. (2009) · Journal of Periodontology · doi:10.1902/jop.2009.080645 · Cited by 85
Self-citationThe Influence of 13- cis Retinoic Acid on Human Meibomian Gland Epithelial Cells
Ding, J., Kam, W. R., Dieckow, J., & Sullivan, D. A. (2013) · Investigative Ophthalmology & Visual Science · doi:10.1167/iovs.13-11863 · Cited by 89
Age and sex dependent changes in liver gene expression during the life cycle of the rat
Kwekel, J. C., Desai, V. G., Moland, C. L., Branham, W. S., & Fuscoe, J. C. (2010) · BMC Genomics · doi:10.1186/1471-2164-11-675 · Cited by 81
“The MicroArray Quality Control (MAQC) Consortium [58] suggests the use of a fold-change cut off along with a non-stringent p-value cut off as a baseline practice to improve reproducibility in microarray data processing.”
MethodSelf-citationEstimation of sequencing error rates in short reads
Wang, V., Blades, N. J., Ding, J., Sultana, R., & Parmigiani, G. (2012) · BMC Bioinformatics · doi:10.1186/1471-2105-13-185 · Cited by 108
“mRNA-seq: MAQC brain experiment 2 using auto calibration We applied shadow regression to Illumina mRNA-seq data from the MAQC project [23].”
MethodOn the importance of small changes in RNA expression
Laurent, G. S., Shtokalo, D. N., Tackett, M., Yang, Z., Vyatkin, Y. V., Milos, P. M., Seilheimer, B., McCaffrey, T. A., & Kapranov, P. (2013) · Methods · doi:10.1016/j.ymeth.2013.03.027 · Cited by 89
Mane, S. P., Evans, C., Cooper, K. L., Crasta, O. R., Folkerts, O., Hutchison, S. K., Harkins, T. T., Thierry-Mieg, D., Thierry-Mieg, J., & Jensen, R. V. (2009) · BMC Genomics · doi:10.1186/1471-2164-10-264 · Cited by 83
Self-citationRelative power and sample size analysis on gene expression profiling data
van Iterson, M., Hoen, P. A. C. ’., Pedotti, P., Hooiveld, G. J. E. J., den Dunnen, J. T., van Ommen, G.-J., de Boer, J. C., & Menezes, R. M. (2009) · BMC Genomics · doi:10.1186/1471-2164-10-439 · Cited by 92
Sárvári, M., Hrabovszky, E., Kalló, I., Solymosi, N., Tóth, K., Likó, I., Széles, J., Mahó, S., Molnár, B., & Liposits, Z. (2011) · Journal of Neuroinflammation · doi:10.1186/1742-2094-8-82 · Cited by 78
“After outlier identification [26], linear models combined with empirical Bayesian methods were applied [27] and the raw fold change values were used to select differentially expressed genes [28].”
MethodLi, H., Torres-Garcia, S., Latrasse, D., Benhamed, M., Schilderink, S., Zhou, W., Kulikova, O., Hirt, H., & Bisseling, T. (2017) · The Plant Cell · doi:10.1105/tpc.17.00366 · Cited by 84
“The Arabidopsis GFP reporter lines used are pSCR:SCR-GFP (Helariutta et al., 2000), pRGA: GFP-RGA (Silverstone et al., 2001), pPIN1:PIN1-GFP (Benková et al., 2003), and pPIN2:PIN2-GFP (Blilou et al., 2005).”
BackgroundMethodInfluentialAnalysis of Microarray Data: A Network‐Based Approach
(2008) · Wiley eBooks · doi:10.1002/9783527622818 · Cited by 85
Multigene Classifiers, Prognostic Factors, and Predictors of Breast Cancer Clinical Outcome
Ross, J. S. (2009) · Advances in Anatomic Pathology · doi:10.1097/pap.0b013e3181a9d4bf · Cited by 91
“The FDA is currently leading an effort to further standardize the microarray technology and related analytical methodology to facilitate broader clinical adaptation.(51) The anticipated costs for the RT-PCR and microarray based multigene predictors for breast cancer is in the $3000 range per test.”
BackgroundStatistical methods of background correction for Illumina BeadArray data
Xie, Y., Wang, X., & Story, M. (2009) · Bioinformatics · doi:10.1093/bioinformatics/btp040 · Cited by 82
Bossé, Y., Mathieu, P., & Pibarot, P. (2008) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2007.12.031 · Cited by 82
Wu, P., Castner, D. G., & Grainger, D. W. (2008) · Journal of Biomaterials Science Polymer Edition · doi:10.1163/156856208784522092 · Cited by 89
Sárvári, M., Hrabovszky, E., Kalló, I., Solymosi, N., Likó, I., Berchtold, N., Cotman, C., & Liposits, Z. (2012) · Journal of Neuroinflammation · doi:10.1186/1742-2094-9-264 · Cited by 77
Ravo, M., Mutarelli, M., Ferraro, L., Grober, O. M. V., Paris, O., Tarallo, R., Vigilante, A., Cimino, D., Bortoli, M. D., Nola, E., et al. (2008) · Laboratory Investigation · doi:10.1038/labinvest.2008.11 · Cited by 81
“The latter problem is being addressed by the MicroArray Quality Control (MAQC) project”
BackgroundBroad Up-Regulation of Innate Defense Factors during Acute Cholera
Flach, C., Qadri, F., Bhuiyan, T. R., Alam, N. H., Jennische, E., Lönnroth, I., & Holmgren, J. R. (2007) · Infection and Immunity · doi:10.1128/iai.01900-06 · Cited by 87
“A recent joint effort of the FDA and several microarray manufacturers, the Microarray Quality Control study, also showed that there was both a high degree of intraplatform consistency and interplatform concordance when the same RNA samples were analyzed on different microarray platforms at different test sites in para…”
BackgroundReview of the literature examining the correlation among DNA microarray technologies
Yauk, C. L., & Berndt, M. L. (2007) · Environmental and Molecular Mutagenesis · doi:10.1002/em.20290 · Cited by 79
Geyer, F. C., & Reis-Filho, J. S. (2008) · International Journal of Surgical Pathology · doi:10.1177/1066896908328577 · Cited by 80
“For instance, up to 15% of error in qualitative assessment of gene expression has been demonstrated when the optimal samples were analyzed with the same platform and protocols in different laboratories.(107) Furthermore, the applicability of microarrays to readily available, formalin-fixed, paraffin-embedded material…”
Diez, D., Grijota-Martinez, C., Agretti, P., De Marco, G., Tonacchera, M., Pinchera, A., Morreale de Escobar, G., Bernal, J., & Morte, B. (2008) · Endocrinology · doi:10.1210/en.2008-0350 · Cited by 79
“On the other hand, it has been noted that selection on the basis on fold change achieves better reproducibility among different platforms and biological groups than a selection based on t statistics (44, 45).”
BackgroundArray feature size influences nucleic acid surface capture in DNA microarrays
Dandy, D. S., Wu, P., & Grainger, D. W. (2007) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0606054104 · Cited by 77
ArrayTrack: An FDA and Public Genomic Tool
Fang, H., Harris, S., Su, Z., Chen, M., Qian, F., Shi, L. M., Perkins, R., & Tong, W. (2009) · Methods in molecular biology · doi:10.1007/978-1-60761-175-2_20 · Cited by 80
Self-citationLIQA: long-read isoform quantification and analysis
Hu, Y., Li, F., Chen, X., Zhong, J. F., Li, M., & Wang, K. (2021) · Genome biology · doi:10.1186/s13059-021-02399-8 · Cited by 88
“As part of the MAQC project, the expression levels of 894 isoforms were measured by TaqMan Gene Expression Assay based qRT-PCR.”
MethodInfluentialA systematic survey of lipids across mouse tissues
Jain, M., Ngoy, S., Sheth, S. A., Swanson, R. A., Rhee, E. P., Liao, R., Clish, C. B., Mootha, V. K., & Nilsson, R. (2014) · American Journal of Physiology-Endocrinology and Metabolism · doi:10.1152/ajpendo.00371.2013 · Cited by 74
“such as microarrays and gel proteomics (32, 47).”
BackgroundEU Framework 6 Project: Predictive Toxicology (PredTox)—overview and outcome
Suter‐Dick, L., Schroeder, S., Meyer, K., Gautier, J., Amberg, A., Wendt, M., Gmuender, H., Mally, A., Boitier, E., Ellinger-Ziegelbauer, H. C., et al. (2010) · Toxicology and Applied Pharmacology · doi:10.1016/j.taap.2010.10.008 · Cited by 79
Nota, B., Bosse, M., Ylstra, B., van Straalen, N. M., & Roelofs, D. (2009) · BMC Genomics · doi:10.1186/1471-2164-10-236 · Cited by 82
“(2006) showed high consistency across different microarray platforms [32], supporting data comparison between different platforms to some extent.”
BackgroundBeyer, R. P., Fry, R. C., Lasarev, M. R., McConnachie, L. A., Meira, L. B., Palmer, V. S., Powell, C. L., Ross, P. K., Bammler, T. K., Bradford, B. U., et al. (2007) · Toxicological Sciences · doi:10.1093/toxsci/kfm150 · Cited by 88
Self-citationKinaret, P., Serra, A., Federico, A., Kohonen, P. J., Nymark, P. E. H., Liampa, I., Ha, M. K., Choi, J., Jagiełło, K., Sanabria, N. M., et al. (2020) · Nanomaterials · doi:10.3390/nano10040750 · Cited by 81
“DNA microarrays can capture relevant transcriptomic responses with concordant biological signals, independently from the platforms and laboratories [100,101].”
BackgroundBiomarkers in solid organ transplantation: establishing personalized transplantation medicine
Roedder, S., Vitalone, M. J., Khatri, P., & Sarwal, M. (2011) · Genome Medicine · doi:10.1186/gm253 · Cited by 78
Walter, S. R., & Geiger, F. M. (2009) · The Journal of Physical Chemistry Letters · doi:10.1021/jz9001086 · Cited by 78
Zhao, B., Erwin, A., & Xue, B. (2017) · Genomics · doi:10.1016/j.ygeno.2017.08.007 · Cited by 78
The U.S. Environmental Protection Agency Strategic Plan for Evaluating the Toxicity of Chemicals
Firestone, M. P., Kavlock, R. J., Zenick, H., Kramer, M. G., & the U.S. EPA Working Group on the F (2010) · Journal of Toxicology and Environmental Health Part B · doi:10.1080/10937404.2010.483178 · Cited by 81
“NCTR has coordinated the Microarray Quality Control (MAQC) project, with numerous partners including the U.S. EPA (Shi et al., 2006).”
BackgroundStatistical expression deconvolution from mixed tissue samples
Clarke, J., Seo, P. H., & Clarke, B. S. (2010) · Bioinformatics · doi:10.1093/bioinformatics/btq097 · Cited by 77
Hammer, P., Banck, M. S., Amberg, R., Wang, C., Petznick, G., Luo, S., Khrebtukova, I., Schroth, G. P., Beyerlein, P., & Beutler, A. S. (2010) · Genome Research · doi:10.1101/gr.101204.109 · Cited by 75
“Fold-change differences among 755 transcripts quantified by mRNA-seq were calculated as described above and compared with established Taqman qRT-PCR data (Shi et al. 2006) as shown in Supplemental Figure 3.”
MethodSelf-citationEvaluation of connectivity map shows limited reproducibility in drug repositioning
Lim, N., & Pavlidis, P. (2021) · Scientific Reports · doi:10.1038/s41598-021-97005-z · Cited by 90
Generalized reporter score-based enrichment analysis for omics data
Chen, P., Chen, Q., Tan, S., Shen, X., & Jiang, C. (2024) · Briefings in Bioinformatics · doi:10.1093/bib/bbae116 · Cited by 80
“The top 400 genes with the highest unsigned log-fold changes were considered as differentially expressed genes (DEGs) [20].”
BackgroundBourdon-Lacombe, J., Moffat, I. D., Deveau, M., Husain, M., Auerbach, S. S., Krewski, D. R., Thomas, R. S., Bushel, P. R., Williams, A., & Yauk, C. L. (2015) · Regulatory Toxicology and Pharmacology · doi:10.1016/j.yrtph.2015.04.010 · Cited by 81
Toward an Alzheimer's disease diagnosis via high‐resolution blood gene expression
Fehlbaum-Beurdeley, P., Jarrige-Le Prado, A. C., Pallares, D., Carrière, J., Guihal, C., Soucaille, C., Rouet, F., Drouin, D., Sol, O., Jordan, H., et al. (2010) · Alzheimer's & Dementia · doi:10.1016/j.jalz.2009.07.001 · Cited by 75
“Its performances were measured against standards recently described for microarray quality control [12], and were found to be similar to the expression platforms reported in the Microarray Quality Control Study [13; personal communication, W.”
ResultYin, M., Soikkeli, J., Jahkola, T., Virolainen, S., Saksela, O., & Hölttä, E. (2012) · American Journal Of Pathology · doi:10.1016/j.ajpath.2012.08.027 · Cited by 79
Gene Expression Profiling from Formalin-Fixed Paraffin-Embedded Tumors of Pediatric Glioblastoma
Haque, T., Faury, D., Albrecht, S., Lopez-Aguilar, E., Hauser, P., Garami, M., Hanzély, Z., Bognár, L., Del Maestro, R. F., Atkinson, J., et al. (2007) · Clinical Cancer Research · doi:10.1158/1078-0432.ccr-07-0525 · Cited by 76
Oligonucleotide microarrays: widely applied poorly understood
Pozhitkov, A. E., Tautz, D., & Noble, P. A. (2007) · Briefings in Functional Genomics and Proteomics · doi:10.1093/bfgp/elm014 · Cited by 88
“However, this expectation was not confirmed in the respective comparative studies [21, 22].”
ResultHurley, D., Araki, H., Tamada, Y., Dunmore, B., Sanders, D., Humphreys, S., Affara, M., Imoto, S., Yasuda, K., Tomiyasu, Y., et al. (2012) · Nucleic Acids Research · doi:10.1093/nar/gkr902 · Cited by 77
“However, format incompatibilities and experimental differences can make it hard to combine data from different microarray platforms and studies, and the reproducibility of microarray results from different laboratories has been problematic in the past (18).”
BackgroundPeeters, E., Sass, A. M., Mahenthiralingam, E., Nelis, H. J. C. F., & Coenye, T. (2010) · BMC Genomics · doi:10.1186/1471-2164-11-90 · Cited by 75
“To this end, the log2 fold changes were subjected to bivariate regression analysis and Pearson correlation coefficients were determined using SPSS 15 [72, 73].”
van Delft, J. H. M., Gaj, S., Lienhard, M., Albrecht, M. W., Кирпий, А. А., Brauers, K. J. J., Claessen, S. M. H., Lizárraga, D., Lehrach, H., Herwig, R., et al. (2012) · Toxicological Sciences · doi:10.1093/toxsci/kfs250 · Cited by 84
Liu, F., Jenssen, T.-K., Trimarchi, J., Punzo, C., Cepko, C. L., Ohno-Machado, L., Hovig, E., & Patrick Kuo, W. (2007) · BMC Genomics · doi:10.1186/1471-2164-8-153 · Cited by 74
Self-citationAccurate Expression Profiling of Very Small Cell Populations
Gonzalez-Roca, E., Garcia-Albéniz, X., Rodriguez-Mulero, S., Gomis, R. R., Kornacker, K., & Auer, H. (2010) · PLoS ONE · doi:10.1371/journal.pone.0014418 · Cited by 75
“qPCR data of Laboratory 1 of the MAQC study [3] were downloaded from Gene Expression Omnibus (GSE5350, samples GSM129638-44).”
BackgroundMethodInfluentialPhysiological and toxicological transcriptome changes in HepG2 cells exposed to copper
Song, M. O., Li, J., & Freedman, J. H. (2009) · Physiological Genomics · doi:10.1152/physiolgenomics.00083.2009 · Cited by 77
Application of next generation qPCR and sequencing platforms to mRNA biomarker analysis
Devonshire, A. S., Sanders, R., Wilkes, T. M., Taylor, M. S., Foy, C. A., & Huggett, J. F. (2012) · Methods · doi:10.1016/j.ymeth.2012.07.021 · Cited by 81
Measurement and Quality Control Issues in Multiplex Protein Assays: A Case Study
Ellington, A. A., Kullo, I. J., Bailey, K. R., & Klee, G. G. (2009) · Clinical Chemistry · doi:10.1373/clinchem.2008.120717 · Cited by 75
“We divided QC materials into 3 sets: (1) the study volunteer’s plasma control, (2) recombinant QC1 and QC2 that were developed by SearchLight for our study, and (3) QC3, a recombinant protein control used routinely by SearchLight to monitor assay performance.”
MethodWilson, C. M., Rodríguez, M., Johnson, C. M., Martin, S., Chu, T., Wolfinger, R. D., Hauser, L., Land, M., Klingeman, D. M., Syed, M. H., et al. (2013) · Biotechnology for Biofuels · doi:10.1186/1754-6834-6-179 · Cited by 80
“However, DNA microarray testing and analysis has evolved over many years through studies such as the MicroArray Quality Control (MAQC) project [65,66] and further studies and cost reductions in sequencing are similarly required to develop RNA-seq analyses.”
Self-citationy Fortéa, J. O., Giraud, É., Regnault, B., Coppée, J., Milon, G., Lang, T., & Prina, E. (2009) · BMC Genomics · doi:10.1186/1471-2164-10-119 · Cited by 79
Micro- and nanobiosensors—state of the art and trends
Urban, G. A. (2008) · Measurement Science and Technology · doi:10.1088/0957-0233/20/1/012001 · Cited by 95
den broeck, A. V., Vankelecom, H. E. J., Delm, W. V., Gremeaux, L., Wouters, J., Allemeersch, J., Govaere, O., Roskams, T., & Topal, B. (2013) · PLoS ONE · doi:10.1371/journal.pone.0073968 · Cited by 74
“001 was used as an alternative, pragmatic approach to balance the number of false positives and false negatives [15].”
MethodAssessing the Metabolic Competence of Sandwich-Cultured Mouse Primary Hepatocytes
Mathijs, K., Kienhuis, A. S., Brauers, K. J. J., Jennen, D. G. J., Lahoz, A., Kleinjans, J. C. S., & van Delft, J. H. M. (2009) · Drug Metabolism and Disposition · doi:10.1124/dmd.108.025775 · Cited by 78
Liu, H., Zeeberg, B. R., Qu, G., Koru, A. G., Ferrucci, A., Kahn, A., Ryan, M. C., Nuhanovic, A., Munson, P. J., Reinhold, W. C., et al. (2007) · Bioinformatics · doi:10.1093/bioinformatics/btm360 · Cited by 75
A Reduced Transcriptome Approach to Assess Environmental Toxicants Using Zebrafish Embryo Test
Wang, P., Xia, P., Yang, J., Wang, Z., Peng, Y., Shi, W. F., Villeneuve, D. L., Yu, H., & Zhang, X. (2017) · Environmental Science & Technology · doi:10.1021/acs.est.7b04073 · Cited by 75
Applications of transcriptomics and proteomics in understanding fish immunity
Ye, H., Lin, Q., & Luo, H. (2018) · Fish & Shellfish Immunology · doi:10.1016/j.fsi.2018.03.046 · Cited by 81
Reversal of the Transcriptome by Prostaglandin E2 during Myofibroblast Dedifferentiation
Wettlaufer, S. H., Scott, J. P., McEachin, R. C., Peters-Golden, M. L., & Huang, S. K. (2015) · American Journal of Respiratory Cell and Molecular Biology · doi:10.1165/rcmb.2014-0468oc · Cited by 76
Gene Expression and Isoform Variation Analysis using Affymetrix Exon Arrays
Bemmo, A., Benovoy, D., Kwan, T., Gaffney, D. J., Jensen, R. V., & Majewski, J. (2008) · BMC Genomics · doi:10.1186/1471-2164-9-529 · Cited by 70
Self-citationBrain progranulin expression in GRN-associated frontotemporal lobar degeneration
Chen-Plotkin, A. S., Xiao, J., Geser, F., Martinez-Lage, M., Grossman, M., Unger, T., Wood, E. M., Van Deerlin, V. M., Trojanowski, J. Q., & Lee, V. M.-Y. (2010) · Acta Neuropathologica · doi:10.1007/s00401-009-0576-2 · Cited by 73
“2), although, as others have seen for array data and likely resulting from increased technical noise [44], the”
Ischemia/reperfusion induces interferon stimulated gene expression in microglia
McDonough, A., Lee, R. V., Noor, S., Lee, C., Le, T., Iorga, M., Phillips, J. L. H., Murphy, S., Möller, T., & Weinstein, J. R. (2017) · The Journal of Neuroscience · doi:10.1523/jneurosci.0725-17.2017 · Cited by 71
Vito, R. D., Bellio, R., Trippa, L., & Parmigiani, G. (2018) · Biometrics · doi:10.1111/biom.12974 · Cited by 72
Fortina, P. M., & Surrey, S. (2008) · Nature Biotechnology · doi:10.1038/nbt0308-293 · Cited by 76
Gender-Specific Molecular and Clinical Features Underlie Malignant Pleural Mesothelioma
Rienzo, A. D., Archer, M. A., Yeap, B. Y., Dao, N. T., Sciaranghella, D., Sideris, A. C., Zheng, Y., Holman, A. G., Wang, Y. E., Cin, P. S. D., et al. (2015) · Cancer Research · doi:10.1158/0008-5472.can-15-0751 · Cited by 74
“For quality control across platforms, two MAQC samples were included in the analysis (24).”
MethodSelf-citationInfluence of DNA copy number and mRNA levels on the expression of breast cancer related proteins
Myhre, S., Lingjærde, O.-C., Hennessy, B. T., Aure, M. R., Carey, M. S., Alsner, J., Tramm, T., Overgaard, J., Mills, G. B., Børresen-Dale, A.-L., et al. (2013) · Molecular Oncology · doi:10.1016/j.molonc.2013.02.018 · Cited by 72
“Gene probes on themicroarray might also detect alternative splice variants which can contribute noise into the comparisons (Shi et al., 2006).”
BackgroundDNA Microarrays Are Predictive of Cancer Prognosis: A Re-evaluation
Fan, X., Shi, L. M., Fang, H., Cheng, Y., Perkins, R., & Tong, W. (2010) · Clinical Cancer Research · doi:10.1158/1078-0432.ccr-09-1815 · Cited by 76
Self-citationGenomics and pharmacogenomics of pediatric acute lymphoblastic leukemia
Wu, C., & Li, W. (2018) · Critical Reviews in Oncology/Hematology · doi:10.1016/j.critrevonc.2018.04.002 · Cited by 73
Zachariou, V., Liu, R., LaPlant, Q., Xiao, G., Renthal, W., Chan, G. C., Storm, D. R., Aghajanian, G., & Nestler, E. J. (2008) · Biological Psychiatry · doi:10.1016/j.biopsych.2007.11.021 · Cited by 70
Gene expression profiling in MDS and AML: potential and future avenues
Theilgaard‐Mönch, K., Boultwood, J., Ferrari, S., Giannopoulos, K., Hernández‐Rivas, J. M., Kohlmann, A., Morgan, M. A., Porse, B. T., Tagliafico, E., Zwaan, C. M., et al. (2011) · Leukemia · doi:10.1038/leu.2011.48 · Cited by 77
Borisov, N. M., Suntsova, M., Sorokin, M., Garazha, A., Kovalchuk, O., Aliper, A. M., Ilnitskaya, E., Lezhnina, K. V., Korzinkin, M., Tkachev, V., et al. (2017) · Cell Cycle · doi:10.1080/15384101.2017.1361068 · Cited by 72
“essing method is badly needed to enable data harmonization among different platforms and experiments.(4,5) Recently we showed that aggregation of gene expression data into molecular pathways, each containing dozens or hundreds”
MethodPeterson, S. M., Zhang, J., Weber, G. J., & Freeman, J. L. (2010) · Environmental Health Perspectives · doi:10.1289/ehp.1002590 · Cited by 74
“Criteria used to classify genes as being differentially expressed were adapted from recent recommendations from the Microarray Quality Control Consortium (Guo et al. 2006; Shi et al. 2006).”
Exploring genomic profiles of hepatocellular carcinoma
Woo, H. G., Park, E. S., Thorgeirsson, S. S., & Kim, Y. J. (2011) · Molecular Carcinogenesis · doi:10.1002/mc.20691 · Cited by 74
“bility of microarray data across different sites and platforms [26], many of microarray studies generated only a small number of overlapping genes [24,27].”
BackgroundCREB Binding and Activity in Brain: Regional Specificity and Induction by Electroconvulsive Seizure
Tanis, K. Q., Duman, R. S., & Newton, S. S. (2007) · Biological Psychiatry · doi:10.1016/j.biopsych.2007.08.003 · Cited by 76
“05, and statistical analysis of microarrays (SAM) false discovery rates (FDR) of less than 5% (27,28).”
BackgroundTranscriptional responses to complex mixtures—A review
Sen, B., Mahadevan, B., & DeMarini, D. M. (2007) · Mutation Research/Reviews in Mutation Research · doi:10.1016/j.mrrev.2007.08.002 · Cited by 70
McHale, C. M., Zhang, L., Lan, Q., Li, G., Hubbard, A., Forrest, M. S., Vermeulen, R. C. H., Chen, J., Shen, M., Rappaport, S. M., et al. (2009) · Genomics · doi:10.1016/j.ygeno.2008.12.006 · Cited by 70
“Recent reports have shown good inter-platform reproducibility of gene expression measurements between these two platforms (13).”
BackgroundSelf-citationEarly Blood Profiles of Virus Infection in a Monkey Model for Lassa Fever
Djavani, M. M., Crasta, O. R., Zapata, J. C., Fei, Z., Folkerts, O., Sobral, B., Swindells, M. B., Bryant, J., Davis, H., Pauza, C. D., et al. (2007) · Journal of Virology · doi:10.1128/jvi.00536-07 · Cited by 72
“whereas the next closest commercial platform had a mean of 10% (71).”
BackgroundKang, C. (2010) · International Journal of Oncology · doi:10.3892/ijo_00000637 · Cited by 77
Deleterious effects of formalin-fixation and delays to fixation on RNA and miRNA-Seq profiles
Jones, W. D., Greytak, S. R., Odeh, H. M., Guan, P., Powers, J., Bavarva, J. H., & Moore, H. M. (2019) · Scientific Reports · doi:10.1038/s41598-019-43282-8 · Cited by 74
Self-citationExpectations, validity, and reality in gene expression profiling
Kim, K., Zakharkin, S. O., & Allison, D. B. (2010) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.02.018 · Cited by 79
“[67] Shi L, Reid LH, Jones WD, Shippy R, Warrington JA, Baker SC, et al”
BackgroundA Genome-Wide Expression Analysis in Blood Identifies Pre-Elafin as a Biomarker in ARDS
Wang, Z., Beach, D. F., Su, L., Zhai, R., & Christiani, D. C. (2008) · American Journal of Respiratory Cell and Molecular Biology · doi:10.1165/rcmb.2007-0354oc · Cited by 70
“05), which was proved to be a more reliable ranking criterion for gene selection in previous studies (23, 24).”
BackgroundToxicogenomic profiling of chemically exposed humans in risk assessment
McHale, C. M., Zhang, L., Hubbard, A., & Smith, M. T. (2010) · Mutation Research/Reviews in Mutation Research · doi:10.1016/j.mrrev.2010.04.001 · Cited by 72
“…FDA-led MicroArray Quality Control (MAQC) consortium, a widespread collaboration conceived to broadly address performance, quality, and data analysis issues related to the use of DNA microarrays [37] and the development of Minimum Information About a Microarray Experiment (MIAME) standards [38].”
BackgroundSelf-citationPromises of Biomarkers in Drug Development – A Reality Check
Marrer, E., & Dieterle, F. (2007) · Chemical Biology & Drug Design · doi:10.1111/j.1747-0285.2007.00522.x · Cited by 69
Chen, X., Lü, P., Wang, D., Yang, S., Wu, Y., Shen, H.-Y., Zhong, S., Zhao, J., & Tang, J.-H. (2016) · Gene · doi:10.1016/j.gene.2016.10.015 · Cited by 76
“…of miRNAs mainly because of its recognized precision and superiority for gene expression analysis(Canales et al. 2006; AC C EP TE D M AN U SC R IP T Shi et al. 2006; Wang et al. 2006) and found out that miR-222, miR-29a, miR-34a, miR-423, miR-140, miR-3178, miR-574, miR-6780b and miR-744 showed…”
BackgroundSelf-citationMathijs, K., Brauers, K. J. J., Jennen, D. G. J., Boorsma, A., van Herwijnen, M. H. M., Gottschalk, R. W. H., Kleinjans, J. C. S., & van Delft, J. H. M. (2009) · Toxicological Sciences · doi:10.1093/toxsci/kfp229 · Cited by 68
“5 (Shi et al., 2006).”
Eklund, A. C., & Szallasi, Z. (2008) · Genome Biology · doi:10.1186/gb-2008-9-2-r26 · Cited by 69
“Controlled reference data sets featuring spike-ins or mixtures have demonstrated the possibility of high accuracy in measurements and have been valuable for comparison of normalization algorithms and analytical techniques [2-4].”
MethodSelf-citationCrossNorm: a novel normalization strategy for microarray data in cancers
Cheng, L., Lo, L.-Y., Tang, N. L. S., Wang, D., & Leung, K. (2016) · Scientific Reports · doi:10.1038/srep18898 · Cited by 68
Bioinformatics Analysis for Circulating Cell-Free DNA in Cancer
Huang, C., Du, M., & Wang, L. (2019) · Cancers · doi:10.3390/cancers11060805 · Cited by 73
“A joint effort from the scientific community for the MicroArray Quality Control (MAQC) project [99] is an excellent example to follow to attain this goal.”
BackgroundLee, J., Romero, R. J., Chaiworapongsa, T., Dong, Z., Tarca, A. L., Xu, Y., Chiang, P. J., Kusanovic, J. P., Hassan, S. S., Yeo, L., et al. (2013) · American Journal of Reproductive Immunology · doi:10.1111/aji.12142 · Cited by 70
Transcriptomics of the human endometrium
Díaz-Gimeno, P., Ruíz-Alonso, M., Blesa, D., & Simón, C. (2014) · The International Journal of Developmental Biology · doi:10.1387/ijdb.130340pd · Cited by 79
Metadata
- Authors
- MAQC Consortium, Shi, L., Shi, L., Reid, L. H., Jones, W. D., Shippy, R., Warrington, J. A., Baker, S. C., Collins, P. J., de Longueville, F., Kawasaki, E. S., Lee, K. Y., Luo, Y., Sun, Y. A., Willey, J. C., Setterquist, R. A., Fischer, G. M., Tong, W., Dragan, Y. P., Dix, D. J., Frueh, F. W., Goodsaid, F. M., Herman, D., Jensen, R. V., Johnson, C. D., Lobenhofer, E. K., Puri, R. K., Scherf, U., Thierry-Mieg, J., Wang, C., Wilson, M., Wolber, P. K., Zhang, L., Amur, S., Bao, W., Barbacioru, C. C., Lucas, A. B., Bertholet, V., Boysen, C., Bromley, B., Brown, D., Brunner, A., Canales, R., Cao, X. M., Cebula, T. A., Chen, J. J., Cheng, J., Chu, T.-M., Chudin, E., Corson, J., Corton, J. C., Croner, L. J., Davies, C., Davison, T. S., Delenstarr, G., Deng, X., Dorris, D., Eklund, A. C., Fan, X.-H., Fang, H., Fulmer-Smentek, S., Fuscoe, J. C., Gallagher, K., Ge, W., Guo, L., Guo, X., Hager, J., Haje, P. K., Han, J., Han, T., Harbottle, H. C., Harris, S. C., Hatchwell, E., Hauser, C. A., Hester, S., Hong, H., Hurban, P., Jackson, S. A., Ji, H., Knight, C. R., Kuo, W. P., LeClerc, J. E., Levy, S., Li, Q.-Z., Liu, C., Liu, Y., Lombardi, M. J., Ma, Y., Magnuson, S. R., Maqsodi, B., McDaniel, T., Mei, N., Myklebost, O., Ning, B., Novoradovskaya, N., Orr, M. S., Osborn, T. W., Papallo, A., Patterson, T. A., Perkins, R. G., Peters, E. H., Peterson, R., Philips, K. L., Pine, P. S., Pusztai, L., Qian, F., Ren, H., Rosen, M., Rosenzweig, B. A., Samaha, R. R., Schena, M., Schroth, G. P., Shchegrova, S., Smith, D. D., Staedtler, F., Su, Z., Sun, H., Szallasi, Z., Tezak, Z., Thierry-Mieg, D., Thompson, K. L., Tikhonova, I., Turpaz, Y., Vallanat, B., Van, C., Walker, S. J., Wang, S. J., Wang, Y., Wolfinger, R., Wong, A., Wu, J., Xiao, C., Xie, Q., Xu, J., Yang, W., Zhang, L., Zhong, S., Zong, Y., & Slikker, W.
- Affiliations
- Affymetrix (United States) (US)
- Illumina (United States) (US)
- Agilent Technologies (United States) (US)
- National Cancer Institute (US)
- National Center for Toxicological Research (US)
- Environmental Protection Agency (US)
- United States Food and Drug Administration (US)
- National Institutes of Health (US)
- University of Massachusetts Boston (US)
- Asuragen (United States) (US)
- Center for Biologics Evaluation and Research (US)
- Cedars-Sinai Medical Center (US)
- SAS Institute (United States) (US)
- Center for Food Safety and Applied Nutrition (US)
- Biogen (United States) (US)
- Yale University (US)
- Center for Veterinary Medicine (US)
- Cold Spring Harbor Laboratory (US)
- Sanford Burnham Prebys Medical Discovery Institute (US)
- Stanford Medicine (US)
- Harvard University (US)
- Vanderbilt University (US)
- The University of Texas Southwestern Medical Center (US)
- The University of Texas at Dallas (US)
- GenUs BioSystems (United States) (US)
- Oslo University Hospital (NO)
- The University of Texas MD Anderson Cancer Center (US)
- Luminex (United States) (US)
- Wake Forest University Health Sciences (US)
- University of Illinois Urbana-Champaign (US)
- ORCID
- Jones, Wendell D
- Collins, Patrick J
- Luo, Yuling
- Sun, Yongming Andrew
- Willey, James C
- Tong, Weida
- Frueh, Felix W
- Goodsaid, Federico M
- Johnson, Charles D
- Puri, Raj K
- Scherf, Uwe
- Wang, Charles
- Bao, Wenjun
- Brown, Donna
- Chen, James J
- Cheng, Jing
- Corton, J Christopher
- Croner, Lisa J
- Deng, Xutao
- Eklund, Aron C
- Fang, Hong
- Guo, Lei
- Guo, Xu
- Han, Jing
- Han, Tao
- Harbottle, Heather C
- Harris, Stephen C
- Hatchwell, Eli
- Hester, Susan
- Hong, Huixiao
- Jackson, Scott A
- Ji, Hanlee
- Levy, Shawn
- Li, Quan-Zhen
- Liu, Chunmei
- Liu, Ying
- Ma, Yunqing
- Mei, Nan
- Myklebost, Ola
- Ning, Baitang
- Patterson, Tucker A
- Pusztai, Lajos
- Qian, Feng
- Schroth, Gary P
- Su, Zhenqiang
- Sun, Hongmei
- Tikhonova, Irina
- Turpaz, Yaron
- Walker, Stephen J
- Wang, Sue Jane
- Wang, Yonghong
- Wong, Alex
- Wu, Jie
- Xiao, Chunlin
- Xie, Qian
- Xu, Jun
- Yang, Wen
- Zhang, Liang
- Zhong, Sheng
- Slikker, William
- Venue
- Nature Biotechnology
- Published
- 2006-09-01
- Volume
- 24
- Issue
- 9
- Pages
- 1151-1161
- ISSN
- 0733-222X (ISSN-L)
- 1087-0156
- 1546-1696
- PMC
- PMC3272078
- OpenAlex
- W2010637767
- Cited by
- 2,126
- References
- 43
- Citations by year
- 2026: 21 · 2025: 25 · 2024: 23 · 2023: 39 · 2022: 37 · 2021: 60 · 2020: 43 · 2019: 59 · 2018: 68 · 2017: 87 · 2016: 87 · 2015: 98 · 2014: 122 · 2013: 155 · 2012: 160
- FWCI
- 87.96
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Licence
- www.springer.com/tdm (tdm)
- www.springer.com/tdm (vor)
- Locations
- Nature Biotechnology (publishedVersion)
- PubMed (publishedVersion)
- Europe PMC (PubMed Central) (acceptedVersion)
- PubMed Central (acceptedVersion)
- Digital Access to Libraries (Université catholique de Louvain (UCL), l'Université de Namur (UNamur) and the Université Saint-Louis (USL-B)) (submittedVersion)
- Funders
- U.S. Environmental Protection Agency
- National Institutes of Health (Z99 LM999999, Z99 HL999999, Z99 MH999999, Z99CL999999)
- Topics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Gene expression and cancer classification
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Molecular Biology Techniques and Applications
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Genomics and Phylogenetic Studies
- Keywords
- probe mapping, differentially expressed genes, microarray data analysis
- MeSH terms
- Equipment Design, Quality Assurance, Health Care, Quality Control, Sensitivity and Specificity, United States, Reproducibility of Results, Equipment Failure Analysis, Oligonucleotide Array Sequence Analysis, Gene Expression Profiling
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.