The MicroArray Quality Control (MAQC) project shows inter- and intraplatform reproducibility of gene expression measurements
Open accessThe DOI matches its Crossref record; 43 of the 43 references it cites were found in a registry or an index.
Abstract
Over the last decade, the introduction of microarray technology has had a profound impact on gene expression research. The publication of studies with dissimilar or altogether contradictory results, obtained using different microarray platforms to analyze identical RNA samples, has raised concerns about the reliability of this technology. The MicroArray Quality Control (MAQC) project was initiated to address these concerns, as well as other performance and data analysis issues. Expression data on four titration pools from two distinct reference RNA samples were generated at multiple test sites using a variety of microarray-based and alternative technology platforms. Here we describe the experimental design and probe mapping efforts behind the MAQC project. We show intraplatform consistency across test sites as well as a high level of interplatform concordance in terms of genes identified as differentially expressed. This study provides a resource that represents an important first step toward establishing a framework for the use of microarrays in clinical and regulatory settings.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Next, we applied the |log 2 fold-change|>1 filter advocated in the MAQC study on microarrays, 10 and for microarrays observed a reduction in the eFDR to below 1.5%, save for one outlier site (Figs 3c,d).”
From A comprehensive assessment of RNA-seq accuracy, reproducibility and information content by the Sequencing Quality Control Consortium (2014) · cites this paper as [10]
In the citing paper
“Studies on microarrays have shown that results of typical statistical differential expression tests thresholded by p -value need to be filtered and sorted by effect strength (fold-change) in order to attain robust comparisons across platforms and sites. 10”
From A comprehensive assessment of RNA-seq accuracy, reproducibility and information content by the Sequencing Quality Control Consortium (2014) · cites this paper as [10]
In the citing paper
“The Microarray Quality Control (MAQC) Consortium landmark study10 examined the degree of variance within and across many different microarray platforms and found similar coefficients of variation between platforms.”
From Multi-platform assessment of transcriptome profiling using RNA-seq in the ABRF next-generation sequencing study (2014) · cites this paper as [10]
In the citing paper
“prior work comparing microarray platforms and methods showed high levels of inter-platform concordance for the ability to detect differentially expressed genes.”
From Multi-platform assessment of transcriptome profiling using RNA-seq in the ABRF next-generation sequencing study (2014) · cites this paper as [10]
Citations
Cites
In silico prediction of protein-protein interactions in human macrophages
Souiai, O., Guerfali, F., Ben Miled, S., Brun, C., & Benkahla, A. (2014) · BMC Research Notes · doi:10.1186/1756-0500-7-157 · Cited by 10,803
Significance analysis of microarrays applied to the ionizing radiation response
Tusher, V. G., Tibshirani, R., & Chu, G. (2001) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.091062498 · Cited by 10,568
Pruitt, K. D., Tatusova, T., & Maglott, D. R. (2007) · Nucleic Acids Research · doi:10.1093/nar/gkl842 · Cited by 4,481 · cited 1×
“We mapped the probes (see Supplementary Methods online and Supplementary Notes online) to the RefSeq human mRNA databas”
Method"Stemness": Transcriptional Profiling of Embryonic and Adult Stem Cells
Ramalho-Santos, M., Yoon, S., Matsuzaki, Y., Mulligan, R. C., & Melton, D. A. (2002) · Science · doi:10.1126/science.1072530 · Cited by 1,709
Pruitt, K. D. (2004) · Nucleic Acids Research · doi:10.1093/nar/gki025 · Cited by 1,660 · cited 1×
“We mapped the probes (see Supplementary Methods online and Supplementary Notes online) to the RefSeq human mRNA database 31 (http://www.ncbi.nlm.nih.gov/RefSeq) and to the AceView database 32 (http://www.ncbi.nlm.nih.gov/IEB/Research/Acembly), a less curated but more comprehensive database, which…”
MethodA Stem Cell Molecular Signature
Ivanova, N. B., Dimos, J. T., Schaniel, C., Hackney, J. A., Moore, K. A., & Lemischka, I. R. (2002) · Science · doi:10.1126/science.1073823 · Cited by 1,542
Prediction of cancer outcome with microarrays: a multiple random validation strategy
Michiels, S., Koscielny, S., & Hill, C. (2005) · The Lancet · doi:10.1016/s0140-6736(05)17866-0 · Cited by 1,025 · cited 1×
“, raising doubts about the repeatability, reproducibility and comparability of microarray technolog”
BackgroundMultiple-laboratory comparison of microarray platforms
Irizarry, R. A., Warren, D., Spencer, F., Kim, I. F., Biswal, S., Frank, B. C., Gabrielson, E., Garcia, J. G. N., Geoghegan, J., Germino, G., et al. (2005) · Nature Methods · doi:10.1038/nmeth756 · Cited by 862
Self-citationEvaluation of gene expression measurements from commercial microarray platforms
Tan, P. K. (2003) · Nucleic Acids Research · doi:10.1093/nar/gkg763 · Cited by 706
Thousands of samples are needed to generate a robust gene list for predicting outcome in cancer
Ein-Dor, L., Zuk, O., & Domany, E. (2006) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0601231103 · Cited by 689 · cited 1×
“…to appear in peer-reviewed journals 9,10 , raising doubts about the repeatability, reproducibility and comparability of microarray technology 11–13 , several studies have also been recently published showing increased reproducibility of microarray data generated at different test sites and/or…”
Background
Research works printed but not linked: 1
- Reid, L.H. The value of a proficiency testing program to monitor performance in microarray laboratories. Pharm. Discov. 5, 20–25 (2005).
Cited by
Items 901–1,000 of 1,004
Caboux, É., Paciencia, M., Durand, G., Robinot, N. R., Wozniak, M. B., Galateau-Sallé, F., Byrnes, G. B., Hainaut, P. L., & Calvez‐Kelm, F. L. (2013) · PLoS ONE · doi:10.1371/journal.pone.0079826 · Cited by 23
“Whole-genome expression profiling using microarrays has proven to be a powerful and reliable tool for classifying tumors and identifying predictors of therapeutic responses [1,2].”
BackgroundMethodvan Laar, R. K. (2010) · British Journal of Cancer · doi:10.1038/sj.bjc.6605970 · Cited by 23
dbZach Toxicogenomic Information Management System
Burgoon, L. D., & Zacharewski, T. R. (2007) · Pharmacogenomics · doi:10.2217/14622416.8.3.287 · Cited by 23
“This is critical within the area of toxicogenomics, where good laboratory practice is not necessarily required [3,4], but questions regarding data quality and consistency have been raised [5–13].”
BackgroundIn vitro gene expression analysis of hepatotoxic drugs in rat primary hepatocytes
Suzuki, H., Inoue, T., Matsushita, T., Kobayashi, K., Horii, I., Hirabayashi, Y., & Inoue, T. (2008) · Journal of Applied Toxicology · doi:10.1002/jat.1328 · Cited by 23
“The MicroArray Quality Control (MAQC) consortium showed reproducibility across different microarray platforms, and revealed good correlation between TaqMan and microarrays including the Affymetrix GeneChip (Shi et al., 2006; Patterson et al., 2006).”
MethodSasaki, E., Takahashi, C., Asami, T., & Shimada, Y. (2010) · Plant and Cell Physiology · doi:10.1093/pcp/pcq185 · Cited by 26
“For permissions, please email: journals.permissions@oup.com (Bammler et al. 2005, Irizarry et al. 2005, Shi et al. 2006).”
BackgroundEstimating the proportion of microarray probes expressed in an RNA sample
Shi, W., de Graaf, C. A., Kinkel, S., Achtman, A. H., Baldwin, T. M., Schofield, L., Scott, H. S., Hilton, D. J., & Smyth, G. K. (2010) · Nucleic Acids Research · doi:10.1093/nar/gkp1204 · Cited by 23
PLIDA: cross-platform gene expression normalization using perturbed topic models
Deshwar, A. G., & Morris, Q. (2013) · Bioinformatics · doi:10.1093/bioinformatics/btt574 · Cited by 23
“However, even microarrays from the same provider can differ in the probes used to target specific genes, and these differences can lead to systematic differences in the relationship between their target abundance and the measured probe intensity (Shi et al., 2006).”
BackgroundMethodInfluentialWang, D., Done, S. J., McCready, D. R., Boerner, S. L., Kulkarni, S. R., & Leong, W. L. (2011) · Breast Cancer Research · doi:10.1186/bcr3017 · Cited by 23
Willems, E., Guerrero‐Bosagna, C., Decuypere, E. M. P., Janssens, S., Buyse, J. G., Buys, N., Jensen, P., & Everaert, N. (2016) · Scientific Reports · doi:10.1038/srep20837 · Cited by 26
Feswick, A., Isaacs, M. A., Biales, A. D., Flick, R. W., Bencic, D. C., Wang, R.-L., Vulpe, C. D., Brown‐Augustine, M., Loguinov, A. V., Falciani, F., et al. (2017) · Environmental Toxicology and Chemistry · doi:10.1002/etc.3799 · Cited by 23
“It is important to recognize that a high level of concordance of differentially expressed genes across platforms and laboratories dependsmainly on the statistical criteria used to rank and select them; using a fold change or ratio (vs a p value) generated the most robust results [20].”
BackgroundYoon, S., & Nam, D. (2017) · BMC Genomics · doi:10.1186/s12864-017-3809-0 · Cited by 25
Sheng, X., Fan, T. T., & Jin, X. (2020) · BioMed Research International · doi:10.1155/2020/1470867 · Cited by 23
“Following this previous study [15], we used t-test and fold change methods to identify differentially expressed genes.”
MethodClinical Application of Gene Expression Profiling in Breast Cancer
Sparano, J. A., Fazzari, M. J., & Kenny, P. A. (2010) · Surgical Oncology Clinics of North America · doi:10.1016/j.soc.2010.03.008 · Cited by 23
The Scandinavian Sarcoma Group: 30 years' experience
Alvegård, T. A., Hall, K. S., Bauer, H. C. F., & Rydholm, A. (2009) · Acta Orthopaedica · doi:10.1080/17453690610046602 · Cited by 24
Vogl, C., Flatt, T., Fuhrmann, B., Hofmann, E., Wallner, B., Stiefvater, R., Kovarik, P., Strobl, B., & Müller, M. M. (2010) · BMC Genomics · doi:10.1186/1471-2164-11-199 · Cited by 24
Sohal, D. P. S., Yeatts, A. B., Ye, K. Q., Pellagatti, A., Zhou, L., Pahanish, P., Mo, Y., Bhagat, T. D., Mariadason, J. M., Boultwood, J., et al. (2008) · PLoS ONE · doi:10.1371/journal.pone.0002965 · Cited by 23
“The MicroArray Quality Control consortium (MAQC) was formed to address these questions and recently reported that reproducibility can be enhanced by better matching of microarray probes between platforms [32].”
BackgroundHigh-throughput Gene and Protein Expression Analysis in Pulp Biologic Research: Review
Pääkkönen, V., & Tjäderhane, L. (2009) · Journal of Endodontics · doi:10.1016/j.joen.2009.10.016 · Cited by 23
Aarhus, M., Bruland, O., Bredholt, G., Lybæk, H., Husebye, E. S., Krossnes, B. K., Vedeler, C. A., Wester, K. G., Lund‐Johansen, M., & Knappskog, P. M. (2008) · Journal of Neuro-Oncology · doi:10.1007/s11060-008-9569-6 · Cited by 23
“The ABI microarray platform used in this study of 27 meningiomas and 4 arachnoid cysts has recently been subject to thorough validation in the MicroArray Quality Control project [ 10 ].”
MethodAliferis, C., Statnikov, A., Tsamardinos, I., Schildcrout, J. S., Shepherd, B. E., & Harrell, F. E. (2009) · PLoS ONE · doi:10.1371/journal.pone.0004922 · Cited by 24
“The reproducibility of gene expression microarrays across laboratories for individual gene expression measurements and the ability to differentiate between disease subtypes are well established in recent studies [2,10,11].”
BackgroundFan, J., & Niu, Y. (2007) · Bioinformatics · doi:10.1093/bioinformatics/btm361 · Cited by 25
“If within-array replications have a common correlation g ¼ and observations across arrays are independent, Smyth et al. (2005) introduced the residual maximum likelihood (REML) estimator of 2 g as follows: s 2 g ¼ 1 IJ À 1 n ðJ À 1Þs 2 B;g 1 þ ðI À 1Þ ^ þ JðI À 1Þs 2 W;g 1 À ^ o ; ð14Þ where s 2…”
xMAN: extreme MApping of OligoNucleotides
Li, W., Carroll, J. S., Brown, M. A., & Liu, X. S. (2008) · BMC Genomics · doi:10.1186/1471-2164-9-s1-s20 · Cited by 23
Unifying Gene Expression Measures from Multiple Platforms Using Factor Analysis
Wang, X. V., Verhaak, R. G. W., Purdom, E. A., Spellman, P. T., & Speed, T. P. (2011) · PLoS ONE · doi:10.1371/journal.pone.0017691 · Cited by 24
“Our approach will not apply in a setting such as the MAQC project [9], where the number of samples and the number of platforms are very similar.”
MethodTympanic membrane wound healing in rats assessed by transcriptome profiling
Maria, P. L. S., Redmond, S. L., McInnes, R. L., Atlas, M. D., & Ghassemifar, R. (2011) · The Laryngoscope · doi:10.1002/lary.22150 · Cited by 22
Bioinformatics analysis of a TF-miRNA-lncRNA regulatory network in major depressive disorder
Bian, Q., Chen, J., Wu, J., Ding, F., Li, X., Ma, Q., Zhang, L., Zou, X., & Chen, J. (2021) · Psychiatry Research · doi:10.1016/j.psychres.2021.113842 · Cited by 24
Self-citationThe Quartet Data Portal: integration of community-wide resources for multiomics quality control
Yang, J., Liu, Y., Shang, J., Chen, Q., Chen, Q., Ren, L., Zhang, N., Yu, Y., Li, Z., Song, Y., et al. (2023) · Genome biology · doi:10.1186/s13059-023-03091-9 · Cited by 29
“Intra-and inter-lab performance varies considerably in terms of instruments, experimental protocols, and computational pipelines [14, 22].”
BackgroundMethodSelf-citationBidard, F., Imbeaud, S., Reymond, N., Lespinet, O., Silar, P., Clavé, C., Delacroix, H., Berteaux‐Lecellier, V., & Debuchy, R. (2010) · BMC Research Notes · doi:10.1186/1756-0500-3-171 · Cited by 22
“Progression in microarray v.3 was confirmed by its signal CV which was lower than that obtained with v.2 upon self-to-self hybridization with the common reference cRNA pool (Figure 4). The median CV of microarray v.3 is similar to those obtained in the MAQC study with the commercial Agilent human microarray platform […”
ResultGenome-wide expression quantitative trait loci analysis in asthma
Bossé, Y. (2013) · Current Opinion in Allergy and Clinical Immunology · doi:10.1097/aci.0b013e328364e951 · Cited by 23
“of all known genes in a single experiment [3].”
BackgroundMeta-Analysis of Genetic Programs between Idiopathic Pulmonary Fibrosis and Sarcoidosis
Leng, D., Huan, C., Xie, T., Liang, J., Wang, J., Dai, H., Wang, C., & Jiang, D. (2013) · PLoS ONE · doi:10.1371/journal.pone.0071059 · Cited by 23
“Probes and their measurements of each expression profile from different platforms were all mapped to a common gene list as previously described [25].”
MethodSelf-citationGeneration of a non-small cell lung cancer transcriptome microarray
Tanney, A., Oliver, G. R., Farztdinov, V. M., Kennedy, R. D., Mulligan, J. M., Fulton, C. E., Farragher, S., Field, J. K., Johnston, P. G., Harkin, D. P., et al. (2008) · BMC Medical Genomics · doi:10.1186/1755-8794-1-20 · Cited by 22
“The technical assessment experiment shows that data generated by the Lung Cancer DSA is highly reproducible and exhibits extremely good correlation and low coefficient of variance, similar to those obtained from generic Affymetrix arrays in our own [See Additional file 4] and in previously published studies [ 50 ].”
ResultMorgan, A. A., Khatri, P., Jones, R. H., Sarwal, M., & Butte, A. J. (2010) · BMC Bioinformatics · doi:10.1186/1471-2105-11-s9-s6 · Cited by 22
“However, the foldchange is still an important measure of effect size, and it is substantially more reproducible across studies [ 12 ].”
Kadam, U. S., Moeller, C. A., Irudayaraj, J., & Schulz, B. (2014) · Plant Biotechnology Journal · doi:10.1111/pbi.12161 · Cited by 22
Cekaite, L., Peng, Q., Reiner, A. H., Shahzidi, S., Tveito, S., Furre, I. E., & Hovig, E. J. (2007) · BMC Genomics · doi:10.1186/1471-2164-8-273 · Cited by 22
“Recently, different microarray platforms have been reported to have a good agreement [79,80].”
BackgroundApplication of DNA array technology to mammalian embryos
Niemann, H., Carnwath, J. W., & Kues, W. A. (2007) · Theriogenology · doi:10.1016/j.theriogenology.2007.05.041 · Cited by 27
Ishii, T., Mitsui, T., Suzuki, S., Matsuzaki, Y., & Hasegawa, T. (2012) · Endocrinology · doi:10.1210/en.2011-1627 · Cited by 22
Kinoshita, K., & Obayashi, T. (2009) · Bioinformatics · doi:10.1093/bioinformatics/btp442 · Cited by 22
“Our new indexes provided a good means to determine the functional relationships of the genes, by examining a few examples, and higher performance of Gene Ontology term prediction by using the support vector machine and the multidimensional correlation.”
MethodZhou, Y.-H., Raj, V. R., Siegel, E. R., & Yu, L. (2010) · Biomarker Insights · doi:10.4137/bmi.s5596 · Cited by 25
Sontrop, H. M. J., Moerland, P. D., van den Ham, R., Reinders, M. J. T., & Verhaegh, W. (2009) · BMC Bioinformatics · doi:10.1186/1471-2105-10-389 · Cited by 23
“Based on data from the MAQC study [16], however, Klebanov et al. [17] claim that for Affymetrix arrays the magnitude of technical variation has been gravely exaggerated in the literature and that the effects on the results of statistical inference from Affymetrix GeneChip microarray data are…”
Use of normalization methods for analysis of microarrays containing a high degree of gene effects
Ni, T. T., Lemon, W. J., Shyr, Y., & Zhong, T. P. (2008) · BMC Bioinformatics · doi:10.1186/1471-2105-9-505 · Cited by 22
“Large community-wide efforts including The Microarray Quality Control (MAQC) Project [3] and The Published: 28 November 2008”
BackgroundEssential guidelines for computational method benchmarking
Weber, L. M., Saelens, W., Cannoodt, R., Soneson, C., Hapfelmeier, A., Gardner, P. P., Boulesteix, A.-L., Saeys, Y., & Robinson, M. D. (2019) · Genome Biology · doi:10.1186/s13059-019-1738-8 · Cited by 21
Deregulation of Ion Channel and Transporter Encoding Genes in Pediatric Gliomas
Masselli, M., Laise, P., Tonini, G., Fanelli, D., Pillozzi, S., Cetica, V., Ros, M. D., Sardi, I., Buccoliero, A. M., Aricò, M., et al. (2012) · Frontiers in Oncology · doi:10.3389/fonc.2012.00053 · Cited by 22
LFCseq: a nonparametric approach for differential expression analysis of RNA-seq data
Lin, B., Zhang, L., & Chen, X. (2014) · BMC Genomics · doi:10.1186/1471-2164-15-s10-s7 · Cited by 22
“MAQC dataset [24] contains two RNA sample types, Stratagene’s human universal reference RNA (UHR) and Ambion’s human brain reference RNA (brain).”
BackgroundSelf-citationIntegrative Biological Analysis For Neuropsychopharmacology
Emmett, M. R., Kroes, R. A., Moskal, J. R., Conrad, C. A., Priebe, W., Laezza, F., Meyer‐Baese, A., & Nilsson, C. L. (2013) · Neuropsychopharmacology · doi:10.1038/npp.2013.156 · Cited by 22
Quality Assessment of Transcriptome Data Using Intrinsic Statistical Properties
Brysbaert, G., Pellay, F., Noth, S., & Benecke, A. (2010) · Genomics Proteomics & Bioinformatics · doi:10.1016/s1672-0229(10)60006-x · Cited by 21
Llorens, F., Hummel, M., Hostench, X. P., Ferrer, A., Pluvinet, R., Vivancos, A. P., Castillo, E., Iraola‐Guzmán, S., Mosquera, A. M., González‐Roca, E., et al. (2011) · BMC Genomics · doi:10.1186/1471-2164-12-326 · Cited by 22
“to integrate data derived from different platforms [41,42].”
BackgroundBacher, U., Kohlmann, A., & Haferlach, T. (2010) · Cancer Treatment Reviews · doi:10.1016/j.ctrv.2010.05.002 · Cited by 21
Influence of Aromatase Absence on the Gene Expression and Histology of the Mouse Meibomian Gland
Darabad, R. R., Suzuki, T., Richards, S. M., Jensen, R. V., Jakobiec, F. A., Zakka, F. R., Liu, S., & Sullivan, D. A. (2012) · Investigative Ophthalmology & Visual Science · doi:10.1167/iovs.12-10992 · Cited by 22
Self-citationTranscriptome Tomography for Brain Analysis in the Web-Accessible Anatomical Space
Okamura-Oho, Y., Shimokawa, K., Takemoto, S., Hirakiyama, A., Nakamura, S., Tsujimura, Y., Nishimura, M., Kasukawa, T., Masumoto, K.-H., Nikaido, I., et al. (2012) · PLoS ONE · doi:10.1371/journal.pone.0045373 · Cited by 21
“Considering the previous inter-platform comparison study of microarray datasets [13], good agreement was observed between the two datasets in a scatter plot (Figure 3B, r= 0.73, p , 1.0 6 10 2 12 ).”
ResultInfluentialBayesian ranking and selection methods using hierarchical mixture models in microarray studies
Noma, H., Matsui, S., Omori, T., & Sato, T. S. (2009) · Biostatistics · doi:10.1093/biostatistics/kxp047 · Cited by 24
Transcriptome assembly and quantification from Ion Torrent RNA-Seq data
Mangul, S., Caciula, A., Seesi, S. A., Brinza, D., Măndoiu, I., & Zelikovsky, A. (2014) · BMC Genomics · doi:10.1186/1471-2164-15-s5-s7 · Cited by 22
“As described in [41], each TaqMan Assay was run in four replicates for each measured gene.”
MethodRebuli, M. E., Pawlak, E. A., Walsh, D. M., Martin, E. M., & Jaspers, I. (2018) · Scientific Reports · doi:10.1038/s41598-018-21443-5 · Cited by 21
Gene expression signature for detection of gastric cancer in peripheral blood
Shi, J., Cheng, C., Ma, J., Liew, C., & Geng, X. (2018) · Oncology Letters · doi:10.3892/ol.2018.8577 · Cited by 21
“These probe sets were also present within the MAQC list for Affymetrix U133 P2 microarray, as reported by the MAQC Consortium (28) and were repeatable within ±15% in whole blood technical replicates (unpublished data).”
Ama, O. M., Anku, W. W., & Ray, S. S. (2019) · International Journal of Electrochemical Science · doi:10.20964/2019.10.41 · Cited by 21
Anderson, G. D., Peterson, T. C., Haar, C. V., Farin, F. M., Bammler, T. K., MacDonald, J. W., Kantor, E. D., & Hoane, M. R. (2015) · The AAPS Journal · doi:10.1208/s12248-015-9792-y · Cited by 21
“We used a combination of p value and foldchange criteria based on observations made by the MAQC consortium (31).”
Transcriptomic biomarkers of human ageing in peripheral blood mononuclear cell total RNA
Vo, T. K. D., Godard, P., de Saint‐Hubert, M., Morrhaye, G., Bauwens, É., Debacq‐Chainiaux, F., Glupczynski, Y., Swine, C., Geenen, V. G., & Martens, H. (2009) · Experimental Gerontology · doi:10.1016/j.exger.2009.12.001 · Cited by 21
Study of gene expression profiles in TK6 human cells exposed to DNA-oxidizing agents
Platel, A., Gervais, V., Sajot, N., Nesslany, F., Marzin, D. R., & Claude, N. (2010) · Mutation research. Fundamental and molecular mechanisms of mutagenesis · doi:10.1016/j.mrfmmm.2010.04.004 · Cited by 23
Strategies for biomarker discovery in fibrotic disease
Marshall, R. P., Simpson, J. K., & Lukey, P. T. (2013) · Biochimica et Biophysica Acta (BBA) - Molecular Basis of Disease · doi:10.1016/j.bbadis.2013.01.018 · Cited by 23
Miclaus, K. J., Wolfinger, R. D., Vega, S. C., Chierici, M., Furlanello, C., Lambert, C. G., Hong, H., Zhang, L., Yin, S., & Goodsaid, F. M. (2010) · The Pharmacogenomics Journal · doi:10.1038/tpj.2010.36 · Cited by 21
Self-citationLiterature-related discovery: common factors for Parkinson’s Disease and Crohn’s Disease
Kostoff, R. N. (2014) · Scientometrics · doi:10.1007/s11192-014-1298-3 · Cited by 23
Extended analysis of benchmark datasets for Agilent two-color microarrays
Kerr, K. F. (2007) · BMC Bioinformatics · doi:10.1186/1471-2105-8-371 · Cited by 21
“Background Recently, the MicroArray Quality Control (MAQC) Consortium published a series of papers on an important effort to address ongoing issues concerning the reliability of microarray data [1-6].”
BackgroundLyu, Y., & Li, Q. (2016) · BMC Bioinformatics · doi:10.1186/s12859-015-0847-y · Cited by 24
Self-citationCodelink: an R package for analysis of GE healthcare gene expression bioarrays
Díez, D., Álvarez, R., & Dopazo, A. (2007) · Bioinformatics · doi:10.1093/bioinformatics/btm072 · Cited by 21
Cross-platform ultradeep transcriptomic profiling of human reference RNA samples by RNA-Seq
Xu, J., Su, Z., Hong, H., Thierry‐Mieg, J., Thierry‐Mieg, D., Kreil, D. P., Mason, C. E., Tong, W., & Shi, L. M. (2014) · Scientific Data · doi:10.1038/sdata.2014.20 · Cited by 24
Self-citationMicroarray test results should not be compensated for multiplicity of gene contents
Konishi, T. (2011) · BMC Systems Biology · doi:10.1186/1752-0509-5-s2-s6 · Cited by 22
“While the quality of microarray assays has been questioned [1,2], the actual level of noise is therefore low enough to reflect biological differences among samples; thus, considerable improvements have been made to chips, reagents, and experimental protocols [29], and advances in data analysis have resolved many of th…”
BackgroundGao, J., Wang, Q., Fu, F., Zhao, Y., Yang, T., Li, X., Sun, Y., Hu, H., Ma, L., Miao, L., et al. (2025) · Signal Transduction and Targeted Therapy · doi:10.1038/s41392-025-02306-8 · Cited by 23
Self-citationStreit, S., Michalski, C. W., Erkan, M., Friess, H., & Kleeff, J. H. (2009) · Pancreatology · doi:10.1159/000212084 · Cited by 20
Peters, T. J., French, H. J., Bradford, S. T., Pidsley, R., Stirzaker, C., Varinli, H., Nair, S. S., Qu, W., Song, J. Z., Giles, K. A., et al. (2018) · Bioinformatics · doi:10.1093/bioinformatics/bty675 · Cited by 21
Random gene sets in predicting survival of patients with hepatocellular carcinoma
Itzel, T., Spang, R., Maass, T., Munker, S., Roessler, S. K., Ebert, M. P. A., Schlitt, H. J., Herr, W., Evert, M., & Teufel, A. (2019) · Journal of Molecular Medicine · doi:10.1007/s00109-019-01764-2 · Cited by 24
Covariance adjustment for batch effect in gene expression data
Lee, J. A., Dobbin, K. K., & Ahn, J. (2014) · Statistics in Medicine · doi:10.1002/sim.6157 · Cited by 21
Microarray Meta-Analysis: From Data to Expression to Biological Relationships
Feichtinger, J., Thallinger, G., McFarlane, R. J., & Larcombe, L. D. (2012) · Springer eBooks · doi:10.1007/978-3-7091-0947-2_4 · Cited by 22
The use of neuroproteomics in drug abuse research
Lull, M. E., Freeman, W. M., VanGuilder, H. D., & Vrana, K. E. (2009) · Drug and Alcohol Dependence · doi:10.1016/j.drugalcdep.2009.10.001 · Cited by 22
“These approaches include qPCR and bead-based multiplex assays with the flexibility to examine a large number of samples (10s–100s) and a large number of genes (10s–100s) (Shi et al., 2006).”
BackgroundCampo, E. (2013) · Modern Pathology · doi:10.1038/modpathol.2012.179 · Cited by 20
Yang, D., Parrish, R. S., & Brock, G. N. (2013) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2013.12.002 · Cited by 20
“This result agrees with conclusions from the MicroArray Quality Control (MAQC) project [42, 40], which found that DE gene rankings based on fold-changes (FCs) produced the most consistent gene lists.”
ResultDégletagne, C., Keime, C., Rey, B., de Dinechin, M., Forcheron, F., Chuchana, P., Jouventin, P., Gautier, C., & Duchamp, C. (2010) · BMC Genomics · doi:10.1186/1471-2164-11-344 · Cited by 20
“For the validated genes, the fold change assessed by qPCR was not always the same as assessed by the microarrays, and the change was more often higher with qPCR than with microarrays, as previously described for homologous hybridization [13,14].”
Shi, X., Meng, X., Sun, L., Liu, J., Zheng, J., Gai, H., Yang, R., & Yeung, E. S. (2010) · Lab on a Chip · doi:10.1039/c005258b · Cited by 20
Methods of Analysis and Meta-Analysis for Identifying Differentially Expressed Genes
Kontou, P. I., Pavlopoulou, A., & Bagos, P. G. (2018) · Methods in molecular biology · doi:10.1007/978-1-4939-7868-7_12 · Cited by 26
Dean, B., Parkin, G. M., & Gibbons, A. S. (2020) · Experimental Brain Research · doi:10.1007/s00221-020-05730-0 · Cited by 21
Multisite Evaluation of Next-Generation Methods for Small RNA Quantification
Herbert, Z. T., Thimmapuram, J., Xie, S., Kershner, J. P., Kolling, F., Ringelberg, C. S., LeClerc, A. L., Alekseyev, Y. O., Fan, J., Podnar, J. W., et al. (2020) · Journal of Biomolecular Techniques JBT · doi:10.7171/jbt.20-3102-001 · Cited by 22
Toward Personalized Medicine in the Neuropsychiatric Field
Wong, E. H. F., Fox, J. C., Ng, M., & Lee, C. (2011) · International review of neurobiology · doi:10.1016/b978-0-12-387718-5.00013-4 · Cited by 21
Gower, A. C., Spira, A. E., & Lenburg, M. E. (2011) · BMC Bioinformatics · doi:10.1186/1471-2105-12-381 · Cited by 20
“Although comparison of differential gene expression across more distinct array platforms is possible [19], we chose to restrict our analysis to data from a single family of related arrays to avoid challenges in data comparability.”
Coordinated Regulation of Growth Genes in Saccharomyces cerevisiae
Slattery, M. G., & Heideman, W. (2007) · Cell Cycle · doi:10.4161/cc.6.10.4257 · Cited by 20
Comprehensive analysis of Arabidopsis expression level polymorphisms with simple inheritance
Plantegenet, S., Weber, J., Goldstein, D. R., Zeller, G., Nussbaumer, C., Thomas, J., Weigel, D., Harshman, K., & Hardtke, C. S. (2009) · Molecular Systems Biology · doi:10.1038/msb.2008.79 · Cited by 22
Chipping away at diagnostics for neurodegenerative diseases
Scherzer, C. R. (2009) · Neurobiology of Disease · doi:10.1016/j.nbd.2009.02.016 · Cited by 22
“This consortium of 51 academic institutions and industrial partners systematically addressed the technical reproducibility of microarray measurements within and between laboratories, as well as across different gene expression platforms (Applied Biosystems, Affymetrix, Agilent, Eppendorf, GE Healthcare, Illumina, NCI_…”
BackgroundMethodInfluentialBriggs, J., Mason, E. A., Ovchinnikov, D. A., Wells, C. A., & Wolvetang, E. J. (2013) · Stem Cells Translational Medicine · doi:10.5966/sctm.2012-0117 · Cited by 21
Serum from humans on long-term calorie restriction enhances stress resistance in cell culture
Omodei, D., Licastro, D., Salvatore, F., Crosby, S. D., & Fontana, L. (2013) · Aging · doi:10.18632/aging.100584 · Cited by 21
“Differentially expressed genes were selected using a moderate t-test approach [10] and multiple test correction [11], and these genes were used to identify transcriptionally altered pathways.”
MethodA Primer on the Current State of Microarray Technologies
Trachtenberg, A. J., Robert, J.-H., Abdalla, A. E. A., Fraser, A., He, S. Y., Lacy, J. N., Rivas‐Morello, C., Truong, A. K., Hardiman, G., Lucila Ohno-machado, A., et al. (2011) · Methods in molecular biology · doi:10.1007/978-1-61779-400-1_1 · Cited by 20
Self-citationPrimer: strategies for identifying genes involved in renal disease
de Borst, M. H., Benigni, A., & Remuzzi, G. (2008) · Nature Reviews Nephrology · doi:10.1038/ncpneph0785 · Cited by 22
“The consortium published a series of papers that reported comparable performance by different gene array platforms, intraplatform consistency across test sites, and interplatform consistency in terms of genes that were found to be differentially expressed”
BackgroundRežen, T., Juvan, P., Tacer, K. F., Kuzman, D., Roth, A. B., Pompon, D., Aggerbeck, L. P., Meyer, U., & Rozman, D. (2008) · BMC Genomics · doi:10.1186/1471-2164-9-76 · Cited by 22
Gene expression profiling as a tool for basic analysis and clinical application of human cancer
García‐Escudero, R., & Paramio, J. M. (2008) · Molecular Carcinogenesis · doi:10.1002/mc.20430 · Cited by 20
“platforms, and the consistency of experiments made with same platforms, and within same and between different laboratories [35].”
BackgroundPapapetropoulos, S., Shehadeh, L. A., & McCorquodale, D. S. (2007) · Journal of Neuroscience Research · doi:10.1002/jnr.21355 · Cited by 20
Selective discussion and transparency in microarray research findings for cancer outcomes
Ioannidis, J. P. A., Polyzos, N. P., & Trikalinos, T. A. (2007) · European Journal of Cancer · doi:10.1016/j.ejca.2007.05.019 · Cited by 20
Quality Control and Protein Microarrays
Kricka, L. J., & Master, S. R. (2009) · Clinical Chemistry · doi:10.1373/clinchem.2009.126557 · Cited by 20
An international comparability study on quantification of mRNA gene expression ratios: CCQM-P103.1
Devonshire, A. S., Sanders, R., Whale, A. S., Nixon, G. J., Cowen, S., Ellison, S. L. R., Parkes, H., Pine, P. S., Salit, M., McDaniel, J., et al. (2016) · Biomolecular Detection and Quantification · doi:10.1016/j.bdq.2016.05.003 · Cited by 19
Self-citationMarko, N. F., Quackenbush, J. F., & Weil, R. J. (2011) · PLoS ONE · doi:10.1371/journal.pone.0020826 · Cited by 24
“These effects have been studied extensively [39,40,41,42,43,44,45,46,47,48], and numerous modeling and”
BackgroundAn optimized protocol for generation and analysis of Ion Proton sequencing reads for RNA-Seq
Yuan, Y., Xu, H., & Leung, R. K. (2016) · BMC Genomics · doi:10.1186/s12864-016-2745-8 · Cited by 22
“This sample is composed of total RNA from 10 human cell lines, and the corresponding differential gene expression data measured by Taqman array in MAQCI [53], which is widely used for assessing the accuracy of genes expression quantification in RNA-seq are available [11, 12, 53].”
BackgroundMethodResultInfluentialLiang, K. (2013) · Elsevier eBooks · doi:10.1533/9781908818232.49 · Cited by 19
Borisov, N. M., Sorokin, M., Zolotovskaya, M., Borisov, C., & Buzdin, A. A. (2022) · Current Protocols · doi:10.1002/cpz1.444 · Cited by 20
“Third, we compared clustering of human brain and non-brain samples profiled using different MH platforms in the Microarray Quality control (MAQC; Shi et al., 2006), Sequencing Quality Control (SEQC; Xu et al., 2016), GTEx, and TCGA projects, for 449 samples total.”
MethodShang, J., Jiang, H., Zhao, Y., Lai, J., Shi, L. M., Yang, J., Chen, H., & Zheng, Y. (2023) · EBioMedicine · doi:10.1016/j.ebiom.2023.104728 · Cited by 22
Self-citationNormalization of RNA-Seq data using adaptive trimmed mean with multi-reference
Singh, V., Kirtipal, N., Song, B., & Lee, S. (2024) · Briefings in Bioinformatics · doi:10.1093/bib/bbae241 · Cited by 20
“In the second study,we analyzed the performance of each method on the MAQC2 and MAQC3 datasets of the microarray quality control project (MAQC) Project [33].”
MethodChen, Z., Liu, Z., Deng, X., Warden, C., Li, W., & García‐Aguilar, J. (2012) · Diseases of the Colon & Rectum · doi:10.1097/dcr.0b013e31824f873f · Cited by 21
“Predictive biomarkers for persistent lymph node metastasis were derived by use of a combination of methods including feature (candidate biomarker) selection, classification model fitting, and cross-validation, as described previously.(20,21) Differential features were selected from all probes on the aCGH array based o…”
MethodSelf-citation
Metadata
- Authors
- MAQC Consortium, Shi, L., Shi, L., Reid, L. H., Jones, W. D., Shippy, R., Warrington, J. A., Baker, S. C., Collins, P. J., de Longueville, F., Kawasaki, E. S., Lee, K. Y., Luo, Y., Sun, Y. A., Willey, J. C., Setterquist, R. A., Fischer, G. M., Tong, W., Dragan, Y. P., Dix, D. J., Frueh, F. W., Goodsaid, F. M., Herman, D., Jensen, R. V., Johnson, C. D., Lobenhofer, E. K., Puri, R. K., Scherf, U., Thierry-Mieg, J., Wang, C., Wilson, M., Wolber, P. K., Zhang, L., Amur, S., Bao, W., Barbacioru, C. C., Lucas, A. B., Bertholet, V., Boysen, C., Bromley, B., Brown, D., Brunner, A., Canales, R., Cao, X. M., Cebula, T. A., Chen, J. J., Cheng, J., Chu, T.-M., Chudin, E., Corson, J., Corton, J. C., Croner, L. J., Davies, C., Davison, T. S., Delenstarr, G., Deng, X., Dorris, D., Eklund, A. C., Fan, X.-H., Fang, H., Fulmer-Smentek, S., Fuscoe, J. C., Gallagher, K., Ge, W., Guo, L., Guo, X., Hager, J., Haje, P. K., Han, J., Han, T., Harbottle, H. C., Harris, S. C., Hatchwell, E., Hauser, C. A., Hester, S., Hong, H., Hurban, P., Jackson, S. A., Ji, H., Knight, C. R., Kuo, W. P., LeClerc, J. E., Levy, S., Li, Q.-Z., Liu, C., Liu, Y., Lombardi, M. J., Ma, Y., Magnuson, S. R., Maqsodi, B., McDaniel, T., Mei, N., Myklebost, O., Ning, B., Novoradovskaya, N., Orr, M. S., Osborn, T. W., Papallo, A., Patterson, T. A., Perkins, R. G., Peters, E. H., Peterson, R., Philips, K. L., Pine, P. S., Pusztai, L., Qian, F., Ren, H., Rosen, M., Rosenzweig, B. A., Samaha, R. R., Schena, M., Schroth, G. P., Shchegrova, S., Smith, D. D., Staedtler, F., Su, Z., Sun, H., Szallasi, Z., Tezak, Z., Thierry-Mieg, D., Thompson, K. L., Tikhonova, I., Turpaz, Y., Vallanat, B., Van, C., Walker, S. J., Wang, S. J., Wang, Y., Wolfinger, R., Wong, A., Wu, J., Xiao, C., Xie, Q., Xu, J., Yang, W., Zhang, L., Zhong, S., Zong, Y., & Slikker, W.
- Affiliations
- Affymetrix (United States) (US)
- Illumina (United States) (US)
- Agilent Technologies (United States) (US)
- National Cancer Institute (US)
- National Center for Toxicological Research (US)
- Environmental Protection Agency (US)
- United States Food and Drug Administration (US)
- National Institutes of Health (US)
- University of Massachusetts Boston (US)
- Asuragen (United States) (US)
- Center for Biologics Evaluation and Research (US)
- Cedars-Sinai Medical Center (US)
- SAS Institute (United States) (US)
- Center for Food Safety and Applied Nutrition (US)
- Biogen (United States) (US)
- Yale University (US)
- Center for Veterinary Medicine (US)
- Cold Spring Harbor Laboratory (US)
- Sanford Burnham Prebys Medical Discovery Institute (US)
- Stanford Medicine (US)
- Harvard University (US)
- Vanderbilt University (US)
- The University of Texas Southwestern Medical Center (US)
- The University of Texas at Dallas (US)
- GenUs BioSystems (United States) (US)
- Oslo University Hospital (NO)
- The University of Texas MD Anderson Cancer Center (US)
- Luminex (United States) (US)
- Wake Forest University Health Sciences (US)
- University of Illinois Urbana-Champaign (US)
- ORCID
- Jones, Wendell D
- Collins, Patrick J
- Luo, Yuling
- Sun, Yongming Andrew
- Willey, James C
- Tong, Weida
- Frueh, Felix W
- Goodsaid, Federico M
- Johnson, Charles D
- Puri, Raj K
- Scherf, Uwe
- Wang, Charles
- Bao, Wenjun
- Brown, Donna
- Chen, James J
- Cheng, Jing
- Corton, J Christopher
- Croner, Lisa J
- Deng, Xutao
- Eklund, Aron C
- Fang, Hong
- Guo, Lei
- Guo, Xu
- Han, Jing
- Han, Tao
- Harbottle, Heather C
- Harris, Stephen C
- Hatchwell, Eli
- Hester, Susan
- Hong, Huixiao
- Jackson, Scott A
- Ji, Hanlee
- Levy, Shawn
- Li, Quan-Zhen
- Liu, Chunmei
- Liu, Ying
- Ma, Yunqing
- Mei, Nan
- Myklebost, Ola
- Ning, Baitang
- Patterson, Tucker A
- Pusztai, Lajos
- Qian, Feng
- Schroth, Gary P
- Su, Zhenqiang
- Sun, Hongmei
- Tikhonova, Irina
- Turpaz, Yaron
- Walker, Stephen J
- Wang, Sue Jane
- Wang, Yonghong
- Wong, Alex
- Wu, Jie
- Xiao, Chunlin
- Xie, Qian
- Xu, Jun
- Yang, Wen
- Zhang, Liang
- Zhong, Sheng
- Slikker, William
- Venue
- Nature Biotechnology
- Published
- 2006-09-01
- Volume
- 24
- Issue
- 9
- Pages
- 1151-1161
- ISSN
- 0733-222X (ISSN-L)
- 1087-0156
- 1546-1696
- PMC
- PMC3272078
- OpenAlex
- W2010637767
- Cited by
- 2,126
- References
- 43
- Citations by year
- 2026: 21 · 2025: 25 · 2024: 23 · 2023: 39 · 2022: 37 · 2021: 60 · 2020: 43 · 2019: 59 · 2018: 68 · 2017: 87 · 2016: 87 · 2015: 98 · 2014: 122 · 2013: 155 · 2012: 160
- FWCI
- 87.96
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Licence
- www.springer.com/tdm (tdm)
- www.springer.com/tdm (vor)
- Locations
- Nature Biotechnology (publishedVersion)
- PubMed (publishedVersion)
- Europe PMC (PubMed Central) (acceptedVersion)
- PubMed Central (acceptedVersion)
- Digital Access to Libraries (Université catholique de Louvain (UCL), l'Université de Namur (UNamur) and the Université Saint-Louis (USL-B)) (submittedVersion)
- Funders
- U.S. Environmental Protection Agency
- National Institutes of Health (Z99 LM999999, Z99 HL999999, Z99 MH999999, Z99CL999999)
- Topics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Gene expression and cancer classification
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Molecular Biology Techniques and Applications
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Genomics and Phylogenetic Studies
- Keywords
- probe mapping, differentially expressed genes, microarray data analysis
- MeSH terms
- Equipment Design, Quality Assurance, Health Care, Quality Control, Sensitivity and Specificity, United States, Reproducibility of Results, Equipment Failure Analysis, Oligonucleotide Array Sequence Analysis, Gene Expression Profiling
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.