The MicroArray Quality Control (MAQC) project shows inter- and intraplatform reproducibility of gene expression measurements
Open accessThe DOI matches its Crossref record; 43 of the 43 references it cites were found in a registry or an index.
Abstract
Over the last decade, the introduction of microarray technology has had a profound impact on gene expression research. The publication of studies with dissimilar or altogether contradictory results, obtained using different microarray platforms to analyze identical RNA samples, has raised concerns about the reliability of this technology. The MicroArray Quality Control (MAQC) project was initiated to address these concerns, as well as other performance and data analysis issues. Expression data on four titration pools from two distinct reference RNA samples were generated at multiple test sites using a variety of microarray-based and alternative technology platforms. Here we describe the experimental design and probe mapping efforts behind the MAQC project. We show intraplatform consistency across test sites as well as a high level of interplatform concordance in terms of genes identified as differentially expressed. This study provides a resource that represents an important first step toward establishing a framework for the use of microarrays in clinical and regulatory settings.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Next, we applied the |log 2 fold-change|>1 filter advocated in the MAQC study on microarrays, 10 and for microarrays observed a reduction in the eFDR to below 1.5%, save for one outlier site (Figs 3c,d).”
From A comprehensive assessment of RNA-seq accuracy, reproducibility and information content by the Sequencing Quality Control Consortium (2014) · cites this paper as [10]
In the citing paper
“Studies on microarrays have shown that results of typical statistical differential expression tests thresholded by p -value need to be filtered and sorted by effect strength (fold-change) in order to attain robust comparisons across platforms and sites. 10”
From A comprehensive assessment of RNA-seq accuracy, reproducibility and information content by the Sequencing Quality Control Consortium (2014) · cites this paper as [10]
In the citing paper
“The Microarray Quality Control (MAQC) Consortium landmark study10 examined the degree of variance within and across many different microarray platforms and found similar coefficients of variation between platforms.”
From Multi-platform assessment of transcriptome profiling using RNA-seq in the ABRF next-generation sequencing study (2014) · cites this paper as [10]
In the citing paper
“prior work comparing microarray platforms and methods showed high levels of inter-platform concordance for the ability to detect differentially expressed genes.”
From Multi-platform assessment of transcriptome profiling using RNA-seq in the ABRF next-generation sequencing study (2014) · cites this paper as [10]
Citations
Cites
In silico prediction of protein-protein interactions in human macrophages
Souiai, O., Guerfali, F., Ben Miled, S., Brun, C., & Benkahla, A. (2014) · BMC Research Notes · doi:10.1186/1756-0500-7-157 · Cited by 10,803
Significance analysis of microarrays applied to the ionizing radiation response
Tusher, V. G., Tibshirani, R., & Chu, G. (2001) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.091062498 · Cited by 10,568
Pruitt, K. D., Tatusova, T., & Maglott, D. R. (2007) · Nucleic Acids Research · doi:10.1093/nar/gkl842 · Cited by 4,481 · cited 1×
“We mapped the probes (see Supplementary Methods online and Supplementary Notes online) to the RefSeq human mRNA databas”
Method"Stemness": Transcriptional Profiling of Embryonic and Adult Stem Cells
Ramalho-Santos, M., Yoon, S., Matsuzaki, Y., Mulligan, R. C., & Melton, D. A. (2002) · Science · doi:10.1126/science.1072530 · Cited by 1,709
Pruitt, K. D. (2004) · Nucleic Acids Research · doi:10.1093/nar/gki025 · Cited by 1,660 · cited 1×
“We mapped the probes (see Supplementary Methods online and Supplementary Notes online) to the RefSeq human mRNA database 31 (http://www.ncbi.nlm.nih.gov/RefSeq) and to the AceView database 32 (http://www.ncbi.nlm.nih.gov/IEB/Research/Acembly), a less curated but more comprehensive database, which…”
MethodA Stem Cell Molecular Signature
Ivanova, N. B., Dimos, J. T., Schaniel, C., Hackney, J. A., Moore, K. A., & Lemischka, I. R. (2002) · Science · doi:10.1126/science.1073823 · Cited by 1,542
Prediction of cancer outcome with microarrays: a multiple random validation strategy
Michiels, S., Koscielny, S., & Hill, C. (2005) · The Lancet · doi:10.1016/s0140-6736(05)17866-0 · Cited by 1,025 · cited 1×
“, raising doubts about the repeatability, reproducibility and comparability of microarray technolog”
BackgroundMultiple-laboratory comparison of microarray platforms
Irizarry, R. A., Warren, D., Spencer, F., Kim, I. F., Biswal, S., Frank, B. C., Gabrielson, E., Garcia, J. G. N., Geoghegan, J., Germino, G., et al. (2005) · Nature Methods · doi:10.1038/nmeth756 · Cited by 862
Self-citationEvaluation of gene expression measurements from commercial microarray platforms
Tan, P. K. (2003) · Nucleic Acids Research · doi:10.1093/nar/gkg763 · Cited by 706
Thousands of samples are needed to generate a robust gene list for predicting outcome in cancer
Ein-Dor, L., Zuk, O., & Domany, E. (2006) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0601231103 · Cited by 689 · cited 1×
“…to appear in peer-reviewed journals 9,10 , raising doubts about the repeatability, reproducibility and comparability of microarray technology 11–13 , several studies have also been recently published showing increased reproducibility of microarray data generated at different test sites and/or…”
Background
Research works printed but not linked: 1
- Reid, L.H. The value of a proficiency testing program to monitor performance in microarray laboratories. Pharm. Discov. 5, 20–25 (2005).
Cited by
featureCounts: an efficient general purpose program for assigning sequence reads to genomic features
Liao, Y., Smyth, G. K., & Shi, W. (2014) · Bioinformatics · doi:10.1093/bioinformatics/btt656 · Cited by 27,219
RSEM: accurate transcript quantification from RNA-Seq data with or without a reference genome
Li, B., & Dewey, C. N. (2011) · BMC Bioinformatics · doi:10.1186/1471-2105-12-323 · Cited by 22,179
“(cid:129) Project name: RSEM (cid:129) Project home page : http://deweylab.biostat.wisc. edu/rsem (cid:129) Operating systems: Any POSIX-compatible platform (e.g., Linux, Mac OS X, Cygwin) (cid:129) Programming languages: C++, Perl (cid:129) Other requirements: Pthreads; Bowtie [29] for the default alignment mode of r…”
Methodvoom: precision weights unlock linear model analysis tools for RNA-seq read counts
Law, C. W., Chen, Y., Shi, W., & Smyth, G. K. (2014) · Genome Biology · doi:10.1186/gb-2014-15-2-r29 · Cited by 6,155
“The groups are labeled A–D and are closely analogous to the similarly labeled RNA samples used in the earlier microarray quality control study [57].”
ResultAlternative isoform regulation in human tissue transcriptomes
Wang, E. T., Sandberg, R., Luo, S., Khrebtukova, I., Zhang, L., Mayr, C., Kingsmore, S. F., Schroth, G. P., & Burge, C. B. (2008) · Nature · doi:10.1038/nature07509 · Cited by 5,181
Self-citationGyörffy, B., Lanczky, A., Eklund, A. C., Denkert, C., Budczies, J., Li, Q., & Szallasi, Z. (2010) · Breast Cancer Research and Treatment · doi:10.1007/s10549-009-0674-9 · Cited by 4,169
“The accuracy of microarray-based gene expression measurements has been evaluated by a wide array of diverse studies [6–8], leading to the general conclusion that it is a powerful surveyor of gene expression changes when its limitations are considered properly.”
BackgroundSelf-citationThe Subread aligner: fast, accurate and scalable read mapping by seed-and-vote
Liao, Y., Smyth, G. K., & Shi, W. (2013) · Nucleic Acids Research · doi:10.1093/nar/gkt214 · Cited by 3,064
“The SEQC project, which is the third stage of the wellknown MAQC project (31), is producing benchmark nextgen sequencing data.”
BackgroundRNA-seq: An assessment of technical reproducibility and comparison with gene expression arrays
Marioni, J. C., Mason, C. E., Mane, S. M., Stephens, M., & Gilad, Y. (2008) · Genome Research · doi:10.1101/gr.079558.108 · Cited by 2,771
“…for 81% of genes called significantly differentially expressed from the array data, and the correlation of fold change ratios between the two technologies (Spearman correlation = 0.73) is similar or higher than observed in comparisons across different microarray platforms (Shi et al. 2006).”
ResultInfluentialDu, P., Zhang, X., Huang, C.-C., Jafari, N., Kibbe, W. A., Hou, L., & Lin, S. M. (2010) · BMC Bioinformatics · doi:10.1186/1471-2105-11-587 · Cited by 2,136
“One of the major statistical paradigms in expression microarray analysis has been the “Fold change-ranking with a non-stringent p-value cutoff” [18-20].”
BackgroundDirect multiplexed measurement of gene expression with color-coded probe pairs
Geiss, G. K., Bumgarner, R. E., Birditt, B., Dahl, T., Dowidar, N., Dunaway, D. L., Fell, H. P., Ferree, S., George, R. D., Grogan, T., et al. (2008) · Nature Biotechnology · doi:10.1038/nbt1385 · Cited by 2,066
“NM_014484 MOCS3 206141_at Hs00819330_s1 183 (6) 83 (9) D/D 42 (2) 40 (2) D/U 28.”
InfluentialThe Immunological Genome Project: networks of gene expression in immune cells
The Immunological Genome Project Consortium, Heng, T. S. P., Painter, M. W., Elpek, K., Lukacs-Kornek, V., Mauermann, N., Turley, S. J., Koller, D., Kim, F. S., Wagers, A. J., et al. (2008) · Nature Immunology · doi:10.1038/ni1008-1091 · Cited by 1,864
Improving RNA-Seq expression estimates by correcting for fragment bias
Roberts, A., Trapnell, C., Donaghey, J., Rinn, J. L., & Pachter, L. (2011) · Genome Biology · doi:10.1186/gb-2011-12-3-r22 · Cited by 1,725
“After quanti cation with Cufflinks , the subset of transcripts with 1-to-1 mappings to the TaqMan qRT-PCR probes were selected (as listed in the supplement to [16]) to be used in the correlation tests.”
MethodBullard, J. H., Purdom, E., Hansen, K. D., & Dudoit, S. (2010) · BMC Bioinformatics · doi:10.1186/1471-2105-11-94 · Cited by 1,680
Differential expression in RNA-seq: A matter of depth
Tarazona, S., García-Alcalde, F., Dopazo, J., Ferrer, A., & Conesa, A. (2011) · Genome Research · doi:10.1101/gr.124321.111 · Cited by 1,632
EBSeq: an empirical Bayes hierarchical model for inference in RNA-seq experiments
Leng, N., Dawson, J. A., Thomson, J. A., Ruotti, V., Rissman, A. I., Smits, B. M. G., Haag, J. D., Gould, M. N., Stewart, R. M., & Kendziorski, C. (2013) · Bioinformatics · doi:10.1093/bioinformatics/btt087 · Cited by 1,302
Sultan, M., Schulz, M. H., Richard, H., Magen, A., Klingenhoff, A., Scherf, M., Seifert, M., Borodina, T., Soldatov, A., Parkhomchuk, D., et al. (2008) · Science · doi:10.1126/science.1160342 · Cited by 1,293
Trait-associated SNPs are more likely to be eQTLs: annotation to enhance discovery from GWAS
Nicolae, D. L., Gamazon, E., Zhang, W., Duan, S., Dolan, M. E., & Cox, N. J. (2010) · PLoS Genetics · doi:10.1371/journal.pgen.1000888 · Cited by 1,292
“) that they might be considered unreliable for genomic studies [24], while others highlight the reproducibility of results of transcriptome studies [25,26] and find that LCLs show little variability as a consequence of non-genetic”
ResultDNA Biosensors and Microarrays
Sassolas, A., Leca-Bouvier, B. D., & Blum, L. J. (2008) · Chemical Reviews · doi:10.1021/cr0684467 · Cited by 1,246
Standardizing immunophenotyping for the Human Immunology Project
Maecker, H. T., McCoy, J. P., & Nussenblatt, R. (2012) · Nature Reviews Immunology · doi:10.1038/nri3158 · Cited by 1,176
Measuring reproducibility of high-throughput experiments
Li, Q., Brown, J. B., Huang, H., & Bickel, P. J. (2011) · The Annals of Applied Statistics · doi:10.1214/11-aoas466 · Cited by 1,009
Self-citationSEQC/MAQC-III Consortium (2014) · Nature Biotechnology · doi:10.1038/nbt.2957 · Cited by 983
Self-citationMeasuring reproducibility of high-throughput experiments
Li, Q., Brown, J. B., Huang, H., & Bickel, P. J. (2010) · UC Berkeley · Cited by 915
Self-citationWirapati, P., Sotiriou, C., Kunkel, S., Farmer, P., Pradervand, S., Haibe-Kains, B., Desmedt, C., Ignatiadis, M., Sengstag, T., Schütz, F., et al. (2008) · Breast Cancer Research · doi:10.1186/bcr2124 · Cited by 891
“Hybridization probes were mapped to Entrez GeneID [26] through sequence alignment against RefSeq mRNA in the (NM) subset, similar to the approach of Shi and colleagues [27], using RefSeq version 21 (2007.”
MethodGeeleher, P., Cox, N. J., & Huang, R. S. (2014) · Genome Biology · doi:10.1186/gb-2014-15-3-r47 · Cited by 890
“One of the major concerns is that gene expression estimates, generated on different microarray platforms or even in different batches, are not always consistent [11].”
BackgroundMAQC Consortium (2010) · Nature Biotechnology · doi:10.1038/nbt.1665 · Cited by 889
Self-citationGC-Content Normalization for RNA-Seq Data
Risso, D., Schwartz, K., Sherlock, G., & Dudoit, S. (2011) · BMC Bioinformatics · doi:10.1186/1471-2105-12-480 · Cited by 867
Biological Processes Associated with Breast Cancer Clinical Outcome Depend on the Molecular Subtypes
Desmedt, C., Haibe-Kains, B., Wirapati, P., Buyse, M., Larsimont, D., Bontempi, G., Delorenzi, M., Piccart, M., & Sotiriou, C. (2008) · Clinical Cancer Research · doi:10.1158/1078-0432.ccr-07-4756 · Cited by 828
An Abundance of Ubiquitously Expressed Genes Revealed by Tissue Transcriptome Sequence Data
Ramsköld, D., Wang, E. T., Burge, C. B., & Sandberg, R. (2009) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1000598 · Cited by 820
“2 exons and only one annotated isoform in RefSeq whose expressions had been measured by the MicroArray Quality Control project [50] in the same two samples, UHR (universal human reference) RNA and Figure 4.”
't Hoen, P. A. C., Ariyurek, Y., Thygesen, H. H., Vreugdenhil, E., Vossen, R. H. A. M., de Menezes, R. X., Boer, J. M., van Ommen, G.-J. B., & den Dunnen, J. T. (2008) · Nucleic Acids Research · doi:10.1093/nar/gkn705 · Cited by 742
“In samples with larger differences in gene expression, like the samples analyzed by the MAQC consortium (46), the correlation is likely higher.”
MethodComprehensive evaluation of differential gene expression analysis methods for RNA-seq data
Rapaport, F., Khanin, R., Liang, Y., Pirun, M., Krek, A., Zumbo, P., Mason, C. E., Socci, N. D., & Betel, D. (2013) · Genome Biology · doi:10.1186/gb-2013-14-9-r95 · Cited by 733
“ray technology [11,12] as well as the SEQC effort to characterize RNA-seq technology and include close to 1,000 genes that were validated by TaqMan qPCR.”
MethodGit, A., Dvinge, H., Salmon-Divon, M., Osborne, M., Kutter, C., Hadfield, J., Bertone, P., & Caldas, C. (2010) · RNA · doi:10.1261/rna.1947110 · Cited by 689
“2009), stem cell maintenance, and tissue differentiation (Shi et al. 2006).”
BackgroundRNA-Seq differential expression analysis: An extended review and a software tool
Costa-Silva, J., Domingues, D., & Lopes, F. M. (2017) · PLOS ONE · doi:10.1371/journal.pone.0190152 · Cited by 626
A quantitative atlas of polyadenylation in five mammals
Derti, A., Garrett-Engele, P., MacIsaac, K. D., Stevens, R. C., Sriram, S., Chen, R., Rohl, C. A., Johnson, J. M., & Babak, T. (2012) · Genome Research · doi:10.1101/gr.132563.111 · Cited by 654
“To assess the quantitative potential of PolyA-seq, we assayed the Human Brain Reference and Universal Human Reference (UHR) MAQC samples (Shi et al. 2006) in technical replicates.”
MethodInfluentialA Modular Analysis Framework for Blood Genomics Studies: Application to Systemic Lupus Erythematosus
Chaussabel, D., Quinn, C., Shen, J., Patel, P., Glaser, C., Baldwin, N., Stichweh, D., Blankenship, D., Li, L., Munagala, I., et al. (2008) · Immunity · doi:10.1016/j.immuni.2008.05.012 · Cited by 649
“…results of microarray analyses are prone to include noise (i.e., false-positive results—see Ioannidis, 2005) and do not compare well between laboratories and/or platforms (Bammler et al., 2005; Hyatt et al., 2006; Irizarry et al., 2005; Jarvinen et al., 2004; Larkin et al., 2005; Shi et al., 2006).”
BackgroundResultInfluentialAntidepressant Actions of Histone Deacetylase Inhibitors
Covington, H. E., Maze, I., LaPlant, Q. C., Vialou, V. F., Ohnishi, Y. N., Berton, O., Fass, D. M., Renthal, W., Rush, A. J., Wu, E. Y., et al. (2009) · The Journal of Neuroscience · doi:10.1523/jneurosci.1758-09.2009 · Cited by 624
“Third, the data analysis criteria used for our study are recommended by the MicroArray Quality Control (MAQC) project because these criteria have been validated to have the highest intersite reproducibility and intraplatform reproducibility (Guo et al., 2006; Shi et al., 2006).”
MethodEvaluation of DNA microarray results with quantitative gene expression platforms
Canales, R. D., Luo, Y., Willey, J. C., Austermiller, B., Barbacioru, C. C., Boysen, C., Hunkapiller, K., Jensen, R. V., Knight, C. R., Lee, K. Y., et al. (2006) · Nature Biotechnology · doi:10.1038/nbt1236 · Cited by 615
Self-citationRemoving technical variability in RNA-seq data using conditional quantile normalization
Hansen, K. D., Irizarry, R. A., & WU, Z. (2012) · Biostatistics · doi:10.1093/biostatistics/kxr054 · Cited by 607
“The same samples have been assayed extensively on microarrays, and we used data from the MAQC Consortium (2006), in which each of the 2 samples was hybridized to 5 different Affymetrix U133 Plus 2.0 arrays (Affymetrix, Santa Clara, CA).”
MethodInfluentialAlternative (non-animal) methods for cosmetics testing: current status and future prospects—2010
Adler, S., Basketter, D., Creton, S., Pelkonen, O., van Benthem, J., Zuang, V., Andersen, K. E., Angers-Loustau, A., Aptula, A., Bal-Price, A., et al. (2011) · Archives of Toxicology · doi:10.1007/s00204-011-0693-2 · Cited by 584
Patro, R., Mount, S. M., & Kingsford, C. (2014) · Nature Biotechnology · doi:10.1038/nbt.2862 · Cited by 586
Xia, Z., Donehower, L. A., Cooper, T. A., Neilson, J. R., Wheeler, D. A., Wagner, E. J., & Li, W. (2014) · Nature Communications · doi:10.1038/ncomms6274 · Cited by 576
Molecular Genetics of Adult Acute Myeloid Leukemia: Prognostic and Therapeutic Implications
Marcucci, G., Haferlach, T., & Döhner, H. (2011) · Journal of Clinical Oncology · doi:10.1200/jco.2010.30.2554 · Cited by 570
Microarrays, deep sequencing and the true measure of the transcriptome
Malone, J. H., & Oliver, B. (2011) · BMC Biology · doi:10.1186/1741-7007-9-34 · Cited by 548
Population Genomics of Early Events in the Ecological Differentiation of Bacteria
Shapiro, B. J., Friedman, J., Cordero, O. X., Preheim, S. P., Timberlake, S. C., Szabó, G., Polz, M. F., & Alm, E. J. (2012) · Science · doi:10.1126/science.1218198 · Cited by 562
Enabling personalized cancer medicine through analysis of gene-expression patterns
van 't Veer, L. J., & Bernards, R. (2008) · Nature · doi:10.1038/nature06915 · Cited by 554
“A second impediment is that DNA-microarray technology was initially not very robust (at least around the year 2000) and, to many scientists and doctors, it still has a poor reputation, which has been unfounded since industry became involved in productio”
BackgroundRepeatability of published microarray gene expression analyses
Ioannidis, J. P. A., Allison, D. B., Ball, C. A., Coulibaly, I., Cui, X., Culhane, A. C., Falchi, M., Furlanello, C., Game, L., Jurman, G., et al. (2009) · Nature Genetics · doi:10.1038/ng.295 · Cited by 551
Guidelines for reporting an fMRI study
Poldrack, R. A., Fletcher, P. C., Henson, R. N., Worsley, K. J., Brett, M., & Nichols, T. E. (2008) · NeuroImage · doi:10.1016/j.neuroimage.2007.11.048 · Cited by 550
Key Issues in Conducting a Meta-Analysis of Gene Expression Microarray Datasets
Ramasamy, A., Mondry, A., Holmes, C. C., & Altman, D. G. (2008) · PLoS Medicine · doi:10.1371/journal.pmed.0050184 · Cited by 536
“Recently, the MicroArray Quality Control (MAQC) project [”
BackgroundMolecular markers of early Parkinson's disease based on gene expression in blood
Scherzer, C. R., Eklund, A. C., Morse, L. J., Liao, Z., Locascio, J. J., Fefer, D., Schwarzschild, M. A., Schlossmacher, M. G., Hauser, M. A., Vance, J. M., et al. (2007) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0610204104 · Cited by 512
Self-citationZhao, Y., Li, M.-C., Konaté, M. M., Chen, L., Das, B., Karlovich, C., Williams, P. M., Evrard, Y. A., Doroshow, J. H., & McShane, L. M. (2021) · Journal of Translational Medicine · doi:10.1186/s12967-021-02936-w · Cited by 489
Wang, C., Gong, B., Bushel, P. R., Thierry-Mieg, J., Thierry-Mieg, D., Xu, J., Fang, H., Hong, H., Shen, J., Su, Z., et al. (2014) · Nature Biotechnology · doi:10.1038/nbt.3001 · Cited by 512
Self-citationPatterson, T. A., Lobenhofer, E. K., Fulmer-Smentek, S. B., Collins, P. J., Chu, T.-M., Bao, W., Fang, H., Kawasaki, E. S., Hager, J., Tikhonova, I. R., et al. (2006) · Nature Biotechnology · doi:10.1038/nbt1242 · Cited by 492
Self-citationSotiriou, C., & Piccart, M. J. (2007) · Nature Reviews Cancer · doi:10.1038/nrc2173 · Cited by 461
“Microarrays: ready for the clinic? The initial enthusiasm for the application of microarray technology was tempered by the publication of several studies reporting contradictory results on the analysis of the same RNA samples hybridized on different microarray platform”
ResultUsing control genes to correct for unwanted variation in microarray data
Gagnon-Bartsch, J. A., & Speed, T. P. (2012) · Biostatistics · doi:10.1093/biostatistics/kxr034 · Cited by 442
“Section A in the supplementary material 1 (SM) provides a brief example from the MAQC study (Shi et al., 2006) of substantial within-batch unwanted variation.”
BackgroundInfluentialBenchmarking of RNA-sequencing analysis workflows using whole-transcriptome RT-qPCR expression data
Everaert, C., Luypaert, M., Maag, J. L. V., Cheng, Q. X., Dinger, M. E., Hellemans, J., & Mestdagh, P. (2017) · Scientific Reports · doi:10.1038/s41598-017-01617-3 · Cited by 393
Rat toxicogenomic study reveals analytical consistency across microarray platforms
Guo, L., Lobenhofer, E. K., Wang, C., Shippy, R., Harris, S. C., Zhang, L., Mei, N., Chen, T., Herman, D., Goodsaid, F. M., et al. (2006) · Nature Biotechnology · doi:10.1038/nbt1238 · Cited by 406
Self-citationShi, L., Jones, W. D., Jensen, R. V., Harris, S. C., Perkins, R. G., Goodsaid, F. M., Guo, L., Croner, L. J., Boysen, C., Fang, H., et al. (2008) · BMC Bioinformatics · doi:10.1186/1471-2105-9-s9-s10 · Cited by 401
“For more information about the MAQC project and the data sets, refer to Shi et al [13].”
BackgroundMethodInfluentialSelf-citationToxicity testing in the 21st century: progress in the past decade and future perspectives
Krewski, D., Andersen, M. E., Tyshenko, M. G., Krishnan, K., Hartung, T., Boekelheide, K., Wambaugh, J. F., Jones, D., Whelan, M., Thomas, R., et al. (2020) · Archives of Toxicology · doi:10.1007/s00204-019-02613-4 · Cited by 383
A rat RNA-Seq transcriptomic BodyMap across 11 organs and 4 developmental stages
Yu, Y., Fuscoe, J. C., Zhao, C., Guo, C., Jia, M., Qing, T., Bannon, D. I., Lancashire, L., Bao, W., Du, T., et al. (2014) · Nature Communications · doi:10.1038/ncomms4230 · Cited by 395
Self-citationComparison of RNA-seq and microarray-based models for clinical endpoint prediction
Zhang, W., Yu, Y., Hertwig, F., Thierry-Mieg, J., Zhang, W., Thierry-Mieg, D., Wang, J., Furlanello, C., Devanarayan, V., Cheng, J., et al. (2015) · Genome Biology · doi:10.1186/s13059-015-0694-1 · Cited by 393
“%) of the DEGs identified by microarrays using the MAQC-I and Magic methods, respectively (Additional file 1: Figure S5).”
MethodInfluentialSelf-citationA multigene predictor of outcome in glioblastoma
Colman, H., Zhang, L., Sulman, E. P., McDonald, J. M., Shooshtari, N. L., Rivera, A., Popoff, S., Nutt, C. L., Louis, D. N., Cairncross, J. G., et al. (2010) · Neuro-Oncology · doi:10.1093/neuonc/nop007 · Cited by 386
“These findings are consistent with results from the Microarray Quality Control (MAQC) Project, which also found that gene rankings based on P-values or SAM did not generate more reproducible results compared to fold-change.(18) Importantly, an examination of each consensus gene identified by the fold-change method usi…”
BackgroundResultSelf-citationMacromolecule biosynthesis: a key function of sleep
Mackiewicz, M., Shockley, K. R., Romer, M. A., Galante, R. J., Zimmerman, J. E., Naidoo, N., Baldwin, D. A., Jensen, S. T., Churchill, G. A., & Pack, A. I. (2007) · Physiological Genomics · doi:10.1152/physiolgenomics.00275.2006 · Cited by 377
“20) are not supported by our data, a conclusion that is in line with recent studies on the topic of reliability of microarray results (26, 70).”
ResultCameron, M. J., Ran, L., Xu, L., Danesh, A., Bermejo-Martin, J. F., Cameron, C. M., Muller, M. P., Gold, W. L., Richardson, S. E., Poutanen, S. M., et al. (2007) · Journal of Virology · doi:10.1128/jvi.00527-07 · Cited by 386
A Systems Biology View of Responses to Lignin Biosynthesis Perturbations in Arabidopsis
Vanholme, R., Storme, V., Vanholme, B., Sundin, L., Christensen, J. H., Goeminne, G., Halpin, C., Rohde, A., Morreel, K., & Boerjan, W. (2012) · The Plant Cell · doi:10.1105/tpc.112.102574 · Cited by 365
A Comparative Study of Techniques for Differential Expression Analysis on RNA-Seq Data
Zhang, Z. H., Jhaveri, D. J., Marshall, V. M., Bauer, D. C., Edson, J., Narayanan, R. K., Robinson, G. J., Lundberg, A. E., Bartlett, P. F., Wray, N. R., et al. (2014) · PLoS ONE · doi:10.1371/journal.pone.0103207 · Cited by 360
“The MAQC dataset: MicroArray Quality Control Project (MAQC) data [31] includes biological samples for human brain reference RNA (hbr) and universal human reference RNA (uhr).”
MethodResistance Exercise Reverses Aging in Human Skeletal Muscle
Melov, S., Tarnopolsky, M. A., Beckman, K., Felkey, K., & Hubbard, A. (2007) · PLoS ONE · doi:10.1371/journal.pone.0000465 · Cited by 347
“Recent studies by the MAQC (microarray quality control) project, show substantial agreement between illumina microarray data and real-time PCR data on the same samples[49–51].”
BackgroundNext-generation sequencing in the clinic: Promises and challenges
Xuan, J., Yu, Y., Qing, T., Guo, L., & Shi, L. (2013) · Cancer Letters · doi:10.1016/j.canlet.2012.11.025 · Cited by 344
Self-citationA global map of human gene expression
Lukk, M., Kapushesky, M., Nikkilä, J., Parkinson, H., Goncalves, A., Huber, W., Ukkonen, E., & Brazma, A. (2010) · Nature Biotechnology · doi:10.1038/nbt0410-322 · Cited by 350
Pivotal Role of Innate and Adaptive Immunity in Anthracycline Chemotherapy of Established Tumors
Mattarollo, S. R., Loi, S., Duret, H., Ma, Y., Zitvogel, L., & Smyth, M. J. (2011) · Cancer Research · doi:10.1158/0008-5472.can-11-0753 · Cited by 333
“, (23) using RefSeq and Entrez database version 2007.”
MethodSelf-citationFold change and p-value cutoffs significantly alter microarray interpretations
Dalman, M. R., Deeter, A., Nimishakavi, G., & Duan, Z.-H. (2012) · BMC Bioinformatics · doi:10.1186/1471-2105-13-s2-s11 · Cited by 332
“Array technology has improved these conditions yet problems of standardizing statistical analyses are lacking, along with observed differences when comparing microarray platforms [2], though others have found significant reproducibility [3].”
BackgroundTranscriptomic and metabolomic data integration
Cavill, R., Jennen, D., Kleinjans, J., & Briedé, J. J. (2016) · Briefings in Bioinformatics · doi:10.1093/bib/bbv090 · Cited by 330
Distinctive MicroRNA Profiles Relating to Patient Survival in Esophageal Squamous Cell Carcinoma
Guo, Y., Chen, Z., Zhang, L., Zhou, F., Shi, S., Feng, X., Li, B., Meng, X., Ma, X., Luo, M., et al. (2008) · Cancer Research · doi:10.1158/0008-5472.can-06-4418 · Cited by 339
“Several previous in depth comparative studies between microarray platforms and analysis procedures have indicated the very high reproducibility, sensitivity, and specificity of similar expression microarrays using their recommended procedures (38, 39).”
MethodSelf-citationOuyang, Z., Zhou, Q., & Wong, W. H. (2009) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0904863106 · Cited by 339
“Considering that the ChIP-Seq data do not directly measure transcript abundance, this is a surprisingly high correlation which is comparable to those observed between measurements made on the same samples by different types of expression arrays (23, 24).”
ResultA Three-Gene Model to Robustly Identify Breast Cancer Molecular Subtypes
Haibe-Kains, B., Desmedt, C., Loi, S., Culhane, A. C., Bontempi, G., Quackenbush, J., & Sotiriou, C. (2012) · JNCI: Journal of the National Cancer Institute · doi:10.1093/jnci/djr545 · Cited by 313
Molecular Classification and Novel Targets in Hepatocellular Carcinoma: Recent Advancements
Hoshida, Y., Toffanin, S., Lachenmayer, A., Villanueva, A., Minguez, B., & Llovet, J. (2010) · Seminars in Liver Disease · doi:10.1055/s-0030-1247131 · Cited by 310
Cumulus and granulosa cell markers of oocyte and embryo quality
Uyar, A., Torrealday, S., & Seli, E. (2013) · Fertility and Sterility · doi:10.1016/j.fertnstert.2013.01.129 · Cited by 308
Suárez-Fariñas, M., Li, K., Fuentes-Duculan, J., Hayden, K., Brodmerkel, C., & Krueger, J. G. (2012) · Journal of Investigative Dermatology · doi:10.1038/jid.2012.184 · Cited by 305
Molecular biomarkers in acute myeloid leukemia
Prada-Arismendy, J., Arroyave, J. C., & Röthlisberger, S. (2017) · Blood Reviews · doi:10.1016/j.blre.2016.08.005 · Cited by 304
Vassena, R., Boué, S., González-Roca, E., Aran, B., Auer, H., Veiga, A., & Belmonte, J. C. I. (2011) · Development · doi:10.1242/dev.064741 · Cited by 300
Human Disease-Drug Network Based on Genomic Expression Profiles
Hu, G., & Agarwal, P. (2009) · PLoS ONE · doi:10.1371/journal.pone.0006536 · Cited by 310
“05) were more reliable (and more consistent with the results from similar studies) than those simply based on P value or fold change alone [23].”
ResultHewitt, S. M., Lewis, F. A., Cao, Y., Conrad, R. C., Cronin, M., Danenberg, K. D., Goralski, T. J., Langmore, J. P., Raja, R. G., Williams, P. M., et al. (2008) · Archives of Pathology & Laboratory Medicine · doi:10.5858/132.12.1929 · Cited by 281
Self-citationMetabolic adaptation establishes disease tolerance to sepsis
Weis, S., Carlos, A. R., Moita, M. R., Singh, S., Blankenhaus, B., Cardoso, S., Larsen, R., Rebelo, S., Schäuble, S., Del Barrio, L., et al. (2017) · Cell · doi:10.1016/j.cell.2017.05.031 · Cited by 284
The microbiome quality control project: baseline study design and future directions
Sinha, R., Abnet, C. C., White, O., Knight, R., & Huttenhower, C. (2015) · Genome Biology · doi:10.1186/s13059-015-0841-8 · Cited by 265
“The project was inspired by earlier work in transcriptomics, for example, the Microarray Quality Control project (MAQC) [3], and we hope to improve the state-of-thescience in microbial community sample collection, DNA”
MethodInfluentialPrognostic and Predictive Molecular Biomarkers for Colorectal Cancer: Updates and Challenges
Koncina, E., Haan, S., Rauh, S., & Letellier, E. (2020) · Cancers · doi:10.3390/cancers12020319 · Cited by 259
“The disparity, caused by the use of different probes and labels between the various platforms, was addressed as an important issue at the FDA’s Microarray Quality Control workshop [145].”
BackgroundSex-specific and lineage-specific alternative splicing in primates
Blekhman, R., Marioni, J. C., Zumbo, P., Stephens, M., & Gilad, Y. (2010) · Genome Research · doi:10.1101/gr.099226.109 · Cited by 267
“As typical correlations between different array platforms in controlled experiments, using the same RNA samples, are ;0.70 (Shi et al. 2006), the level of consistency across studies in this case is satisfying (for Figure 2.”
BackgroundMethodInfluentialLee, E.-S., Son, D.-S., Kim, S.-H., Lee, J., Jo, J., Han, J., Kim, H., Lee, H. J., Choi, H. Y., Jung, Y., et al. (2008) · Clinical Cancer Research · doi:10.1158/1078-0432.ccr-07-4937 · Cited by 265
Self-citationMetabolic regulation in the lactating mammary gland: a lipid synthesizing machine
Rudolph, M. C., McManaman, J. L., Phang, T., Russell, T., Kominsky, D. J., Serkova, N. J., Stein, T., Anderson, S. M., & Neville, M. C. (2007) · Physiological Genomics · doi:10.1152/physiolgenomics.00020.2006 · Cited by 257
Commercialized Multigene Predictors of Clinical Outcome for Breast Cancer
Ross, J. S., Hatzis, C., Symmans, W. F., Pusztai, L., & Hortobágyi, G. N. (2008) · The Oncologist · doi:10.1634/theoncologist.2007-0248 · Cited by 259
Self-citationMegason, S. G., & Fraser, S. E. (2007) · Cell · doi:10.1016/j.cell.2007.08.031 · Cited by 254
Barbosa-Morais, N. L., Dunning, M. J., Samarajiwa, S. A., Darot, J. F. J., Ritchie, M. E., Lynch, A. G., & Tavaré, S. (2010) · Nucleic Acids Research · doi:10.1093/nar/gkp942 · Cited by 257
“The MAQC project (43,44) selected two human RNA samples for the comparison of differential expression on microarray platforms.”
BackgroundReproducibility of high-throughput mRNA and small RNA sequencing across laboratories
The GEUVADIS Consortium, 't Hoen, P. A. C., Friedländer, M. R., Almlöf, J., Sammeth, M., Pulyakhina, I., Anvar, S. Y., Laros, J. F. J., Buermans, H. P. J., Karlberg, O., et al. (2013) · Nature Biotechnology · doi:10.1038/nbt.2702 · Cited by 250
Criteria for the use of omics-based predictors in clinical trials
McShane, L. M., Cavenagh, M. M., Lively, T. G., Eberhard, D. A., Bigbee, W. L., Williams, P. M., Mesirov, J. P., Polley, M.-Y. C., Kim, K. Y., Tricoli, J. V., et al. (2013) · Nature · doi:10.1038/nature12564 · Cited by 241
A new shrinkage estimator for dispersion improves differential expression detection in RNA-seq data
Wu, H., Wang, C., & Wu, Z. (2013) · Biostatistics · doi:10.1093/biostatistics/kxs033 · Cited by 242
Self-citationAssessing and mitigating batch effects in large-scale omics studies
Yu, Y., Mai, Y., Zheng, Y., & Shi, L. (2024) · Genome Biology · doi:10.1186/s13059-024-03401-9 · Cited by 202
“Specifically, ratio-based expression profiles, defined as a fold-change or a ratio of expression levels between two sample groups for the same gene, agreed well across multiple transcriptomic technologies, including RNA-seq, microarray, and qPCR [73, 146].”
BackgroundMethodSelf-citationLi, S., Tighe, S. W., Nicolet, C. M., Grove, D., Levy, S., Farmerie, W., Viale, A., Wright, C., Schweitzer, P. A., Gao, Y., et al. (2014) · Nature Biotechnology · doi:10.1038/nbt.2972 · Cited by 240
Self-citationRobust global micro-RNA profiling with formalin-fixed paraffin-embedded breast cancer tissues
Hui, A. B. Y., Shi, W., Boutros, P. C., Miller, N., Pintilie, M., Fyles, T., McCready, D., Wong, D., Gerster, K., Jurisica, I., et al. (2009) · Laboratory Investigation · doi:10.1038/labinvest.2009.12 · Cited by 232
“There have been extensive evaluations conducted on the inter- and intra-experimental replicabilities for mRNA expression profiling platforms”
BackgroundKoedrith, P., Kim, H., Weon, J.-I., & Seo, Y. R. (2013) · International Journal of Hygiene and Environmental Health · doi:10.1016/j.ijheh.2013.02.010 · Cited by 225
Gene expression profiling in human neurodegenerative disease
Cooper-Knock, J., Kirby, J., Ferraiuolo, L., Heath, P. R., Rattray, M., & Shaw, P. J. (2012) · Nature Reviews Neurology · doi:10.1038/nrneurol.2012.156 · Cited by 223
Sasaki-Sekimoto, Y., Jikumaru, Y., Obayashi, T., Saito, H., Masuda, S., Kamiya, Y., Ohta, H., & Shirasu, K. (2013) · Plant Physiology · doi:10.1104/pp.113.220129 · Cited by 220
“A previous report demonstrated that a straightforward approach of fold change ranking plus a nonstringent P cutoff is effective to identify reproducible gene lists (Shi et al., 2006).”
BackgroundSequencing technology does not eliminate biological variability
Hansen, K. D., Wu, Z., Irizarry, R. A., & Leek, J. T. (2011) · Nature Biotechnology · doi:10.1038/nbt.1910 · Cited by 226
“Well-known sources of technical variability in both sequencing and microarray studies are laboratory 10, 11 and batch 12 effects.”
Optimizing the noise versus bias trade-off for Illumina Whole Genome Expression Beadchips
Shi, W., Oshlack, A., & Smyth, G. K. (2010) · Nucleic Acids Research · doi:10.1093/nar/gkq871 · Cited by 225
Macrophage Responses to Silica Nanoparticles are Highly Conserved Across Particle Sizes
Waters, K. M., Masiello, L. M., Zangar, R. C., Tarasevich, B. J., Karin, N. J., Quesenberry, R. D., Bandyopadhyay, S., Teeguarden, J. G., Pounds, J. G., & Thrall, B. D. (2009) · Toxicological Sciences · doi:10.1093/toxsci/kfn250 · Cited by 221
“Biological interpretation of genome-level data can significantly depend on the arbitrary statistical thresholds for gene expression that are set (Shi et al., 2006).”
Metadata
- Authors
- MAQC Consortium, Shi, L., Shi, L., Reid, L. H., Jones, W. D., Shippy, R., Warrington, J. A., Baker, S. C., Collins, P. J., de Longueville, F., Kawasaki, E. S., Lee, K. Y., Luo, Y., Sun, Y. A., Willey, J. C., Setterquist, R. A., Fischer, G. M., Tong, W., Dragan, Y. P., Dix, D. J., Frueh, F. W., Goodsaid, F. M., Herman, D., Jensen, R. V., Johnson, C. D., Lobenhofer, E. K., Puri, R. K., Scherf, U., Thierry-Mieg, J., Wang, C., Wilson, M., Wolber, P. K., Zhang, L., Amur, S., Bao, W., Barbacioru, C. C., Lucas, A. B., Bertholet, V., Boysen, C., Bromley, B., Brown, D., Brunner, A., Canales, R., Cao, X. M., Cebula, T. A., Chen, J. J., Cheng, J., Chu, T.-M., Chudin, E., Corson, J., Corton, J. C., Croner, L. J., Davies, C., Davison, T. S., Delenstarr, G., Deng, X., Dorris, D., Eklund, A. C., Fan, X.-H., Fang, H., Fulmer-Smentek, S., Fuscoe, J. C., Gallagher, K., Ge, W., Guo, L., Guo, X., Hager, J., Haje, P. K., Han, J., Han, T., Harbottle, H. C., Harris, S. C., Hatchwell, E., Hauser, C. A., Hester, S., Hong, H., Hurban, P., Jackson, S. A., Ji, H., Knight, C. R., Kuo, W. P., LeClerc, J. E., Levy, S., Li, Q.-Z., Liu, C., Liu, Y., Lombardi, M. J., Ma, Y., Magnuson, S. R., Maqsodi, B., McDaniel, T., Mei, N., Myklebost, O., Ning, B., Novoradovskaya, N., Orr, M. S., Osborn, T. W., Papallo, A., Patterson, T. A., Perkins, R. G., Peters, E. H., Peterson, R., Philips, K. L., Pine, P. S., Pusztai, L., Qian, F., Ren, H., Rosen, M., Rosenzweig, B. A., Samaha, R. R., Schena, M., Schroth, G. P., Shchegrova, S., Smith, D. D., Staedtler, F., Su, Z., Sun, H., Szallasi, Z., Tezak, Z., Thierry-Mieg, D., Thompson, K. L., Tikhonova, I., Turpaz, Y., Vallanat, B., Van, C., Walker, S. J., Wang, S. J., Wang, Y., Wolfinger, R., Wong, A., Wu, J., Xiao, C., Xie, Q., Xu, J., Yang, W., Zhang, L., Zhong, S., Zong, Y., & Slikker, W.
- Affiliations
- Affymetrix (United States) (US)
- Illumina (United States) (US)
- Agilent Technologies (United States) (US)
- National Cancer Institute (US)
- National Center for Toxicological Research (US)
- Environmental Protection Agency (US)
- United States Food and Drug Administration (US)
- National Institutes of Health (US)
- University of Massachusetts Boston (US)
- Asuragen (United States) (US)
- Center for Biologics Evaluation and Research (US)
- Cedars-Sinai Medical Center (US)
- SAS Institute (United States) (US)
- Center for Food Safety and Applied Nutrition (US)
- Biogen (United States) (US)
- Yale University (US)
- Center for Veterinary Medicine (US)
- Cold Spring Harbor Laboratory (US)
- Sanford Burnham Prebys Medical Discovery Institute (US)
- Stanford Medicine (US)
- Harvard University (US)
- Vanderbilt University (US)
- The University of Texas Southwestern Medical Center (US)
- The University of Texas at Dallas (US)
- GenUs BioSystems (United States) (US)
- Oslo University Hospital (NO)
- The University of Texas MD Anderson Cancer Center (US)
- Luminex (United States) (US)
- Wake Forest University Health Sciences (US)
- University of Illinois Urbana-Champaign (US)
- ORCID
- Jones, Wendell D
- Collins, Patrick J
- Luo, Yuling
- Sun, Yongming Andrew
- Willey, James C
- Tong, Weida
- Frueh, Felix W
- Goodsaid, Federico M
- Johnson, Charles D
- Puri, Raj K
- Scherf, Uwe
- Wang, Charles
- Bao, Wenjun
- Brown, Donna
- Chen, James J
- Cheng, Jing
- Corton, J Christopher
- Croner, Lisa J
- Deng, Xutao
- Eklund, Aron C
- Fang, Hong
- Guo, Lei
- Guo, Xu
- Han, Jing
- Han, Tao
- Harbottle, Heather C
- Harris, Stephen C
- Hatchwell, Eli
- Hester, Susan
- Hong, Huixiao
- Jackson, Scott A
- Ji, Hanlee
- Levy, Shawn
- Li, Quan-Zhen
- Liu, Chunmei
- Liu, Ying
- Ma, Yunqing
- Mei, Nan
- Myklebost, Ola
- Ning, Baitang
- Patterson, Tucker A
- Pusztai, Lajos
- Qian, Feng
- Schroth, Gary P
- Su, Zhenqiang
- Sun, Hongmei
- Tikhonova, Irina
- Turpaz, Yaron
- Walker, Stephen J
- Wang, Sue Jane
- Wang, Yonghong
- Wong, Alex
- Wu, Jie
- Xiao, Chunlin
- Xie, Qian
- Xu, Jun
- Yang, Wen
- Zhang, Liang
- Zhong, Sheng
- Slikker, William
- Venue
- Nature Biotechnology
- Published
- 2006-09-01
- Volume
- 24
- Issue
- 9
- Pages
- 1151-1161
- ISSN
- 0733-222X (ISSN-L)
- 1087-0156
- 1546-1696
- PMC
- PMC3272078
- OpenAlex
- W2010637767
- Cited by
- 2,126
- References
- 43
- Citations by year
- 2026: 21 · 2025: 25 · 2024: 23 · 2023: 39 · 2022: 37 · 2021: 60 · 2020: 43 · 2019: 59 · 2018: 68 · 2017: 87 · 2016: 87 · 2015: 98 · 2014: 122 · 2013: 155 · 2012: 160
- FWCI
- 87.96
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Licence
- www.springer.com/tdm (tdm)
- www.springer.com/tdm (vor)
- Locations
- Nature Biotechnology (publishedVersion)
- PubMed (publishedVersion)
- Europe PMC (PubMed Central) (acceptedVersion)
- PubMed Central (acceptedVersion)
- Digital Access to Libraries (Université catholique de Louvain (UCL), l'Université de Namur (UNamur) and the Université Saint-Louis (USL-B)) (submittedVersion)
- Funders
- U.S. Environmental Protection Agency
- National Institutes of Health (Z99 LM999999, Z99 HL999999, Z99 MH999999, Z99CL999999)
- Topics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Gene expression and cancer classification
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Molecular Biology Techniques and Applications
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Genomics and Phylogenetic Studies
- Keywords
- probe mapping, differentially expressed genes, microarray data analysis
- MeSH terms
- Equipment Design, Quality Assurance, Health Care, Quality Control, Sensitivity and Specificity, United States, Reproducibility of Results, Equipment Failure Analysis, Oligonucleotide Array Sequence Analysis, Gene Expression Profiling
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.