The DOI matches its Crossref record; 34 of the 34 references it cites were found in a registry or an index.
Ontological analysis of gene expression data: current tools, limitations, and open problems
Open accessAbstract
Summary: Independent of the platform and the analysis methods used, the result of a microarray experiment is, in most cases, a list of differentially expressed genes. An automatic ontological analysis approach has been recently proposed to help with the biological interpretation of such results. Currently, this approach is the de facto standard for the secondary analysis of high throughput experiments and a large number of tools have been developed for this purpose. We present a detailed comparison of 14 such tools using the following criteria: scope of the analysis, visualization capabilities, statistical model(s) used, correction for multiple comparisons, reference microarrays available, installation issues and sources of annotation data. This detailed analysis of the capabilities of these tools will help researchers choose the most appropriate tool for a given type of analysis. More importantly, in spite of the fact that this type of analysis has been generally adopted, this approach has several important intrinsic drawbacks. These drawbacks are associated with all tools discussed and represent conceptual limitations of the current state-of-the-art in ontological analysis. We propose these as challenges for the next generation of secondary data analysis tools. Contact: sod@cs.wayne.edu
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing
Benjamini, Y., & Hochberg, Y. (1995) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1995.tb02031.x · Cited by 109,482
Cytoscape: A software Environment for integrated models of biomolecular interaction networks
Shannon, P., Markiel, A., Ozier, O., Baliga, N. S., Wang, J. T., Ramage, D., Amin, N., Schwikowski, B., & Ideker, T. (2003) · Genome Research · doi:10.1101/gr.1239303 · Cited by 51,888
A Simple Sequentially Rejective Multiple Test Procedure
(1979) · Scandinavian Journal of Statistics · doi:10.2307/4615733 · Cited by 21,526
The control of the false discovery rate in multiple testing under dependency
Benjamini, Y., & Yekutieli, D. (2001) · The Annals of Statistics · doi:10.1214/aos/1013699998 · Cited by 10,490
Multiple Comparison Procedures
Hochberg, Y., & Tamhane, A. C. (1987) · Wiley Series in Probability and Statistics · doi:10.1002/9780470316672 · Cited by 2,389
The Database of Interacting Proteins: 2004 update
Salwinski, L. (2004) · Nucleic Acids Research · doi:10.1093/nar/gkh086 · Cited by 2,271
Multiple Comparison Procedures
(2013) · An Introduction to Statistical Concepts · doi:10.4324/9780203137819-17 · Cited by 2,109
GOstat: find statistically overrepresented Gene Ontologies within a group of genes
Beißbarth, T., & Speed, T. P. (2004) · Bioinformatics · doi:10.1093/bioinformatics/bth088 · Cited by 1,257
GoMiner: a resource for biological interpretation of genomic and proteomic data
Zeeberg, B. R., Feng, W., Wang, G., Wang, M. D., Fojo, A. T., Sunshine, M., Narasimhan, S., Kane, D. W., Reinhold, W. C., Lababidi, S., et al. (2003) · Genome Biology · doi:10.1186/gb-2003-4-4-r28 · Cited by 1,198
FatiGO: a web tool for finding significant associations of Gene Ontology terms with groups of genes
Al-Shahrour, F., Díaz-Uriarte, R., & Dopazo, J. (2004) · Bioinformatics · doi:10.1093/bioinformatics/btg455 · Cited by 1,122
Cited by
Systematic and integrative analysis of large gene lists using DAVID bioinformatics resources
Huang, D. W., Sherman, B. T., & Lempicki, R. A. (2009) · Nature Protocols · doi:10.1038/nprot.2008.211 · Cited by 35,890
GSVA: gene set variation analysis for microarray and RNA-Seq data
Hänzelmann, S., Castelo, R., & Guinney, J. (2013) · BMC Bioinformatics · doi:10.1186/1471-2105-14-7 · Cited by 15,440
Huang, D. W., Sherman, B. T., & Lempicki, R. A. (2009) · Nucleic Acids Research · doi:10.1093/nar/gkn923 · Cited by 14,108
Metascape provides a biologist-oriented resource for the analysis of systems-level datasets
Zhou, Y., Zhou, B., Pache, L., Chang, M., Khodabakhshi, A. H., Tanaseichuk, O., Benner, C., & Chanda, S. K. (2019) · Nature Communications · doi:10.1038/s41467-019-09234-6 · Cited by 14,611
GREAT improves functional interpretation of cis-regulatory regions
McLean, C. Y., Bristor, D., Hiller, M., Clarke, S. L., Schaar, B. T., Lowe, C. B., Wenger, A. M., & Bejerano, G. (2010) · Nature Biotechnology · doi:10.1038/nbt.1630 · Cited by 4,502
High-throughput functional annotation and data mining with the Blast2GO suite
Gotz, S., Garcia-Gomez, J. M., Terol, J., Williams, T. D., Nagaraj, S. H., Nueda, M. J., Robles, M., Talon, M., Dopazo, J., & Conesa, A. (2008) · Nucleic Acids Research · doi:10.1093/nar/gkn176 · Cited by 4,423
GOrilla: a tool for discovery and visualization of enriched GO terms in ranked gene lists
Eden, E., Navon, R., Steinfeld, I., Lipson, D., & Yakhini, Z. (2009) · BMC Bioinformatics · doi:10.1186/1471-2105-10-48 · Cited by 3,356
Huang, D. W., Sherman, B. T., Tan, Q., Collins, J. R., Alvord, W. G., Roayaei, J., Stephens, R., Baseler, M. W., Lane, H. C., & Lempicki, R. A. (2007) · Genome Biology · doi:10.1186/gb-2007-8-9-r183 · Cited by 2,479
Enrichment Map: A Network-Based Method for Gene-Set Enrichment Visualization and Interpretation
Merico, D., Isserlin, R., Stueker, O., Emili, A., & Bader, G. D. (2010) · PLoS ONE · doi:10.1371/journal.pone.0013984 · Cited by 2,235
“Over 60 enrichment analysis methods and tools have been developed in the last few years [5,9,10].”
MethodHuang, D. W., Sherman, B. T., Tan, Q., Kir, J., Liu, D., Bryant, D., Guo, Y., Stephens, R., Baseler, M. W., Lane, H. C., et al. (2007) · Nucleic Acids Research · doi:10.1093/nar/gkm415 · Cited by 2,213
Improved scoring of functional groups from gene expression data by decorrelating GO graph structure
Alexa, A., Rahnenführer, J., & Lengauer, T. (2006) · Bioinformatics · doi:10.1093/bioinformatics/btl140 · Cited by 2,188
STEM: a tool for the analysis of short time series gene expression data
Ernst, J., & Bar-Joseph, Z. (2006) · BMC Bioinformatics · doi:10.1186/1471-2105-7-191 · Cited by 1,703
Ten Years of Pathway Analysis: Current Approaches and Outstanding Challenges
Khatri, P., Sirota, M., & Butte, A. J. (2012) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1002375 · Cited by 1,566
Self-citationg:Profiler—a web-based toolset for functional profiling of gene lists from large-scale experiments
Reimand, J., Kull, M., Peterson, H., Hansen, J., & Vilo, J. (2007) · Nucleic Acids Research · doi:10.1093/nar/gkm226 · Cited by 1,545
A systems biology approach for pathway level analysis
Draghici, S., Khatri, P., Tarca, A. L., Amin, K., Done, A., Voichita, C., Georgescu, C., & Romero, R. (2007) · Genome Research · doi:10.1101/gr.6202607 · Cited by 1,249
Self-citationA novel signaling pathway impact analysis
Tarca, A. L., Draghici, S., Khatri, P., Hassan, S. S., Mittal, P., Kim, J.-S., Kim, C. J., Kusanovic, J. P., & Romero, R. (2009) · Bioinformatics · doi:10.1093/bioinformatics/btn577 · Cited by 1,031
Self-citationThe Gene Ontology (GO) project in 2006
Gene Ontology Consortium (2006) · Nucleic Acids Research · doi:10.1093/nar/gkj021 · Cited by 1,016
Manzoni, C., Kia, D. A., Vandrovcova, J., Hardy, J., Wood, N. W., Lewis, P. A., & Ferrari, R. (2018) · Briefings in Bioinformatics · doi:10.1093/bib/bbw114 · Cited by 804
Analyzing gene expression data in terms of gene sets: methodological issues
Goeman, J. J., & Bühlmann, P. (2007) · Bioinformatics · doi:10.1093/bioinformatics/btm051 · Cited by 802
MSEA: a web-based tool to identify biologically meaningful patterns in quantitative metabolomic data
Xia, J., & Wishart, D. S. (2010) · Nucleic Acids Research · doi:10.1093/nar/gkq329 · Cited by 782
Gene regulatory network inference: Data integration in dynamic models—A review
Hecker, M., Lambeck, S., Toepfer, S., van Someren, E., & Guthke, R. (2009) · Biosystems · doi:10.1016/j.biosystems.2008.12.004 · Cited by 755
Enrichment or depletion of a GO category within a class of genes: which test?
Rivals, I., Personnaz, L., Taing, L., & Potier, M.-C. (2007) · Bioinformatics · doi:10.1093/bioinformatics/btl633 · Cited by 718
A new measure for functional similarity of gene products based on Gene Ontology
Schlicker, A., Domingues, F. S., Rahnenführer, J., & Lengauer, T. (2006) · BMC Bioinformatics · doi:10.1186/1471-2105-7-302 · Cited by 692
GENECODIS: a web-based tool for finding significant concurrent annotations in gene lists
Carmona-Saez, P., Chagoyen, M., Tirado, F., Carazo, J. M., & Pascual-Montano, A. (2007) · Genome Biology · doi:10.1186/gb-2007-8-1-r3 · Cited by 592
Reliability and reproducibility issues in DNA microarray measurements
DRAGHICI, S., KHATRI, P., EKLUND, A., & SZALLASI, Z. (2006) · Trends in Genetics · doi:10.1016/j.tig.2005.12.005 · Cited by 590
Self-citationIntrinsic Disorder Is a Common Feature of Hub Proteins from Four Eukaryotic Interactomes
Haynes, C., Oldfield, C. J., Ji, F., Klitgord, N., Cusick, M. E., Radivojac, P., Uversky, V. N., Vidal, M., & Iakoucheva, L. M. (2006) · PLoS Computational Biology · doi:10.1371/journal.pcbi.0020100 · Cited by 582
GeneCodis3: a non-redundant and modular enrichment analysis tool for functional genomics
Tabas-Madrid, D., Nogales-Cadenas, R., & Pascual-Montano, A. (2012) · Nucleic Acids Research · doi:10.1093/nar/gks402 · Cited by 536
Use and misuse of the gene ontology annotations
Yon Rhee, S., Wood, V., Dolinski, K., & Draghici, S. (2008) · Nature Reviews Genetics · doi:10.1038/nrg2363 · Cited by 506
Ontologizer 2.0—a multifunctional tool for GO term enrichment analysis and data exploration
Bauer, S., Grossmann, S., Vingron, M., & Robinson, P. N. (2008) · Bioinformatics · doi:10.1093/bioinformatics/btn250 · Cited by 495
Nogales-Cadenas, R., Carmona-Saez, P., Vazquez, M., Vicente, C., Yang, X., Tirado, F., Carazo, J. M., & Pascual-Montano, A. (2009) · Nucleic Acids Research · doi:10.1093/nar/gkp416 · Cited by 437
Sequeira, A., Mamdani, F., Ernst, C., Vawter, M. P., Bunney, W. E., Lebel, V., Rehal, S., Klempan, T., Gratton, A., Benkelfat, C., et al. (2009) · PLoS ONE · doi:10.1371/journal.pone.0006585 · Cited by 388
“Functional analysis using gene ontologies [82] was performed across the 17 regions using a new tool (ErmineJ) that efficiently addresses many of the limitations and problems of the initial gene ontology tools [83], by implementing more comprehensive algorithms and the possibility of performing analyses in parallel.”
Kaimal, V., Bardes, E. E., Tabar, S. C., Jegga, A. G., & Aronow, B. J. (2010) · Nucleic Acids Research · doi:10.1093/nar/gkq418 · Cited by 375
Automated protein function prediction--the genomic challenge
Friedberg, I. (2006) · Briefings in Bioinformatics · doi:10.1093/bib/bbl004 · Cited by 366
A general modular framework for gene set enrichment analysis
Ackermann, M., & Strimmer, K. (2009) · BMC Bioinformatics · doi:10.1186/1471-2105-10-47 · Cited by 358
AUBIN‐HORTH, N., & RENN, S. C. P. (2009) · Molecular Ecology · doi:10.1111/j.1365-294x.2009.04313.x · Cited by 350
Gene-set approach for expression pattern analysis
Nam, D., & Kim, S.-Y. (2008) · Briefings in Bioinformatics · doi:10.1093/bib/bbn001 · Cited by 338
Usadel, B., Nagel, A., Steinhauser, D., Gibon, Y., Bläsing, O. E., Redestig, H., Sreenivasulu, N., Krall, L., Hannah, M. A., Poree, F., et al. (2006) · BMC Bioinformatics · doi:10.1186/1471-2105-7-535 · Cited by 337
Kuhn, A., Goldstein, D. R., Hodges, A., Strand, A. D., Sengstag, T., Kooperberg, C., Becanovic, K., Pouladi, M. A., Sathasivam, K., Cha, J.-H. J., et al. (2007) · Human Molecular Genetics · doi:10.1093/hmg/ddm133 · Cited by 328
Gu, Z., & Hübschmann, D. (2023) · Genomics, Proteomics & Bioinformatics · doi:10.1016/j.gpb.2022.04.008 · Cited by 307
Analysis of microarray experiments of gene expression profiling
Tarca, A. L., Romero, R., & Draghici, S. (2006) · American Journal of Obstetrics and Gynecology · doi:10.1016/j.ajog.2006.07.001 · Cited by 304
The United Kingdom Brain Expression Consortium, Botía, J. A., Vandrovcova, J., Forabosco, P., Guelfi, S., D’Sa, K., Hardy, J., Lewis, C. M., Ryten, M., & Weale, M. E. (2017) · BMC Systems Biology · doi:10.1186/s12918-017-0420-6 · Cited by 289
Biomedical ontologies: a functional perspective
Rubin, D. L., Shah, N. H., & Noy, N. F. (2007) · Briefings in Bioinformatics · doi:10.1093/bib/bbm059 · Cited by 287
ErmineJ: Tool for functional analysis of gene expression data sets
Lee, H. K., Braynen, W., Keshav, K., & Pavlidis, P. (2005) · BMC Bioinformatics · doi:10.1186/1471-2105-6-269 · Cited by 278
Concept annotation in the CRAFT corpus
Bada, M., Eckert, M., Evans, D., Garcia, K., Shipley, K., Sitnikov, D., Baumgartner, W. A., Cohen, K. B., Verspoor, K., Blake, J. A., et al. (2012) · BMC Bioinformatics · doi:10.1186/1471-2105-13-161 · Cited by 272
BABELOMICS: a systems biology perspective in the functional annotation of genome-scale experiments
Al-Shahrour, F., Minguez, P., Tarraga, J., Montaner, D., Alloza, E., Vaquerizas, J. M., Conde, L., Blaschke, C., Vera, J., & Dopazo, J. (2006) · Nucleic Acids Research · doi:10.1093/nar/gkl172 · Cited by 269
Gene set enrichment analysis: performance evaluation and usage guidelines
Hung, J.-H., Yang, T.-H., Hu, Z., Weng, Z., & DeLisi, C. (2012) · Briefings in Bioinformatics · doi:10.1093/bib/bbr049 · Cited by 266
Pathway Analysis: State of the Art
García-Campos, M. A., Espinal-Enríquez, J., & Hernández-Lemus, E. (2015) · Frontiers in Physiology · doi:10.3389/fphys.2015.00383 · Cited by 265
Evaluation and comparison of bioinformatic tools for the enrichment analysis of metabolomics data
Marco-Ramell, A., Palau-Rodriguez, M., Alay, A., Tulipani, S., Urpi-Sarda, M., Sanchez-Pla, A., & Andres-Lacueva, C. (2018) · BMC Bioinformatics · doi:10.1186/s12859-017-2006-0 · Cited by 259
Al-Shahrour, F., Minguez, P., Tárraga, J., Medina, I., Alloza, E., Montaner, D., & Dopazo, J. (2007) · Nucleic Acids Research · doi:10.1093/nar/gkm260 · Cited by 257
A gene ontology inferred from molecular networks
Dutkowski, J., Kramer, M., Surma, M. A., Balakrishnan, R., Cherry, J. M., Krogan, N. J., & Ideker, T. (2013) · Nature Biotechnology · doi:10.1038/nbt.2463 · Cited by 184
Integrating genetic and gene expression evidence into genome-wide association analysis of gene sets
Xiong, Q., Ancona, N., Hauser, E. R., Mukherjee, S., & Furey, T. S. (2012) · Genome Research · doi:10.1101/gr.124370.111 · Cited by 103
Consistency, comprehensiveness, and compatibility of pathway databases
Soh, D., Dong, D., Guo, Y., & Wong, L. (2010) · BMC Bioinformatics · doi:10.1186/1471-2105-11-449 · Cited by 74
“Indeed, most of the new generation of algorithms incorporate information from biological pathways into microarray data analysis [2-4].”
BackgroundIdentifying biological concepts from a protein-related corpus with a probabilistic topic model
Zheng, B., McLean, D. C., & Lu, X. (2006) · BMC Bioinformatics · doi:10.1186/1471-2105-7-58 · Cited by 68
“, semantic information has be used to cluster genes [10] and evaluate the functional coherence of a group of genes [11-13].”
Pathway analysis of dilated cardiomyopathy using global proteomic profiling and enrichment maps
Isserlin, R., Merico, D., Alikhani‐Koupaei, R., Gramolini, A., Bader, G. D., & Emili, A. (2010) · PROTEOMICS · doi:10.1002/pmic.200900412 · Cited by 63
Mortensen, J. M., Minty, E. P., Januszyk, M., Sweeney, T. E., Rector, A. L., Noy, N. F., & Musen, M. A. (2015) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2014-002901 · Cited by 63
Navigating Transcriptomic Connectivity Mapping Workflows to Link Chemicals with Bioactivities
Shah, I., Bundy, J., Chambers, B., Everett, L. J., Haggard, D., Harrill, J., Judson, R. S., Nyffeler, J., & Patlewicz, G. (2022) · Chemical Research in Toxicology · doi:10.1021/acs.chemrestox.2c00245 · Cited by 19
Metadata
- Authors
- Khatri, P., & Drăghici, S.
- Venue
- Bioinformatics
- Publisher
- Oxford University Press (OUP)
- Published
- 2005-06-30
- Volume
- 21
- Issue
- 18
- Pages
- 3587-3595
- ISSN
- 1367-4803 (ISSN-L)
- 1367-4811
- OpenAlex
- W2135230352
- Cited by
- 854
- References
- 34
- FWCI
- 25.01
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.