The DOI matches its Crossref record; 55 of the 55 references it cites were found in a registry or an index.
A comprehensive evaluation of multicategory classification methods for microarray gene expression cancer diagnosis
Open accessAbstract
Motivation: Cancer diagnosis is one of the most important emerging clinical applications of gene expression microarray technology. We are seeking to develop a computer system for powerful and reliable cancer diagnostic model creation based on microarray data. To keep a realistic perspective on clinical applications we focus on multicategory diagnosis. To equip the system with the optimum combination of classifier, gene selection and cross-validation methods, we performed a systematic and comprehensive evaluation of several major algorithms for multicategory classification, several gene selection methods, multiple ensemble classifier methods and two cross-validation designs using 11 datasets spanning 74 diagnostic categories and 41 cancer types and 12 normal tissue types. Results: Multicategory support vector machines (MC-SVMs) are the most effective classifiers in performing accurate cancer diagnosis from gene expression data. The MC-SVM techniques by Crammer and Singer, Weston and Watkins and one-versus-rest were found to be the best methods in this domain. MC-SVMs outperform other popular machine learning algorithms, such as k-nearest neighbors, backpropagation and probabilistic neural networks, often to a remarkable degree. Gene selection techniques can significantly improve the classification performance of both MC-SVMs and other non-SVM learning algorithms. Ensemble classifiers do not generally improve performance of the best non-ensemble models. These results guided the construction of a software system GEMS (Gene Expression Model Selector) that automates high-quality model construction and enforces sound optimization and performance estimation procedures. This is the first such system to be informed by a rigorous comparative analysis of the available algorithms and datasets. Availability: The software system GEMS is available for download from http://www.gems-system.org for non-commercial use. Contact: alexander.statnikov@vanderbilt.edu
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
LIBSVM: A library for support vector machines
Chang, C.-C., & Lin, C.-J. (2011) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/1961189.1961199 · Cited by 40,918
Wu, Y. (1999) · Technometrics · doi:10.1080/00401706.1999.10485951 · Cited by 27,221
The Elements of Statistical Learning
Hastie, T., Tibshirani, R., & Friedman, J. (2009) · Springer Series in Statistics · doi:10.1007/978-0-387-84858-7 · Cited by 25,970
The Elements of Statistical Learning
Hastie, T., Friedman, J., & Tibshirani, R. (2001) · Springer Series in Statistics · doi:10.1007/978-0-387-21606-5 · Cited by 24,162
Advances in neural information processing systems 7
Cios, K., & Shields, M. (1997) · Neurocomputing · doi:10.1016/s0925-2312(97)90018-7 · Cited by 22,059
(2012) · Encyclopedia of the Sciences of Learning · doi:10.1007/978-1-4419-1428-6_5195 · Cited by 19,370
Golub, T. R., Slonim, D. K., Tamayo, P., Huard, C., Gaasenbeek, M., Mesirov, J. P., Coller, H., Loh, M. L., Downing, J. R., Caligiuri, M. A., et al. (1999) · Science · doi:10.1126/science.286.5439.531 · Cited by 11,516
A Study of Cross-Validation and Bootstrap for Accuracy Estimation and Model Selection
Kohavi, R. (1995) · Zenodo · doi:10.5281/zenodo.19712698 · Cited by 10,909
Gene selection for cancer classification using support vector machines
Guyon, I., Weston, J., Barnhill, S., & Vapnik, V. (2002) · Machine Learning · doi:10.1023/a:1012487302797 · Cited by 9,954
A comparison of methods for multiclass support vector machines
Hsu, C.-W., & Lin, C.-J. (2002) · IEEE Transactions on Neural Networks · doi:10.1109/72.991427 · Cited by 6,838
Advances in kernel methods: support vector learning
Schölkopf, B., Burges, C. J. C., & Smola, A. J. (1999) · doi:10.5555/299094 · Cited by 5,789
Fast Training of Support Vector Machines Using Sequential Minimal Optimization
Platt, J. C. (1998) · Advances in Kernel Methods · doi:10.7551/mitpress/1130.003.0016 · Cited by 5,437
Making Large-Scale SVM Learning Practical
Joachims, T. (1998) · RePEc: Research Papers in Economics · doi:10.17877/de290r-14262 · Cited by 4,411
Approximate Statistical Tests for Comparing Supervised Classification Learning Algorithms
Dietterich, T. G. (1998) · Neural Computation · doi:10.1162/089976698300017197 · Cited by 3,840
Specht, D. F. (1990) · Neural Networks · doi:10.1016/0893-6080(90)90049-q · Cited by 3,762
Comparison of Discrimination Methods for the Classification of Tumors Using Gene Expression Data
Dudoit, S., Fridlyand, J., & Speed, T. P. (2002) · Journal of the American Statistical Association · doi:10.1198/016214502753479248 · Cited by 2,672
Khan, J., Wei, J. S., Ringnér, M., Saal, L. H., Ladanyi, M., Westermann, F., Berthold, F., Schwab, M., Antonescu, C. R., Peterson, C., et al. (2001) · Nature Medicine · doi:10.1038/89044 · Cited by 2,644
Bhattacharjee, A., Richards, W. G., Staunton, J., Li, C., Monti, S., Vasa, P., Ladd, C., Beheshti, J., Bueno, R., Gillette, M., et al. (2001) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.191502998 · Cited by 2,576
Gene expression correlates of clinical prostate cancer behavior
Singh, D., Febbo, P. G., Ross, K., Jackson, D. G., Manola, J., Ladd, C., Tamayo, P., Renshaw, A. A., D'Amico, A. V., Richie, J. P., et al. (2002) · Cancer Cell · doi:10.1016/s1535-6108(02)00030-2 · Cited by 2,512
Furey, T. S., Cristianini, N., Duffy, N., Bednarski, D. W., Schummer, M., & Haussler, D. (2000) · Bioinformatics · doi:10.1093/bioinformatics/16.10.906 · Cited by 2,443
Shipp, M. A., Ross, K. N., Tamayo, P., Weng, A. P., Kutok, J. L., Aguiar, R. C. T., Gaasenbeek, M., Angelo, M., Reich, M., Pinkus, G. S., et al. (2002) · Nature Medicine · doi:10.1038/nm0102-68 · Cited by 2,428
Prediction of central nervous system embryonal tumour outcome based on gene expression
Pomeroy, S. L., Tamayo, P., Gaasenbeek, M., Sturla, L. M., Angelo, M., McLaughlin, M. E., Kim, J. Y. H., Goumnerova, L. C., Black, P. M., Lau, C., et al. (2002) · Nature · doi:10.1038/415436a · Cited by 2,371
Multiclass cancer diagnosis using tumor gene expression signatures
Ramaswamy, S., Tamayo, P., Rifkin, R., Mukherjee, S., Yeang, C.-H., Angelo, M., Ladd, C., Reich, M., Latulippe, E., Mesirov, J. P., et al. (2001) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.211566398 · Cited by 2,023
Boosting a Weak Learning Algorithm by Majority
Freund, Y. (1995) · Information and Computation · doi:10.1006/inco.1995.1136 · Cited by 1,674
Large Margin DAG's for Multiclass Classification
Platt, J. C., Cristianini, N., & Shawe-Taylor, J. S. (1999) · Cited by 1,674
Neural Network Toolbox™ User's Guide
Beale, M. H., Hagan, M., & Demuth, H. B. (2015) · Cited by 1,639
Decision combination in multiple classifier systems
Ho, T. K., Hull, J. J., & Srihari, S. N. (1994) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.273716 · Cited by 1,511
Permutation Tests: A Practical Guide to Resampling Methods for Testing Hypotheses
Niknian, M. (1995) · Technometrics · doi:10.1080/00401706.1995.10484341 · Cited by 1,462
Weiss, S. M. (1990) · Cited by 1,058
Pairwise Classification and Support Vector Machines
Kreßel, U. (1998) · Advances in Kernel Methods · doi:10.7551/mitpress/1130.003.0020 · Cited by 1,020
Automatic Construction of Decision Trees from Data: A Multi-Disciplinary Survey
Murthy, S. K. (1998) · Data Mining and Knowledge Discovery · doi:10.1023/a:1009744630224 · Cited by 991
Nutt, C. L., Mani, D. R., Betensky, R. A., Tamayo, P., Cairncross, J. G., Ladd, C., Pohl, U., Hartmann, C. J., McLaughlin, M. E., Batchelor, T. T., et al. (2003) · PubMed · Cited by 959
Support vector machines for multi-class pattern recognition.
Weston, J., & Watkins, C. (1999) · Cited by 796
Molecular classification of human carcinomas by use of gene expression signatures.
Su, A. I., Welsh, J. B., Sapinoso, L. M., Kern, S. G., Dimitrov, P., Lapp, H., Schultz, P. G., Powell, S. M., Moskaluk, C. A., Frierson, H. F., et al. (2001) · PubMed · Cited by 688
Chemosensitivity prediction by transcriptional profiling
Staunton, J. E., Slonim, D. K., Coller, H. A., Tamayo, P., Angelo, M. J., Park, J., Scherf, U., Lee, J. K., Reinhold, W. O., Weinstein, J. N., et al. (2001) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.191368598 · Cited by 684
On the Learnability and Design of Output Codes for Multiclass Problems
Crammer, K., & Singer, Y. (2002) · Machine Learning · doi:10.1023/a:1013637720281 · Cited by 650
Another approach to polychotomous classification
Friedman, J. H. (1996) · Medical Entomology and Zoology · Cited by 577
Learning Kernel Classifiers: Theory and Algorithms
Herbrich, R. (2001) · The MIT Press eBooks · doi:10.7551/mitpress/4170.001.0001 · Cited by 547
On Combining Artificial Neural Nets
SHARKEY, A. J. C. (1996) · Connection Science · doi:10.1080/095400996116785 · Cited by 513
Lee, Y., & Lee, C.-K. (2003) · Bioinformatics · doi:10.1093/bioinformatics/btg102 · Cited by 367
Time and sample efficient discovery of Markov blankets and direct causal relations
Tsamardinos, I., Aliferis, C. F., & Statnikov, A. (2003) · Proceedings of the ninth ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/956750.956838 · Cited by 343
Self-citationPredictive ability of DNA microarrays for cancer outcomes and correlates: an empirical assessment
Ntzani, E. E., & Ioannidis, J. P. (2003) · The Lancet · doi:10.1016/s0140-6736(03)14686-7 · Cited by 323
Newton's Method for Large Bound-Constrained Optimization Problems
Lin, C., & Morè, J. J. (1999) · SIAM Journal on Optimization · doi:10.1137/s1052623498345075 · Cited by 322
HITON: a novel Markov Blanket algorithm for optimal variable selection.
Aliferis, C., Tsamardinos, I., & Statnikov, A. (2003) · Europe PMC (PubMed Central) · Cited by 315
Self-citationMossman, D. (1999) · Medical Decision Making · doi:10.1177/0272989x9901900110 · Cited by 302
Molecular classification of multiple tumor types
Yeang, C.-H., Ramaswamy, S., Tamayo, P., Mukherjee, S., Rifkin, R. M., Angelo, M., Reich, M., Lander, E., Mesirov, J., & Golub, T. (2001) · Bioinformatics · doi:10.1093/bioinformatics/17.suppl_1.s316 · Cited by 272
Schwarzer, G., Vach, W., & Schumacher, M. (2000) · Statistics in Medicine · doi:10.1002/(sici)1097-0258(20000229)19:4<541::aid-sim355>3.0.co;2-v · Cited by 251
Overfitting in making comparisons between variable selection methods
ReunanenJuha (2003) · Journal of Machine Learning Research · doi:10.5555/944919.944978 · Cited by 205
Volume under the ROC Surface for Multi-class Problems
Ferri, C., Hernández-Orallo, J., & Salido, M. A. (2003) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-39857-8_12 · Cited by 162
(1993) · Cited by 128
GeneCluster 2.0: an advanced toolset for bioarray analysis
Reich, M. M., Ohm, K., Angelo, M., et al. (2004) · Bioinformatics · doi:10.1093/bioinformatics/bth138 · Cited by 97
Graphical Exploration of Gene Expression Data: A Comparative Study of Three Multivariate Methods
Wouters, L., Göhlmann, H. W. H., Bijnens, L., Kass, S. U., Molenberghs, G., & Lewi, P. J. (2003) · Biometrics · doi:10.1111/j.0006-341x.2003.00130.x · Cited by 84
MULTICLASS CANCER CLASSIFICATION USING GENE EXPRESSION PROFILING AND PROBABILISTIC NEURAL NETWORKS
Berrar, D., Downes, C. S., & Dubitzky, W. (2002) · Pacific Symposium on Biocomputing (PSB) · doi:10.1142/9789812776303_0002 · Cited by 69
Romualdi, C. (2003) · Human Molecular Genetics · doi:10.1093/hmg/ddg093 · Cited by 63
Information Theoretic Feature Crediting in Multiclass Support Vector Machines
Sindhwani, V., Bhattacharya, P., & Rakshit, S. (2001) · SIAM International Conference on Data Mining · doi:10.1137/1.9781611972719.16 · Cited by 63
Cited by
Machine learning in bioinformatics
Larrañaga, P., Calvo, B., Santana, R., Bielza, C., Galdiano, J., Inza, I., Lozano, J. A., Armañanzas, R., Santafé, G., Pérez, A., et al. (2006) · Briefings in Bioinformatics · doi:10.1093/bib/bbk007 · Cited by 867
Statnikov, A., Wang, L., & Aliferis, C. F. (2008) · BMC Bioinformatics · doi:10.1186/1471-2105-9-319 · Cited by 686
Self-citationRecursive feature elimination with random forest for PTR-MS analysis of agroindustrial products
Granitto, P. M., Furlanello, C., Biasioli, F., & Gasperi, F. (2006) · Chemometrics and Intelligent Laboratory Systems · doi:10.1016/j.chemolab.2006.01.007 · Cited by 663
Machine Learning Meta-analysis of Large Metagenomic Datasets: Tools and Biological Insights
Pasolli, E., Truong, D. T., Malik, F., Waldron, L., & Segata, N. (2016) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1004977 · Cited by 659
Usadel, B., Nagel, A., Thimm, O., Redestig, H., Blaesing, O. E., Palacios-Rojas, N., Selbig, J., Hannemann, J., Piques, M. C., Steinhauser, D., et al. (2005) · Plant Physiology · doi:10.1104/pp.105.060459 · Cited by 618
Metadata
- Authors
- Statnikov, A., Aliferis, C. F., Tsamardinos, I., Hardin, D., & Levy, S.
- Venue
- Bioinformatics
- Publisher
- Oxford University Press (OUP)
- Published
- 2004-09-16
- Volume
- 21
- Issue
- 5
- Pages
- 631-643
- ISSN
- 1367-4803 (ISSN-L)
- 1367-4811
- OpenAlex
- W2115358726
- Cited by
- 839
- References
- 55
- FWCI
- 15.09
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.