The DOI matches its Crossref record; 45 of the 45 references it cites were found in a registry or an index.
Manifold Regularized Discriminative Nonnegative Matrix Factorization With Fast Gradient Descent
Abstract
Nonnegative matrix factorization (NMF) has become a popular data-representation method and has been widely used in image processing and pattern-recognition problems. This is because the learned bases can be interpreted as a natural parts-based representation of data and this interpretation is consistent with the psychological intuition of combining parts to form a whole. For practical classification tasks, however, NMF ignores both the local geometry of data and the discriminative information of different classes. In addition, existing research results show that the learned basis is unnecessarily parts-based because there is neither explicit nor implicit constraint to ensure the representation parts-based. In this paper, we introduce the manifold regularization and the margin maximization to NMF and obtain the manifold regularized discriminative NMF (MD-NMF) to overcome the aforementioned problems. The multiplicative update rule (MUR) can be applied to optimizing MD-NMF, but it converges slowly. In this paper, we propose a fast gradient descent (FGD) to optimize MD-NMF. FGD contains a Newton method that searches the optimal step length, and thus, FGD converges much faster than MUR. In addition, FGD includes MUR as a special case and can be applied to optimizing NMF and its variants. For a problem with 165 samples in R(1600), FGD converges in 28 s, while MUR requires 282 s. We also apply FGD in a variant of MD-NMF and experimental results confirm its efficiency. Experimental results on several face image datasets suggest the effectiveness of MD-NMF.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Maximum Likelihood from Incomplete Data Via the EM Algorithm
Dempster, A. P., Laird, N. M., & Rubin, D. B. (1977) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1977.tb01600.x · Cited by 50,444
THE USE OF MULTIPLE MEASUREMENTS IN TAXONOMIC PROBLEMS
FISHER, R. A. (1936) · Annals of Eugenics · doi:10.1111/j.1469-1809.1936.tb02137.x · Cited by 15,227
Learning the parts of objects by non-negative matrix factorization
Lee, D. D., & Seung, H. S. (1999) · Nature · doi:10.1038/44565 · Cited by 13,868
Eigenfaces vs. Fisherfaces: recognition using class specific linear projection
Belhumeur, P. N., Hespanha, J. P., & Kriegman, D. J. (1997) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.598228 · Cited by 11,707
Introduction to Statistical Pattern Recognition
(1990) · Elsevier eBooks · doi:10.1016/c2009-0-27872-x · Cited by 11,031
ALGORITHMS FOR NON-NEGATIVE MATRIX FACTORIZATION
Lee, D. D., & Seung, H. S. (2000) · Cited by 7,952
Chung, F. (1996) · CBMS Regional Conference Series in Mathematics · doi:10.1090/cbms/092 · Cited by 5,524
Nonnegative Matrices in the Mathematical Sciences
(1979) · Elsevier eBooks · doi:10.1016/c2013-0-10361-3 · Cited by 5,451
Maximum likelihood estimation from incomplete data via the EM algorithm
Dempster, A. P. (1977) · Journal of the Royal Statistical Society Series A (Statistics in Society) · Cited by 4,543
Laplacian Eigenmaps and Spectral Techniques for Embedding and Clustering
Belkin, M., & Niyogi, P. (2002) · Advances in Neural Information Processing Systems 14 · doi:10.7551/mitpress/1120.003.0080 · Cited by 4,412
Locality Preserving Projections
He, X., & Niyogi, P. (2003) · Cited by 4,119
Sparse coding with an overcomplete basis set: A strategy employed by V1?
Olshausen, B. A., & Field, D. J. (1997) · Vision Research · doi:10.1016/s0042-6989(97)00169-7 · Cited by 3,642
(2006) · Cited by 3,624
Nonnegative Matrices in the Mathematical Sciences
Berman, A., & Plemmons, R. J. (1994) · Society for Industrial and Applied Mathematics eBooks · doi:10.1137/1.9781611971262 · Cited by 3,573
Graph Embedding and Extensions: A General Framework for Dimensionality Reduction
Yan, S., Xu, D., Zhang, B., Zhang, H.-J., Yang, Q., & Lin, S. (2007) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2007.250598 · Cited by 3,031
Parameterisation of a stochastic model for human face identification
Samaria, F. S., & Harter, A. C. (2002) · Proceedings of 1994 IEEE Workshop on Applications of Computer Vision · doi:10.1109/acv.1994.341300 · Cited by 2,593
Non-negative matrix factorization with sparseness constraints
Hoyer, P. O. (2004) · arXiv (Cornell University) · doi:10.48550/arxiv.cs/0408058 · Cited by 2,591
The CMU pose, illumination, and expression database
Sim, T., Baker, S., & Bsat, M. (2003) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2003.1251154 · Cited by 1,783
Projected Gradient Methods for Nonnegative Matrix Factorization
Lin, C.-J. (2007) · Neural Computation · doi:10.1162/neco.2007.19.10.2756 · Cited by 1,737
Logothetis, N. K., & Sheinberg, D. L. (1996) · Annual Review of Neuroscience · doi:10.1146/annurev.ne.19.030196.003045 · Cited by 1,351
Orthogonal nonnegative matrix t-factorizations for clustering
Ding, C., Li, T., Peng, W., & Park, H. (2006) · Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/1150402.1150420 · Cited by 1,245
General Tensor Discriminant Analysis and Gabor Features for Gait Recognition
Tao, D., Li, X., Wu, X., & Maybank, S. J. (2007) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2007.1096 · Cited by 1,140
Self-citationHoyer, P. O. (2003) · Proceedings of the 12th IEEE Workshop on Neural Networks for Signal Processing · doi:10.1109/nnsp.2002.1030067 · Cited by 804
Learning spatially localized, parts-based representation
Li, S. Z., Hou, X. W., Zhang, H. J., & Cheng, Q. S. (2005) · Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001 · doi:10.1109/cvpr.2001.990477 · Cited by 776
When Does Non-Negative Matrix Factorization Give a Correct Decomposition into Parts?
Donoho, D. L., & Stodden, V. (2004) · doi:10.7916/d88d05n7 · Cited by 685
Hierarchical structure in perceptual representation
Palmer, S. E. (1977) · Cognitive Psychology · doi:10.1016/0010-0285(77)90016-0 · Cited by 630
Characterising Virtual Eigensignatures for General Purpose Face Recognition
Graham, D. B., & Allinson, N. M. (1998) · Face Recognition · doi:10.1007/978-3-642-72201-1_25 · Cited by 607
Geometric Mean for Subspace Selection
Tao, D., Li, X., Wu, X., & Maybank, S. J. (2009) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2008.70 · Cited by 583
Self-citationOn the Goldstein-Levitin-Polyak gradient projection method
Bertsekas, D. P. (1976) · IEEE Transactions on Automatic Control · doi:10.1109/tac.1976.1101194 · Cited by 583
Bregman Divergence-Based Regularization for Transfer Subspace Learning
Si, S., Tao, D., & Geng, B. (2010) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2009.126 · Cited by 477
Self-citationOn the Convergence of Multiplicative Update Algorithms for Nonnegative Matrix Factorization
Lin, C.-J. (2007) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2007.895831 · Cited by 416
Non-negative Matrix Factorization on Manifold
Cai, D., He, X., Wu, X., & Han, J. (2008) · 2008 Eighth IEEE International Conference on Data Mining · doi:10.1109/icdm.2008.57 · Cited by 405
Patch Alignment for Dimensionality Reduction
Zhang, T., Tao, D., Li, X., & Yang, J. (2009) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2008.212 · Cited by 381
Self-citationZafeiriou, S., Tefas, A., Buciu, I., & Pitas, I. (2006) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2006.873291 · Cited by 329
Text Mining using Non-Negative Matrix Factorizations
Pauca, V. P., Shahnaz, F., Berry, M. W., & Plemmons, R. J. (2004) · Proceedings of the 2004 SIAM International Conference on Data Mining · doi:10.1137/1.9781611972740.45 · Cited by 302
Wachsmuth, E., Oram, M. W., & Perrett, D. I. (1994) · Cerebral Cortex · doi:10.1093/cercor/4.5.509 · Cited by 278
Xu, D., Yan, S., Tao, D., Lin, S., & Zhang, H.-J. (2007) · IEEE Transactions on Image Processing · doi:10.1109/tip.2007.906769 · Cited by 276
Self-citationManifold elastic net: a unified framework for sparse dimension reduction
Zhou, T., Tao, D., & Wu, X. (2011) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-010-0182-x · Cited by 195
Self-citationBiased Discriminant Euclidean Embedding for Content-Based Image Retrieval
Bian, W., & Tao, D. (2009) · IEEE Transactions on Image Processing · doi:10.1109/tip.2009.2035223 · Cited by 154
Self-citationTopology Preserving Non-negative Matrix Factorization for Face Recognition
Zhang, T., Fang, B., Tang, Y. Y., He, G., & Wen, J. A. (2008) · IEEE Transactions on Image Processing · doi:10.1109/tip.2008.918957 · Cited by 145
Learning on Graph with Laplacian Regularization
Ando, R. K., & Zhang, T. (2007) · The MIT Press eBooks · doi:10.7551/mitpress/7503.003.0009 · Cited by 125
Problems of learning on manifolds
Niyogi, P., & Belkin, M. A. (2003) · Cited by 124
Yang, J., Yan, S., Fu, Y., Li, X., & Huang, T. S. (2008) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2008.4587665 · Cited by 103
Interior-Point Gradient Method for Large-Scale Totally Nonnegative Least Squares Problems
Merritt, M., & Zhang, Y. (2005) · Journal of Optimization Theory and Applications · doi:10.1007/s10957-005-2668-z · Cited by 70
Nonlinear Non-Negative Component Analysis Algorithms
Zafeiriou, S. P., & Petrou, M. (2009) · IEEE Transactions on Image Processing · doi:10.1109/tip.2009.2038816 · Cited by 69
Cited by
Nonnegative Matrix Factorization: A Comprehensive Review
Wang, Y.-X., & Zhang, Y.-J. (2013) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2012.51 · Cited by 981
MicroRNAs and complex diseases: from experimental results to computational models
Chen, X., Xie, D., Zhao, Q., & You, Z.-H. (2019) · Briefings in Bioinformatics · doi:10.1093/bib/bbx130 · Cited by 638
3-D Object Retrieval and Recognition With Hypergraph Analysis
Gao, Y., Wang, M., Tao, D., Ji, R., & Dai, Q. (2012) · IEEE Transactions on Image Processing · doi:10.1109/tip.2012.2199502 · Cited by 633
Self-citationNeNMF: An Optimal Gradient Method for Nonnegative Matrix Factorization
Guan, N., Tao, D., Luo, Z., & Yuan, B. (2012) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2012.2190406 · Cited by 468
Self-citationAdaptive Hypergraph Learning and its Application in Image Classification
Yu, J., Tao, D., & Wang, M. (2012) · IEEE Transactions on Image Processing · doi:10.1109/tip.2012.2190083 · Cited by 405
Self-citation
Metadata
- Authors
- Guan, N., Tao, D., Luo, Z., & Yuan, B.
- Venue
- IEEE Transactions on Image Processing
- Published
- 2011-07
- Volume
- 20
- Issue
- 7
- Pages
- 2030-2048
- ISSN
- 1057-7149 (ISSN-L)
- 1941-0042
- OpenAlex
- W2144719328
- Cited by
- 318
- References
- 45
- FWCI
- 43.48
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.