The DOI matches its Crossref record; 44 of the 44 references it cites were found in a registry or an index.
Online Nonnegative Matrix Factorization With Robust Stochastic Approximation
Abstract
Nonnegative matrix factorization (NMF) has become a popular dimension-reduction method and has been widely applied to image processing and pattern recognition problems. However, conventional NMF learning methods require the entire dataset to reside in the memory and thus cannot be applied to large-scale or streaming datasets. In this paper, we propose an efficient online RSA-NMF algorithm (OR-NMF) that learns NMF in an incremental fashion and thus solves this problem. In particular, OR-NMF receives one sample or a chunk of samples per step and updates the bases via robust stochastic approximation. Benefitting from the smartly chosen learning rate and averaging technique, OR-NMF converges at the rate of in each update of the bases. Furthermore, we prove that OR-NMF almost surely converges to a local optimal solution by using the quasi-martingale. By using a buffering strategy, we keep both the time and space complexities of one step of the OR-NMF constant and make OR-NMF suitable for large-scale or streaming datasets. Preliminary experimental results on real-world datasets show that OR-NMF outperforms the existing online NMF (ONMF) algorithms in terms of efficiency. Experimental results of face recognition and image annotation on public datasets confirm the effectiveness of OR-NMF compared with the existing ONMF algorithms.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Learning the parts of objects by non-negative matrix factorization
Lee, D. D., & Seung, H. S. (1999) · Nature · doi:10.1038/44565 · Cited by 13,868
ALGORITHMS FOR NON-NEGATIVE MATRIX FACTORIZATION
Seung, D., & Lee, L.-W. (2001) · Cited by 4,753
Vaart, A. W. V. D. (1998) · doi:10.1017/cbo9780511802256 · Cited by 4,159
Advances in neural information processing systems 11
Peter Sollich, A. (1999) · Cited by 3,168
Parameterisation of a stochastic model for human face identification
Samaria, F. S., & Harter, A. C. (2002) · Proceedings of 1994 IEEE Workshop on Applications of Computer Vision · doi:10.1109/acv.1994.341300 · Cited by 2,593
Non-negative matrix factorization with sparseness constraints
Hoyer, P. O. (2004) · arXiv (Cornell University) · doi:10.48550/arxiv.cs/0408058 · Cited by 2,591
Graph Regularized Nonnegative Matrix Factorization for Data Representation
Cai, D., He, X., Han, J., & Huang, T. S. (2011) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2010.231 · Cited by 2,451
Online Learning for Matrix Factorization and Sparse Coding
Mairal, J., Bach, F., Ponce, J., & Sapiro, G. (2009) · arXiv (Cornell University) · doi:10.48550/arxiv.0908.0050 · Cited by 2,258
Robust Stochastic Approximation Approach to Stochastic Programming
Nemirovski, A., Juditsky, A., Lan, G., & Shapiro, A. (2009) · SIAM Journal on Optimization · doi:10.1137/070704277 · Cited by 1,945
Parlar, M. (2000) · Interactive Operations Research with Maple · doi:10.1007/978-1-4612-1356-7_5 · Cited by 1,741
Projected Gradient Methods for Nonnegative Matrix Factorization
Lin, C.-J. (2007) · Neural Computation · doi:10.1162/neco.2007.19.10.2756 · Cited by 1,737
Object Recognition as Machine Translation: Learning a Lexicon for a Fixed Image Vocabulary
Duygulu, P., Barnard, K., de Freitas, J. F. G., & Forsyth, D. A. (2002) · Lecture Notes in Computer Science · doi:10.1007/3-540-47979-1_7 · Cited by 1,608
Algorithms and applications for approximate nonnegative matrix factorization
Berry, M. W., Browne, M., Langville, A. N., Pauca, V. P., & Plemmons, R. J. (2007) · Computational Statistics & Data Analysis · doi:10.1016/j.csda.2006.11.006 · Cited by 1,512
The Existence of Probability Measures with Given Marginals
Strassen, V. (1965) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177700153 · Cited by 1,301
Efficient projections onto the l1-ball for learning in high dimensions
Duchi, J., Shalev-Shwartz, S., Singer, Y., & Chandra, T. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390191 · Cited by 1,184
The Tradeoffs of Large-Scale Learning
Bottou, L., & Bousquet, O. (2011) · Optimization for Machine Learning · doi:10.7551/mitpress/8996.003.0015 · Cited by 1,128
Févotte, C., Bertin, N., & Durrieu, J.-L. (2009) · Neural Computation · doi:10.1162/neco.2008.04-08-771 · Cited by 1,119
Hoyer, P. O. (2003) · Proceedings of the 12th IEEE Workshop on Neural Networks for Signal Processing · doi:10.1109/nnsp.2002.1030067 · Cited by 804
(2000) · CrossRef Listing of Deleted DOIs · doi:10.1162/153244303322533214 · Cited by 615
Bregman Divergence-Based Regularization for Transfer Subspace Learning
Si, S., Tao, D., & Geng, B. (2010) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2009.126 · Cited by 477
Self-citationNeNMF: An Optimal Gradient Method for Nonnegative Matrix Factorization
Guan, N., Tao, D., Luo, Z., & Yuan, B. (2012) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2012.2190406 · Cited by 468
Self-citationA New Baseline for Image Annotation
Makadia, A., Pavlovic, V., & Kumar, S. (2008) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-88690-7_24 · Cited by 436
On the Convergence of Multiplicative Update Algorithms for Nonnegative Matrix Factorization
Lin, C.-J. (2007) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2007.895831 · Cited by 416
Component-Based Face Recognition with 3D Morphable Models
Weyrauch, B., Heisele, B., Huang, J., & Blanz, V. (2005) · 2004 Conference on Computer Vision and Pattern Recognition Workshop · doi:10.1109/cvpr.2004.315 · Cited by 330
Zafeiriou, S., Tefas, A., Buciu, I., & Pitas, I. (2006) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2006.873291 · Cited by 329
Manifold Regularized Discriminative Nonnegative Matrix Factorization With Fast Gradient Descent
Guan, N., Tao, D., Luo, Z., & Yuan, B. (2011) · IEEE Transactions on Image Processing · doi:10.1109/tip.2011.2105496 · Cited by 318
Self-citationText Mining using Non-Negative Matrix Factorizations
Pauca, V. P., Shahnaz, F., Berry, M. W., & Plemmons, R. J. (2004) · Proceedings of the 2004 SIAM International Conference on Data Mining · doi:10.1137/1.9781611972740.45 · Cited by 302
The IAPR Benchmark : A New Evaluation Resource for Visual Information Systems
Grubinger, M. (2006) · RWTH Publications (RWTH Aachen) · Cited by 285
Linear and Nonlinear Projective Nonnegative Matrix Factorization
Yang, Z., & Oja, E. (2010) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2010.2041361 · Cited by 270
Laplacian Regularized Gaussian Mixture Model for Data Clustering
He, X., Cai, D., Shao, Y., Bao, H., & Han, J. (2011) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2010.259 · Cited by 246
Ensemble Manifold Regularization
Geng, B., Tao, D., Xu, C., Yang, L., & Hua, X.-S. (2012) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2012.57 · Cited by 227
Self-citationToward Faster Nonnegative Matrix Factorization: A New Algorithm and Comparisons
Kim, J., & Park, H. (2008) · 2008 Eighth IEEE International Conference on Data Mining · doi:10.1109/icdm.2008.149 · Cited by 220
Non-Negative Patch Alignment Framework
Guan, N., Tao, D., Luo, Z., & Yuan, B. (2011) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2011.2157359 · Cited by 181
Self-citationm-SNE: Multiview Stochastic Neighbor Embedding
Xie, B., Mu, Y., Tao, D., & Huang, K. (2011) · IEEE Transactions on Systems Man and Cybernetics Part B (Cybernetics) · doi:10.1109/tsmcb.2011.2106208 · Cited by 131
Self-citationIncremental subspace learning via non-negative matrix factorization
Bucak, S. S., & Günsel, B. (2008) · Pattern Recognition · doi:10.1016/j.patcog.2008.09.002 · Cited by 127
Sparse transfer learning for interactive video search reranking
Tian, X., Tao, D., & Rui, Y. (2012) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/2240136.2240139 · Cited by 120
Self-citationColor to Gray: Visual Cue Preservation
Song, M., Tao, D., Chen, C., Li, X., & Chen, C. W. (2010) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2009.74 · Cited by 111
Self-citationSubspaces Indexing Model on Grassmann Manifold for Image Search
Wang, X., Li, Z., & Tao, D. (2011) · IEEE Transactions on Image Processing · doi:10.1109/tip.2011.2114354 · Cited by 106
Self-citationYang, J., Yan, S., Fu, Y., Li, X., & Huang, T. S. (2008) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2008.4587665 · Cited by 103
Fisk, D. L. (1965) · Transactions of the American Mathematical Society · doi:10.1090/s0002-9947-1965-0192542-5 · Cited by 92
Detect and track latent factors with online nonnegative matrix factorization
Cao, B., Shen, D., Sun, J.-T., et al. (2007) · Cited by 85
Online algorithms for nonnegative matrix factorization with the Itakura-Saito divergence
Lefèvre, A., Bach, F., & Févotte, C. (2011) · IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA) · doi:10.1109/aspaa.2011.6082314 · Cited by 79
Three-Dimensional Face Reconstruction From a Single Image by a Coupled RBF Network
Song, M., Tao, D., Huang, X., Chen, C., & Bu, J. (2012) · IEEE Transactions on Image Processing · doi:10.1109/tip.2012.2183882 · Cited by 75
Self-citationStatistical Optimization of Non-Negative Matrix Factorization
Balan, A. K., Boyles, L., Welling, M., et al. (2011) · International Conference on Artificial Intelligence and Statistics · Cited by 11
Cited by
Data-intensive applications, challenges, techniques and technologies: A survey on Big Data
Philip Chen, C. L., & Zhang, C.-Y. (2014) · Information Sciences · doi:10.1016/j.ins.2014.01.015 · Cited by 3,047
3-D Object Retrieval and Recognition With Hypergraph Analysis
Gao, Y., Wang, M., Tao, D., Ji, R., & Dai, Q. (2012) · IEEE Transactions on Image Processing · doi:10.1109/tip.2012.2199502 · Cited by 633
Self-citationSingle Image Super-Resolution With Non-Local Means and Steering Kernel Regression
Zhang, K., Gao, X., Tao, D., & Li, X. (2012) · IEEE Transactions on Image Processing · doi:10.1109/tip.2012.2208977 · Cited by 576
Self-citationLuo, X., Zhou, M., Li, S., You, Z., Xia, Y., & Zhu, Q. (2016) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2015.2415257 · Cited by 322
Understanding and Enhancement of Internal Clustering Validation Measures
Liu, Y., Li, Z., Xiong, H., Gao, X., Wu, J., & Wu, S. (2013) · IEEE Transactions on Cybernetics · doi:10.1109/tsmcb.2012.2220543 · Cited by 315
Metadata
- Authors
- Guan, N., Tao, D., Luo, Z., & Yuan, B.
- Venue
- IEEE Transactions on Neural Networks and Learning Systems
- Published
- 2012-07
- Volume
- 23
- Issue
- 7
- Pages
- 1087-1099
- ISSN
- 2162-237X (ISSN-L)
- 2162-2388
- OpenAlex
- W2121281940
- Cited by
- 287
- References
- 44
- FWCI
- 40.00
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.