The DOI matches its Crossref record; 40 of the 40 references it cites were found in a registry or an index.
Automatic Relevance Determination in Nonnegative Matrix Factorization with the /spl beta/-Divergence
Abstract
This paper addresses the estimation of the latent dimensionality in nonnegative matrix factorization (NMF) with the β-divergence. The β-divergence is a family of cost functions that includes the squared euclidean distance, Kullback-Leibler (KL) and Itakura-Saito (IS) divergences as special cases. Learning the model order is important as it is necessary to strike the right balance between data fidelity and overfitting. We propose a Bayesian model based on automatic relevance determination (ARD) in which the columns of the dictionary matrix and the rows of the activation matrix are tied together through a common scale parameter in their prior. A family of majorization-minimization (MM) algorithms is proposed for maximum a posteriori (MAP) estimation. A subset of scale parameters is driven to a small lower bound in the course of inference, with the effect of pruning the corresponding spurious components. We demonstrate the efficacy and robustness of our algorithms by performing extensive experiments on synthetic data, the swimmer dataset, a music decomposition example, and a stock price prediction task.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Maximum Likelihood from Incomplete Data Via the EM Algorithm
Dempster, A. P., Laird, N. M., & Rubin, D. B. (1977) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1977.tb01600.x · Cited by 50,444
Estimating the Dimension of a Model
Schwarz, G. (1978) · The Annals of Statistics · doi:10.1214/aos/1176344136 · Cited by 47,033
Learning the parts of objects by non-negative matrix factorization
Lee, D. D., & Seung, H. S. (1999) · Nature · doi:10.1038/44565 · Cited by 13,868
Model Selection and Estimation in Regression with Grouped Variables
Yuan, M., & Lin, Y. (2006) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.1467-9868.2005.00532.x · Cited by 7,261
Paatero, P., & Tapper, U. (1994) · Environmetrics · doi:10.1002/env.3170050203 · Cited by 6,023
Enhancing Sparsity by Reweighted ℓ 1 Minimization
Candès, E. J., Wakin, M. B., & Boyd, S. P. (2008) · Journal of Fourier Analysis and Applications · doi:10.1007/s00041-008-9045-x · Cited by 4,863
Probabilistic Matrix Factorization
Mnih, A., & Salakhutdinov, R. (2007) · neural information processing systems · Cited by 3,798
Online Learning for Matrix Factorization and Sparse Coding
Mairal, J., Bach, F., Ponce, J., & Sapiro, G. (2009) · arXiv (Cornell University) · doi:10.48550/arxiv.0908.0050 · Cited by 2,258
Hunter, D. R., & Lange, K. (2004) · The American Statistician · doi:10.1198/0003130042836 · Cited by 1,783
Cichocki, A., Zdunek, R., Phan, A. H., & Amari, S. (2009) · doi:10.1002/9780470747278 · Cited by 1,540
Clustering with Bregman Divergences
Banerjee, A., Merugu, S., Dhillon, I., & Ghosh, J. (2004) · Proceedings of the 2004 SIAM International Conference on Data Mining · doi:10.1137/1.9781611972740.22 · Cited by 1,472
Févotte, C., Bertin, N., & Durrieu, J.-L. (2009) · Neural Computation · doi:10.1162/neco.2008.04-08-771 · Cited by 1,119
Probabilistic Matrix Factorization
Salakhutdinov, R., & Mnih, A. (2007) · Cited by 1,075
Mackay, D. J. C. (1995) · Network Computation in Neural Systems · doi:10.1088/0954-898x/6/3/011 · Cited by 915
Non-negative matrix factorization for polyphonic music transcription
Smaragdis, P., & Brown, J. C. (2004) · 2003 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (IEEE Cat. No.03TH8684) · doi:10.1109/aspaa.2003.1285860 · Cited by 860
Optimization with Sparsity-Inducing Penalties
Bach, F., Jenatton, R., Mairal, J., & Obozinski, G. (2025) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000015 · Cited by 730
When Does Non-Negative Matrix Factorization Give a Correct Decomposition into Parts?
Donoho, D. L., & Stodden, V. (2004) · doi:10.7916/d88d05n7 · Cited by 685
Jørgensen, B. (1987) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1987.tb01685.x · Cited by 679
Algorithms for Nonnegative Matrix Factorization with the β-Divergence
Févotte, C., & Idier, J. (2011) · Neural Computation · doi:10.1162/neco_a_00168 · Cited by 663
Optimization with Sparsity-Inducing Penalties
Bach, F. (2011) · now publishers, Inc. eBooks · doi:10.1561/9781601985118 · Cited by 608
Families of Alpha- Beta- and Gamma- Divergences: Flexible and Robust Measures of Similarities
Cichocki, A., & Amari, S.-I. (2010) · Entropy · doi:10.3390/e12061532 · Cited by 459
A Generalization of Principal Components Analysis to the Exponential Family
Collins, M., Dasgupta, S., & Schapire, R. E. (2001) · Cited by 382
Improving molecular cancer class discovery through sparse non-negative matrix factorization
Gao, Y., & Church, G. (2005) · Bioinformatics · doi:10.1093/bioinformatics/bti653 · Cited by 339
Bayesian Inference for Nonnegative Matrix Factorisation Models
Cemgil, A. T. (2009) · Computational Intelligence and Neuroscience · doi:10.1155/2009/785152 · Cited by 337
A new view of automatic relevance determination
Wipf, D., & Nagarajan, S. S. (2008) · Cited by 312
Latent Variable Bayesian Models for Promoting Sparsity
Wipf, D. P., Rao, B. D., & Nagarajan, S. (2011) · IEEE Transactions on Information Theory · doi:10.1109/tit.2011.2162174 · Cited by 305
Eggert, J., & Korner, E. (2005) · 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No.04CH37541) · doi:10.1109/ijcnn.2004.1381036 · Cited by 272
Csiszár’s Divergences for Non-negative Matrix Factorization: Family of New Algorithms
Cichocki, A., Zdunek, R., & Amari, S.-I. (2006) · Lecture Notes in Computer Science · doi:10.1007/11679363_5 · Cited by 271
Linear and Nonlinear Projective Nonnegative Matrix Factorization
Yang, Z., & Oja, E. (2010) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2010.2041361 · Cited by 270
Bishop, C. (1998) · neural information processing systems · Cited by 224
Generalized Alpha-Beta Divergences and Their Application to Robust Nonnegative Matrix Factorization
Cichocki, A., Cruces, S., & Amari, S.-I. (2011) · Entropy · doi:10.3390/e13010134 · Cited by 213
Bayesian Non-negative Matrix Factorization
Schmidt, M. N., Winther, O., & Hansen, L. K. (2009) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-00599-2_68 · Cited by 197
Automatic relevance determination for multi‐way models
Mørup, M., & Hansen, L. K. (2009) · Journal of Chemometrics · doi:10.1002/cem.1223 · Cited by 161
Bayesian Nonparametric Matrix Factorization for Recorded Music
Blei, D. M., Cook, P. R., & Hoffman, M. D. (2010) · International Conference on Machine Learning · Cited by 128
Nonnegative Matrix Factorisations As Probabilistic Inference In Composite Models
Cemgil, A. T., & Févotte, C. (2009) · doi:10.5281/zenodo.41636 · Cited by 106
Online algorithms for nonnegative matrix factorization with the Itakura-Saito divergence
Lefèvre, A., Bach, F., & Févotte, C. (2011) · IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA) · doi:10.1109/aspaa.2011.6082314 · Cited by 79
Information theoretic methods in probability and statistics
Csiszár, I. (2002) · IEEE International Symposium on Information Theory (ISIT) · doi:10.1109/isit.1997.612917 · Cited by 31
Tuning pruning in sparse non-negative matrix factorization
Mørup, M., & Hansen, L. K. (2009) · Technical University of Denmark, DTU Orbit (Technical University of Denmark, DTU) · Cited by 26
Reversible Jump MCMC for Non-Negative Matrix Factorization
Zhong, M., & Girolami, M. (2009) · Cited by 18
Automatic Rank Determination in Projective Nonnegative Matrix Factorization
Yang, Z., Zhu, Z., & Erkki Oja, A. (2010) · Lecture notes in computer science · doi:10.1007/978-3-642-15995-4_64 · Cited by 13
Cited by
The repertoire of mutational signatures in human cancer
Alexandrov, L. B., Kim, J., Haradhvala, N. J., Huang, M. N., Tian Ng, A. W., Wu, Y., Boot, A., Covington, K. R., Gordenin, D. A., Bergstrom, E. N., et al. (2020) · Nature · doi:10.1038/s41586-020-1943-3 · Cited by 3,496
Comprehensive and Integrative Genomic Characterization of Hepatocellular Carcinoma
Ally, A., Balasundaram, M., Carlsen, R., Chuah, E., Clarke, A., Dhalla, N., Holt, R. A., Jones, S. J. M., Lee, D., Ma, Y., et al. (2017) · Cell · doi:10.1016/j.cell.2017.05.046 · Cited by 2,490
Comprehensive Molecular Characterization of Muscle-Invasive Bladder Cancer
Robertson, A. G., Kim, J., Al-Ahmadie, H., Bellmunt, J., Guo, G., Cherniack, A. D., Hinoue, T., Laird, P. W., Hoadley, K. A., Akbani, R., et al. (2017) · Cell · doi:10.1016/j.cell.2017.09.007 · Cited by 2,443
Scalable whole-exome sequencing of cell-free DNA reveals high concordance with metastatic tumors
Adalsteinsson, V. A., Ha, G., Freeman, S. S., Choudhury, A. D., Stover, D. G., Parsons, H. A., Gydush, G., Reed, S. C., Rotem, D., Rhoades, J., et al. (2017) · Nature Communications · doi:10.1038/s41467-017-00965-y · Cited by 1,051
Proteogenomic Characterization Reveals Therapeutic Vulnerabilities in Lung Adenocarcinoma
Gillette, M. A., Satpathy, S., Cao, S., Dhanasekaran, S. M., Vasaikar, S. V., Krug, K., Petralia, F., Li, Y., Liang, W.-W., Reva, B., et al. (2020) · Cell · doi:10.1016/j.cell.2020.06.013 · Cited by 820
Metadata
- Authors
- Tan, V. Y. F., & Fevotte, C.
- Venue
- IEEE Transactions on Pattern Analysis and Machine Intelligence
- Published
- 2013-07
- Volume
- 35
- Issue
- 7
- Pages
- 1592-1605
- ISSN
- 0162-8828 (ISSN-L)
- 2160-9292
- OpenAlex
- W2116542188
- Cited by
- 271
- References
- 40
- FWCI
- 12.79
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.