Fast Monte Carlo Algorithms for Matrices III: Computing a Compressed Approximate Matrix Decomposition
The DOI matches its Crossref record; 22 of the 22 references it cites were found in a registry or an index.
Abstract
In many applications, the data consist of (or may be naturally formulated as) an $m \times n$ matrix A which may be stored on disk but which is too large to be read into random access memory (RAM) or to practically perform superlinear polynomial time computations on it. Two algorithms are presented which, when given an $m \times n$ matrix A, compute approximations to A which are the product of three smaller matrices, C, U, and R, each of which may be computed rapidly. Let $A' = CUR$ be the computed approximate decomposition; both algorithms have provable bounds for the error matrix $A-A'$. In the first algorithm, c columns of A and r rows of A are randomly chosen. If the $m \times c$ matrix C consists of those c columns of A (after appropriate rescaling) and the $r \times n$ matrix R consists of those r rows of A (also after appropriate rescaling), then the $c \times r$ matrix U may be calculated from C and R. For any matrix X, let $\|X\|_F$ and $\|X\|_2$ denote its Frobenius norm and its spectral norm, respectively. It is proven that $$ \left\|A-A'\right\|_\xi \le \min_{D:\mathrm{rank}(D)\le k} \left\|A-D\right\|_\xi + poly(k,1/c) \left\|A\right\|_F $$ holds in expectation and with high probability for both $\xi = 2,F$ and for all $k=1,\ldots,\mbox{rank}(A)$; thus by appropriate choice of k $$ \left\|A-A'\right\|_2 \le \epsilon \left\|A\right\|_F $$ also holds in expectation and with high probability. This algorithm may be implemented without storing the matrix A in RAM, provided it can make two passes over the matrix stored in external memory and use $O(m+n)$ additional RAM (assuming that c and r are constants, independent of the size of the input). The second algorithm is similar except that it approximates the matrix C by randomly sampling a constant number of rows of C. Thus, it has additional error but it can be implemented in three passes over the matrix using only constant additional RAM. To achieve an additional error (beyond the best rank-k approximation) that is at most $\epsilon \|A\|_F$, both algorithms take time which is a low-degree polynomial in k, $1/\epsilon$, and $1/\delta$, where $\delta>0$ is a failure probability; the first takes time linear in $\mbox{max}(m,n)$ and the second takes time independent of m and n. The proofs for the error bounds make important use of matrix perturbation theory and previous work on approximating matrix multiplication and computing low-rank approximations to a matrix. The probability distribution over columns and rows and the rescaling are crucial features of the algorithms and must be chosen judiciously.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Golub, G. (2012) · doi:10.56021/9781421407944 · Cited by 34,505
Horn, R. A., & Johnson, C. R. (1985) · Cited by 22,203
Horn, R. A., & Johnson, C. R. (1985) · Cambridge University Press eBooks · doi:10.1017/cbo9780511810817 · Cited by 10,280
Bhatia, R. (1996) · Cited by 2,847
(1991) · Mathematics and Computers in Simulation · doi:10.1016/0378-4754(91)90038-5 · Cited by 2,437
Bhatia, R. (1997) · Graduate Texts in Mathematics · doi:10.1007/978-1-4612-0653-8 · Cited by 2,336
Bogaevski, V. N., & Povzner, A. (1991) · Applied Mathematical Sciences · doi:10.1007/978-1-4612-4438-7_1 · Cited by 1,384
On the Nyström Method for Approximating a Gram Matrix for Improved Kernel-Based Learning
Drineas, P., & Mahoney, M. W. (2005) · Journal of Machine Learning Research · doi:10.5555/1046920.1194916 · Cited by 1,181
Self-citationFast Monte Carlo Algorithms for Matrices II: Computing a Low-Rank Approximation to a Matrix
Drineas, P., Kannan, R., & Mahoney, M. W. (2006) · SIAM Journal on Computing · doi:10.1137/s0097539704442696 · Cited by 552
Self-citationA theory of pseudoskeleton approximations
Goreinov, S. A., Tyrtyshnikov, E. E., & Замарашкин, Н. Л. (1997) · Linear Algebra and its Applications · doi:10.1016/s0024-3795(96)00301-1 · Cited by 498
Fast Monte Carlo Algorithms for Matrices I: Approximating Matrix Multiplication
Drineas, P., Kannan, R., & Mahoney, M. W. (2006) · SIAM Journal on Computing · doi:10.1137/s0097539704442684 · Cited by 487
Self-citationTwo algorithms for nearest-neighbor search in high dimensions
Kleinberg, J. M. (1997) · Proceedings of the twenty-ninth annual ACM symposium on Theory of computing - STOC '97 · doi:10.1145/258533.258653 · Cited by 348
Fast computation of low-rank matrix approximations
Achlioptas, D., & Mcsherry, F. (2007) · Journal of the ACM · doi:10.1145/1219092.1219097 · Cited by 292
Stable distributions, pseudorandom generators, embeddings and data stream computation
Indyk, P. (2002) · Proceedings 41st Annual Symposium on Foundations of Computer Science · doi:10.1109/sfcs.2000.892082 · Cited by 238
Seacrest, T., & Gu, W. (2005) · Cited by 192
Nolan, J. P. (2002) · Cited by 178
Sahu, P. K. (2013) · Research Methodology: A Guide for Researchers In Agricultural Science, Social Science and Other Related Fields · doi:10.1007/978-81-322-1020-7_12 · Cited by 169
Competitive recommendation systems
Drineas, P., Kerenidis, I., & Raghavan, P. (2002) · ACM Symposium on Theory of Computing (STOC) · doi:10.1145/509907.509922 · Cited by 149
Self-citationApproximating a Gram Matrix for Improved Kernel-Based Learning
Petros Drineas, B., & Mahoney, M. W. (2005) · Lecture notes in computer science · doi:10.1007/11503415_22 · Cited by 107
Self-citationThe complexity of massive data set computations
Bar-Yossef, Z., & Papadimitriou, C. H. (2002) · Cited by 65
Sampling lower bounds via information theory
Bar-Yossef, Z. (2003) · ACM Symposium on Theory of Computing (STOC) · doi:10.1145/780542.780593 · Cited by 41
Sampling Sub-problems of Heterogeneous Max-cut Problems and Approximation Algorithms
Petros Drineas, B., Kannan, R., & Mahoney, M. W. (2005) · Lecture notes in computer science · doi:10.1007/978-3-540-31856-9_5 · Cited by 6
Self-citation
Cited by
Halko, N., Martinsson, P. G., & Tropp, J. A. (2011) · SIAM Review · doi:10.1137/090771806 · Cited by 4,002
Rajaraman, A., & Ullman, J. D. (2011) · Cambridge University Press eBooks · doi:10.1017/cbo9781139058452 · Cited by 2,262
Introduction to the non-asymptotic analysis of random matrices
Vershynin, R. (2012) · Compressed Sensing · doi:10.1017/cbo9780511794308.006 · Cited by 2,150
Algorithms and applications for approximate nonnegative matrix factorization
Berry, M. W., Browne, M., Langville, A. N., Pauca, V. P., & Plemmons, R. J. (2007) · Computational Statistics & Data Analysis · doi:10.1016/j.csda.2006.11.006 · Cited by 1,512
Graph based anomaly detection and description: a survey
Akoglu, L., Tong, H., & Koutra, D. (2015) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-014-0365-y · Cited by 1,504
Metadata
- Authors
- Drineas, P., Kannan, R., & Mahoney, M. W.
- Venue
- SIAM Journal on Computing
- Published
- 2006-01
- Volume
- 36
- Issue
- 1
- Pages
- 184-206
- ISSN
- 0097-5397 (ISSN-L)
- 1095-7111
- OpenAlex
- W2003690406
- Cited by
- 310
- References
- 22
- FWCI
- 18.45
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.