The DOI matches its Crossref record; 9 of the 9 references it cites were found in a registry or an index.
Benchmarking GPUs to tune dense linear algebra
Abstract
We present performance results for dense linear algebra using recent NVIDIA GPUs. Our matrix-matrix multiply routine (GEMM) runs up to 60% faster than the vendor's implementation and approaches the peak of hardware capabilities. Our LU, QR and Cholesky factorizations achieve up to 80-90% of the peak GEMM rate. Our parallel LU running on two GPUs achieves up to ~540 Gflop/s. These results are accomplished by challenging the accepted view of the GPU architecture and programming guidelines. We argue that modern GPUs should be viewed as multithreaded multicore vector units. We exploit blocking similarly to vector computers and heterogeneity of the system by computing both on GPU and CPU. This study includes detailed benchmarking of the GPU memory system that reveals sizes and latencies of caches and TLB. We present a couple of algorithmic optimizations aimed at increasing parallelism and regularity in the problem that provide us with slightly higher performance.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A set of level 3 basic linear algebra subprograms
Dongarra, J. J., Du Croz, J., Hammarling, S., & Duff, I. S. (1990) · ACM Transactions on Mathematical Software · doi:10.1145/77626.79170 · Cited by 1,821
Optimization principles and application performance evaluation of a multithreaded GPU using CUDA
Ryoo, S., Rodrigues, C. I., Baghsorkhi, S. S., Stone, S. S., Kirk, D. B., & Hwu, W.-M. W. (2008) · Proceedings of the 13th ACM SIGPLAN Symposium on Principles and practice of parallel programming · doi:10.1145/1345206.1345220 · Cited by 903
Understanding the efficiency of GPU algorithms for matrix-matrix multiplication
Fatahalian, K., Sugerman, J., & Hanrahan, P. (2004) · Proceedings of the ACM SIGGRAPH/EUROGRAPHICS conference on Graphics hardware · doi:10.1145/1058129.1058148 · Cited by 309
A compiler framework for optimization of affine loop nests for gpgpus
Baskaran, M. M., Bondhugula, U., Krishnamoorthy, S., Ramanujam, J., Rountev, A., & Sadayappan, P. (2008) · Proceedings of the 22nd annual international conference on Supercomputing · doi:10.1145/1375527.1375562 · Cited by 216
LU-GPU: Efficient Algorithms for Solving Dense Linear Systems on Graphics Hardware
Galoppo, N., Govindaraju, N. K., Henson, M., & Manocha, D. (2005) · ACM/IEEE SC 2005 Conference (SC'05) · doi:10.1109/sc.2005.42 · Cited by 202
LAPACK: A portable linear algebra library for high-performance computers
Angerson, E., Sorensen, D., Bai, Z., Dongarra, J., Greenbaum, A., McKenney, A., Du Croz, J., Hammarling, S., Demmel, J., & Bischof, C. (1990) · Proceedings SUPERCOMPUTING '90 · doi:10.1109/superc.1990.129995 · Cited by 202
Self-citationMemory---A memory model for scientific algorithms on graphics processors
Govindaraju, N. K., Larsen, S., Gray, J., & Manocha, D. (2006) · Proceedings of the 2006 ACM/IEEE conference on Supercomputing - SC '06 · doi:10.1145/1188455.1188549 · Cited by 174
Efficient gather and scatter operations on graphics processors
He, B., Govindaraju, N. K., Luo, Q., et al. (2007) · Proceedings - Supercomputing · doi:10.1145/1362622.1362684 · Cited by 128
An adaptive blocking strategy for matrix factorizations
Bischof, C. H., & Lacroute, P. (1990) · Lecture notes in computer science · doi:10.1007/3-540-53065-7_101 · Cited by 12
Cited by
Billion-scale similarity search with GPUs
Johnson, J., Douze, M., & Jégou, H. (2017) · arXiv · doi:10.48550/arxiv.1702.08734 · Cited by 4,140
Billion-scale similarity search with GPUs
Johnson, J., Douze, M., & Jegou, H. (2021) · IEEE Transactions on Big Data · doi:10.1109/tbdata.2019.2921572 · Cited by 4,228
StarPU: a unified platform for task scheduling on heterogeneous multicore architectures
Augonnet, C., Thibault, S., Namyst, R., & Wacrenier, P. (2011) · Concurrency and Computation: Practice and Experience · doi:10.1002/cpe.1631 · Cited by 1,275
Towards dense linear algebra for hybrid GPU accelerated manycore systems
Tomov, S., Dongarra, J., & Baboulin, M. (2010) · Parallel Computing · doi:10.1016/j.parco.2009.12.005 · Cited by 451
Demystifying GPU microarchitecture through microbenchmarking
Wong, H., Papadopoulou, M.-M., Sadooghi-Alvandi, M., & Moshovos, A. (2010) · 2010 IEEE International Symposium on Performance Analysis of Systems & Software (ISPASS) · doi:10.1109/ispass.2010.5452013 · Cited by 439
Metadata
- Authors
- Volkov, V., & Demmel, J. W.
- Venue
- 2008 SC - International Conference for High Performance Computing, Networking, Storage and Analysis
- Publisher
- IEEE
- Published
- 2008-11
- Pages
- 1-11
- ISSN
- 2167-4337 (ISSN-L)
- OpenAlex
- W3141650078
- Cited by
- 489
- References
- 9
- FWCI
- 28.34
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Benchmarking GPUs to tune dense linear algebra. https://app.cochat.ai/papers/10.1109/sc.2008.5214359
- The paper
- Volkov, V., & Demmel, J. W. (2008). Benchmarking GPUs to tune dense linear algebra. 2008 SC - International Conference for High Performance Computing, Networking, Storage and Analysis. https://doi.org/10.1109/sc.2008.5214359
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.