Implementing sparse matrix-vector multiplication on throughput-oriented processors
Nathan Bell, Michael Garland · 2009
Sparse matrix-vector multiplication (SpMV) is of singular importance in sparse linear algebra. In contrast to the uniform regularity of dense linear algebra, sparse operations encounter a broad spectrum of matrices ranging from the regular to the highly irregular. Harnessing the tremendous potential of throughput-oriented processors for sparse operations requires that we expose substantial fine-grained parallelism and impose sufficient regularity on execution paths and memory access patterns. We explore SpMV methods that are well-suited to throughput-oriented architectures like the GPU and which exploit several common sparsity classes. The techniques we propose are efficient, successfully utilizing large percentages of peak bandwidth. Furthermore, they deliver excellent total throughput, averaging 16 GFLOP/s and 10 GFLOP/s in double precision for structured grid and unstructured mesh matrices, respectively, on a GeForce GTX 285. This is roughly 2.8 times the throughput previously achieved on Cell BE and more than 10 times that of a quad-core Intel Clovertown system.
Cited by 900 works. doi:10.1145/1654059.1654078
Authors
References
- Iterative Methods for Sparse Linear Systems
- Basic Linear Algebra Subprograms for Fortran Usage
- LAPACK Users' Guide
- Scalable parallel programming with CUDA
- Scalable Parallel Programming with CUDA
- NVIDIA Tesla: A Unified Graphics and Computing Architecture
- Benchmarking GPUs to tune dense linear algebra
- Optimization of sparse matrix-vector multiplication on emerging multicore platforms
- Ecient Sparse Matrix-Vector Multiplication on CUDA
- Sparsity: Optimization Framework for Sparse Matrix Kernels