Analyzing CUDA workloads using a detailed GPU simulator
Ali Bakhoda, George L. Yuan, Wilson Wai Lun Fung et al. · IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS) · 2009
Modern Graphic Processing Units (GPUs) provide sufficiently flexible programming models that understanding their performance can provide insight in designing tomorrow's manycore processors, whether those are GPUs or otherwise. The combination of multiple, multithreaded, SIMD cores makes studying these GPUs useful in understanding tradeoffs among memory, data, and thread level parallelism. While modern GPUs offer orders of magnitude more raw computing power than contemporary CPUs, many important applications, even those with abundant data level parallelism, do not achieve peak performance. This paper characterizes several non-graphics applications written in NVIDIA's CUDA programming model by running them on a novel detailed microarchitecture performance simulator that runs NVIDIA's parallel thread execution (PTX) virtual instruction set. For this study, we selected twelve non-trivial CUDA applications demonstrating varying levels of performance improvement on GPU hardware (versus a CPU-only sequential version of the application). We study the performance of these applications on our GPU performance simulator with configurations comparable to contemporary high-end graphics cards. We characterize the performance impact of several microarchitecture design choices including choice of interconnect topology, use of caches, design of memory controller, parallel workload distribution mechanisms, and memory request coalescing hardware. Two observations we make are (1) that for the applications we study, performance is more sensitive to interconnect bisection bandwidth rather than latency, and (2) that, for some applications, running fewer threads concurrently than on-chip resources might otherwise allow can improve performance by reducing contention in the memory system.
Cited by 1,663 works. doi:10.1109/ispass.2009.4919648
Authors
References
- The design and implementation of a first-generation CELL processor
- Scalable parallel programming with CUDA
- Scalable Parallel Programming with CUDA
- NVIDIA Tesla: A Unified Graphics and Computing Architecture
- Lockup-free instruction fetch/prefetch cache organization
- Optimization principles and application performance evaluation of a multithreaded GPU using CUDA
- Memory access scheduling
- The microarchitecture of the Pentium 4 processor
- Accelerating Large Graph Algorithms on the GPU Using CUDA
- Dynamic Warp Formation and Scheduling for Efficient GPU Control Flow