Enabling preemptive multiprogramming on GPUs
Ivan Tanasić, Isaac Gelado, Javier Cabezas et al. · ACM SIGARCH Computer Architecture News · 2014
GPUs are being increasingly adopted as compute accelerators in many domains, spanning environments from mobile systems to cloud computing. These systems are usually running multiple applications, from one or several users. However GPUs do not provide the support for resource sharing traditionally expected in these scenarios. Thus, such systems are unable to provide key multiprogrammed workload requirements, such as responsiveness, fairness or quality of service. In this paper, we propose a set of hardware extensions that allow GPUs to efficiently support multiprogrammed GPU workloads. We argue for preemptive multitasking and design two preemption mechanisms that can be used to implement GPU scheduling policies. We extend the architecture to allow concurrent execution of GPU kernels from different user processes and implement a scheduling policy that dynamically distributes the GPU cores among concurrently running kernels, according to their priorities. We extend the NVIDIA GK110 (Kepler) like GPU architecture with our proposals and evaluate them on a set of multiprogrammed workloads with up to eight concurrent processes. Our proposals improve execution time of high-priority processes by 15.6x, the average application turnaround time between 1.5x to 2x, and system fairness up to 3.4x
Cited by 168 works. doi:10.1145/2678373.2665702 Open-access PDF
Authors
References
- NVIDIA Tesla: A Unified Graphics and Computing Architecture
- GPU Computing
- GPU Computing
- Dynamic Warp Formation and Scheduling for Efficient GPU Control Flow
- Implementation of precise interrupts in pipelined processors
- Microarchitectural Design Space Exploration Using an Architecture-Centric Approach
- MCUDA: An Efficient Implementation of CUDA Kernels for Multi-core CPUs
- Understanding the efficiency of ray traversal on GPUs
- System-Level Performance Metrics for Multiprogram Workloads
- TimeGraph: GPU scheduling for real-time multi-tasking environments