SwitchFlow
Xiaofeng Wu, Jia Rao, Wei Chen et al. · International Middleware Conference · 2021
Accelerators, such as GPU, are a scarce resource in deep learning (DL). Effectively and efficiently sharing GPU leads to improved hardware utilization as well as user experiences, who may need to wait for hours to access GPU before a long training job is done. Spatial and temporal multitasking on GPU have been studied in the literature, but popular deep learning frameworks, such as Tensor-Flow and PyTorch, lack the support of GPU sharing among multiple DL models, which are typically represented as computation graphs, heavily optimized by underlying DL libraries, and run on a complex pipeline spanning CPU and GPU. Our study shows that GPU kernels, spawned from computation graphs, can barely execute simultaneously on a single GPU and time slicing may lead to low GPU utilization.
Cited by 10 works. doi:10.1145/3464298.3493391 Open-access PDF
Authors
References
- Deep Residual Learning for Image Recognition
- Deep learning
- In-Datacenter Performance Analysis of a Tensor Processing Unit
- cuDNN: Efficient Primitives for Deep Learning
- Multitask Learning: A Knowledge-Based Source of Inductive Bias
- Evolving Deep Neural Networks
- vDNN: Virtualized deep neural networks for scalable, memory-efficient neural network design
- A generic communication scheduler for distributed DNN training acceleration
- Improving GPGPU concurrency with elastic kernels
- Exploiting concurrent kernel execution on graphic processing units