tf.data service
Andrew Audibert, Yang Chen, Dan Graur et al. · ACM Symposium on Cloud Computing (SoCC) · 2023
Machine learning (ML) computations commonly execute on expensive specialized hardware, such as GPUs and TPUs, which provide high FLOPs and performance-per-watt. For cost efficiency, it is essential to keep these accelerators highly utilized. This requires preprocessing input data at the rate at which the accelerators can ingest and perform ML computations on the data. To avoid data stalls, the host CPU and RAM required for input data processing per accelerator core used for ML computations varies across jobs. Hence, the traditional approach of processing input data on ML accelerator hosts with a fixed hardware ratio leads to either under-utilizing the accelerators or the host CPU and RAM. In this paper, we address these concerns by building a disaggregated ML data processing system.
Cited by 12 works. doi:10.1145/3620678.3624666 Open-access PDF
Authors
References
- Deep Residual Learning for Image Recognition
- Focal Loss for Dense Object Detection
- Advances in neural information processing systems 7
- A survey on Image Data Augmentation for Deep Learning
- Large-Scale Machine Learning with Stochastic Gradient Descent
- In-Datacenter Performance Analysis of a Tensor Processing Unit
- Large-scale cluster management at Google with Borg
- Text Data Augmentation for Deep Learning
- Logic-based approach to semantic query optimization
- Scaling Distributed Machine Learning with the Parameter Server