A Learned Performance Model for Tensor Processing Units
Samuel J. Kaufman, Phitchaya Mangpo Phothilimthana, Yanqi Zhou et al. · arXiv (Cornell University) · 2020
Accurate hardware performance models are critical to efficient code generation. They can be used by compilers to make heuristic decisions, by superoptimizers as a minimization objective, or by autotuners to find an optimal configuration for a specific program. However, they are difficult to develop because contemporary processors are complex, and the recent proliferation of deep learning accelerators has increased the development burden. We demonstrate a method of learning performance models from a corpus of tensor computation graph programs for Tensor Processing Unit (TPU) instances. We show that our learned model outperforms a heavily-optimized analytical performance model on two tasks -- tile-size selection and operator fusion -- and that it helps an autotuner discover faster programs in a setting where access to TPUs is limited or expensive.
Cited by 39 works. doi:10.48550/arxiv.2008.01040 Open-access PDF
Authors
- Samuel J. Kaufman
- Phitchaya Mangpo Phothilimthana
- Yanqi Zhou
- Charith Mendis
- Sudip Roy
- Amit Sabne
- Mike Burrows
References
- Long Short-Term Memory
- Attention Is All You Need
- Learning to rank using gradient descent
- Inductive Representation Learning on Large Graphs
- PipeDream
- Graph Transformer Networks
- Learning to optimize halide with tree search and random programs
- Learning to Optimize Tensor Programs
- End-to-End Deep Learning of Optimization Heuristics
- Neural Predictor for Neural Architecture Search