Bao: Making Learned Query Optimization Practical
Ryan Marcus, Parimarjan Negi, Hongzi Mao et al. · Proceedings - ACM-SIGMOD International Conference on Management of Data · 2021
Recent efforts applying machine learning techniques to query optimization have shown few practical gains due to substantive training overhead, inability to adapt to changes, and poor tail performance. Motivated by these difficulties, we introduce Bao (the \underlineBa ndit \underlineo ptimizer). Bao takes advantage of the wisdom built into existing query optimizers by providing per-query optimization hints. Bao combines modern tree convolutional neural networks with Thompson sampling, a well-studied reinforcement learning algorithm. As a result, Bao automatically learns from its mistakes and adapts to changes in query workloads, data, and schema. Experimentally, we demonstrate that Bao can quickly learn strategies that improve end-to-end query execution performance, including tail latency, for several workloads containing long-running queries. In cloud environments, we show that Bao can offer both reduced costs and better performance compared with a commercial system.
Cited by 241 works. doi:10.1145/3448016.3452838 Open-access PDF
Authors
References
- AutoResearch Lite: a reproducible paper-inspired experiment runner (Adam mechanism reproduction and convergence-speed comparison)
- Bagging Predictors
- Human-level control through deep reinforcement learning
- Access path selection in a relational database management system
- Deep Sparse Rectifier Neural Networks
- International Conference on Learning Representations (ICLR 2013)
- ON THE LIKELIHOOD THAT ONE UNKNOWN PROBABILITY EXCEEDS ANOTHER IN VIEW OF THE EVIDENCE OF TWO SAMPLES
- Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI)
- Deep Reinforcement Learning: A Brief Survey
- The tail at scale