High Dimensional Bayesian Optimization using Dropout
Cheng Li, Sunil Gupta, Santu Rana et al. · International Joint Conference on Artificial Intelligence · 2017
Scaling Bayesian optimization to high dimensions is challenging task as the global optimization of high-dimensional acquisition function can be expensive and often infeasible. Existing methods depend either on limited “active” variables or the additive form of the objective function. We propose a new method for high-dimensional Bayesian optimization, that uses a drop-out strategy to optimize only a subset of variables at each iteration. We derive theoretical bounds for the regret and show how it can inform the derivation of our algorithm. We demonstrate the efficacy of our algorithms for optimization on two benchmark functions and two real-world applications - training cascade classifiers and optimizing alloy composition.
Cited by 108 works. doi:10.24963/ijcai.2017/291 Open-access PDF
Authors
References
- Dropout: a simple way to prevent neural networks from overfitting
- Rapid object detection using a boosted cascade of simple features
- Practical Bayesian Optimization of Machine Learning Algorithms
- Lipschitzian optimization without the Lipschitz constant
- Efficiency of Coordinate Descent Methods on Huge-Scale Optimization Problems
- Accelerated search for materials with targeted properties by adaptive design
- Bayesian optimization in high dimensions via random embeddings
- Statistical Approximations for Multidisciplinary Design Optimization: The Problem of Size
- High Dimensional Bayesian Optimisation and Bandits via Additive Models
- High-Dimensional Gaussian Process Bandits