Hyperopt: a Python library for model selection and hyperparameter optimization
James Bergstra, Brent Komer, Chris Eliasmith et al. · Computational Science & Discovery · 2015
Sequential model-based optimization (also known as Bayesian optimization) is one of the most efficient methods (per function evaluation) of function minimization. This efficiency makes it appropriate for optimizing the hyperparameters of machine learning algorithms that are slow to train. The Hyperopt library provides algorithms and parallelization infrastructure for performing hyperparameter optimization (model selection) in Python. This paper presents an introductory tutorial on the usage of the Hyperopt library, including the description of search spaces, minimization (in serial and parallel), and the analysis of the results collected in the course of minimization. This paper also gives an overview of Hyperopt-Sklearn, a software project that provides automatic algorithm configuration of the Scikit-learn machine learning library. Following Auto-Weka, we take the view that the choice of classifier and even the choice of preprocessing module can be taken together to represent a single large hyperparameter optimization problem . We use Hyperopt to define a search space that encompasses many standard components (e.g. SVM, RF, KNN, PCA, TFIDF) and common patterns of composing them together. We demonstrate, using search algorithms in Hyperopt and standard benchmarking data sets (MNIST, 20-newsgroups, convex shapes), that searching this space is practical and effective. In particular, we improve on best-known scores for the model space for both MNIST and convex shapes. The paper closes with some discussion of ongoing and future work.
Cited by 1,010 works. doi:10.1088/1749-4699/8/1/014008 Open-access PDF
Authors
References
- Gradient-based learning applied to document recognition
- Learning Multiple Layers of Features from Tiny Images
- Scikit-learn: Machine Learning in Python
- The Elements of Statistical Learning: Data Mining, Inference, and Prediction
- The WEKA data mining software
- Labeled Faces in the Wild: A Database forStudying Face Recognition in Unconstrained Environments
- Multi-column deep neural networks for image classification
- Practical Bayesian Optimization of Machine Learning Algorithms
- Random search for hyper-parameter optimization
- An empirical evaluation of deep architectures on problems with many factors of variation