Hyperparameter tuning for big data using Bayesian optimisation
Tinu Theckel Joy, Santu Rana, Sunil Gupta et al. · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · 2016
Hyperparameters play a crucial role in the model selection of machine learning algorithms. Tuning these hyperparameters can be exhaustive when the data is large. Bayesian optimisation has emerged as an efficient tool for hyperparameter tuning of machine learning algorithms. In this paper, we propose a novel framework for tuning the hyperparameters for big data using Bayesian optimisation. We divide the big data into chunks and generate hyperparameter configurations for the chunks using the standard Bayesian optimisation. We utilise this information from the chunks for hyperparameter tuning on big data using a transfer learning setting. We evaluate the performance of the proposed method on the task of tuning hyperparameters of two machine learning algorithms. We show that our method achieves the best available hyperparameter configuration within less computational time compared to the state-of-art hyperparameter tuning methods.
Cited by 102 works. doi:10.1109/icpr.2016.7900023
Authors
References
- Gradient-based learning applied to document recognition
- LIBSVM
- Libsvm : A library for support vector machines
- Practical Bayesian Optimization of Machine Learning Algorithms
- Gaussian Processes for Machine Learning
- Auto-WEKA
- Gaussian Processes in Machine Learning
- A Tutorial on Bayesian Optimization of Expensive Cost Functions, with Application to Active User Modeling and Hierarchical Reinforcement Learning
- Information-Theoretic Regret Bounds for Gaussian Process Optimization in the Bandit Setting
- A New Method of Locating the Maximum Point of an Arbitrary Multipeak Curve in the Presence of Noise