Interactive Bayesian optimization : learning user preferences for graphics and animation
Eric Brochu · cIRcle (University of British Columbia) · 2011
Bayesian optimization with Gaussian processes has become increasingly popular in the machine learning community. It is efficient and can be used when very little is known about the objective function, making it useful for optimizing expensive black box functions. We examine the case of using Bayesian optimization when the objective function requires feedback from a human. We call this class of problems \emph{interactive} Bayesian optimization. Here, we assume a parameterized model, and a user whose task is to find an acceptable set of parameters according to some perceptual value function that cannot easily be articulated. This requires special attention to the qualities that make this a unique problem, and so, we introduce three novel extensions: the application of Bayesian optimization to "preference galleries", where human feedback is in the form of preferences over a set of instances; a particle-filter method for learning the distribution of model hyperparameters over heterogeneous users and tasks; and a bandit-based method of using a portfolio of utility functions to select sample points. Using a variety of test functions, we validate our extensions empirically on both low- and high-dimensional objective functions. We also present graphics and animation applications that use interactive Bayesian optimization techniques to help artists find parameters on difficult problems. We show that even with minimal domain knowledge, an interface using interactive Bayesian optimization is much more efficient and effective than traditional "parameter twiddling" techniques on the same problem.
Cited by 12 works. doi:10.14288/1.0051462
Authors
References
- Nonparametric Statistics for the Behavioral Sciences
- Reducing the Dimensionality of Data with Neural Networks
- The Elements of Statistical Learning
- Reinforcement Learning: An Introduction
- Rank Correlation Methods.
- A Practical Bayesian Framework for Backpropagation Networks
- Advances in prospect theory: Cumulative representation of uncertainty
- Design and Analysis of Computer Experiments
- Discrete Choice Methods with Simulation
- Efficient Global Optimization of Expensive Black-Box Functions