Efficient Bayesian optimisation using derivative meta-model

Bayesian optimisation is an efficient method for global optimisation of expensive black-box functions. However, the current Gaussian process based methods cater to functions with arbitrary smoothness, and do not explicitly model the fact that most of the real world optimisation problems are well-behaved functions with only a few peaks. In this paper, we incorporate such shape constraints through the use of a derivative meta-model. The derivative meta-model is built using a Gaussian process with a polynomial kernel and derivative samples from this meta-model are used as extra observations to the standard Bayesian optimisation procedure. We provide a Bayesian framework to infer the degree of the polynomial kernel. Experiments on both benchmark functions and hyperparameter tuning problems demonstrate the superiority of our approach over baselines.