KBS_main-2.pdf (3.04 MB)
Incorporating expert prior in Bayesian optimisation via space warping
Version 3 2024-02-09, 05:29
Version 2 2024-01-19, 04:25
Version 1 2020-03-16, 08:36
journal contribution
posted on 2024-02-09, 05:29 authored by A Ramachandran, S Gupta, S Rana, C Li, S VenkateshBayesian optimisation is a well-known sample-efficient method for the optimisation of expensive black-box functions. However when dealing with big search spaces the algorithm goes through several low function value regions before reaching the optimum of the function. Since the function evaluations are expensive in terms of both money and time, it may be desirable to alleviate this problem. One approach to subside this cold start phase is to use prior knowledge that can accelerate the optimisation. In its standard form, Bayesian optimisation assumes the likelihood of any point in the search space being the optimum is equal. Therefore any prior knowledge that can provide information about the optimum of the function would elevate the optimisation performance. In this paper, we represent the prior knowledge about the function optimum through a prior distribution. The prior distribution is then used to warp the search space in such a way that space gets expanded around the high probability region of function optimum and shrinks around low probability region of optimum. We incorporate this prior directly in function model (Gaussian process), by redefining the kernel matrix, which allows this method to work with any acquisition function, i.e. acquisition agnostic approach. We show the superiority of our method over standard Bayesian optimisation method through optimisation of several benchmark functions and hyperparameter tuning of two algorithms: Support Vector Machine (SVM) and Random forest.
History
Journal
Knowledge-Based SystemsVolume
195Article number
ARTN 105663Pagination
1 - 11Location
Amsterdam, The NetherlandsPublisher DOI
Open access
- Yes
ISSN
0950-7051eISSN
1872-7409Language
EnglishPublication classification
C1 Refereed article in a scholarly journalPublisher
ELSEVIERUsage metrics
Categories
No categories selectedKeywords
Science & TechnologyTechnologyComputer Science, Artificial IntelligenceComputer ScienceBayesian optimisationGaussian processBlack-box functionProbability integrity transformSpace warping4602 Artificial intelligenceInformation and Computing SciencesCommerce, Management, Tourism and ServicesPsychology and Cognitive Sciences
Licence
Exports
RefWorks
BibTeX
Ref. manager
Endnote
DataCite
NLM
DC