main_postprint.pdf (1010.12 kB)
Bayesian optimisation in unknown bounded search domains
Version 3 2023-11-09, 06:02
Version 2 2023-11-09, 04:43
Version 1 2020-03-16, 08:24
journal contribution
posted on 2023-11-09, 06:02 authored by J Berk, S Gupta, S Rana, V Nguyen, S VenkateshBayesian optimisation (BO) is one of the most sample efficient methods for determining the optima of expensive, noisy black-box functions. Despite its tremendous success in scientific discovery and hyperparameter tuning, it still requires a bounded search space. The search spaces boundaries are, however, often chosen heuristically with an educated guess. If the boundaries are misspecified, then the search space may either be unnecessarily large and hence more expensive to optimise, or it may simply not contain the global optimum. In this paper, we introduce a method for dynamically determining the bound directly from the data. This is done using a distribution of the bound derived in a Bayesian setting. The prior is chosen by the user and the likelihood is derived with Thompson sampling. This results in a bound that is both cheap to optimise and has a high probability of containing the global optimum. We compare the performance of our method with the alternative methods on a range of synthetic and real-world problems and demonstrate that our method achieves consistently superior results.
History
Journal
Knowledge-Based SystemsVolume
195Article number
ARTN 105645Pagination
1 - 9Location
Amsterdam, The NetherlandsPublisher DOI
Open access
- Yes
ISSN
0950-7051eISSN
1872-7409Language
EnglishPublication classification
C1 Refereed article in a scholarly journalPublisher
ELSEVIERUsage metrics
Categories
No categories selectedKeywords
Science & TechnologyTechnologyComputer Science, Artificial IntelligenceComputer ScienceBayesian optimisationExperimental designHyperparameter tuningGLOBAL OPTIMIZATIONFL1701000064602 Artificial intelligenceInformation and Computing SciencesCommerce, Management, Tourism and ServicesPsychology and Cognitive Sciences
Licence
Exports
RefWorks
BibTeX
Ref. manager
Endnote
DataCite
NLM
DC