File(s) under permanent embargo
Budgeted batch Bayesian optimization
conference contribution
posted on 2016-01-01, 00:00 authored by Tien Vu Nguyen, Santu RanaSantu Rana, Sunil GuptaSunil Gupta, Cheng Li, Svetha VenkateshSvetha VenkateshParameter settings profoundly impact the performance of machine learning algorithms and laboratory experiments. The classical trial-error methods are exponentially expensive in large parameter spaces, and Bayesian optimization (BO) offers an elegant alternative for global optimization of black box functions. In situations where the functions can be evaluated at multiple points simultaneously, batch Bayesian optimization is used. Current batch BO approaches are restrictive in fixing the number of evaluations per batch, and this can be wasteful when the number of specified evaluations is larger than the number of real maxima in the underlying acquisition function. We present the budgeted batch Bayesian optimization (B3O) for hyper-parameter tuning and experimental design - we identify the appropriate batch size for each iteration in an elegant way. In particular, we use the infinite Gaussian mixture model (IGMM) for automatically identifying the number of peaks in the underlying acquisition functions. We solve the intractability of estimating the IGMM directly from the acquisition function by formulating the batch generalized slice sampling to efficiently draw samples from the acquisition function. We perform extensive experiments for benchmark functions and two real world applications - machine learning hyper-parameter tuning and experimental design for alloy hardening. We show empirically that the proposed B3O outperforms the existing fixed batch BO approaches in finding the optimum whilst requiring a fewer number of evaluations, thus saving cost and time.
History
Event
IEEE Data Mining. International Conference (16th : 2016 : Barcelona, Spain)Pagination
1107 - 1112Publisher
IEEELocation
Barcelona, SpainPlace of publication
Piscataway, N.J.Publisher DOI
Start date
2016-12-12End date
2016-12-15ISSN
1550-4786ISBN-13
9781509054725Language
engNotes
DOI updated from 10.1109/ICDM.2016.52 (on paper) to 10.1109/ICDM.2016.0144 (on website)Publication classification
E Conference publication; E1 Full written paper - refereedCopyright notice
2016, IEEETitle of proceedings
ICDM 2016: Proceedings of the 16th IEEE International Conference on Data MiningUsage metrics
Categories
No categories selectedKeywords
Licence
Exports
RefWorks
BibTeX
Ref. manager
Endnote
DataCite
NLM
DC