[Bayesian Optimisation over Multiple Continuous and Categorical Inputs]
(B. Ru et al., ICML-2020)
Multi-armed bandits and Bayesian optimization are closely related problems.
In general, Bayesian optimization can be seen as an infinite-bandit extension with dependent arms. While Bayesian optimization with Gaussian process surrogate models assume real-valued inputs, we often face problems where both continuous and categorical inputs exist together.
This paper proposes a new approach, Continuous and Categorical Bayesian Optimization (CoCaBO), which combines the strengths of multi-armed bandits and Bayesian optimization to select values for both categorical and continuous inputs. It models this mixed-type space using a Gaussian process kernel, designed to allow sharing of information across multiple categorical variables, each with multiple possible values; this allows CoCaBO to leverage all available data efficiently. It is empirically demonstrated that the method outperforms existing approaches on both synthetic and real-world problems.
1. Juho Lee, Yoonho Lee, Jungtaek Kim, Adam R. Kosiorek, Seungjin Choi, Yee Whye Teh (2019),
Proceedings of the Thirty-Sixth International Conference on Machine Learning (ICML-2019),
Long Beach, California, USA, June 9-15, 2019.
(earlier version in preprint arXiv:1810.00825 )
2. Juho Lee, Lancelot James, Seungjin Choi, and François Caron (2019),
Proceedings of the 22nd International Conference on Artificial Intelligence and Statistics (AISTATS-2019),
Naha, Okinawa, Japan, April 16-18, 2019. (oral)
(earlier version in preprint arXiv:1810.01778 )
3. Yoonho Lee and Seungjin Choi (2018),
in Proceedings of the Thirty-Fifth International Conference on Machine Learning (ICML-2018),
Stockholm, Sweden, July 10-15, 2018.
(earlier version in preprint arXiv:1810.05558 )
4. Saehoon Kim, Jungtaek Kim, and Seungjin Choi (2018),
in Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence (AAAI-2018),
New Orleans, Louisiana, USA, February 2-7, 2018.
5. Juho Lee, Creighton Heaukulani, Zoubin Ghahramani, Lancelot James, and Seungjin Choi (2017),
in Proceedings of the International Conference on Machine Learning (ICML-2017),
Sydney, Australia, August 6-11, 2017.
(earlier version in preprint arXiv:1702.08239 )
6. Saehoon Kim and Seungjin Choi (2017),
in Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence (AAAI-2017),
San Francisco, California, USA, February 4-9, 2017.