Stein Boltzmann Sampling: A Variational Approach for Global Optimization

In this paper, we introduce a new flow-based method for global optimization of continuous Sobolev functions, called Stein Boltzmann Sampling (SBS). Our method samples from the Boltzmann distribution, as it becomes asymptotically supported over the set of the minimizers of the function to be optimized. Candidate solutions are sampled via the Stein Variational Gradient Descent (SVGD) algorithm. We prove the asymptotic convergence of our method by introducing a novel framework of the SVGD theory, suitable for global optimization, that allows to address more general target distributions over a compact subset of $\mathbb{R}^d$. We present two SBS variants and provide a detailed comparison with several state-of-the-art global optimization algorithms on various benchmark functions, showing that SBS and its variants are highly competitive. Its design of our method, the theoretical results, and the experiments suggest that SBS is particularly well-suited to be used as a continuation for particles or distribution-based methods, conjointly with particles filtering strategies, to produce sharp approximations while making a good use of the budget.