Global Optimization with Parametric Function Approximation

This paper introduces GO-UCB, a novel algorithm for high-dimensional global optimization leveraging parametric models such as neural networks. Traditional Bayesian optimization methods, especially those based on Gaussian processes, struggle with the curse of dimensionality, limiting their effectiveness in complex, large-scale problems. In contrast, GO-UCB employs a two-phase approach: initial uniform exploration to estimate model parameters, followed by an optimistic exploitation phase that constructs a confidence set over parameters based on gradient information. This confidence set, termed Ballt, ensures the true parameter remains within it with high probability, enabling the algorithm to perform targeted, optimistic exploration. The core technical innovation lies in adapting linear bandit techniques—such as self-normalized martingale concentration and elliptical potential lemmas—to the non-linear, neural network setting through local strong convexity and growth conditions. Theoretical analysis demonstrates that, under mild assumptions, GO-UCB achieves a cumulative regret of \~O(√T), independent of input dimension, outperforming existing Gaussian process-based methods especially in high-dimensional spaces. Empirical results on synthetic benchmarks, hyperparameter tuning, and material design validate the method's robustness and efficiency, even under model misspecification. This work significantly advances the field of scalable black-box optimization, providing a practical and theoretically grounded framework for tackling complex, high-dimensional problems across scientific and industrial domains. Future research will focus on relaxing assumptions, improving computational efficiency, and extending to multi-objective and constrained settings, promising broad impacts in machine learning, materials science, and beyond.