Kernel-Based LMI Approaches to Solving the Hamilton-Jacobi-Bellman Equation and Nonlinear Optimal Control
Kernel-based LMI approach for solving nonlinear HJB equations, integrating Riccati-Hessian equality for accurate approximation.
Key Findings
Methodology
The approach employs RKHS to represent the value function gradient, transforming the nonlinear HJB inequality into a convex LMI via Schur complement. A key innovation is imposing the Riccati-Hessian equality at equilibrium, aligning the Hessian with the linearized system's algebraic Riccati solution, thus removing trivial solutions. The method parameterizes the value function using kernel centers, solving a convex SDP that enforces boundary, gradient, Hessian, and stability constraints. This framework guarantees convergence and provides error bounds, combining kernel approximation theory with control-theoretic insights.
Key Results
- In a corrected 1D polynomial benchmark, the method reconstructed V* with an error of 3×10^{-7}, achieving near-perfect approximation and 0% suboptimality.
- On the Van der Pol oscillator, it attained the smallest HJB residual (ε≈2.62), outperformed LQR across all initial conditions, and kept the closed-loop cost within 0.42% of the optimal.
- When V* is outside the RKHS, the residual saturates but the suboptimality remains bounded within 13%, demonstrating robustness and graceful degradation.
Significance
This work advances the numerical solution of high-dimensional nonlinear optimal control problems by integrating kernel methods with convex optimization. It addresses the curse of dimensionality and provides theoretical guarantees for stability and approximation accuracy. The explicit Riccati-Hessian constraint ensures the local behavior matches linear theory, enabling reliable control synthesis. The approach bridges the gap between data-driven learning and classical control, opening pathways for scalable, interpretable, and provably stable control design in complex systems.
Technical Contribution
The key contribution is embedding the Riccati-Hessian equality as a linear constraint within a kernel-based SDP framework, ensuring the Hessian of the approximation aligns with the linearized system's Riccati solution. This enforces local quadratic behavior and eliminates trivial solutions, providing a unique, well-posed convex problem. Theoretical analysis yields suboptimality bounds and convergence rates, while numerical experiments validate the approach on benchmark systems. The method's interpretability, stability guarantees, and scalability mark a significant step forward in nonlinear control theory.
Novelty
This is the first work to incorporate the Riccati-Hessian equality explicitly as a linear constraint in a kernel-based convex formulation for HJB equations. Unlike prior approaches relying solely on approximation or sum-of-squares methods, this technique ensures local quadratic accuracy and stability, even when the value function lies outside the RKHS. It combines classical control insights with modern convex optimization and kernel approximation, offering a new paradigm for scalable, provably stable nonlinear control.
Limitations
- When the true value function is not in the chosen RKHS, residuals saturate, limiting global approximation accuracy. Kernel selection becomes critical, and poor choices degrade performance.
- Computational complexity grows with the number of kernel centers and system dimension, posing challenges for very high-dimensional systems.
- The method assumes a known equilibrium point and local linearization; extending to systems with multiple equilibria or strong nonlinearities away from equilibrium remains an open challenge.
Future Work
Future directions include developing multi-kernel and deep kernel approaches to enhance approximation capacity, extending the framework to multi-equilibrium and non-stationary systems, and integrating with real-time control schemes like MPC. Improving scalability through distributed algorithms and exploring adaptive kernel selection strategies will further broaden applicability. Additionally, combining this approach with data-driven system identification could enable fully model-free control solutions for complex, uncertain environments.
AI Executive Summary
This paper introduces a novel kernel-based linear matrix inequality (LMI) framework for approximating solutions to the Hamilton-Jacobi-Bellman (HJB) equation in nonlinear optimal control. Traditional methods struggle with high-dimensional systems due to the curse of dimensionality, limiting their practical utility. The proposed approach leverages reproducing kernel Hilbert spaces (RKHS) to represent the value function's gradient, transforming the nonlinear HJB inequality into a convex LMI via Schur complement. A key innovation is the explicit imposition of the Riccati-Hessian equality at the equilibrium point, aligning the Hessian of the approximation with the algebraic Riccati solution of the linearized system. This constraint effectively removes trivial solutions and ensures the local quadratic behavior of the value function, which is critical for stability and accuracy. Numerical experiments on a corrected 1D polynomial benchmark demonstrate near-perfect reconstruction of the true value function, with errors as low as 3×10^{-7} and suboptimality approaching zero. On the Van der Pol oscillator, the method achieves the smallest HJB residual among tested methods and outperforms LQR across all initial conditions, with the closed-loop cost within 0.42% of the optimal. When the true value function lies outside the chosen RKHS, the residual saturates but the suboptimality remains bounded within 13%, confirming robustness. The approach combines classical control theory with modern convex optimization and kernel approximation, offering a scalable, interpretable, and theoretically guaranteed solution for complex nonlinear systems. Future work will focus on multi-kernel fusion, real-time implementation, and extension to systems with multiple equilibria, aiming to make high-dimensional nonlinear control both practical and provably stable.
Deep Dive
Abstract
We present a kernel-based linear matrix inequality (LMI) approach for the approximate solution of Hamilton--Jacobi--Bellman (HJB) equations arising in nonlinear optimal control. The method represents the gradient of the value function in a reproducing kernel Hilbert space (RKHS) and uses a Schur-complement reformulation to convert the quadratic HJB inequality into an LMI that is linear in the kernel coefficients, yielding a convex semidefinite program. The novel ingredient is an explicit Riccati--Hessian \emph{equality} constraint at the equilibrium, which removes the trivial solution and forces the Hessian of the approximation to match the algebraic Riccati equation solution of the linearised system. We give a suboptimality bound $J(x_0;\hat u) - V^*(x_0)\le \varepsilon\,T(x_0)$ in which $T(x_0)$ depends only on the problem data and the working domain (not on the approximation), and an RKHS approximation rate. Numerical experiments on a corrected 1D polynomial benchmark and on the Van der Pol oscillator measure $\varepsilon$, the RKHS approximation error, and the closed-loop cost $J(x_0;\hat u)$ versus the optimal value $V^*(x_0)$. On the 1D problem with $V^*$ in the polynomial-kernel RKHS the method recovers $V^*$ to within $3\times10^{-7}$ and achieves $0.000\%$ suboptimality. On Van der Pol it achieves the smallest HJB residual ($\varepsilon\approx 2.62$) of any method tested, beats LQR on every initial condition, and is within $0.42\%$ of the best per-IC cost (Albrekht order 6). When $V^*$ is not in the chosen RKHS, the method degrades gracefully: residuals stop improving with more centres but suboptimality remains bounded ($\le 13\%$ on the 1D test).