Finite-Particle Convergence Rates for Conservative and Non-Conservative Drifting Models

Generative modeling has witnessed significant advances in recent years, particularly with diffusion and score-based models demonstrating impressive capabilities in generating high-fidelity images, audio, and other modalities. These models typically rely on multi-step sampling procedures, which, while effective, incur high computational costs and latency during inference. To address these challenges, Deng et al. (2026) introduced drifting models, which shift the computational burden to training by directly moving the model distribution via a drift velocity field, enabling one-step generation at inference. The core idea is to design a drift velocity that attracts particles toward the data distribution while repelling them from the model distribution, thereby evolving the particle cloud toward the target.

However, the original displacement-based drifting velocities often lack a conservative (gradient) structure, complicating theoretical analysis and potentially causing instability in finite-particle systems. This non-conservatism arises from position-dependent normalization factors inherent in the velocity definition, except in special cases like Gaussian kernels. Recent efforts have sought to restore conservatism by introducing sharp normalizations or kernel gradient constructions, but a comprehensive finite-particle convergence analysis remained elusive. This paper builds upon and extends these foundational works by rigorously analyzing KDE-gradient conservative drifting velocities and their finite-particle behavior.

The central problem addressed is the non-conservatism of displacement-based drifting velocities in one-step generative models. Non-conservative velocity fields lack a potential function whose gradient they represent, leading to dynamics that are harder to analyze and potentially unstable. This issue is exacerbated by position-dependent normalization terms in the velocity, which break gradient structure except for Gaussian kernels. Consequently, finite-particle systems driven by such velocities may exhibit uncontrolled behavior, impeding theoretical guarantees on convergence and stability. The challenge is to design a drifting velocity that is both conservative and effective at guiding particles toward the data distribution, while enabling rigorous finite-particle convergence analysis and practical algorithmic implementation.

The paper introduces several key innovations:

1) Conservative KDE-gradient drifting velocity: The authors replace the displacement-based velocity with the difference between kernel-smoothed data and model scores, forming a gradient field that restores conservatism and enables stable particle dynamics.

2) Joint-entropy framework: By leveraging and extending the joint entropy approach from Stein Variational Gradient Descent (SVGD) literature, the authors derive continuous-time finite-particle convergence bounds that tightly control empirical Stein drift, smoothed Fisher discrepancy, and center velocity.

3) Reciprocal-KDE self-interaction term: The analysis identifies a novel finite-particle correction term arising from the KDE denominator’s dependence on particle positions. The authors establish deterministic and high-probability local occupancy conditions to control this term, ensuring stability.

4) Explicit bandwidth-dependent quadrature constants: The work tracks how kernel bandwidth influences quadrature errors and convergence rates, enabling optimized bandwidth selection strategies.

5) Non-conservative Laplace kernel analysis: The paper provides a companion analysis for the original displacement-based Laplace kernel drifting method, decomposing its velocity into a sharp-score mismatch and an unavoidable scale-mismatch residual, clarifying intrinsic limitations.

These innovations collectively advance the theoretical foundations and practical understanding of drifting models for one-step generative modeling.

• �� Conservative velocity construction: Define the KDE q_x(z) = (1/N) ∑_{j=1}^N K_h(z - x_j) and smoothed densities ν_h, μ_h. The conservative velocity is b_{ν,μ,h}(z) = ∇ log ν_h(z) - ∇ log μ_h(z), ensuring a gradient field.

• �� Finite-particle dynamics: Model particle evolution via ODEs ˙X_i(t) = b_x(X_i(t)), where b_x depends on the full particle configuration through KDE scores.

• �� Joint entropy and Stein divergence: Introduce joint relative entropy H_N(t) = KL(P_N(t) || ρ^{⊗N}) to measure particle distribution divergence. Define empirical Stein drift S_N and smoothed Fisher discrepancy I_N to quantify convergence.

• �� Entropy identity derivation: Prove d/dt H_N(t) = E_t[S_N(X_t)] + ΔK(0)/N E_t[R_N(X_t)], where R_N is the reciprocal KDE term, highlighting the self-interaction correction.

• �� Control of reciprocal-KDE term: Establish local occupancy conditions ensuring R_N remains bounded, preventing denominator singularities.

• �� Quadrature error analysis: Track constants A_{B,T}(h), V_{T}(h) related to kernel derivatives, showing their dependence on bandwidth h and their impact on convergence rates.

• �� Bandwidth optimization: Balance entropy initialization, self-interaction, and quadrature errors to select optimal h* minimizing residual velocity, yielding convergence rates N^{-(2-β)/(2(d+4-β))}.

• �� Non-conservative Laplace kernel analysis: Decompose velocity into sharp companion kernel components, identify scale-mismatch residual Δ_h, and derive finite-particle rates including unavoidable residual terms.

• �� One-step generation guarantee: Translate continuous-time residual velocity bounds into explicit Wasserstein distance bounds for one-step generated distributions via drift size η.

The paper is primarily theoretical and does not present empirical experiments on specific datasets. The analysis applies generally to finite-particle systems in ℝ^d with arbitrary dimension d, kernel functions K_h, and bandwidth h. The authors compare Gaussian and Laplace kernels theoretically, demonstrating how kernel smoothness affects conservatism and convergence. Assumptions on local occupancy and kernel smoothness guide practical algorithm design. Although no numerical simulations or benchmark datasets are reported, the theoretical results provide foundational insights for future empirical validation and algorithm development.

The authors establish a fundamental finite-particle entropy identity linking the time derivative of joint relative entropy to empirical Stein drift and a reciprocal-KDE self-interaction term. Under suitable regularity and occupancy assumptions, they prove that the time-averaged empirical Stein drift is bounded by the initial entropy scaled by 1/(N T) plus a term proportional to 1/N, quantifying convergence speed. By relating empirical Stein drift to the smoothed Fisher discrepancy and squared center velocity via quadrature bounds, they derive explicit convergence rates. With bandwidth h chosen optimally, the root residual velocity converges at rate N^{-1/(d+4)} under uniform quadrature regularity, and more generally at N^{-(2-β)/(2(d+4-β))} where β reflects kernel score field regularity. For the non-conservative Laplace kernel drifting method, velocity decomposition reveals an unavoidable scale-mismatch residual Δ_h, which imposes a lower bound on convergence rates. These results rigorously quantify the trade-offs in kernel choice, bandwidth selection, and finite-particle effects in drifting models.

The conservative drifting method is applicable to designing efficient one-step generative models in domains requiring rapid sample generation with stability guarantees, such as image synthesis, speech generation, and reinforcement learning policy sampling. By restoring gradient field structure, the method enhances training stability and theoretical interpretability, facilitating integration into existing generative frameworks. The explicit analysis of bandwidth and particle number informs practical parameter tuning, improving robustness. Moreover, the theoretical insights into non-conservative methods guide kernel selection and highlight limitations, aiding practitioners in method choice. These contributions support deployment in real-time inference scenarios and high-dimensional generative tasks.

The method’s reliance on kernels with high smoothness excludes direct application to non-smooth kernels like the Laplace kernel, limiting generality. Controlling the reciprocal-KDE self-interaction term requires stringent local occupancy and bandwidth conditions, which may be difficult to satisfy in practice, potentially affecting numerical stability. The non-conservative Laplace drifting approach inherently contains scale-mismatch residuals that cannot be eliminated, restricting achievable convergence rates and generation quality.