Quantizing Time-Series Models As Dynamical Systems: Trajectory-Based Quantization Sensitivity Score
Introduces Trajectory Sensitivity Score (TQS) based on dynamical systems stability to evaluate quantization impact on time-series models.
Key Findings
Methodology
This paper proposes TQS by modeling the model's forward pass as a discrete dynamical system, analyzing how quantization errors propagate and amplify over the forecast horizon. It defines the model map Fθ, introduces quantization perturbation Q(θ), and computes the exponential growth rate γ of trajectory divergence over Tmax steps. Lyapunov-based, the γ index quantifies stability. A Gaussian noise proxy γgauss estimates sensitivity without coupling to specific quantization schemes, enabling a priori layer ranking. The TQS-PTQ algorithm then allocates mixed precision based on this ranking without calibration data or second-order approximations, supporting black-box models.
Key Results
- Experiments on Aurora-small, TimesFM-2.5, and Pangu-Weather show TQS-PTQ achieves up to 32× compression with only 1% increase in MAE. It effectively identifies sensitive layers, optimizing quantization strategies, outperforming baseline methods.
- Sensitivity analysis reveals that critical layers are concentrated at input/output modules, diverging from typical fully-connected layer focus in LLMs. The layer ranking is highly consistent across models, supporting transferability.
- Single sensitivity sweep supports multiple compression targets, reducing tuning costs by 3-5×, enabling fast deployment without repeated calibration.
Significance
This work reinterprets quantization as a dynamical stability problem, providing a robust theoretical framework that overcomes reliance on gradients or Hessians. It addresses the challenge of long-term error accumulation in weather and climate models, facilitating resource-efficient deployment with physical consistency. The calibration-free approach simplifies industrial application, broadening accessibility and scalability of low-precision models in critical domains.
Technical Contribution
The core innovation is the Lyapunov exponent-based sensitivity metric, which captures the exponential divergence of quantization errors over time. The Gaussian proxy γgauss enables unbiased, calibration-free layer ranking. The TQS-PTQ algorithm leverages this ranking for multi-target mixed precision scheduling, offering a flexible, theoretically grounded approach to model compression that generalizes across architectures.
Novelty
This is the first systematic application of dynamical systems theory, specifically Lyapunov exponents, to quantify quantization sensitivity in time-series models. Unlike gradient-based methods, TQS directly measures trajectory divergence, providing a universal, model-agnostic sensitivity metric. Its transferability across models and tasks marks a significant advance over prior work focused on static or local sensitivity analysis.
Limitations
- The method primarily applies to discrete-time models; extension to continuous-time systems requires further development. In highly nonlinear or high-dimensional systems, Lyapunov estimates may be less accurate, affecting sensitivity ranking reliability.
- Sensitivity predictions could be affected by model structure variations and parameter initialization, especially in complex or poorly trained models. Further robustness validation is needed.
- While calibration-free, the approach assumes the model's dynamics are well-behaved; in cases with chaotic or unstable behavior, the Lyapunov-based assessment may be less reliable.
Future Work
Future directions include extending TQS to continuous-time and nonlinear systems, integrating adaptive perturbation strategies, and exploring real-time sensitivity monitoring. Combining this framework with physical constraints and multi-modal data could further enhance robustness. Additionally, investigating the method's applicability to other domains like control systems or robotics is promising.
AI Executive Summary
This study introduces Trajectory Sensitivity Score (TQS), a novel metric grounded in dynamical systems theory, to evaluate the impact of quantization on time-series models. Traditional post-training quantization (PTQ) techniques often rely on gradient or Hessian information to assess layer sensitivity, but these approaches struggle to capture long-term error propagation, especially in models like weather forecasts that require stability over extended horizons.
TQS models the model's forward pass as a discrete dynamical system, where quantization introduces bounded perturbations. By analyzing the exponential divergence rate of trajectories via Lyapunov exponents, TQS provides a robust measure of layer sensitivity that is decoupled from specific quantization schemes. This allows for a priori sensitivity ranking, enabling efficient mixed-precision scheduling.
Building on this, the authors develop TQS-PTQ, a calibration-free, multi-target compression algorithm that leverages a single sensitivity sweep to support various bit-width configurations. Extensive experiments on Aurora-small, TimesFM-2.5, and Pangu-Weather demonstrate that TQS-PTQ achieves up to 32× compression with only a 1% increase in MAE, outperforming baseline methods such as GPTQ and uniform quantization. The sensitivity analysis reveals that critical layers are predominantly at input/output interfaces, a pattern consistent across models, supporting transferability.
This approach significantly advances the field by providing a theoretically sound, computationally efficient framework for low-precision deployment of resource-intensive models. It addresses the challenge of long-term error accumulation, ensuring physical and statistical consistency in scientific applications. While promising, future work will focus on extending the methodology to continuous systems, nonlinear dynamics, and real-time adaptive scenarios, broadening its impact across AI and scientific domains.
Deep Dive
Abstract
We introduce the Trajectory-based Quantization Sensitivity Score (TQS), a metric that reframes post-training quantization (PTQ) through the lens of dynamical-systems stability. By modeling the network's rollout as a discrete-time dynamical system, TQS characterizes how quantization-induced errors propagate and amplify over the rollout horizon. Unlike conventional PTQ methods, where sensitivity analysis is often coupled to the quantization procedure, TQS enables a priori sensitivity estimation decoupled from quantizer selection and bit-width assignment. This separation allows for quantization budget planning even for black-box or compiled networks with fused operators. Building on this, we present TQS-PTQ, a flexible mixed-precision framework that requires no calibration data or costly second-order approximations. Our experiments show that a dynamical-systems perspective provides a robust, high-performing pathway for low-precision deployment in resource-constrained settings.