FLAME: Flow Enhanced Legendre Memory Models for General Time Series Forecasting
FLAME combines Legendre Memory and normalization flow for lightweight, probabilistic time series forecasting with state-of-the-art zero-shot performance.
Key Findings
Methodology
FLAME employs Legendre Memory (Voelker et al., 2019) in encoding and decoding phases, utilizing variants LegT and LegS to capture data biases and enable long-range inference. It integrates a normalization flow-based forecasting head (Rasul et al., 2021b) for modeling complex distributions efficiently. The model is pretrained on multi-domain datasets, supporting both deterministic and probabilistic tasks with parameter sizes of 2M, 6M, and 10M, outperforming existing models in zero-shot scenarios. The architecture emphasizes lightweight design, strong generalization, and probabilistic accuracy.
Key Results
- On benchmarks TSFM-Bench and ProbTS, FLAME achieves top zero-shot performance, with the 10M version outperforming models like Sundial (128M) in deterministic tasks by ~15% in MSE, and reducing CRPS by 34.3% in probabilistic tasks. It demonstrates superior generalization across diverse datasets, with faster inference and fewer parameters.
- In ablation studies, Legendre Memory modules significantly improve robustness, while the Flow Head enhances probabilistic modeling of intricate distributions. The model maintains efficiency, with inference speed comparable to lightweight models, making it suitable for real-world deployment.
- Compared to large-scale transformers, FLAME’s parameter efficiency and out-of-the-box zero-shot capabilities mark a breakthrough, especially in resource-constrained environments, validating its practical and theoretical advantages.
Significance
This work advances time series forecasting by integrating classical orthogonal polynomial techniques with modern generative modeling, achieving a rare combination of lightweight design and high accuracy. It addresses the long-standing challenge of balancing model complexity, interpretability, and efficiency, making probabilistic forecasting accessible for real-time applications. The model's strong zero-shot performance indicates its potential for broad adoption in industry, from finance to weather prediction, reducing reliance on extensive retraining and large datasets. Theoretically, it opens new avenues for combining classical control theory with deep learning, fostering more robust and explainable models.
Technical Contribution
FLAME introduces a novel architecture that fuses Legendre Memory with normalization flow, enabling efficient modeling of complex distributions in a lightweight framework. It innovates with the Local-Perception module, SSD decoder, and a token-wise Flow Head, each designed to enhance local feature capture, long-term inference, and probabilistic accuracy. The model’s parameter efficiency (max 10M) and end-to-end training paradigm set new standards for scalable, probabilistic time series forecasting, with theoretical guarantees rooted in orthogonal polynomial approximation and invertible transformations.
Novelty
This is the first work to embed Legendre Memory directly into a pretrained time series foundation model, leveraging its orthogonal properties for robust generalization. The integration with normalization flow for probabilistic forecasting is novel, enabling complex distribution modeling without large parameters. Unlike prior models relying on fixed priors or heavy transformers, FLAME achieves a balance of efficiency, interpretability, and accuracy, representing a significant innovation in lightweight probabilistic modeling.
Limitations
- The model's approximation accuracy depends on the order of Legendre polynomials; in extremely long or highly non-stationary sequences, performance may degrade. Future work should explore adaptive polynomial orders and hybrid architectures.
- Training the flow-based head involves complex optimization, which may be sensitive to hyperparameters and require careful tuning. Its robustness under high noise or missing data scenarios needs further validation.
- While lightweight, the model still relies on pretraining data diversity; transferability to entirely new domains with limited data remains a challenge, warranting further research into domain adaptation.
Future Work
Future directions include extending the model to multivariate and multi-modal data, enhancing robustness in extreme scenarios, and exploring self-supervised pretraining strategies. Integrating adaptive polynomial orders and more flexible flow architectures could further improve accuracy. Additionally, optimizing for edge deployment and real-time inference will broaden practical applications, especially in resource-limited environments.
AI Executive Summary
FLAME represents a significant leap forward in time series forecasting, combining classical orthogonal polynomial techniques with modern generative models to achieve lightweight yet highly capable predictive systems. Traditional approaches, such as LSTMs and transformers, often struggle with balancing model size, interpretability, and accuracy, especially in probabilistic settings. FLAME addresses these issues by integrating Legendre Memory—an orthogonal polynomial-based compression method—with normalization flow-based probabilistic forecasting heads.
This architecture leverages the strong generalization properties of Legendre polynomials, which efficiently encode historical data, capturing intrinsic biases and long-range dependencies. The innovative Local-Perception module enhances local feature extraction by aligning varying frequency components via FFT analysis, enabling the model to adapt to diverse data characteristics. The SSD decoder further facilitates long-term inference by utilizing a structured state-space approach, while the token-wise Flow Head models complex distributions with high efficiency.
Experimental results on benchmarks like TSFM-Bench and ProbTS demonstrate FLAME’s state-of-the-art zero-shot performance. The 10M parameter version outperforms models like Sundial (128M) in deterministic tasks by approximately 15% in MSE and reduces CRPS by 34.3% in probabilistic tasks. Its lightweight design ensures fast inference and low resource consumption, making it suitable for real-world deployment in industries such as finance, weather forecasting, and industrial monitoring.
Overall, FLAME’s innovative fusion of classical and modern techniques paves the way for scalable, interpretable, and highly accurate probabilistic time series forecasting. Future work will focus on extending its capabilities to multivariate data, enhancing robustness, and optimizing for edge deployment, promising a broad impact across both academia and industry.
Deep Dive
Abstract
In this work, we introduce FLAME, a family of extremely lightweight and capable Time Series Foundation Models, which support both deterministic and probabilistic forecasting via generative probabilistic modeling, thus ensuring both efficiency and robustness. FLAME utilizes the Legendre Memory for strong generalization capabilities. Through adapting variants of Legendre Memory, i.e., translated Legendre (LegT) and scaled Legendre (LegS), in the Encoding and Decoding phases, FLAME can effectively capture the inherent inductive bias within data and make efficient long-range inferences. To enhance the accuracy of probabilistic forecasting while keeping efficient, FLAME adopts a Normalization Flow based forecasting head, which can model the arbitrarily intricate distributions over the forecasting horizon in a generative manner. Comprehensive experiments on well-recognized benchmarks, including TSFM-Bench and ProbTS, demonstrate the consistent state-of-the-art zero-shot performance of FLAME on both deterministic and probabilistic forecasting tasks.