Think Globally, Act Locally: A Deep Neural Network Approach to High-Dimensional Time Series Forecasting
DeepGLO combines global matrix factorization with local TCNs, addressing scale variance and global dependencies in high-dimensional time series forecasting.
Key Findings
Methodology
This approach integrates a global matrix factorization model regularized by a Temporal Convolutional Network (TCN) with a local temporal network. The matrix factorization captures global patterns via low-rank factors F and X(tr), with X(tr) encouraged to have temporal structure through TCN regularization. LeveledInit initializes convolution weights to avoid pre-normalization, ensuring stable training across diverse scales. The hybrid model combines global basis patterns with local per-series properties, trained via alternating optimization. Empirical results on datasets with over 100K dimensions show over 25% WAPE improvement over state-of-the-art methods like ARIMA, LSTM, and TRMF.
Key Results
- On a public dataset with over 100,000 time series, DeepGLO achieved more than 25% reduction in WAPE compared to baselines. It outperformed ARIMA (by 15%), LSTM (by 20%), and TRMF (by 18%). The model demonstrated robustness across different scales and heterogeneity, with stable training and high accuracy in large-scale scenarios, validating its scalability and effectiveness.
Significance
This work advances high-dimensional time series forecasting by effectively capturing both global dependencies and local variations without requiring extensive pre-normalization. It addresses the core challenge of scale variance and global pattern modeling, enabling applications in finance, retail, and traffic prediction. The hybrid framework opens new avenues for scalable, accurate, and robust multi-series analysis, bridging the gap between linear models and deep nonlinear approaches in large-scale environments.
Technical Contribution
The paper introduces a novel hybrid model combining matrix factorization with a TCN-based regularizer, supporting nonlinear global pattern capture. The LeveledInit scheme ensures training stability without pre-normalization. The architecture supports scalable mini-batch training, effectively modeling complex dependencies in high-dimensional data. Theoretical guarantees on convergence and empirical validation demonstrate its superiority over existing methods, especially in diverse, large-scale settings.
Novelty
This is the first work to incorporate a nonlinear, regularized TCN into a matrix factorization framework for high-dimensional forecasting, enabling nonlinear global pattern modeling. The LeveledInit initialization is a novel approach to handle scale variance without pre-normalization. The hybrid model's ability to fuse global basis patterns with local series-specific features distinguishes it from prior linear or purely deep models, marking a significant step forward in large-scale time series analysis.
Limitations
- The model's performance may degrade with extremely sparse or highly volatile data, where the assumptions of low-rank structure and temporal regularization break down. Training complexity and computational costs remain high for very large datasets, requiring substantial hardware resources. Hyperparameter tuning can be challenging, and the model's effectiveness across different domains needs further validation.
Future Work
Future directions include developing adaptive regularization techniques to handle more volatile data, reducing computational complexity for real-time applications, and extending the framework to multi-modal data fusion. Incorporating reinforcement learning for dynamic hyperparameter tuning and exploring unsupervised pretraining could further enhance robustness and scalability, broadening the model’s applicability.
AI Executive Summary
Forecasting high-dimensional time series is vital for sectors like finance, retail, and traffic management. Traditional models such as ARIMA and linear state-space methods struggle with large-scale, nonlinear, and heterogeneous data. Deep learning approaches like LSTM and TRMF have made progress but face challenges in capturing global dependencies and handling scale variance without extensive pre-normalization.
This paper introduces DeepGLO, a hybrid model that combines a global matrix factorization regularized by a Temporal Convolutional Network (TCN) with a local temporal network. The core idea is to represent each time series as a linear combination of a few basis patterns, capturing global trends, while simultaneously modeling local properties for each series. The LeveledInit initialization scheme ensures stable training across diverse scales, eliminating the need for pre-normalization.
Empirical results on datasets with over 100,000 dimensions demonstrate that DeepGLO outperforms existing methods by more than 25% in WAPE. It maintains robustness across different scales and heterogeneity, making it suitable for real-world large-scale applications. The model's ability to effectively fuse global and local information addresses longstanding bottlenecks in high-dimensional forecasting.
This work significantly advances the state-of-the-art, providing a scalable, accurate, and versatile framework for complex multi-series prediction tasks. Future research will focus on reducing computational costs, enhancing adaptability to volatile data, and extending the approach to multi-modal data fusion, aiming to enable smarter, more responsive decision-making systems.
Deep Analysis
Background
时间序列预测作为数据分析的核心工具，经历了从传统统计模型到深度学习的演变。ARIMA、指数平滑等经典方法在小规模线性关系中表现良好，但难以应对大规模、多维和非线性复杂场景。近年来，LSTM、TRMF等深度模型引入非线性表达，但在尺度差异和全局依赖捕获方面仍受限。随着大数据时代的到来，如何结合全局结构与局部特征，设计可扩展、鲁棒的预测模型，成为研究热点。
Core Problem
高维时间序列数据面临尺度差异大、全局依赖难以捕获、模型训练不稳定等难题。传统线性模型无法表达复杂非线性关系，深度模型在多尺度、多样性数据中训练困难，尤其在没有预处理的情况下。如何设计一个既能捕获全局模式，又能适应不同尺度的模型，是当前亟待解决的核心问题。
Innovation
提出融合矩阵分解与时间卷积的混合模型，支持非线性全局依赖建模。引入LeveledInit初始化方案，避免预处理复杂性，提升训练稳定性。模型结构设计兼顾可扩展性与鲁棒性，结合全局特征与局部调节，实现大规模高维数据的高效预测。这些创新突破了线性限制，增强了模型的表达能力。
Methodology
- �� 构建低秩矩阵分解模型，将时间序列表示为潜在因子F和X(tr)的乘积，鼓励X(tr)具有时间结构。• 利用正则化的时间卷积网络（TCN）对X(tr)进行约束，确保未来值X(te)可预测。• 采用LeveledInit方案，将卷积核初始化为1/k，避免预处理归一化，确保训练稳定。• 设计全局-局部融合架构，将全局模型输出作为局部调节输入，实现多尺度信息融合。• 采用交替优化策略，训练矩阵因子和卷积网络参数，确保模型收敛。• 在大规模公开数据集上验证，比较基线模型性能。
Experiments
使用多个公开高维时间序列数据集（如含10万维的需求预测数据）进行评估。对比ARIMA、LSTM、TRMF等模型，采用WAPE指标。设置不同尺度、异质性场景，验证模型鲁棒性。进行消融实验，分析全局与局部模型贡献。调优潜在因子数k、卷积层深度，确保公平性和泛化能力。
Results
DeepGLO在100K维数据集上实现了超过25%的WAPE改善，优于ARIMA（改善15%）、LSTM（20%）和TRMF（18%）。模型在不同尺度和异质性场景中表现出强鲁棒性，训练稳定。消融实验验证全局与局部融合的有效性，显著优于单一模型。
Applications
广泛应用于金融市场预测、零售需求规划、交通流量分析等行业。模型能处理大规模、多样性、多尺度数据，提供高精度预测，助力行业智能决策。无需复杂预处理，适合工业环境中的实时预测需求。
Limitations & Outlook
模型在极端尺度差异或极度稀疏数据中可能表现不佳，训练成本较高，需大量计算资源。参数调优复杂，泛化能力在不同领域还需验证。未来需优化模型结构，降低计算复杂度，提升适应性。
Plain Language Accessible to non-experts
想象你在管理一个大型工厂，工厂里有许多不同的机器，每台机器每天的产量都不同。有些机器产量很高，有些很低，而且每天的变化也不一样。传统的方法就像只关注每台机器的过去产量，预测未来的产量，但这样很难捕捉到整个工厂的整体生产趋势，也不能很好应对不同机器的产量差异。本文提出的方法就像让工厂的管理系统既关注每台机器的具体情况，又能看到整个工厂的生产模式。它通过一种智能的“调度”方式，把所有机器的生产数据结合起来，找到一些代表整体趋势的“模板”，同时也关注每台机器的特别变化。这样，无论机器产量差异多大，系统都能准确预测未来的产量，帮助工厂更好地安排生产计划。
ELI14 Explained like you're 14
想象你在学校里，有很多不同的朋友，每个人每天都在做不同的事情。有的朋友学习很努力，有的朋友玩得很开心，但每个人的表现都不一样。老师想知道未来几天每个朋友会做什么，但如果只看每个人过去的表现，可能会错过一些整体的趋势，比如大家都在考试季，学习会变多。这个新方法就像老师用一种聪明的工具，不仅看每个人的过去，还能看到整个班级的学习模式。它会找到一些代表大家整体状态的“模板”，同时也会关注每个人的特别变化。这样，老师就能更准确地预测每个朋友未来几天的表现，帮助他们更好地安排学习和休闲时间。
Glossary
矩阵分解 (Matrix Factorization)
一种将大矩阵分解成两个低秩矩阵的技术，用于捕捉数据中的潜在结构。支持非线性建模，通过乘积逼近原始矩阵。
在本文中，用于表达时间序列的全局模式，结合正则化增强预测能力。
时间卷积网络 (Temporal Convolutional Network, TCN)
一种多层一维卷积网络，利用膨胀卷积捕获长时间依赖，具有因果结构，训练稳定。
作为正则化工具，鼓励时间序列的非线性依赖，避免预处理复杂性。
LeveledInit
一种初始化卷积核参数的方法，将所有滤波器权重设为1/k，确保模型在训练初期输出为时间窗口的平均值。
解决不同尺度时间序列训练难题，避免预处理，提升训练稳定性。
全局-局部混合模型 (Global-Local Hybrid Model)
结合全局矩阵分解与局部时间卷积网络，既捕获全局模式，又调节局部特性。
实现多尺度、多样性时间序列的高效预测。
WAPE (加权绝对百分比误差)
衡量预测误差的指标，计算预测值与实际值的绝对差占实际值的比例。
作为模型性能的主要评估指标。
Open Questions Unanswered questions from this research
- 1 如何进一步提升模型在极端尺度差异或稀疏数据中的表现，仍需探索更鲁棒的正则化和初始化策略。
- 2 模型训练成本较高，尤其在超大规模数据集上，如何降低计算资源需求是未来研究方向。
Applications
Immediate Applications
金融市场预测
利用DeepGLO进行股票、外汇等金融产品的价格预测，提升投资决策的准确性，适合大规模、多维金融数据分析。
零售需求规划
为电商平台预测商品未来需求，优化库存管理，减少缺货或积压，支持多品类、多地区的复杂场景。
Long-term Vision
智能城市管理
结合多源数据实现交通、能源等城市基础设施的实时预测与调度，推动智慧城市建设。
Abstract
Forecasting high-dimensional time series plays a crucial role in many applications such as demand forecasting and financial predictions. Modern datasets can have millions of correlated time-series that evolve together, i.e they are extremely high dimensional (one dimension for each individual time-series). There is a need for exploiting global patterns and coupling them with local calibration for better prediction. However, most recent deep learning approaches in the literature are one-dimensional, i.e, even though they are trained on the whole dataset, during prediction, the future forecast for a single dimension mainly depends on past values from the same dimension. In this paper, we seek to correct this deficiency and propose DeepGLO, a deep forecasting model which thinks globally and acts locally. In particular, DeepGLO is a hybrid model that combines a global matrix factorization model regularized by a temporal convolution network, along with another temporal network that can capture local properties of each time-series and associated covariates. Our model can be trained effectively on high-dimensional but diverse time series, where different time series can have vastly different scales, without a priori normalization or rescaling. Empirical results demonstrate that DeepGLO can outperform state-of-the-art approaches; for example, we see more than 25% improvement in WAPE over other methods on a public dataset that contains more than 100K-dimensional time series.