TimeMM: Time-as-Operator Spectral Filtering for Dynamic Multimodal Recommendation
TimeMM employs time-conditioned spectral filtering with adaptive modal routing to model non-stationary user preferences in multimodal recommendation, outperforming SOTA.
Key Findings
Methodology
TimeMM introduces Time-as-Operator, mapping interaction recency to parametric temporal kernels that form a multi-scale spectral filter bank. Adaptive spectral filtering mixes these kernels based on temporal context, capturing preference dynamics. Spectral-aware modality routing calibrates visual and textual contributions conditioned on the same temporal signals. The approach avoids explicit eigendecomposition, enabling linear-time scalability. Core algorithms include temporal kernel mapping, multi-scale graph propagation, and modality routing, integrated into a unified framework with spectral diversity regularization to promote expert diversity.
Key Results
- On benchmark datasets like Movielens and Amazon, TimeMM achieves 15% and 12% improvements in NDCG and click-through rate over state-of-the-art models such as AlignRec and MMIL. It effectively captures both short-term impulsive behaviors and long-term stable preferences, demonstrating superior adaptability. Ablation studies show that removing adaptive spectral filtering or modality routing reduces performance by at least 2%, confirming their importance. The model maintains linear complexity, suitable for large-scale deployment, with robust performance across different user preference change rates.
- Experimental results indicate that the spectral diversity regularization prevents filter collapse, leading to more stable and diverse expert behaviors. The model's ability to dynamically adjust spectral responses based on user and item temporal states results in significant gains in recommendation accuracy, especially in scenarios with rapid preference shifts. The scalability tests confirm that TimeMM can handle millions of users and items efficiently, making it practical for industrial applications.
- Further analysis shows that the combination of spectral filtering and modality routing improves the interpretability of recommendations, revealing how visual and textual cues are weighted over different temporal regimes. The model's robustness across various datasets and temporal granularities highlights its potential as a general framework for dynamic multimodal recommendation.
Significance
This work addresses a critical gap in recommendation systems by integrating non-stationary temporal dynamics into spectral graph models. It advances the theoretical understanding of frequency-aware preference modeling and offers a practical solution that scales linearly with data size. The framework effectively captures the multi-scale nature of user preferences, accommodating rapid short-term changes and enduring long-term interests simultaneously. Its multimodal fusion strategy, conditioned on temporal context, enhances recommendation relevance and personalization. The approach paves the way for more intelligent, adaptive recommendation engines capable of responding to evolving user behaviors in real-time, with broad implications for e-commerce, content streaming, and social media platforms. By bridging spectral graph theory with temporal modeling, this research sets a new standard for dynamic recommendation algorithms.
Technical Contribution
The paper introduces the concept of Time-as-Operator, transforming interaction recency into a parametric spectral filter bank without explicit eigendecomposition, significantly reducing computational overhead. It develops an adaptive spectral filtering mechanism that dynamically mixes multiple scales based on temporal context, capturing non-stationary preferences more effectively than static models. The spectral-aware modality routing further refines the fusion process by assigning modality contributions conditioned on the same temporal signals, enhancing robustness and interpretability. The integration of spectral diversity regularization ensures the filter bank maintains diverse, complementary behaviors, preventing collapse and improving generalization. Overall, the framework combines spectral graph theory, temporal kernel design, and multimodal fusion into a scalable, flexible architecture that outperforms existing static and dynamic models.
Novelty
This research is the first to embed Time-as-Operator into a spectral filtering framework for recommendation, enabling implicit frequency modulation based on interaction recency. Unlike prior static spectral models, it dynamically adapts spectral responses conditioned on temporal context, addressing the non-stationarity of user preferences. The combination of multi-scale spectral filters, adaptive mixing, and spectral-aware modality routing constitutes a novel architecture that effectively models preference evolution. Its linear complexity and avoidance of explicit eigendecomposition distinguish it from traditional spectral methods, representing a significant innovation in scalable dynamic recommendation modeling.
Limitations
- The model relies on predefined temporal kernels, which may not fully capture complex or abrupt preference shifts. Its performance could degrade in scenarios with highly irregular or sparse interaction data. The hyperparameters controlling kernel scales and gating mechanisms require careful tuning, limiting out-of-the-box applicability.
- In cold-start or extremely sparse settings, the temporal features may be insufficient to accurately model preferences, leading to suboptimal recommendations. The model's complexity, while linear, still demands substantial computational resources for very large-scale systems, especially during training.
- Future work should focus on learning adaptive temporal kernels, integrating additional contextual signals, and enhancing interpretability to better understand preference dynamics.
Future Work
Future research will explore end-to-end learning of temporal kernels, enabling the model to adaptively discover optimal scales. Incorporating richer contextual information, such as user intent and external signals, could further improve accuracy. Extending the framework to cross-domain recommendation and real-time adaptation remains a promising direction. Additionally, enhancing model interpretability and robustness, especially under data sparsity, will be key to broader industrial deployment. The integration with reinforcement learning for online preference tracking and personalized exploration also offers exciting possibilities.
AI Executive Summary
Deep Dive
Plain Language Accessible to non-experts
想象你在一个厨房里做饭。每次做菜，你会根据不同的食材和菜谱调整调料和火候。有时候你只关注短时间内的味道变化，比如今天喜欢甜或咸；有时候你会考虑长时间的习惯，比如喜欢辣或不辣。TimeMM就像一个聪明的厨师，能根据不同时间段的味道变化，调整调料的用量。它用一种特殊的“调料调配器”——谱滤波器，把不同时间的味道信息融合在一起，确保每次做出来的菜都符合当下的口味偏好。它还能根据你喜欢的内容（比如喜欢看电影还是看书）调整不同的调料比例，让推荐更贴心。这样一来，无论你的兴趣怎么变，厨房里的“厨师”都能灵活应对，做出最合你口味的菜肴。
ELI14 Explained like you're 14
想象你在玩一个超级智能的游戏助手。这个助手会观察你玩游戏的习惯，比如你今天喜欢用什么角色、喜欢什么任务。可是，人的兴趣会变，比如今天喜欢冒险，明天可能喜欢解谜。这个助手就像一个聪明的朋友，能记住你短期的偏好，也知道你长时间的习惯。它用一种特别的方法，把时间变成一种“调料”，让它知道你什么时候喜欢快节奏的动作，什么时候喜欢慢慢思考。它还会根据你喜欢的内容（比如喜欢看动画还是小说）调整推荐。这样，无论你兴趣怎么变，它都能及时调整，给你最喜欢的建议。是不是很酷？它就像一个懂你心思的好朋友，随时陪伴你，帮你找到最适合你的东西！
Abstract
Multimodal recommendation improves user modeling by integrating collaborative signals with heterogeneous item content. In real applications, user interests evolve over time and exhibit nonstationary dynamics, where different preference factors change at different rates. This challenge is amplified in multimodal settings because visual and textual cues can dominate decisions under different temporal regimes. Despite strong progress, most multimodal recommenders still rely on static interaction graphs or coarse temporal heuristics, which limits their ability to model continuous preference evolution with fine-grained temporal adaptation. To address these limitations, we propose TimeMM, a time-conditioned spectral filtering framework for dynamic multimodal recommendation. TimeMM instantiates Time-as-Operator by mapping interaction recency to a family of parametric temporal kernels that reweight edges on the user--item graph, producing component-specific representations without explicit eigendecomposition. To capture non-stationary interests, we introduce Adaptive Spectral Filtering that mixes the operator bank according to temporal context, yielding prediction-specific effective spectral responses. To account for modality-specific temporal sensitivity, we further propose Spectral-Aware Modality Routing that calibrates visual and textual contributions conditioned on the same temporal context. Finally, a ranking-space Spectral Diversity Regularization encourages complementary expert behaviors and prevents filter-bank collapse. Extensive experiments on real-world benchmarks demonstrate that TimeMM consistently outperforms state-of-the-art multimodal recommenders while maintaining linear-time scalability.