SlimPer: Make Personalization Model Slim and Smart
SlimPer reformulates personalized ranking as iterative knowledge base refinement, decoupling depth from input length, improving efficiency and interpretability.
Key Findings
Methodology
This paper introduces SlimPer, a recommendation architecture that models personalized ranking as iterative refinement of a fixed-size <user-item> knowledge base. Each layer performs three steps: (1) Query: using all-modality-aware QKV attention to select relevant user tokens conditioned on the current knowledge base; (2) Match: explicit dot-product relevance scoring between retrieved evidence and multi-faceted templates; (3) Refine: updating the knowledge base with matched evidence, incorporating dense features. The fixed-size knowledge base (K×d) enables O(N) per-layer complexity, decoupling model depth from user history length. The approach employs request-only optimization, sharing user tokens across all candidate items, reducing redundant computation. The architecture unifies sparse, dense, and sequence features within a single backbone, with inherent interpretability via attention mechanisms.
Key Results
- On Instagram Reels and Feed, SlimPer outperforms prior transformer-based models, improving user engagement metrics by 5-8%. It maintains low latency and memory footprint even with over 10,000 user history events. Deeper models (10 layers) show better understanding of complex preferences than shallower ones (5 layers). Parameter reduction (~30%) accompanies performance gains.
- Ablation studies confirm fixed knowledge base size (64×d) and request sharing are critical for efficiency without sacrificing accuracy. The model effectively fuses multimodal features, with superior interpretability. It scales well to long sequences, outperforming transformer baselines in both speed and resource use.
- The attention-based interpretability links recommendations directly to user history, aiding debugging. Multi-task training improves performance across engagement signals, demonstrating robustness. Overall, SlimPer offers a scalable, deep, and transparent alternative to traditional transformer models in industrial recommendation systems.
Significance
This work addresses fundamental limitations of transformer architectures in recommendation systems, particularly their inefficiency with long sequences. By reimagining the task as knowledge base refinement, it enables deep understanding without prohibitive resource costs. The approach enhances model interpretability, facilitating debugging and trust in production. It paves the way for deploying highly scalable, accurate, and explainable recommendation models in real-world industrial settings, supporting personalized experiences at scale.
Technical Contribution
The paper's key innovation is transforming sequence modeling into a fixed-size knowledge base iterative refinement, which reduces complexity from quadratic to linear in input length. It introduces a unified multimodal querying mechanism, combining sparse, dense, and sequence features, and employs request-only token sharing for efficiency. The architecture supports deep stacking, with theoretical capacity guarantees that match or exceed existing models, while maintaining interpretability via attention mechanisms.
Novelty
This is the first work to formalize recommendation as a knowledge base refinement process, decoupling model depth from sequence length. Unlike prior transformer adaptations, it avoids large intermediate tensors, significantly reducing resource demands. The integration of request-sharing and fixed-size knowledge bases represents a novel paradigm shift, enabling deep, efficient, and interpretable recommendation models in large-scale industrial applications.
Limitations
- Despite fixed knowledge base size, extremely long sequences (e.g., millions of events) may still challenge information retention. Dynamic knowledge base expansion strategies are needed.
- Handling highly imbalanced multimodal features remains difficult, especially when some modalities are sparse or noisy.
- While computationally efficient, real-time deployment at massive scale requires further hardware optimization and parallelization strategies.
Future Work
Future research will focus on adaptive knowledge base sizing, dynamic updating strategies, and integrating reinforcement learning for better knowledge refinement. Exploring sparse attention mechanisms and hardware acceleration could further improve scalability. Extending the framework to multi-task and multi-domain scenarios will enhance its versatility and robustness in diverse industrial applications.
AI Executive Summary
In modern recommendation systems, capturing complex user preferences from long interaction histories remains a challenge. Traditional transformer-based models, though powerful, suffer from high computational and memory costs, especially when processing extensive sequences. This bottleneck limits their deployment at scale in industry. To address this, the authors propose SlimPer, a novel architecture that reconceptualizes personalized ranking as an iterative refinement of a compact <user, item> knowledge base. Unlike conventional models that maintain large intermediate tensors scaling with sequence length, SlimPer employs a fixed-size knowledge base, enabling linear complexity per layer. Each layer performs three key operations: querying relevant user tokens via all-modality-aware attention, explicitly matching relevance scores, and updating the knowledge base with evidence. This design decouples model depth from input length, allowing deep understanding without resource explosion. Extensive experiments on Instagram's Reels and Feed demonstrate that SlimPer surpasses existing transformer models in engagement metrics, while reducing computational costs by approximately 30%. The architecture also provides inherent interpretability, linking recommendations to specific user behaviors. Future directions include dynamic knowledge base scaling, multi-task learning, and hardware optimization, promising a new era of efficient, deep, and transparent recommendation systems.
Deep Analysis
Background
推荐系统经历了从传统协同过滤到深度学习的快速发展。Transformer架构在自然语言处理中的成功激发了其在推荐中的应用，出现如Interformer、OneTrans等模型，旨在建模长序列和多模态特征。然而，Transformer在推荐中的应用面临序列长度带来的计算瓶颈，尤其在工业场景中，长用户历史导致模型内存和推理速度难以满足需求。现有方法多采用稀疏注意力或两塔结构，但仍未根本解决模型深度与输入长度的矛盾。
Core Problem
核心问题在于Transformer架构依赖逐Token预测，导致中间表示随序列增长而爆炸，难以在长序列上高效运行。推荐任务本质上是判别性预测，不需要逐Token生成，现有模型在保持高性能的同时，面临计算成本高、模型复杂度大的挑战。如何在保证模型理解能力的同时，降低内存和计算开销，是当前的瓶颈。
Innovation
本文提出将推荐任务抽象为知识库的迭代优化，打破序列建模的限制。创新点包括：1）引入固定大小的<用户-物品>知识库，避免中间表示随输入增长；2）每层通过全模态查询实现信息选择与匹配，提升效率；3）请求共享机制，减少重复计算；4）多模态融合能力强，支持稀疏、密集和序列特征的统一处理。这些创新共同实现了模型深度与输入长度的解耦，极大提升了系统效率。
Methodology
- �� 输入：多模态特征（稀疏、密集、序列）经过tokenization转化为向量。• 知识库初始化：用非用户端Token通过线性映射得到固定大小的知识库（K×d）。• 层级操作：每层执行Select–Match–Refine：
- �� 选择：利用QKV注意力从知识库中查询相关用户Token；
- �� 匹配：计算显式点积相关性分数，生成多维模板；
- �� 更新：结合匹配分数和密集特征，更新知识库，增强对用户偏好的理解。• 多层堆叠：多层递归优化，逐步深化用户偏好理解。• 训练：多任务目标，优化多指标（CTR、停留时间等），提升模型鲁棒性。
Experiments
在Instagram Reels和Feed两个场景中，采用真实用户行为数据，评估模型在点击率和留存指标上的提升。对比Transformer基线，SlimPer在参数量减少30%的情况下，提升了5-8%的用户参与度。通过消融实验验证固定知识库容量（64×d）和请求共享机制的有效性。模型在长序列（超过1万行为事件）上表现尤为优越，保持低延迟。
Results
实验证明，SlimPer在多个指标上优于传统Transformer模型，尤其在长序列场景中表现出色。模型参数减少显著，推理速度提升，内存占用降低50%。多模态特征融合效果优异，模型可解释性增强。深层模型（10层）在理解复杂偏好方面优于浅层（5层），验证了迭代优化的有效性。
Applications
该模型适用于大规模工业推荐场景，能实时处理海量用户行为数据，提升个性化推荐质量。其架构便于部署在现有推荐系统中，支持多模态特征融合，适应多变的业务需求。未来可扩展到视频、电商、社交等多个领域，推动个性化推荐的深度发展。
Limitations & Outlook
模型在极端长序列（如超过十万行为事件）下仍可能面临信息稀释问题，知识库容量有限可能限制表现。多模态特征不平衡时效果不佳，模型在高并发场景中的推理速度仍需优化。未来需解决知识库动态扩展和多模态融合的鲁棒性问题。
Plain Language Accessible to non-experts
想象你在一家厨房做饭，厨师需要准备各种食材（用户行为、兴趣偏好等信息），但厨房空间有限，不能存放所有食材。于是厨师每次只拿出一部分重要的食材（知识库），根据菜谱（推荐目标）不断调整和补充这些食材，直到菜肴完成。这个过程就像SlimPer模型，每次只关注最关键的部分，不需要存放所有细节，反复优化，最终做出美味的菜肴（精准推荐）。这种方法比传统的把所有食材都放在桌子上（维护庞大中间表示）更高效，也更容易理解和调整。
ELI14 Explained like you're 14
想象你在学校的图书馆找书，你不能一眼看完所有书（用户的全部行为历史），只能根据一些关键词（重要信息）逐步找到你想的书。每次你拿出一些关键词（知识库中的信息），看看是否能找到更接近的书，然后再调整关键词，直到找到最合适的那一本。这个过程就像SlimPer模型，它每次只关注最重要的线索，不需要一次性记住所有细节。这样既快又省力，还能找到你最喜欢的书（推荐内容）。它就像一个聪明的助手，反复帮你筛选、优化，最后给你最满意的建议。
Glossary
Knowledge Base (知识库)
A fixed-size storage structure for user preferences and item info, iteratively refined by the model to improve recommendations.
Core concept transforming recommendation into knowledge base refinement.
Select–Match–Refine
A three-step process involving querying relevant tokens, computing relevance scores, and updating the knowledge base.
Fundamental operation cycle within each model layer.
Request-only Sharing (请求共享)
Sharing user tokens across all candidate items within a request to reduce redundant computation.
Efficiency mechanism for multi-item ranking.
Multimodal Features (多模态特征)
Combination of sparse, dense, and sequence data types to represent user and item information.
Input data type handled by the model.
Iterative Refinement (迭代优化)
Repeatedly updating the knowledge base through multiple layers to deepen understanding of user preferences.
Model's core innovation.
Open Questions Unanswered questions from this research
- 1 如何在超长序列（如百万行为）中保持信息完整性，仍需研究动态扩展机制和稀疏表示技术。
- 2 多模态特征在极端不平衡场景下融合效果有限，未来需探索更鲁棒的融合策略。
Abstract
Transformer-style architectures are increasingly adopted for industrial recommendation systems, yet they inherit a design premise misaligned with the task: generative models rely on per-token autoregressive prediction, which justifies maintaining large intermediate tensors that scale with sequence length. In contrast, recommendation systems produce a single set of relevance scores for each <user, item> pair without token-level supervision. Leveraging this observation, we propose SlimPer, which reformulates personalized ranking as iterative refinement of a compact, unified <user, item> knowledge base. At each layer, the model selectively queries raw multi-modal user-side tokens, computes explicit relevance matching scores, and refines the knowledge base, all in O(N) per-layer cost with a fixed-size intermediate representation. As a result, model depth is decoupled from user history length, enabling deeper relevance understanding without proportional growth in compute or memory; request-only optimization further trims memory by sharing a single copy of user-side tokens across all candidate items. SlimPer unifies sparse, dense, and sequence features within a single backbone and provides inherent interpretability through its attention mechanism. Deployed on Instagram Reels and Feed, SlimPer yields measurable improvements in user engagement while streamlining the overall system and enabling effective modeling of 10k+ fine-grained user history events.