MPFormer: Adaptive Framework for Industrial Multi-Task Personalized Sequential Retriever
MPFormer employs a target-conditioned Transformer with dynamic quota adjustment for multi-objective personalized retrieval, boosting coverage and efficiency.
Key Findings
Methodology
MPFormer introduces a target-conditioned Transformer that encodes user behavior sequences and multi-task semantics via learnable attention modulation. It incorporates personalized target weights for dynamic result adjustment and integrates user personalization into token representations, enabling efficient multi-objective retrieval. The framework employs multi-objective loss functions and a dynamic quota mechanism to balance objectives in real-time. Deployed on Kuaishou’s platform, it serves over 400 million daily users, achieving significant performance gains, including a 21.8% increase in multi-objective exposure, 99.99% system availability, and 31% GPU memory reduction. The approach combines multi-target indexes, attention-based encoding, and adaptive resource allocation, addressing the limitations of traditional multi-model solutions.
Key Results
- In large-scale deployment, MPFormer improved multi-objective exposure rate by 21.8%, maintained 99.99% availability under peak loads of 1.2 million QPS, and reduced GPU memory consumption by 31%, outperforming baseline methods in efficiency and scalability.
- Offline evaluation on production logs showed Recall@K increased by 12.71%, and NDCG scores improved across multiple metrics. Online AB tests demonstrated a 15% increase in user dwell time and 8% rise in click-through rate, confirming real-world effectiveness.
- Ablation studies verified the contributions of target-conditioned Transformer and dynamic quota mechanisms, highlighting their roles in enhancing coverage, resource utilization, and system stability in industrial environments.
Significance
This work addresses a fundamental challenge in industrial recommendation systems—integrating multi-objective optimization into the retrieval phase efficiently. Traditional solutions either suffer from resource explosion or poor multi-target coverage. MPFormer’s innovative architecture enables simultaneous multi-objective encoding, dynamic adjustment, and resource-efficient retrieval, significantly advancing the state-of-the-art. Its successful deployment at scale demonstrates its practical value, providing a scalable, fast, and flexible solution for complex industrial scenarios. The framework’s ability to adapt to real-time user behaviors and optimize multiple objectives simultaneously paves the way for smarter, more personalized recommendation systems, with broad implications for e-commerce, advertising, and content platforms.
Technical Contribution
The core technical innovation lies in the target-conditioned Transformer architecture, which employs learnable target embeddings to modulate attention dynamically, enabling multi-task semantic encoding within a unified model. The dynamic quota mechanism introduces a real-time, user-specific resource allocation strategy, optimizing the recall distribution across objectives. The integration of multi-object indexes and efficient retrieval pipelines further reduces resource consumption while maintaining high coverage. These advancements collectively overcome the resource scalability issues of multi-model approaches, offering a theoretically sound and practically deployable solution for large-scale multi-objective recommendation tasks.
Novelty
This research is the first to embed target-specific semantics directly into a Transformer-based retrieval framework, enabling dynamic, personalized multi-objective optimization in real-time industrial systems. Unlike prior multi-model or fixed-weight multi-task methods, MPFormer achieves flexible, scalable, and resource-efficient multi-target encoding and retrieval, representing a significant leap forward in recommendation technology. Its combination of learnable target conditioning and adaptive resource allocation distinguishes it from existing approaches, offering a new paradigm for multi-objective recommendation systems.
Limitations
- The model’s performance in cold-start scenarios remains limited due to reliance on historical behavior data; integrating more robust cold-start strategies is necessary.
- Increased model complexity and computational overhead from multi-target encoding and dynamic quota adjustments may challenge real-time deployment at even larger scales.
- Handling highly conflicting objectives or rapidly changing user preferences requires further refinement of the target adjustment mechanisms, possibly through reinforcement learning or more sophisticated adaptive algorithms.
Future Work
Future directions include integrating reinforcement learning to optimize target weights dynamically, exploring multi-modal data fusion (images, text), and extending the framework to other recommendation domains. Enhancing cold-start capabilities and further reducing computational costs will also be prioritized, aiming to make MPFormer adaptable to even more diverse and dynamic industrial environments.
AI Executive Summary
In the era of large-scale content platforms, delivering personalized recommendations that balance multiple objectives—such as click-through rate, watch time, and conversions—poses a significant challenge. Traditional retrieval methods often rely on multiple independent models or fixed-weight schemes, leading to resource inefficiency and limited multi-target coverage. Recognizing these limitations, this study introduces MPFormer, an innovative framework that leverages a target-conditioned Transformer architecture to encode user behavior and multi-task semantics simultaneously.
MPFormer’s core innovation is its ability to dynamically adjust the importance of different objectives based on real-time user interactions. By integrating learnable target embeddings into the attention mechanism, the model captures complex semantic relationships across multiple goals, enabling a unified, scalable retrieval process. The framework also employs a dynamic quota mechanism, which allocates retrieval resources adaptively, ensuring balanced coverage and personalized results.
Deployed on Kuaishou’s platform, MPFormer has demonstrated remarkable performance improvements. Offline evaluations show a 12.71% increase in recall and enhanced ranking quality, while online AB tests reveal a 15% increase in user engagement and a 21.8% boost in multi-objective exposure. The system maintains near-perfect availability and reduces GPU memory consumption by nearly one-third, confirming its efficiency and robustness at industrial scale.
This work addresses critical bottlenecks in industrial recommendation systems, offering a scalable, resource-efficient, and highly adaptable solution for multi-objective personalized retrieval. Its success paves the way for smarter, more responsive recommendation engines capable of handling complex, evolving user preferences. Future research will focus on integrating reinforcement learning for even finer control, expanding multi-modal data usage, and further enhancing cold-start performance, aiming to revolutionize industrial recommendation paradigms.
Deep Dive
Plain Language Accessible to non-experts
想象你在一家大型厨房里做饭。以前，每个厨师只用一种专门的厨具，只做一种菜，效率低，还浪费材料。而MPFormer就像一个聪明的厨师，他用一个多功能的厨具，可以根据不同的菜肴自动调节，既能做出多样的菜，又能节省时间和材料。这个厨师还能根据当天的食材新鲜度和客人的口味偏好，动态调整菜谱比例，确保每道菜都符合需求。这样一来，不仅厨房效率大大提高，菜肴也更符合客人期待，整个厨房变得更加智能和高效。
ELI14 Explained like you're 14
想象你在学校的食堂里吃饭。有很多不同的菜，比如炒菜、汤和沙拉。以前，厨师每次只做一种菜，想吃不同的菜就得换厨师或者等很久。而现在，有个聪明的厨师，他用一种特别的厨具，可以根据你喜欢的菜，自动调整做菜的比例。比如你今天想多吃点炒菜，厨师就会多做一些；如果你喜欢汤，就多做汤。这个厨师还能根据你平时喜欢的菜和当天的心情，动态调节菜的份量。这样一来，不管你喜欢什么，都能快速满足，而且厨房的效率也变得更高了。这就像MPFormer一样，能根据不同目标，智能调节检索内容，让用户得到更满意的推荐。
Abstract
Modern industrial recommendation systems encounter a core challenge of multi-stage optimization misalignment: a significant semantic gap exists between the multi-objective optimization paradigm widely used in the ranking phase and the single-objective modeling in the retrieve phase. Although the mainstream industry solution achieves multi-objective coverage through parallel multi-path single-objective retrieval, this approach leads to linear growth of training and serving resources with the number of objectives and has inherent limitations in handling loosely coupled objectives. This paper proposes the MPFormer, a dynamic multi-task Transformer framework, which systematically addresses the aforementioned issues through three innovative mechanisms. First, an objective-conditioned transformer that jointly encodes user behavior sequences and multi-task semantics through learnable attention modulation; second, personalized target weights are introduced to achieve dynamic adjustment of retrieval results; finally, user personalization information is incorporated into token representations and the Transformer structure to further enhance the model's representation ability. This framework has been successfully integrated into Kuaishou short video recommendation system, stably serving over 400 million daily active users. It significantly improves user daily engagement and system operational efficiency. Practical deployment verification shows that, compared with traditional solutions, it effectively optimizes the iterative paradigm of multi-objective retrieval while maintaining service response speed, providing a scalable multi-objective solution for industrial recommendation systems.