CS3: Efficient Online Capability Synergy for Two-Tower Recommendation

Recommender systems play a crucial role in modern information retrieval, evolving from simple collaborative filtering to complex deep learning models. The two-tower model, as a lightweight architecture, is widely used for large-scale candidate retrieval. It encodes users and items with two separate networks and computes relevance via dot product or cosine similarity. However, this isolated architecture limits the model's representation capacity and embedding-space alignment. Additionally, with the rise of online learning, recommender systems need to quickly adapt to constantly changing data distributions, posing higher demands on the model's real-time performance and adaptability. Existing solutions like late interaction and knowledge distillation can mitigate these issues but often increase latency or are difficult to deploy in online learning settings.

Traditional two-tower models face limitations in representation capacity, embedding-space alignment, and cross-feature interactions. These issues constrain the model's performance in large-scale candidate retrieval, especially in online learning environments where constantly changing data distributions make it difficult for the model to quickly adapt. Moreover, existing solutions like late interaction and knowledge distillation, while mitigating these issues, often increase latency or are difficult to deploy in online learning settings. Therefore, enhancing the capability synergy of two-tower models while maintaining real-time constraints is a core problem that needs to be addressed.

The CS3 framework offers an efficient solution for capability synergy through three core mechanisms:

• �� Cycle-Adaptive Structure (CAS): Provides self-revision and feature denoising within each tower, improving the model's representation capacity and alignment.

• �� Cross-Tower Synchronization (CTS): Improves alignment through lightweight mutual awareness between towers, enhancing the consistency of user and item embedding spaces.

• �� Cascade-Model Sharing (CMS): Enhances cross-stage consistency by reusing knowledge from downstream models, improving the model's adaptability and performance.

The implementation of the CS3 framework involves several key steps:

• �� Cycle-Adaptive Structure (CAS): Insert CAS layers within each tower to achieve lightweight self-revision and feature denoising through adaptive reweighting and cycle-forward propagation.

• �� Cross-Tower Synchronization (CTS): Maintain cross vectors cache and update user and item positive representations using exponential moving average (EMA), enhancing explicit cross-tower interaction.

• �� Cascade-Model Sharing (CMS): Cache cascade model outputs and update user and item cascade vectors using EMA, enhancing cross-stage computation sharing and reuse.

• �� Online Learning Implementation: Implement efficient CS3 online learning in the production system using parameter servers and embedding servers for distributed synchronization of model parameters and storage of cached vectors.

The experimental design includes testing CS3's performance on three public datasets (TaobaoAd, KuaiRand, RecSys2017). Baseline models include DSSM, IntTower, IHM-DAT, and RCG. Experiments use AUC and LogLoss as evaluation metrics and conduct various ablation studies to verify the effectiveness of each CS3 module. Additionally, online A/B tests are conducted in a large-scale advertising system to evaluate CS3's performance improvement in real-world applications.

Experimental results show that CS3 significantly improves model performance across all tested datasets. For instance, on the KuaiRand dataset, CS3 improves AUC from 0.6194 to 0.6855 and reduces LogLoss from 0.2289 to 0.2198. Deployment in a large-scale advertising system demonstrates up to an 8.36% revenue increase across three scenarios while maintaining millisecond-level latency. Ablation studies further verify the effectiveness of each CS3 module, with Cascade-Model Sharing (CMS) contributing the most to improving model consistency and performance.

The CS3 framework has direct application value in large-scale advertising systems. By enhancing the capability synergy of two-tower models, CS3 can improve the accuracy and user experience of ad recommendations, thereby increasing advertising revenue. Additionally, CS3's design meets real-time and online learning requirements, making it potentially applicable in other types of recommendation systems, such as social media and news recommendations.

Despite CS3's outstanding performance in experiments, its performance improvement on high-dimensional datasets may be limited by computational resources. Additionally, Cascade-Model Sharing may increase model complexity, potentially impacting real-time performance. Future research directions include further optimizing CS3's computational efficiency to accommodate larger datasets and more complex application scenarios. Additionally, exploring CS3's application in other types of recommendation systems, such as social media and news recommendations, could be beneficial.