DADF: A Distribution-Aware Debiasing Framework for Watch-Time Regression in Recommender Systems
DADF framework improves watch-time prediction accuracy by 12.57% using distribution-aware debiasing.
Key Findings
Methodology
DADF is a distribution-aware debiasing framework applied to a frozen watch-time predictor. It employs multiplicative correction and group-specific distribution transformations, using video duration to route correction experts and integrating auxiliary engagement representations. The aim is to correct inference-time predictable conditional residuals rather than global recalibration.
Key Results
- On KuaiRec and WeChat21 datasets, DADF reduces MAE by 4.33% and improves XAUC by 4.01% on average.
- In production, it reduces MAE by 12.57%.
- Online A/B tests show an increase in average time spent per device by 0.649%, 0.235%, and 0.199%.
Significance
The DADF framework enhances watch-time prediction accuracy in recommender systems by correcting inference-time predictable conditional residuals. It provides a low-risk improvement path without replacing mature first-stage models, demonstrating its practicality as a model-agnostic plug-in.
Technical Contribution
DADF introduces a distribution-aware second-stage correction to address long-tail distribution and heterogeneous residuals. Its unique duration-indexed expert routing and multi-label auxiliary representation learning offer new engineering possibilities, achieving accuracy improvements without altering existing model interfaces.
Novelty
DADF is the first to propose a distribution-aware second-stage correction framework, focusing on inference-time predictable conditional residual correction rather than global recalibration.
Limitations
- DADF relies on existing model predictions and cannot eliminate all residuals.
- Correction effectiveness may be limited in extreme long-tail distributions.
Future Work
Future research could explore applying DADF to other types of recommender systems or integrating more auxiliary signals to enhance correction accuracy.
AI Executive Summary
Watch-time prediction in short-video recommender systems often suffers from bias, with short observations overestimated and long ones underestimated. Replacing mature models is costly, necessitating a low-risk improvement method.
The DADF framework employs distribution-aware second-stage correction, using video duration to route correction experts and integrating auxiliary engagement representations to correct inference-time predictable conditional residuals. Experiments on KuaiRec and WeChat21 datasets show that DADF reduces MAE by 4.33% and improves XAUC by 4.01% on average.
DADF's successful application demonstrates its practicality as a model-agnostic plug-in, achieving accuracy improvements without altering existing model interfaces. Future research directions include applying DADF to other recommender systems and integrating more auxiliary signals.
Deep Analysis
Background
In short-video recommender systems, watch-time is a crucial metric for measuring user engagement. Traditional prediction methods often exhibit bias, leading to overestimation of short watch times and underestimation of long ones. Replacing mature prediction models is costly, necessitating a low-risk improvement method.
Core Problem
Existing watch-time prediction models exhibit bias in predicting short and long watch times, affecting the accuracy and user experience of recommender systems. Improving prediction accuracy without replacing existing models is a significant challenge.
Innovation
The DADF framework employs distribution-aware second-stage correction, using video duration to route correction experts and integrating auxiliary engagement representations to correct inference-time predictable conditional residuals. Its innovation lies in avoiding global recalibration, reducing implementation risk.
Methodology
- �� DADF freezes the existing predictor and applies multiplicative correction.
- �� Uses video duration to route correction experts.
- �� Integrates auxiliary engagement representations to enhance correction accuracy.
- �� Stabilizes long-tail correction targets through group-specific distribution transformations.
Experiments
Experiments were conducted on KuaiRec and WeChat21 datasets, using seven first-stage models as baselines. Evaluation metrics include MAE and XAUC, with various comparative experiments to validate DADF's effectiveness.
Results
Results show that DADF reduces MAE by 4.33% and improves XAUC by 4.01% on average across multiple datasets. In production, MAE is reduced by 12.57%, demonstrating its effectiveness in real-world applications.
Applications
DADF can be directly applied to short-video recommender systems to improve watch-time prediction accuracy. As a model-agnostic plug-in, it is compatible with various recommender system architectures.
Limitations & Outlook
DADF relies on existing model predictions and cannot eliminate all residuals. Correction effectiveness may be limited in extreme long-tail distributions. Future research could explore integrating more auxiliary signals to enhance correction accuracy.
Plain Language Accessible to non-experts
Imagine you're in a library where the librarian decides where to display books based on how long they're borrowed. The current system sometimes overestimates short-borrowed books and underestimates long-borrowed ones. DADF acts like a smart assistant that adjusts the display positions of books based on their borrowing duration and other information, ensuring each book is correctly displayed. This assistant doesn't change the library's existing system, just makes adjustments on top of it.
ELI14 Explained like you're 14
Imagine you're playing a game with a leaderboard showing each player's playtime. The current system sometimes overestimates players with short playtimes and underestimates those with long playtimes. DADF is like a smart assistant that adjusts the leaderboard based on each player's playtime and other info, ensuring everyone is correctly ranked. This assistant doesn't change the game's existing system, just tweaks it to make it better.
Glossary
DADF (Distribution-Aware Debiasing Framework)
A framework for correcting watch-time prediction bias using multiplicative correction and group-specific distribution transformations.
Used to improve prediction accuracy in short-video recommender systems.
MAE (Mean Absolute Error)
A metric for evaluating prediction model accuracy, representing the average absolute difference between predicted and true values.
Used to evaluate DADF's performance on different datasets.
XAUC (Extended AUC)
A metric for evaluating ranking model performance, considering the distribution of different category samples.
Used to evaluate DADF's performance in ranking tasks.
Multiplicative Correction
A method of adjusting predictions by multiplying by a correction factor.
Used in the DADF framework to correct watch-time prediction bias.
Auxiliary Engagement Representation
A method of improving model prediction accuracy by integrating representations from other related tasks.
A key component in the DADF framework for enhancing correction accuracy.
Open Questions Unanswered questions from this research
- 1 How can the DADF framework be effectively applied to other types of recommender systems?
- 2 How can DADF's correction effectiveness be optimized in extreme long-tail distributions?
Applications
Immediate Applications
Short Video Recommendation
DADF can be used to improve watch-time prediction accuracy in short-video recommender systems, helping platforms optimize content recommendation strategies.
Long-term Vision
Cross-Domain Application
The DADF framework can be extended to other recommender systems, such as e-commerce and music recommendation, to enhance prediction accuracy and user experience.
Abstract
Watch-time predictors in short-video recommender systems can be approximately calibrated by their own scores while still overestimating short observations and underestimating long ones. We study whether this label-space mean shrinkage contains inference-time-predictable residual structure that can be corrected without replacing a mature first-stage model. We propose DADF, a distribution-aware second-stage framework that applies multiplicative correction to a frozen watch-time predictor. DADF stabilizes long-tailed correction targets with group-specific transformations, uses video duration to route specialized correction experts, and incorporates auxiliary engagement representations. Duration is used only to index heterogeneous residual distributions, not treated as the cause of the observed pattern. Experiments on KuaiRec and WeChat21 with seven first-stage backbones, together with a large-scale industrial ranking system, show that DADF reduces offline MAE by 4.33% and improves XAUC by 4.01% on average. In production, it reduces MAE by 12.57%. Three online A/B tests across full ranking, rough ranking, and degraded serving improve average time spent per device by 0.649%, 0.235%, and 0.199%, respectively, and all three integrations were subsequently deployed to 100% of traffic. These results show that DADF is a practical, model-agnostic plug-in for correcting predictable conditional residuals while preserving the serving interface of mature first-stage models. Code is available at https://github.com/liuzhao09/DADF.