DCMT: A Direct Entire-Space Causal Multi-Task Framework for Post-Click Conversion Estimation
DCMT framework improves CVR AUC by 1.07%, addressing selection bias and data sparsity.
Key Findings
Methodology
The DCMT framework integrates causal inference and multi-task learning to directly debias selection bias across the entire exposure space D. By introducing a counterfactual mechanism, DCMT predicts both factual and counterfactual CVR under a soft constraint of counterfactual prior knowledge, enhancing prediction accuracy.
Key Results
- DCMT improves CVR AUC by an average of 1.07% across five offline datasets and boosts PV-CVR by 0.75% in the Alipay Search online A/B test.
- Compared to existing methods, DCMT excels in addressing data sparsity and selection bias, especially in industrial applications.
- Ablation studies show the counterfactual mechanism significantly enhances model performance.
Significance
The DCMT framework significantly improves CVR prediction accuracy by directly debiasing selection bias across the entire exposure space. This method holds substantial significance in both academia and industry, particularly in e-commerce and advertising recommender systems.
Technical Contribution
Building on existing multi-task learning and causal inference methods, DCMT introduces a novel counterfactual mechanism that enables unbiased estimation across the entire exposure space. This innovation provides new insights into addressing selection bias and data sparsity.
Novelty
DCMT is the first framework to directly debias selection bias across the entire exposure space, offering a new perspective on CVR prediction compared to traditional methods.
Limitations
- DCMT's accuracy in generating counterfactual samples needs improvement, which may affect final predictions.
- Performance may degrade when handling extremely sparse data.
Future Work
Future research could focus on improving the accuracy of counterfactual sample generation and validating DCMT's performance on larger datasets.
AI Executive Summary
In recommender systems, Click-Through Rate (CTR) and post-click Conversion Rate (CVR) predictions are fundamental tasks. However, selection bias and data sparsity have long plagued these tasks, leading to decreased prediction accuracy. Existing methods primarily rely on multi-task learning frameworks or causal debiasing frameworks, but these approaches fail to effectively address the not-missing-at-random problem.
To tackle these issues, this paper proposes a Direct entire-space Causal Multi-Task framework (DCMT), which introduces a counterfactual mechanism to debias selection bias. DCMT can predict both factual and counterfactual CVR under soft constraints, enhancing prediction accuracy. Experimental results demonstrate that DCMT improves CVR AUC by an average of 1.07% across five offline datasets and boosts PV-CVR by 0.75% in the Alipay Search online A/B test.
The DCMT framework holds significant implications for both academia and industry. By directly debiasing selection bias across the entire exposure space, DCMT offers new insights into solving long-standing challenges in recommender systems. However, the accuracy of counterfactual sample generation still needs improvement, and future research could further optimize this mechanism.
Deep Analysis
Background
Recommender systems play a crucial role in fields like e-commerce, search engines, and social networks. Click-Through Rate (CTR) and post-click Conversion Rate (CVR) predictions are core tasks in these systems. However, selection bias and data sparsity have long challenged these tasks. Existing methods mainly rely on multi-task learning frameworks (e.g., ESMM) or causal debiasing frameworks (e.g., Multi-IPW), but they fail to effectively address the not-missing-at-random problem.
Core Problem
In recommender systems, selection bias and data sparsity lead to decreased CTR and CVR prediction accuracy. Selection bias arises from the discrepancy between training data distribution and target population distribution, while data sparsity results from insufficient click and conversion samples. These issues cause discrepancies between offline training performance and online testing metrics.
Innovation
The DCMT framework introduces a counterfactual mechanism to directly debias selection bias across the entire exposure space. This innovation allows DCMT to predict both factual and counterfactual CVR under soft constraints, enhancing prediction accuracy. Compared to traditional methods, DCMT offers a new perspective on CVR prediction.
Methodology
- �� Input features are divided into deep and wide features, combining generalized linear structures and multi-layer perceptrons.
- �� A counterfactual mechanism is used to predict both factual and counterfactual CVR.
- �� Soft constraints of counterfactual prior knowledge ensure prediction accuracy.
- �� Input features are shared through a multi-task learning framework, improving model generalization.
Experiments
Experiments were conducted on five real-world public datasets and an online A/B test on the Alipay Search system. Baselines include ESMM and Multi-IPW. Metrics include CVR AUC and PV-CVR. Ablation studies validate the effectiveness of the counterfactual mechanism.
Results
Results show that DCMT improves CVR AUC by an average of 1.07% across five offline datasets and boosts PV-CVR by 0.75% in the Alipay Search online A/B test. Ablation studies indicate the counterfactual mechanism significantly enhances model performance.
Applications
The DCMT framework can be directly applied to e-commerce platforms and advertising recommender systems to improve user conversion rates. Its advantages in addressing selection bias and data sparsity make it valuable for industrial applications.
Limitations & Outlook
DCMT's accuracy in generating counterfactual samples needs improvement, which may affect final predictions. Additionally, performance may degrade when handling extremely sparse data. Future research could focus on optimizing the counterfactual mechanism and validating DCMT's performance on larger datasets.
Plain Language Accessible to non-experts
Imagine a library with many books (exposure samples), but only a few are borrowed (click samples), and even fewer are recommended (conversion samples). DCMT acts like a smart librarian who not only recommends books based on borrowing records but also considers those that haven't been borrowed but might be of interest. This way, DCMT can more accurately predict which books will be recommended, improving the library's overall recommendation efficiency.
ELI14 Explained like you're 14
Imagine you're playing a game where you have to choose which items other players will like. You have some data, like which items were clicked and which were bought. DCMT is like a super helper that not only looks at this data but also considers items that weren't clicked but might be liked. This way, you can better predict which items will be popular and score higher!
Glossary
Selection Bias
Selection bias refers to the discrepancy between the training data distribution and the target population distribution, leading to inaccurate model predictions.
In the CVR task, selection bias arises from user self-selection behavior.
Data Sparsity
Data sparsity refers to the insufficient number of click and conversion samples, making it difficult to train models.
Data sparsity is a common issue in recommender systems.
Counterfactual Mechanism
A counterfactual mechanism predicts possible outcomes for unobserved samples, improving model prediction accuracy.
DCMT uses a counterfactual mechanism to debias selection bias.
Multi-Task Learning
Multi-task learning is a method that trains multiple related tasks simultaneously, improving model generalization.
DCMT uses a multi-task learning framework to share input features.
Click-Through Rate (CTR)
CTR is the probability of a user clicking on an item, a key metric in recommender systems.
CTR prediction is an auxiliary task in the DCMT framework.
Open Questions Unanswered questions from this research
- 1 How can the accuracy of counterfactual sample generation be improved? Current methods still fall short in this area.
- 2 How can DCMT's performance be further enhanced under extremely sparse data conditions?
- 3 How can DCMT's performance be validated on larger datasets?
Applications
Immediate Applications
E-commerce Platforms
DCMT can improve user conversion rates on e-commerce platforms, helping them better recommend products.
Advertising Recommender Systems
By eliminating selection bias, DCMT can improve the accuracy of ad recommendations, increasing ad revenue.
Long-term Vision
Intelligent Recommender Systems
DCMT's counterfactual mechanism can provide new insights for future intelligent recommender systems, driving personalized recommendation development.
Abstract
In recommendation scenarios, there are two long-standing challenges, i.e., selection bias and data sparsity, which lead to a significant drop in prediction accuracy for both Click-Through Rate (CTR) and post-click Conversion Rate (CVR) tasks. To cope with these issues, existing works emphasize on leveraging Multi-Task Learning (MTL) frameworks (Category 1) or causal debiasing frameworks (Category 2) to incorporate more auxiliary data in the entire exposure/inference space D or debias the selection bias in the click/training space O. However, these two kinds of solutions cannot effectively address the not-missing-at-random problem and debias the selection bias in O to fit the inference in D. To fill the research gaps, we propose a Direct entire-space Causal Multi-Task framework, namely DCMT, for post-click conversion prediction in this paper. Specifically, inspired by users' decision process of conversion, we propose a new counterfactual mechanism to debias the selection bias in D, which can predict the factual CVR and the counterfactual CVR under the soft constraint of a counterfactual prior knowledge. Extensive experiments demonstrate that our DCMT can improve the state-of-the-art methods by an average of 1.07% in terms of CVR AUC on the five offline datasets and 0.75% in terms of PV-CVR on the online A/B test (the Alipay Search). Such improvements can increase millions of conversions per week in real industrial applications, e.g., the Alipay Search.