Click-Conversion Multi-Task Model with Position Bias Mitigation for Sponsored Search in eCommerce
Proposed PACC and PACC-PE models significantly mitigate position bias in eCommerce search.
Key Findings
Methodology
The paper introduces two models: PACC and PACC-PE. PACC is based on probability decomposition, modeling position information as probability; PACC-PE uses neural networks to model product-specific position information as embeddings. These models jointly learn CTR and CVR predictions in a multi-task learning framework, significantly reducing position bias.
Key Results
- PACC-PE shows a 4.92%/4.07% improvement in weighted MRR/MRR for CVR prediction, outperforming baseline models significantly.
- PACC achieves a 3.27%/2.64% improvement in PAUC/AUC for CTR prediction, surpassing existing models.
- PACC-PE effectively mitigates position bias across all positions, especially in lower-ranked positions.
Significance
This research holds significant implications for academia and industry, particularly in eCommerce. By mitigating position bias, the models enhance the fairness and accuracy of CTR and CVR predictions, addressing long-standing issues of unfair ranking.
Technical Contribution
Technical contributions include a novel multi-task learning framework that simultaneously mitigates position bias in CTR and CVR predictions. Compared to existing methods, PACC and PACC-PE offer new theoretical guarantees and engineering possibilities.
Novelty
PACC and PACC-PE are the first to apply position bias mitigation in a multi-task learning framework, innovatively modeling position information as probability and embeddings, offering significant advantages over traditional single-task methods.
Limitations
- The models may perform poorly in extreme position bias scenarios, especially with sparse data.
- PACC-PE's computational complexity is high, potentially affecting real-time applications.
Future Work
Future work includes optimizing the models' computational efficiency, exploring more application scenarios, and validating the models' generalizability in other fields.
AI Executive Summary
Position bias is a common issue in search ranking systems, where users tend to focus on top-ranked results regardless of their actual relevance. Existing solutions are mostly single-task models that fail to effectively address position bias in both CTR and CVR predictions.
This paper proposes the PACC and PACC-PE models, which use a multi-task learning framework to simultaneously mitigate position bias in CTR and CVR predictions. PACC is based on probability decomposition, while PACC-PE uses neural networks to model product-specific position information as embeddings.
Experimental results show that PACC-PE significantly outperforms existing models in CVR prediction, especially in lower-ranked positions. This research not only improves the fairness and accuracy of predictions but also provides a new solution for ranking systems in eCommerce.
Deep Analysis
Background
Position bias is prevalent in many information retrieval systems, particularly in sponsored search in eCommerce. Users often focus on top-ranked results, ignoring their actual relevance. This bias leads to unfair CTR and CVR predictions, affecting model accuracy.
Core Problem
The core problem is effectively mitigating position bias in CTR and CVR predictions. Position bias causes inaccurate predictions for lower-ranked items, affecting subsequent data collection and model training.
Innovation
This paper introduces the PACC and PACC-PE models. PACC models position information through probability decomposition, while PACC-PE uses neural networks to model product-specific position information as embeddings. This approach simultaneously mitigates position bias in CTR and CVR predictions within a multi-task learning framework.
Methodology
- �� PACC model is based on probability decomposition, modeling position information as probability.
- �� PACC-PE model uses neural networks to model product-specific position information as embeddings.
- �� A multi-task learning framework is used to jointly learn CTR and CVR prediction models.
Experiments
Experiments used Walmart's sponsored ads log dataset, containing 4.2M training samples, 1.1M validation samples, and 7.5M testing samples. Evaluation metrics include weighted MRR, MRR, PAUC, and AUC. Results show PACC-PE significantly outperforms baseline models in CVR prediction.
Results
PACC-PE shows a 4.92%/4.07% improvement in weighted MRR/MRR for CVR prediction. PACC achieves a 3.27%/2.64% improvement in PAUC/AUC for CTR prediction. Results indicate PACC-PE effectively mitigates position bias across all positions.
Applications
The models can be directly applied to sponsored search in eCommerce platforms, improving the fairness and accuracy of CTR and CVR predictions. Results can optimize ad placement strategies and enhance user experience.
Limitations & Outlook
PACC-PE's computational complexity is high, potentially affecting real-time applications. The models may perform poorly in extreme position bias scenarios, especially with sparse data. Future work includes optimizing computational efficiency.
Plain Language Accessible to non-experts
Imagine you're shopping in a supermarket with many products on the shelves. Typically, you'll notice the products at eye level first, ignoring those lower down, even if they might be better for you. The PACC and PACC-PE models act like a smart assistant, helping you evaluate each product fairly, not just based on their shelf position. This way, you can find the best products for you without being misled by their placement.
ELI14 Explained like you're 14
Imagine you're playing a game with lots of treasure chests. Usually, you'd open the most obvious chests first, ignoring those hidden in corners. The PACC and PACC-PE models are like a smart helper, helping you evaluate each chest's value fairly, not just based on their location. This way, you won't miss out on the hidden treasures! Cool, right?
Glossary
Position Bias
Users tend to focus on top-ranked results, ignoring their actual relevance.
In search ranking systems, position bias leads to unfair CTR and CVR predictions.
Click-Through Rate (CTR)
Measures the probability of a user clicking on an ad.
CTR prediction is used to evaluate the attractiveness and effectiveness of ads.
Conversion Rate (CVR)
Measures the probability of a user making a purchase after clicking an ad.
CVR prediction is used to assess the actual sales impact of ads.
Multi-task Learning
A machine learning approach that learns multiple related tasks simultaneously.
In this paper, multi-task learning is used to simultaneously mitigate position bias in CTR and CVR predictions.
Embedding
A technique to map high-dimensional data into a lower-dimensional space.
In the PACC-PE model, embeddings are used to model product-specific position information.
Open Questions Unanswered questions from this research
- 1 How to effectively mitigate position bias in sparse data scenarios?
- 2 How to optimize the computational efficiency of the PACC-PE model for real-time applications?
Applications
Immediate Applications
eCommerce Platforms
Improve the fairness and accuracy of CTR and CVR predictions in sponsored search.
Long-term Vision
Ad Placement Optimization
Optimize ad placement strategies to enhance user experience through fairer predictions.
Abstract
Position bias, the phenomenon whereby users tend to focus on higher-ranked items of the search result list regardless of the actual relevance to queries, is prevailing in many ranking systems. Position bias in training data biases the ranking model, leading to increasingly unfair item rankings, click-through-rate (CTR), and conversion rate (CVR) predictions. To jointly mitigate position bias in both item CTR and CVR prediction, we propose two position-bias-free CTR and CVR prediction models: Position-Aware Click-Conversion (PACC) and PACC via Position Embedding (PACC-PE). PACC is built upon probability decomposition and models position information as a probability. PACC-PE utilizes neural networks to model product-specific position information as embedding. Experiments on the E-commerce sponsored product search dataset show that our proposed models have better ranking effectiveness and can greatly alleviate position bias in both CTR and CVR prediction.