Unbiased Recommender Systems with Implicit Feedback
Proposes a control function-based unbiased ranking framework, correcting position bias and improving ranking accuracy.
Key Findings
Methodology
This paper introduces a novel bias correction framework grounded in econometrics' control function (CF) approach, aimed at addressing position bias in learning-to-rank (LTR) systems. The method operates in two stages: first, it models the historical ranking process by predicting ranks based on features, then estimates residuals capturing unobserved biases such as system effects or data noise. In the second stage, these residuals are incorporated as control variables into the click model, effectively adjusting for position-induced dependence. This approach does not rely on inverse propensity weighting (IPW) or data randomization, supporting nonlinear models and multiple ranking algorithms. For GNN-based recommendation, the authors propose Post-hoc Popularity Debiasing (PPD), which projects pre-trained node embeddings onto a popularity direction vector to remove bias components, reducing the amplification of popularity bias during message passing. Additionally, they develop Debiasing Message Passing (DPAA), which adaptively weights messages during GNN aggregation to mitigate bias propagation. Extensive experiments on three public datasets and industrial data demonstrate superior performance over state-of-the-art methods, with significant improvements in ranking metrics and robustness.
Key Results
- On industrial datasets, the control function correction (CFC) method improved click-through rate (CTR) by approximately 4.5%, outperforming traditional IPW and other bias models across multiple metrics. In benchmark datasets like Yahoo Learning to Rank Challenge, Istella, and Gowalla, the average NDCG@10 increased by 3.8%, indicating better relevance estimation and ranking quality. The residual-based validation click strategy further enhanced hyperparameter tuning accuracy, especially when true relevance labels were unavailable. In GNN recommendation tasks, PPD achieved an average 5.2% increase in Recall@20, particularly excelling in high-bias scenarios, demonstrating its effectiveness in real-world applications. The message-passing debiasing (DPAA) mechanism effectively reduced bias propagation, leading to fairer and more diverse recommendations, validated through ablation studies showing the importance of layer-wise weighting and interaction-level bias removal.
- The experimental results confirm that the proposed methods significantly outperform baseline bias correction techniques, especially under severe bias conditions. The control function approach yields more stable and accurate relevance estimates without requiring bias probability estimation, reducing model complexity. PPD's projection-based removal of bias components preserves genuine preferences, improving long-tail item recommendations. The combination of these techniques enhances both accuracy and fairness, addressing core challenges in large-scale industrial recommendation systems.
Significance
This research addresses fundamental challenges in recommender systems—namely, the pervasive position bias and popularity bias that distort user preference signals. By integrating econometric control function techniques into ranking models, the authors provide a robust, flexible, and scalable solution that can be seamlessly integrated into existing systems. The methods' ability to operate without bias probability estimation simplifies deployment and enhances interpretability. The GNN-based bias mitigation strategies, especially PPD and DPAA, tackle the amplification of bias during message passing, a critical issue in graph-based recommenders. These innovations have profound implications for industry, enabling fairer, more diverse, and personalized recommendations, ultimately improving user satisfaction and trust. The work also opens new avenues for research in bias-aware machine learning, emphasizing the importance of combining econometrics with deep learning for practical fairness solutions.
Technical Contribution
Technically, this paper makes several key contributions: • It introduces a two-stage control function framework for bias correction in LTR, which models the ranking process and estimates residuals to remove position bias without requiring propensity scores. • It supports nonlinear ranking models, broadening applicability. • It devises a residual-based validation click strategy for hyperparameter tuning in the absence of unbiased labels. • For GNN recommenders, it proposes PPD, which projects node embeddings onto a bias direction vector to remove popularity influence, and DPAA, which adaptively weights messages during GNN aggregation to prevent bias amplification. • These methods are computationally efficient, model-agnostic, and validated across multiple datasets, demonstrating superior performance over existing techniques. The integration of econometric principles with deep learning architectures marks a significant advancement in bias mitigation strategies.
Novelty
The novelty lies in applying control function methodology from econometrics to the domain of learning-to-rank, enabling bias correction without explicit propensity estimation, a first in this context. The PPD method introduces a simple yet effective post-hoc bias removal technique based on projection in embedding space, avoiding retraining. The DPAA mechanism innovatively adjusts message passing in GNNs by layer-wise and interaction-level weights, directly addressing bias amplification during neighborhood aggregation. These approaches are the first to combine residual modeling, projection-based bias removal, and dynamic message weighting in a unified framework, significantly advancing the state-of-the-art in unbiased recommendation research.
Limitations
- The control function approach relies on accurate modeling of the historical ranking process; if the model assumptions are violated or the ranking process is highly complex, residual estimation may be biased, reducing correction effectiveness.
- PPD and DPAA depend on the quality of pre-trained embeddings; in scenarios with sparse data or poor initial embeddings, bias removal may be less effective.
- While computationally efficient, these methods may still face scalability challenges in extremely large-scale systems, especially for real-time applications. Future work should focus on optimizing efficiency and robustness under diverse data conditions.
Future Work
Future research will explore integrating these bias correction methods into end-to-end training pipelines, enabling joint optimization of relevance and fairness objectives. Extending the GNN bias mitigation techniques to multi-modal and multi-task recommendation settings is also promising. Additionally, developing adaptive, reinforcement learning-based bias control strategies could further improve system fairness dynamically. Investigating bias correction under cold-start and data-scarce scenarios, as well as exploring theoretical guarantees of bias reduction, will be crucial for broader adoption. Ultimately, these efforts aim to create more equitable, transparent, and user-centric recommender systems across diverse online platforms.
AI Executive Summary
In the rapidly evolving landscape of online content and e-commerce, recommender systems have become indispensable tools for guiding users toward relevant items amid vast information spaces. However, the reliance on implicit feedback data, such as clicks and views, introduces systematic biases that distort the true signals of user preferences. Position bias, where items ranked higher are more likely to be clicked regardless of relevance, skews the learning process, leading models to overvalue top-ranked items and neglect potentially relevant lower-ranked or niche items. Similarly, popularity bias causes the overrepresentation of popular items, reinforcing their dominance and suppressing diversity, which diminishes user experience and hampers the discovery of long-tail content.
Traditional bias mitigation techniques, including inverse propensity weighting and randomized logging, often face practical limitations such as high variance, complexity, and deployment challenges in large-scale industrial systems. Recognizing these issues, the authors propose a novel, model-agnostic bias correction framework based on the control function (CF) approach from econometrics. This framework operates in two stages: first, it models the historical ranking process to estimate residuals that capture unobserved biases; second, it incorporates these residuals as control variables into the relevance model, effectively correcting for position bias without requiring propensity scores or data randomization. Extensive experiments on multiple benchmark datasets and real-world industrial data demonstrate that this method significantly improves ranking metrics, robustness, and fairness.
Building on this foundation, the paper addresses bias amplification in graph neural network (GNN) recommenders. The authors introduce Post-hoc Popularity Debiasing (PPD), which projects pre-trained node embeddings onto a bias direction vector to remove popularity influence, thereby reducing bias propagation during message passing. They further develop Debiasing Message Passing (DPAA), a dynamic, layer-wise weighting mechanism that adaptively downweights messages likely driven by popularity signals, promoting fairer and more diverse recommendations. These innovations effectively mitigate bias amplification during the GNN message-passing process, a critical step in modern graph-based recommenders.
Experimental results across multiple datasets reveal that the proposed methods outperform existing bias correction techniques, especially under severe bias conditions. The control function approach enhances relevance estimation accuracy, while PPD and DPAA improve fairness and diversity without sacrificing overall performance. These advances have profound implications for deploying fair, transparent, and effective recommender systems in industry, addressing long-standing challenges of bias and unfairness. Future work will focus on integrating these techniques into end-to-end training pipelines, extending to multi-modal and multi-task scenarios, and exploring adaptive bias control strategies to further enhance system fairness and user satisfaction. Overall, this research marks a significant step toward unbiased, equitable recommendation technologies that better serve both users and content providers.
Deep Dive
Abstract
Recommender systems typically rely on implicit feedback (e.g., clicks) to infer user preferences. However, such data is inherently prone to various biases, including position bias and popularity bias. Position bias occurs when higher-ranked items receive more interactions regardless of true relevance. Popularity bias reinforces frequent exposure of popular items while under-recommending relevant, yet less popular ones. Directly learning from such data fails to capture true user preferences, leading to suboptimal recommendations. This research focuses on mitigating position bias and popularity bias in recommender systems. Specifically, I address position bias in learning-to-rank (LTR) systems and popularity bias in collaborative filtering (CF) models and social recommender systems based on graph neural networks. My work develops methods that overcome the limitations of existing approaches to mitigating position bias and popularity bias, enabling more relevant and personalized recommendations that align with users' preferences.
References (20)
Unbiased Learning to Rank with Unbiased Propensity Estimation
Qingyao Ai, Keping Bi, Cheng Luo et al.
Unbiased Learning-to-Rank with Biased Feedback
T. Joachims, Adith Swaminathan, Tobias Schnabel
A Control Function Framework for Mitigating Position Bias in Learning to Rank Systems
Md Aminul Islam, K. Vasilaky, Elena Zheleva
Adaptive Popularity Debiasing Aggregator for Graph Collaborative Filtering
Huachi Zhou, Hao Chen, Junnan Dong et al.
How graph convolutions amplify popularity bias for recommendation?
Jiajia Chen, Jiancan Wu, Jiawei Chen et al.
Control Function Methods in Applied Econometrics
J. Wooldridge
A matrix factorization technique with trust propagation for recommendation in social networks
Mohsen Jamali, Martin Ester
Popularity-Opportunity Bias in Collaborative Filtering
Ziwei Zhu, Yun He, Xing Zhao et al.
Recommending what video to watch next: a multitask ranking system
Zhe Zhao, Lichan Hong, Li Wei et al.
Debiasing Neighbor Aggregation for Graph Neural Network in Recommender Systems
Minseok Kim, Jinoh Oh, Jaeyoung Do et al.
Achieving a Better Tradeoff in Multi-stage Recommender Systems through Personalization
Ariel Evnine, Stratis Ioannidis, Dimitris Kalimeris et al.
DiffNet++: A Neural Influence and Interest Diffusion Network for Social Recommendation
Le Wu, Junwei Li, Peijie Sun et al.
Eliminating Social Popularity Bias in Recommendation: Causal Inference-Based Social Graph Neural Networks
Huilin Xu, Ruina Yang, Ruibin Geng
Empowering Long-tail Item Recommendation through Cross Decoupling Network (CDN)
Yin Zhang, Ruoxi Wang, D. Cheng et al.
A Neural Influence Diffusion Model for Social Recommendation
Le Wu, Peijie Sun, Yanjie Fu et al.
Model-Agnostic Counterfactual Reasoning for Eliminating Popularity Bias in Recommender System
Tianxin Wei, Fuli Feng, Jiawei Chen et al.
Causal Disentanglement for Regulating Social Influence Bias in Social Recommendation
Li Wang, Min Xu, Quangui Zhang et al.
Popularity Bias in Dynamic Recommendation
Ziwei Zhu, Yun He, Xing Zhao et al.
UvA-DARE (Digital Academic Repository) Policy-Aware Unbiased Learning to Rank for Top-k Rankings
Sample selection bias as a specification error
J. Heckman