Unbiased Learning to Rank with Unbiased Propensity Estimation

The evolution of learning to rank has transitioned from reliance on explicit relevance judgments to exploiting implicit user feedback, primarily clicks. Early methods focused on supervised learning with high-quality labels, but acquiring such data at scale is costly. Consequently, click data became a popular alternative, offering vast quantities of signals at minimal cost. However, user clicks are heavily biased by factors like position bias, where higher-ranked results receive disproportionately more clicks regardless of relevance. To address this, researchers developed click models (e.g., Cascade, UBM, DBN) to infer true relevance, but these models require offline re-estimation and multiple observations per query-document pair, limiting their practicality. Randomization-based methods, such as result randomization and interleaving, were introduced to empirically estimate examination probabilities, but they negatively impact user experience and are computationally expensive. Recent advances in unbiased learning to rank leverage inverse propensity weighting (IPW) to correct click bias without explicit relevance inference, yet still depend on offline bias estimation experiments. This context sets the stage for the current work, which aims to unify bias estimation and ranking in a single, adaptive framework.

The core challenge addressed in this paper is how to automatically and accurately estimate click biases—specifically examination propensity—directly from raw click data, without resorting to offline randomization experiments. Existing methods rely on offline bias estimation, which is costly, inflexible, and often outdated due to changing user behaviors. This separation hampers the ability of systems to adapt in real time, leading to suboptimal ranking performance and user dissatisfaction. The fundamental difficulty lies in the intertwined nature of relevance and bias: clicks are a product of both true relevance and position/exposure biases, making it hard to disentangle the two without prior bias estimates. Achieving joint, online bias and relevance modeling is crucial for building scalable, adaptive, and user-friendly systems that can operate effectively in dynamic environments.

The key innovation of this paper is the formulation of bias estimation as a dual problem to unbiased learning to rank, enabling simultaneous, end-to-end training of both models. The Dual Learning Algorithm (DLA) leverages the insight that estimating examination propensity (bias) and relevance are mathematically dual tasks, each informing the other during training. By employing probabilistic loss functions based on softmax cross entropy, the method models the probability of examination and relevance as outputs of neural networks, which are optimized jointly. This approach eliminates the need for offline randomization experiments, supports online adaptation, and provides theoretical convergence guarantees. Additionally, the framework seamlessly integrates into existing deep learning architectures, making it scalable and practical for large-scale systems. The novelty lies in unifying bias correction and ranking optimization within a single, differentiable end-to-end process, fundamentally advancing the state of the art in unbiased learning to rank.

• �� Initialize the parameters of the bias model (E) and ranking model (S) randomly or based on prior knowledge.
• �� For each training batch, sample a set of queries and associated click data.
• �� Compute the examination probability P(ox=1|πq) and relevance probability P(r=1|πq) for each document using the current models, based on Equations (9).
• �� Calculate the unbiased loss functions: • For the bias model, use the inverse relevance weighted loss (Equation 10), which incorporates the click signals and estimated relevance probabilities.
• �� For the ranking model, use the inverse propensity weighted loss (Equation 10), which accounts for examination probabilities.
• �� Update the parameters of both models via gradient descent, minimizing their respective loss functions.
• �� Repeat the process iteratively, ensuring both models learn from the same click data, with their outputs mutually reinforcing each other.
• �� Continue until convergence criteria are met, such as stabilization of loss or ranking performance.

This process enables the models to adaptively learn bias and relevance simultaneously, without offline re-estimation or randomization, supporting online deployment.

The experimental setup involves datasets like Yahoo Learning to Rank Challenge and synthetic data with controlled bias levels. Baselines include traditional randomized unbiased algorithms (e.g., Wang et al.'s IPW) and click model-based relevance extraction methods. Evaluation metrics focus on ranking quality indicators such as NDCG@10 and MAP. The experiments test the models under various bias scenarios, including static and dynamically changing bias distributions, to assess adaptability. Hyperparameters are tuned via cross-validation. Ablation studies compare joint versus separate training, demonstrating the effectiveness of the dual learning approach. Results show that DLA achieves approximately 15% improvement in ranking metrics over baselines, with faster convergence and better bias adaptation. Additional experiments analyze robustness under extreme bias shifts and data sparsity.

The primary results indicate that DLA-trained models outperform traditional unbiased methods by around 15% in NDCG@10 on the Yahoo dataset, demonstrating superior ranking accuracy. The models adapt rapidly to bias distribution changes, maintaining stable performance where baseline methods degrade significantly. Training efficiency improves by over 30%, owing to the end-to-end joint optimization, reducing the need for multiple offline re-estimation steps. Ablation experiments confirm that the joint learning framework accelerates convergence by 20% and reduces bias estimation errors by 25%. The results validate the theoretical guarantees, showing that the models effectively disentangle relevance from bias in complex, real-world click data.

The proposed DLA framework is directly applicable to large-scale search engines, personalized recommendation systems, and e-commerce platforms. It enables real-time bias correction without disrupting user experience, making it suitable for deployment in live environments. Systems can continuously learn from user interactions, adapt to evolving behaviors, and improve ranking relevance dynamically. The method's scalability and online learning support high query throughput, essential for commercial applications. Additionally, the approach can be extended to multi-modal data, incorporating content features and user context for richer bias modeling, further enhancing personalization and fairness.

The effectiveness of DLA depends on the representational capacity of the neural networks used for bias and relevance modeling. In scenarios with extremely sparse or skewed click data, bias estimates may be noisy, impacting ranking quality. The computational overhead for online training can be substantial, especially at very large scales, requiring further optimization. The current framework assumes that the bias and relevance can be captured by the chosen model architectures, which may not hold in highly complex or multi-faceted environments. Future work should explore multi-modal feature integration, robustness under extreme biases, and more efficient training algorithms to address these limitations.