A Nonparametric Delayed Feedback Model for Conversion Rate Prediction

The evolution of online advertising has increasingly emphasized accurate conversion rate (CVR) prediction. Early models relied on logistic regression or SVMs, which treated click-to-conversion delay as negligible. As understanding deepened, delayed feedback emerged as a key factor affecting model accuracy. Chapelle et al. introduced exponential delay assumptions, but these often failed to match real data's multimodal and complex delay patterns. Recent work extended to Weibull distributions, yet still relied on parametric assumptions. Survival analysis offers a flexible framework for modeling event times without fixed distribution assumptions, but its application in advertising remains limited. This paper builds on these foundations, proposing a nonparametric hazard-based approach to better capture real-world delay behaviors.

The core challenge lies in modeling the diverse and intricate delay distributions from ad click to user conversion. Fixed parametric models, like exponential or Weibull, cannot accommodate multimodal or irregular patterns, leading to biased CVR estimates. Misclassification of unobserved conversions as negatives hampers model training, reducing predictive accuracy. The difficulty is compounded by high-dimensional feature spaces and the need for real-time inference. Developing a flexible, data-driven delay distribution model that adapts to content and user heterogeneity is essential for improving ad effectiveness measurement and user engagement analysis.

First, the paper introduces a nonparametric hazard function based on kernel density ideas, enabling flexible delay distribution modeling. Second, it employs feature-dependent intensity functions, allowing the hazard to vary with ad and user characteristics. Third, the integration of hidden variables with EM algorithm facilitates robust estimation of eventual conversions and delay distributions simultaneously. This approach overcomes the rigidity of parametric assumptions, capturing complex, multimodal delay behaviors. Additionally, the model's modular design allows seamless incorporation of various classifiers, enhancing its adaptability across different prediction tasks. These innovations collectively advance the state-of-the-art in delayed feedback modeling.

• �� Pseudo-point placement: Equally spaced points {tl} on the time axis serve as basis for hazard function. • Hazard function: Defined as a weighted sum of kernel functions \( h(d; x, V) = \sum_{l=1}^L \alpha_l(x; V) k(t_l, d) \), where \( \alpha_l \) depends on features \( x \) and parameters \( V \). • Kernel choice: Gaussian kernels \( k(t_l, d) = \exp(- (t_l - d)^2 / 2h^2) \) enable smooth, flexible density estimation. • Survival function: Computed via \( s(d; x, V) = \exp(- \int_0^d h( au; x, V) d au) \), describing the probability of no conversion before time \( d \). • Hidden variables: \( c_i \in \{0,1\} \) indicating eventual conversion, estimated via EM. • EM steps: E-step computes posterior \( q_{ic} \), M-step updates parameters \( V \) and classifier weights \( w \) using gradient-based optimization. • Regularization: \( \ell_2 \) penalties applied to prevent overfitting. This iterative process refines the hazard and classification models to fit observed data.

• �� Synthetic data: Generated from multimodal distributions with known shapes, testing model’s ability to recover complex delay patterns. • Real data: Criteo logs with click and conversion timestamps, split into training, validation, and test sets. • Evaluation metrics: AUC, log loss, prediction accuracy. • Hyperparameter tuning: Cross-validation used to select pseudo-point number \( L \), kernel bandwidth \( h \), and regularization parameters \( \lambda \). • Baselines: Exponential delay model (Chapelle), naive logistic regression treating unobserved as negative. • Results: NoDeF consistently outperforms baselines, accurately estimates delay distributions, and improves CVR prediction metrics across datasets.

• �� Synthetic experiments show NoDeF accurately captures multimodal delay distributions, with density estimates closely matching true shapes. • On Criteo, NoDeF improves AUC by approximately 0.02 and reduces log loss by 0.01 compared to exponential models. • Hyperparameter sensitivity analysis indicates robustness; model adapts well to different delay patterns, confirming its flexibility and practical effectiveness.