Fairness Begins with State: Purifying Latent Preferences for Hierarchical Reinforcement Learning in Interactive Recommendation

Interactive recommender systems have become the foundation of modern digital ecosystems, widely used in short-video streaming and large-scale e-commerce platforms. To handle the complexity of user-system interactions, reinforcement learning (RL) has emerged as the dominant paradigm. However, as these systems increasingly influence information dissemination, the issue of recommendation fairness has become critical. Existing fairness methods often mitigate biases through reward function adjustments or policy output constraints, assuming the observed user state is a faithful representation of true preferences.

Existing fairness methods assume the observed user state is a faithful representation of true preferences, but in reality, implicit feedback is often contaminated by popularity-driven noise and exposure bias, leading to distorted states that mislead the RL agent. This paper argues that the persistent conflict between accuracy and fairness is not merely a reward design issue but fundamentally a state estimation failure.

The paper proposes the DSRM-HRL framework, redefining fairness-aware recommendation as a latent state purification problem combined with hierarchical decision-making. • DSRM module based on diffusion models recovers low-entropy preference manifolds. • High-level policy regulates long-term fairness trajectories. • Low-level policy optimizes short-term engagement under dynamic constraints. This approach addresses the long-standing conflict between accuracy and fairness in recommendation systems by purifying user states.

• �� DSRM module: uses diffusion models to denoise user states. • High-level policy: regulates long-term fairness. • Low-level policy: optimizes short-term engagement. • Diffusion process: progressively injects Gaussian noise to simulate preference signal degradation. • Reverse diffusion: reconstructs preference structure, recovering low-entropy states.

Experiments were conducted on high-fidelity simulators KuaiRec and KuaiRand to validate the effectiveness of DSRM-HRL. Standard RL baselines and state-of-the-art fairness methods were compared, with evaluation metrics including recommendation utility and exposure equity. Ablation studies were used to analyze the contribution of each component.

On KuaiRec, DSRM-HRL achieved an 88% improvement in Absolute Difference and an 18.4% gain in interaction length. On KuaiRand, DSRM-HRL reached a superior Pareto frontier between recommendation utility and exposure equity. Ablation studies show that the denoising state representation module is crucial for achieving fairness and utility improvements.

DSRM-HRL can be directly applied in scenarios requiring a balance between recommendation utility and fairness, such as short-video platforms and e-commerce websites. Its core lies in improving the exposure opportunities for long-tail items through state purification, thereby enhancing user experience and platform utility.

DSRM-HRL may be unstable in extreme noise environments, affecting recommendation accuracy. It requires significant computational resources for training, potentially unsuitable for resource-constrained applications. The user preference modeling assumptions may not apply to all types of recommendation systems.