SafeCRS: Personalized Safety Alignment for LLM-Based Conversational Recommender Systems
SafeCRS employs Safe-SFT and Safe-GDPO to achieve personalized safety alignment, reducing violations by 96.5% while maintaining recommendation quality.
Key Findings
Methodology
This paper introduces SafeCRS, combining Safe Supervised Fine-Tuning (Safe-SFT) and Safe Group Reward-Decoupled Normalization Policy Optimization (Safe-GDPO). The framework integrates structured safety knowledge from content metadata (IPG, ESRB, DDD) and user implicit signals via a trait inference pipeline. Safe-SFT incorporates a safety reasoning module during fine-tuning to filter unsafe items, while Safe-GDPO employs reward normalization to balance relevance and safety, preventing reward collapse. The SafeRec benchmark dataset, built from Reddit-V2 and r/gamingsuggestions conversations, provides structured safety annotations and user trait profiles, enabling systematic evaluation of personalized safety in recommendation tasks. The training pipeline iteratively refines the model to optimize both recommendation utility and safety constraints, demonstrating significant reductions in safety violations and competitive recommendation performance.
Key Results
- On SafeRec, SafeCRS reduces safety violation rates from 20% in baseline models to 0.735%, a 96.5% decrease. Recommendation metrics such as Recall@5 and NDCG@5 improve by 3.7x and 3.3x respectively over traditional RLHF and GRPO baselines. The model maintains high relevance scores while effectively filtering out unsafe content across domains.
- Ablation studies reveal that reward normalization in Safe-GDPO stabilizes multi-objective optimization, avoiding reward conflicts and model instability. The multi-domain experiments confirm the model's robustness and generalization to both movies and games, demonstrating its broad applicability.
- The experimental results highlight that explicit reasoning over personalized safety constraints is essential for safe recommendation, outperforming global filtering and heuristic approaches. The framework effectively balances safety and utility, setting new standards for personalized safety in LLM-based CRS.
Significance
This work addresses a critical gap in personalized AI safety, moving beyond population-level content moderation to user-specific safety constraints. It introduces a novel benchmark, SafeRec, enabling systematic evaluation of safety violations in personalized recommendation systems. The proposed SafeCRS framework offers a scalable, interpretable, and effective solution, with broad implications for deploying safer AI assistants, content platforms, and personalized services. It advances the state-of-the-art in AI safety, aligning recommendations more closely with individual user sensitivities, thus fostering greater trust and user well-being in AI applications.
Technical Contribution
The paper's main technical innovations include the formalization of personalized safety constraints as a user-level satisfaction problem, the development of a structured safety knowledge base integrating content metadata and user traits, and the design of a two-stage training pipeline combining Safe-SFT and Safe-GDPO. The reward normalization mechanism in Safe-GDPO ensures stable multi-objective optimization, preventing reward hacking and collapse. The framework's modular design allows easy extension to new domains and safety dimensions, providing a practical blueprint for future personalized safety systems in LLMs.
Novelty
This is the first comprehensive approach explicitly modeling personalized safety constraints in conversational recommendation systems. It introduces the SafeRec benchmark, which combines content metadata with user trait inference for safety evaluation. The integration of Safe-SFT and Safe-GDPO, especially the reward normalization strategy, represents a significant departure from existing global or heuristic safety methods, enabling fine-grained, user-centric safety guarantees. This work pioneers a new paradigm in AI safety research, emphasizing individual-level constraint satisfaction.
Limitations
- The model's effectiveness depends on the quality and coverage of safety knowledge bases; rare or emerging safety concerns may be missed. Additionally, the inference of user traits from conversation signals may not capture all personal sensitivities, leading to residual risks.
- The training process involves high computational costs due to multi-stage fine-tuning and reward normalization, which may hinder real-time deployment at scale.
Future Work
Future research will focus on incorporating multimodal data (images, audio) to better infer user sensitivities, developing more efficient training algorithms, and expanding safety knowledge bases to cover diverse cultural contexts. Additionally, integrating user feedback in real-time to adapt safety constraints dynamically will be explored, aiming for more adaptive and scalable personalized safety systems.
AI Executive Summary
The rapid evolution of large language models (LLMs) has transformed conversational recommender systems (CRS), enabling more natural and engaging interactions. However, this progress introduces significant safety challenges, particularly in respecting individual user sensitivities. Existing approaches primarily enforce global content policies, which fail to account for personal safety constraints such as trauma triggers, cultural norms, or mental health considerations. Consequently, recommendation outputs may inadvertently harm users, undermining trust and usability.
To address this critical gap, this study proposes SafeCRS, a novel framework that achieves personalized safety alignment through a two-stage training pipeline. The first stage, Safe-SFT, incorporates a safety reasoning module during supervised fine-tuning, enabling the model to identify and filter unsafe items based on structured content metadata and inferred user traits. The second stage, Safe-GDPO, employs a reward normalization mechanism to balance relevance and safety, preventing reward conflicts and ensuring stable multi-objective optimization. This approach effectively reduces safety violations by up to 96.5%, as demonstrated on the newly constructed SafeRec benchmark dataset.
SafeRec is a comprehensive, multi-domain safety evaluation platform built from Reddit-V2 and r/gamingsuggestions conversations, annotated with structured safety labels derived from content metadata and user trait profiles. It enables systematic assessment of personalized safety in recommendation systems, filling a significant gap in current evaluation standards.
Experimental results show that SafeCRS not only drastically reduces safety violations but also maintains or exceeds the recommendation quality of existing models like GPT-4. The framework's modular design allows for broad applicability across domains such as movies and games, with potential extensions to multimodal data and real-time adaptive safety mechanisms. This work marks a substantial step forward in aligning AI systems with individual safety needs, fostering more trustworthy and user-centric AI applications.
Despite these advances, challenges remain in scaling the approach for real-time deployment and expanding safety knowledge bases across cultures. Future efforts will focus on improving efficiency, robustness, and coverage, aiming to realize truly personalized, safe, and effective conversational AI systems.
Deep Dive
Abstract
Current LLM-based conversational recommender systems (CRS) primarily optimize recommendation accuracy and user satisfaction. We identify an underexplored vulnerability in which recommendation outputs may negatively impact users by violating personalized safety constraints, when individualized safety sensitivities -- such as trauma triggers, self-harm history, or phobias -- are implicitly inferred from the conversation but not respected during recommendation. We formalize this challenge as personalized CRS safety and introduce SafeRec, a new benchmark dataset designed to systematically evaluate safety risks in LLM-based CRS under user-specific constraints. To further address this problem, we propose SafeCRS, a safety-aware training framework that integrates Safe Supervised Fine-Tuning (Safe-SFT) with Safe Group reward-Decoupled Normalization Policy Optimization (Safe-GDPO) to jointly optimize recommendation quality and personalized safety alignment. Extensive experiments on SafeRec demonstrate that SafeCRS reduces safety violation rates by up to 96.5% relative to the strongest recommendation-quality baseline while maintaining competitive recommendation quality. Warning: This paper contains potentially harmful and offensive content.