Interplay: Training Independent Simulators for Reference-Free Conversational Recommendation

Conversational Recommender Systems (CRS) have become a significant research direction in artificial intelligence and natural language processing. CRS can provide personalized, context-sensitive recommendations through natural language conversations. Unlike traditional systems that rely on static user-item interactions, CRS allows dynamic, interactive feedback from users and enables both parties to guide the conversation. However, the development of CRS depends on rich conversational data, and collecting human-annotated conversations is costly and limited in quality. Previous work highlights key issues in crowd-sourced CRS datasets: lack of genuine preferences, lack of depth and context, and lack of domain expertise among crowd workers, resulting in weak recommendations and poor explanations. With the growing popularity of large language models (LLMs), researchers have begun using LLM simulators to generate authentic and context-rich conversations, which are critical for training CRS. However, existing methods generally rely on reference-dependent generation, feeding target items to simulators in advance, leading to data leakage and rigid dialogues.

The core problem is how to generate realistic and diverse conversational data for training conversational recommender systems. Traditional simulation methods often use a single large language model (LLM) that generates entire conversations with prior knowledge of the target items, leading to scripted and artificial dialogues. Additionally, using general-purpose LLMs as static role-players for both conversational parties is problematic, as these models are designed as helpful assistants rather than realistic user simulators with diverse and sometimes inconsistent preferences. The resulting conversations often lack the natural exploration, uncertainty, and flexibility that characterize genuine human-recommender interactions.

The core innovation of this paper is the proposal of a reference-free simulation framework by training two independent large language models (LLMs), one as a user simulator and the other as a conversational recommender. These models interact in real-time without access to predetermined target items, inferring user preferences through dialogue. Specific innovations include: 1) the reference-free design eliminates data leakage by ensuring that simulators discover information through conversation rather than having it predetermined; 2) independent training of specialized models creates more realistic user and recommender behaviors than using general-purpose LLMs for both roles; 3) the interactive nature allows flexible conversations that can naturally evolve in multiple directions, better reflecting the complexity of real recommendation scenarios.

The methodology of this paper includes the following steps:

• �� Independently train user and recommender simulators: fine-tune on existing CRS data using a reference-free framework, ensuring no access to target items during generation.
• �� User simulator operation: operates based on preference summaries and attribute descriptions, providing realistic feedback without knowing specific targets.
• �� Recommender simulator generation: generates contextually appropriate recommendations based solely on the evolving conversation.
• �� Structured action generation: structure the entire model output to separate the action and natural language response, ensuring the model commits to an unambiguous action before generating its utterance.
• �� Role-specific loss masking: use a token masking strategy during loss computation, ensuring each simulator learns exclusively from its own turns.

The experimental design includes comprehensive evaluations of both the user simulator and the recommender simulator. Baselines used include open-source models such as Llama3.1 70B and Qwen3 32B, as well as the modular UniCRS system. Evaluation metrics include success rate in multi-turn dialogues, single-turn response quality, and recommender performance (e.g., Recall@1 and Match Score). Experiments are conducted on a filtered test set to ensure fair comparison of user and recommender roles. Human evaluations are also conducted to verify the realism and fluency of reference-free dialogues.

Experimental results show that the user simulator achieves a success rate of 93-95% in multi-turn dialogues, significantly outperforming larger zero-shot LLMs like Llama3.1 70B (36% success rate) and Qwen3 32B (77% success rate), validating that reference-free operation preserves realistic user behavior while eliminating artificial constraints. The recommender simulator RecSim-Qwen8B achieves the highest performance with Recall@1 of 0.0217 and Match Score of 0.9333, significantly outperforming the larger Qwen3-32B baseline and exceeding the modular UniCRS system, demonstrating that fine-tuning smaller models for specific conversational recommendation tasks is more effective. Human evaluations show that reference-free dialogues exhibit significantly more user control and conversational flow than reference-dependent methods, although they fall slightly short in naturalness.

Direct application scenarios of this method include:

• �� Online movie recommendation: generating more realistic dialogue data through the reference-free simulation framework to improve the performance of recommendation systems.
• �� E-commerce platforms: applying to product recommendations, helping users discover new products without preset targets.
• �� Customer service: applying reference-free simulation in customer service dialogues to improve the naturalness and user satisfaction of conversations.

Although the reference-free simulation framework performs well in the movie recommendation domain, its effectiveness in other domains such as e-commerce, music, and travel has not yet been validated, requiring future application and evaluation in different contexts. Additionally, the method employs a structured, task-oriented dialogue framework, which does not capture the full spectrum of real-world interactions, such as open-domain chit-chat and multi-intent utterances. While Match Score is a useful proxy for recommendation quality, it has not yet been validated as a reliable metric that aligns with real user preferences, necessitating future user studies to evaluate the correlation between automatic metrics and human judgments.