LifeSim: Long-Horizon User Life Simulator for Personalized Assistant Evaluation

In recent years, with the rapid development of large language models (LLMs), the vision of universal AI assistants is becoming increasingly achievable. Existing research has primarily focused on optimizing the model's ability to handle complex and knowledge-intensive tasks, as well as improving its social intelligence. However, there remains a clear gap between current evaluation frameworks and real-world scenarios, constraining advances in personalized intelligence. Ideal user-assistant interactions fundamentally differ from isolated question answering and involve complex external environments and users' cognitive states. User needs vary in terms of situational factors such as time, location, weather, and ongoing life events. User intentions arise from internal cognitive states, jointly shaped by evolving life experiences and relatively stable personalities and preferences. Real-world user data are constrained by privacy and ethical considerations, and publicly available interaction logs spanning multiple years and diverse scenarios remain extremely scarce. Therefore, establishing a realistic testbed with long-term user-assistant interactions at scale poses a fundamental problem.

Existing benchmarks for personalized assistants fail to capture the complexity of external contexts and users' cognitive states, leading to a misalignment with real-world user-assistant interactions. This mismatch constrains advances in personalized intelligence, as ideal user-assistant interactions involve complex external environments and users' cognitive states, rather than isolated question answering. User needs vary in terms of situational factors such as time, location, weather, and ongoing life events. User intentions arise from internal cognitive states, jointly shaped by evolving life experiences and relatively stable personalities and preferences.

The core innovation of LifeSim lies in its use of the Belief-Desire-Intention (BDI) model to simulate user cognition within physical environments, generating coherent life trajectories and simulating intention-driven user interactive behaviors. Compared to existing static or short-context datasets, LifeSim provides a high-fidelity long-horizon user-assistant interaction simulation framework. LifeSim-Eval is a comprehensive benchmark covering 8 life domains and 1,200 diverse scenarios, adopting a multi-turn interactive method to assess models' abilities to complete explicit and implicit intentions, recover user profiles, and produce high-quality responses.

• �� LifeSim models user cognition through the Belief-Desire-Intention (BDI) model within physical environments.
• �� An event engine generates life trajectories, and a user behavior engine produces responses aligned with user cognition and external contexts.
• �� LifeSim-Eval covers 8 life domains and 1,200 diverse scenarios, adopting a multi-turn interactive method to assess models' abilities to complete explicit and implicit intentions, recover user profiles, and produce high-quality responses.
• �� Experiments reveal significant limitations of current large language models in handling implicit intention and long-term user preference modeling.

The experimental design includes evaluating the performance of various open-source and proprietary models on the LifeSim-Eval benchmark. The benchmark covers 8 life domains and 1,200 diverse scenarios, adopting a multi-turn interactive method to assess models' abilities to complete explicit and implicit intentions, recover user profiles, and produce high-quality responses. Experimental results show that while models perform well in explicit intention recognition, there is significant room for improvement in implicit intention recognition and long-term user preference modeling.

Experimental results show that while models perform well in explicit intention recognition, there is significant room for improvement in implicit intention recognition and long-term user preference modeling. Specifically, on the LifeSim-Eval benchmark, GPT-5 achieved 79.5% accuracy in explicit intention recognition but only 52.2% in implicit intention recognition. In long-horizon settings, while models maintain stable performance on explicit intentions, implicit intention completion rates significantly decline as conversation history grows.

LifeSim and LifeSim-Eval provide a more realistic testing platform for evaluating personalized assistants. By simulating user cognition and life trajectories, this research fills the gap in existing benchmarks in capturing the complexity of external contexts and users' cognitive states. It not only advances academic research in personalized intelligence but also offers new evaluation tools for developing smarter AI assistants in the industry.

Current models show significant limitations in handling implicit intentions and long-term user preference modeling, particularly as conversation history grows. LifeSim-Eval currently focuses on everyday life scenarios and does not yet cover high-stakes domains such as healthcare and legal consultation. Lacks multimodal user signals, primarily simulating user behavior dynamics through textual interactions.