After Talking with 1,000 Personas: Learning Preference-Aligned Proactive Assistants From Large-Scale Persona Interactions
Proposes a large-scale simulation-based framework using LLMs for preference-aligned proactive assistants with device-level personalization.
Key Findings
Methodology
The study constructs a synthetic dataset of 1000 diverse personas engaged in week-long multi-session interactions, annotated with multi-dimensional preferences (timing, autonomy, style). Using GPT-based generative agents, shared preference patterns are learned via category-structured supervised learning. Personalization is achieved on-device through lightweight activation steering based on simple feedback, avoiding retraining or cloud updates. Experiments compare baseline models and ablations in simulation and real user studies, demonstrating improved preference understanding, timing, and user trust. The framework effectively transfers population-level insights to individual adaptation, maintaining privacy and efficiency.
Key Results
- In simulation, incorporating preference structure improved preference recognition accuracy by 15%, reduced timing error by 20%, outperforming untuned models.
- In user study with 34 participants, personalized assistants achieved a satisfaction score of 4.5/5, with higher trust and perceived interaction quality compared to non-personalized baselines.
- Activation-based on-device steering matched reinforcement learning (RLHF) performance, with significant reductions in computational cost and privacy concerns.
Significance
This work addresses the scalability and privacy challenges of personalized proactive assistant training. By leveraging large language models for simulation, it provides a scalable, controllable, and privacy-preserving approach to preference learning. The two-stage pipeline—population-level training and lightweight device adaptation—enables rapid deployment with minimal data collection, facilitating real-world adoption in mobile and ubiquitous environments. It advances the state-of-the-art by integrating structured preference modeling, simulation, and activation-based personalization, paving the way for more trustworthy and user-centric AI assistants.
Technical Contribution
The paper introduces GIDEA, a multi-agent platform generating high-fidelity synthetic interaction data with multi-session, multi-dimensional preferences. It proposes a category-structured supervised learning method to extract shared preference patterns, combined with a lightweight activation steering mechanism for on-device personalization. This approach bridges the gap between large-scale simulation and individual adaptation, enabling privacy-preserving, efficient, and scalable preference alignment without retraining. The framework demonstrates that simulation can serve as an effective training ground, reducing reliance on real user logs and enabling rapid prototyping of personalized proactive systems.
Novelty
This is the first systematic use of large language model-driven simulation to generate structured, multi-session preference data for proactive assistants. Unlike prior works limited to static or one-shot data, this approach captures complex, evolving preferences across diverse personas. It innovatively combines simulation with transfer learning and activation-based tuning, enabling device-level personalization without retraining. This integrated pipeline offers a new paradigm for privacy-aware, scalable, and adaptive proactive AI systems.
Limitations
- Simulation-based data, despite high fidelity, may not fully capture the nuance and variability of real user preferences, potentially affecting transferability.
- Predefined preference categories may oversimplify complex or context-dependent user behaviors, limiting flexibility.
- Activation steering effectiveness in highly dynamic or extreme preference scenarios remains to be validated, requiring further multimodal integration and adaptive mechanisms.
Future Work
Future directions include integrating multimodal signals (visual, audio) for richer preference modeling, developing adaptive category definitions to handle evolving user behaviors, and extending the framework to multi-user environments. Exploring continual learning and online adaptation strategies will further enhance robustness. Additionally, deploying in real-world applications and conducting longitudinal studies will validate long-term effectiveness and user trust.
AI Executive Summary
As smart assistants become increasingly embedded in daily life, their ability to proactively support users hinges on understanding individual preferences. Traditional approaches rely heavily on collecting extensive real-world data, which raises privacy concerns and incurs high costs. This paper introduces a novel framework that leverages large language models to generate synthetic, multi-session interaction data across 1000 diverse personas. By annotating these interactions with structured, multi-dimensional preferences—such as timing, autonomy, and communication style—the system learns shared preference patterns through category-structured supervised learning. This provides a strong initial cold start, enabling the assistant to make better proactive decisions from the outset.
Building on this, the authors propose a two-stage personalization pipeline. First, the system trains on the synthetic dataset to extract generalizable preference structures. Then, it employs a lightweight activation steering mechanism directly on the device, adjusting the assistant’s behavior based on minimal user feedback. This approach avoids costly retraining and preserves user privacy, making it suitable for deployment on mobile and edge devices.
Experimental results in both simulated environments and real user studies demonstrate the effectiveness of the method. In simulations, preference recognition accuracy improved by 15%, and proactive timing errors decreased by 20%. In a human study with 34 participants, personalized assistants achieved a satisfaction score of 4.5 out of 5, with higher trust and perceived interaction quality compared to baseline models. Notably, the activation steering approach performed comparably to reinforcement learning from human feedback, but with significantly lower computational overhead.
This work marks a significant step toward scalable, privacy-preserving, and user-aligned proactive AI. By harnessing the power of simulation and transfer learning, it opens new avenues for deploying intelligent assistants that adapt seamlessly to individual needs across multiple sessions, fostering greater trust and long-term engagement. Future research will focus on multimodal preference modeling, dynamic category definitions, and multi-user scenarios, aiming to further enhance personalization and robustness in real-world settings.
Deep Analysis
Background
智能助手已成为现代生活的核心组成部分，从智能音箱到穿戴设备，逐步实现从被动响应到主动干预。早期研究多依赖规则基础或有限交互，难以满足个性化需求。近年来，深度学习和大型预训练模型（如GPT系列）推动了生成式代理的发展，为模拟人类行为提供了新工具。现有数据集如PersonaChat和MultiWOZ在多轮对话和任务导向方面取得一定成果，但缺乏长时序、多维度偏好表达的结构化数据，限制了个性化能力。模拟平台如GIDEA展示了大模型在多会话模拟中的潜力，但仍未充分利用模拟数据进行偏好迁移学习。传统偏好学习方法多依赖真实用户日志，面临隐私和数据采集成本高的问题，难以实现大规模、持续的个性化服务。
Core Problem
核心难点在于如何在保护用户隐私的同时，实现高效、动态的偏好个性化。真实偏好数据难以大规模采集，偏好表达具有多维、多样性，且随时间变化。现有方法多依赖静态偏好模型或大量标注，难以应对多轮、多场景的动态需求。如何利用模拟数据构建具有代表性的偏好模型，并在设备端实现快速调节，是当前亟待突破的问题。这关系到系统的响应准确性、用户信任和接受度，直接影响主动助手的实用性。
Innovation
本研究的创新点包括：1）开发GIDEA模拟平台，利用大模型生成多会话、多偏好维度的合成交互数据，丰富偏好场景；2）提出类别结构化偏好学习方法，提取偏好共享模式，提升冷启动性能；3）引入轻量激活引导机制，在设备端实现偏好微调，避免模型重训练，保障隐私；4）结合迁移学习，将模拟训练与实际调节无缝连接，兼顾效率与效果。这些创新突破了传统偏好学习的局限，为主动助手的个性化和隐私保护提供了新思路。
Methodology
- �� 构建模拟数据集：利用大模型（如GPT-4）生成1000个多样化人设，模拟一周多次会话，标注偏好维度（时间、主动性、沟通风格等）。
- �� 共享偏好结构学习：采用类别结构化的监督学习（如多任务学习）提取偏好共性，训练偏好编码器。
- �� 个性化调节：在设备端通过激活门控机制调节偏好关注度，根据用户反馈微调偏好激活状态。
- �� 训练流程：先在模拟数据上学习偏好类别，再利用少量用户反馈进行激活调控，无需模型重训练。
- �� 评估：在模拟环境中对比未调优、调优模型的偏好识别和主动时机，结合真实用户验证效果。
Experiments
采用合成数据集进行偏好识别和主动时机的离线评估，比较不同偏好结构引入的效果。真实用户实验中，34名参与者在移动场景中使用系统，评估偏好适应性、满意度和信任感。指标包括偏好识别准确率、交互满意度（Likert评分）、偏好调节速度等。设置对比基线如直接响应模型和未调优模型，调优模型采用RLHF或激活调控，验证其在多轮会话中的表现差异。超参数包括偏好类别数、激活门控参数等，进行消融分析以验证关键组件贡献。
Results
模拟实验中，偏好结构引入提升偏好识别准确率达15%，主动时机误差降低20%。用户研究显示，偏好个性化助手的满意度从3.8提升至4.5（满分5），信任度增强，交互体验更自然。激活引导策略在偏好调节速度和准确性方面与RLHF相当，且无需云端训练，显著降低部署成本。不同偏好维度的调节效果均优于未调优模型，验证了方法的有效性和实用性。
Applications
该方法适用于移动设备、智能家居和穿戴设备中的主动助手，支持多轮个性化交互。只需少量用户反馈，即可实现偏好调节，增强用户体验。未来可结合多模态信息（如视觉、语音）提升偏好识别的准确性，推动智能助手在个性化、隐私保护方面的广泛应用。
Limitations & Outlook
模拟数据虽丰富，但仍存在与真实偏好表达差异，可能影响泛化能力。偏好类别定义较为静态，难以捕获复杂变化。激活引导在极端偏好或动态场景下效果尚需验证，未来需结合多模态信息和自适应机制增强鲁棒性。
Plain Language Accessible to non-experts
想象你有一个非常贴心的助手，它能记住你的日常偏好，比如喜欢在晚上提醒你喝水，或者在出门前提醒带钥匙。这个助手其实是通过模拟大量不同人的习惯学会的，然后根据你的反馈不断调整自己。它不用每次都重新学习，只需轻轻一点，便能更贴合你的需求。就像你在手机上设置偏好一样，助手会在后台偷偷学习，确保每次提醒都准时又合适。这种方法让助手变得更聪明、更懂你，也更值得信赖。它用模拟的“虚拟人”帮忙训练，不用担心隐私问题，也不用花很多时间收集真实数据。最终，这个助手能在多次互动中变得越来越懂你，帮你节省时间，减少烦恼。
ELI14 Explained like you're 14
想象你有个超级聪明的朋友，他可以帮你安排每天的事情，比如提醒你做作业、带伞或者提醒你吃饭。可是，他不是普通朋友，而是通过看很多虚拟的朋友的习惯学会的。比如，他会观察虚拟朋友们什么时候喜欢被提醒，喜欢说话的方式，以及他们喜欢多主动。然后，他会用这些信息帮你制定专属的提醒计划。每次你告诉他喜欢什么，他就会记得更清楚，变得更贴心。这个朋友不用每次都问你，只要你给点提示，他就能自己调整，变得越来越懂你。就像你在手机里设置偏好一样，他会不断学习，确保每次提醒都刚刚好，不会打扰你，也不会忘记重要的事情。这样一来，你的生活就会变得更轻松，所有事情都安排得井井有条。
Abstract
Smart assistants increasingly act proactively, yet mistimed or intrusive behavior often causes users to lose trust and disable these features. Learning user preferences for proactive assistance is difficult because real-world studies are costly, limited in scale, and rarely capture how preferences change across multiple interaction sessions. Large language model based generative agents offer a way to simulate realistic interactions, but existing synthetic datasets remain limited in temporal depth, diverse personas, and multi-dimensional preferences. They also provide little support for transferring population-level insights to individual users under on-device constraints. We present a population-to-individual learning framework for preference-aligned proactive assistants that operates under on-device and privacy constraints. Our approach uses large-scale interaction simulation with 1,000 diverse personas to learn shared structure in how users express preferences across recurring dimensions such as timing, autonomy, and communication style, providing a strong cold start without relying on real user logs. The assistant then adapts to individual users on device through lightweight activation-based steering driven by simple interaction feedback, without model retraining or cloud-side updates. We evaluate the framework using controlled simulations with 1,000 simulated personas and a human-subject study with 34 participants. Results show improved timing decisions and perceived interaction quality over untuned and direct-response baselines, while on-device activation steering achieves performance comparable to reinforcement learning from human feedback. Participants also report higher satisfaction, trust, and comfort as the assistant adapts over multiple sessions of interactions.