From Profiling to Synthesis: Benchmarking Implicit Behavioral Alignment in Personalized LLM Agents
Introduces IBA-Bench for implicit behavior alignment, combining deep retrieval and trajectory-level synthesis, significantly improving personalized LLM agent performance.
Key Findings
Methodology
This work constructs IBA-Bench, a benchmark that evaluates agents' ability to align with implicit user constraints derived from long-term interaction histories containing noise, implicit cues, and temporal inconsistencies. The proposed IBA-Agent framework integrates deep retrieval to extract relevant behavioral signals and employs trajectory-level synthesis to fuse dynamic preferences. Extensive multi-domain experiments demonstrate the persistent knowledge-to-action gap in state-of-the-art LLMs, with IBA-Agent substantially closing this gap by enabling more accurate behavior alignment in complex scenarios.
Key Results
- On IBA-Bench, top models like GPT-5.1 achieve around 58% task success, whereas IBA-Agent improves this to over 78%, showing a significant performance boost.
- In scenarios such as healthcare and planning, the introduction of trajectory-level synthesis increased performance by 20-30 percentage points, confirming the importance of dynamic preference integration.
- Ablation studies reveal that deep retrieval, trajectory synthesis, and broad reasoning are all critical, with removal of any component reducing performance by at least 10%.
Significance
This research addresses the longstanding challenge of translating inferred implicit preferences into effective actions, moving beyond static profiling to dynamic synthesis. It advances the field by systematically evaluating and improving behavioral alignment, which is crucial for deploying truly personalized AI assistants in real-world applications. The framework bridges the gap between knowledge understanding and actionable decision-making, promising more adaptive and user-centric intelligent systems.
Technical Contribution
The core innovation lies in the integration of deep retrieval with trajectory-level synthesis within the IBA-Agent framework, enabling robust extraction and fusion of implicit behavioral cues from noisy long-term histories. The approach introduces a novel multi-stage process that combines retrieval, reasoning, and synthesis, providing theoretical guarantees for preference consistency and practical scalability. This work sets a new standard for behavior-aware personalization in large language models, offering a scalable, interpretable, and effective methodology.
Novelty
This is the first benchmark to evaluate implicit behavioral alignment using long-term interaction histories with noise and conflicts. The trajectory-level synthesis strategy is a novel contribution, allowing models to reconcile conflicting preferences dynamically. Unlike prior static profiling methods, this approach captures the evolving nature of user preferences, filling a critical gap in personalized AI research.
Limitations
- The models still struggle under extreme noise or highly conflicting preferences, indicating room for robustness improvements.
- Handling large-scale interaction histories incurs high computational and storage costs, limiting real-time deployment.
- Current experiments are primarily simulated; real user data validation remains a future step to confirm practical effectiveness.
Future Work
Future directions include integrating multimodal signals (images, voice), enhancing real-time processing capabilities, and deploying in real-world systems. Further research will explore cross-domain transferability, improving robustness against noise, and developing explainability features to increase user trust.
AI Executive Summary
The rapid evolution of large language models (LLMs) has transformed autonomous agents from simple chatbots into sophisticated assistants capable of complex tasks. However, personalization remains a core challenge. Traditional approaches rely on static user profiles or limited interaction logs, which fail to capture the dynamic and implicit nature of user preferences. This disconnect leads to a significant knowledge-to-action gap, where models possess knowledge but cannot effectively translate it into tailored actions.
To address this, the authors introduce IBA-Bench, a comprehensive benchmark designed to evaluate implicit behavioral alignment. It simulates realistic long-term interaction histories that contain noise, implicit cues, and conflicting signals, reflecting real-world complexity. The benchmark assesses whether agents can infer and satisfy implicit user constraints during task execution, moving beyond simple preference matching.
Building on this, the paper proposes the IBA-Agent framework, which combines deep retrieval techniques with trajectory-level synthesis. This approach enables the agent to extract relevant behavioral signals from noisy histories and dynamically fuse preferences into actionable plans. The framework's core components include a multi-stage process: retrieving behavioral cues, reasoning over accumulated evidence, and synthesizing personalized actions.
Extensive experiments across nine application domains demonstrate that existing state-of-the-art models, such as GPT-5.1, perform poorly on complex, synthesis-heavy tasks, with success rates around 58%. In contrast, IBA-Agent consistently achieves over 78%, confirming its effectiveness. Ablation studies highlight the importance of each component, especially the trajectory-level synthesis, which significantly boosts performance.
This work has profound implications for personalized AI, enabling systems that understand and adapt to evolving user preferences more accurately. It paves the way for smarter, more intuitive assistants capable of bridging the knowledge-action gap in real-world scenarios. Nonetheless, challenges remain, including handling extreme noise, reducing computational costs, and validating with real user data. Future efforts will focus on multimodal integration, real-time deployment, and enhancing model interpretability, aiming to realize truly adaptive and trustworthy personalized AI systems.
Deep Analysis
Background
近年来，随着GPT-4、Claude等大规模语言模型的出现，自动化智能体在自然语言理解和生成方面取得了突破性进展。早期研究如ReAct和Toolformer强调模型推理和工具调用能力，推动智能体在复杂任务中的应用。随后，个性化代理逐渐兴起，采用静态偏好模型（如PersonaBench）或有限交互历史，试图满足用户个性化需求。然而，这些方法多忽视偏好的动态演变和复杂冲突，难以应对真实场景中用户不断变化的偏好，导致行为与偏好不匹配的问题日益突出。
Core Problem
核心问题在于用户偏好具有隐性、动态、多变的特性，传统静态模型难以捕获偏好的演变过程，导致智能体在实际任务中表现出偏差。现有评估多依赖静态快照或问答匹配，无法衡量模型在复杂交互中的行为表现。如何从长时交互中提取隐性线索，动态融合偏好，确保任务执行符合用户未明确表达的约束，成为亟需解决的难题。这一问题关系到智能体的实用性和用户满意度，亟待新的技术方案。
Innovation
本研究的创新点在于提出基于长时交互历史的隐性行为对齐机制，突破静态偏好模型的局限。首先，构建IBA-Bench，模拟噪声、隐性线索与时间不一致性，真实反映用户偏好的复杂性。其次，设计IBA-Agent，结合深度检索提取行为线索，采用轨迹级合成融合偏好，实现偏好动态调整。最后，建立多域、多场景的评估体系，系统验证模型在复杂环境中的行为对齐能力。这些创新推动个性化代理从静态模型走向动态合成，解决知识-行动鸿沟。
Methodology
- �� 构建IBA-Bench：采集66个场景、超2800个任务实例，模拟真实交互中的噪声、隐性线索与偏好冲突。• 用户画像：定义400个长短期偏好模型，结合背景、性格、状态与角色，丰富用户多样性。• 交互历史生成：利用多智能体框架模拟长时交互，注入噪声与干扰，隐藏偏好线索。• 任务实例设计：基于API调用和文本生成，结合偏好约束，测试模型的行为对齐能力。• IBA-Agent：采用深度检索识别行为线索，轨迹级合成融合偏好，逐步推理任务目标。• 评估指标：结合行为正确率、偏好满足度与任务成功率，全面衡量模型表现。
Experiments
在多域、多场景下，测试包括GPT-5.1、Qwen-3-4B-Instruct等模型，采用统一RAG流程，评估偏好理解与任务执行能力。通过对比不同模型、不同模块（如无检索、无轨迹合成）性能，验证轨迹合成的贡献。采用人工与AI评审结合的评分体系，确保评估的客观性。实验还包括消融分析，验证深度检索、偏好融合对性能的影响，确保方法的有效性与鲁棒性。
Results
模型在IBA-Bench上的任务成功率平均提升20%以上，最优模型达78%以上，显著优于传统静态偏好模型。引入轨迹合成后，多个场景（如医疗、规划）表现提升明显，验证了偏好动态融合的必要性。消融实验显示，缺少深度检索或轨迹合成会导致性能下降至少10个百分点，说明每个模块都不可或缺。这些结果表明，动态偏好融合是实现个性化行为对齐的关键。
Applications
该方法适用于智能助理、个性化推荐、自动化客服等场景，能根据用户历史行为动态调整响应策略。需要丰富的交互数据和偏好线索，适合企业构建个性化服务平台。未来，结合多模态信息和真实用户反馈，将大幅提升系统的适应性和实用性，推动个性化AI的普及。
Limitations & Outlook
模型在极端噪声环境下表现仍有限，偏好冲突复杂时难以完全融合，计算成本较高，存储与检索长时交互历史存在挑战。未来需提升鲁棒性、降低成本，并验证真实用户场景中的效果。
Plain Language Accessible to non-experts
想象你在一家厨房做饭，厨师需要根据不同食材、口味偏好和时间变化调整菜谱。传统方法就像厨师只记住了用户喜欢辣味，忽略了用户最近生病不能吃辣。现在的智能厨师不仅记住用户的偏好，还能根据用户的最新状态，动态调整菜谱。它会从长时间的对话中找到隐含的线索，比如用户最近牙疼，厨师会主动推荐软食。这就像厨师在做饭时不断观察、调整，确保每一道菜都符合用户的实际需求。这个过程比以前单纯记住偏好更智能、更贴心，也更符合真实生活的复杂性。
ELI14 Explained like you're 14
想象你有个超级厨师朋友，他不仅记住你喜欢吃辣，还会偷偷观察你最近是不是生病了。比如你告诉他你饿了，他会问你喜欢什么，然后还会注意到你刚刚说牙疼，知道你不能吃辣。于是，他会主动推荐一些软软的菜，让你吃得舒服。这就像这个研究中的智能助手，不只是记住你说过的话，还会从你平时的行为中猜出你的真实需求，然后帮你做决定。这样一来，你不用每次都告诉他所有细节，他就能帮你做出最贴心的选择。这种智能让我们的生活变得更方便、更温暖，就像有个懂你、会照顾你的好朋友一样。
Abstract
Large Language Models have enabled increasingly capable autonomous agents, yet personalization remains critical for making such agents practically useful. Recent benchmarks have begun evaluating personalization in agents, but they largely rely on static preference snapshots, fixed interaction logs, or question answering over predefined user profiles. Such designs fail to capture the complexity of evolving user preferences and neglect preference-conditioned task execution-a discrepancy we term as the knowledge-to-action gap. To address this challenge, we introduce IBA-Bench, a benchmark for implicit behavioral alignment constructed from longitudinal interaction histories that contain noise, implicit cues, and temporal inconsistencies. Unlike prior work, IBA-Bench evaluates whether an agent can execute tasks while satisfying implicit user constraints inferred from historical interactions. We further propose IBA-Agent, an agent framework that reconciles conflicting priorities through broad retrieval and trajectory-level alignment. Experiment results on IBA-Bench show that effective personalization remains a significant challenge for state-of-the-art LLM agents, and the proposed IBA-Agent substantially improves behavioral alignment in complex scenarios across nine application domains.