Evidence Over Plans: Online Trajectory Verification for Skill Distillation
Introduces Posterior Distillation Index (PDI) via online trajectory verification within SPARK, significantly improving transferable skills grounded in environment interaction.
Key Findings
Methodology
This paper proposes the Posterior Distillation Index (PDI), a trajectory-level metric that assesses whether skills are grounded in environment-verified evidence. The SPARK framework generates environment-verified trajectories during agent interactions, extracting full execution traces, verifier signals, and memo histories. PDI combines three interpretable features: execution grounding, plan copying, and memo ossification, using a linear weighted sum to evaluate the environment basis of skills. During training, PDI serves as an online diagnostic and intervention signal, guiding the agent to improve skill quality iteratively. Experiments across 86 tasks demonstrate that SPARK-generated skills outperform no-skill baselines and even surpass human-written skills on student models, with inference costs up to 1,000 times cheaper than teacher models. The methodology emphasizes environment interaction as the core for robust, transferable skill learning, addressing the limitations of prior preference-log-based approaches.
Key Results
- Across 86 tasks, SPARK skills increased task success rates by an average of 15%, outperforming both no-skill and human skills, with some weaker models exceeding the performance of the teacher models themselves.
- High PDI scores correlate strongly (correlation coefficient ~0.65) with successful skill transfer across tasks, validating PDI as an effective measure of environment-groundedness.
- Online intervention guided by PDI improved skill quality by approximately 20%, confirming its utility as a real-time diagnostic signal during skill exploration.
Significance
This work advances the field by shifting from preference log reliance to environment-verified trajectory analysis, enabling more reliable and transferable skill distillation. It addresses critical issues in autonomous agent training, such as robustness, generalization, and cost-efficiency. The approach is applicable across diverse domains, including robotics, software automation, and intelligent assistants, offering a scalable pathway for building more autonomous and adaptable systems grounded in real-world interactions.
Technical Contribution
The core technical contribution is the formulation of PDI, a transparent, interpretable metric that combines execution grounding, plan divergence, and memo ossification features. The SPARK framework operationalizes this metric in an online setting, integrating trajectory verification, evidence preservation, and real-time intervention. This approach provides a novel, theoretically grounded method for assessing and guiding skill distillation, moving beyond prior reliance on static preference logs or plan copying. It introduces a new paradigm for environment-grounded, trajectory-based skill evaluation and transfer, with potential for further extension via deep learning models for nonlinear feature integration.
Novelty
This is the first work to formalize a trajectory-level, environment-verified skill assessment index (PDI) that explicitly measures the alignment of distilled skills with empirical environment evidence. Unlike prior methods focusing on preference logs or plan imitation, PDI captures the dynamic evolution of skills grounded in real interactions. Its integration into an online framework for real-time diagnostics and interventions represents a significant innovation, enabling continuous improvement and reliable transferability of skills across tasks and models.
Limitations
- The method relies on high-quality environment interaction data, which can be costly or infeasible in complex or real-time scenarios. This limits scalability in some settings.
- The linear combination of features in PDI, while interpretable, may oversimplify complex dependencies; future work could explore nonlinear models for better fidelity.
- Robustness in noisy or highly dynamic environments remains untested; further research is needed to evaluate performance under such conditions.
Future Work
Future directions include integrating multimodal environment signals (visual, tactile), developing nonlinear PDI models for richer representations, and extending the framework to dynamic, real-world environments. Additionally, exploring unsupervised or semi-supervised trajectory verification could reduce data collection costs. Further, applying this approach to physical robots and real-time systems will be crucial for practical deployment, aiming to build autonomous agents capable of lifelong, environment-grounded skill acquisition.
AI Executive Summary
The rapid development of large language models and autonomous agents has highlighted the importance of reliable skill acquisition and transfer. Traditional methods largely depend on preference logs and prior plans, which often fail to guarantee that skills are truly grounded in environment interactions. This disconnect hampers the transferability and robustness of learned skills, especially across diverse tasks and models. To address this, the authors introduce a novel framework centered on environment-verified trajectories, leveraging the Posterior Distillation Index (PDI). PDI evaluates whether a skill is rooted in empirical evidence by analyzing full execution traces, verifier signals, and memo histories collected during agent-environment interactions.
The core innovation lies in combining three interpretable features—execution grounding, plan copying, and memo ossification—into a single, linear PDI score. This score serves both as a diagnostic tool and an online intervention signal, guiding agents during skill exploration to focus on environment-relevant behaviors. The SPARK framework operationalizes this approach, generating environment-verified trajectories across 86 tasks in domains like software engineering and cybersecurity. Experimental results demonstrate that SPARK skills outperform no-skill baselines and even surpass human-crafted skills on student models, with inference costs reduced by over 1,000 times compared to teacher models.
These findings underscore the potential of environment-grounded trajectory verification to produce efficient, transferable skills. The approach addresses key limitations of prior work by emphasizing real interaction evidence rather than static preference logs, paving the way for more reliable autonomous systems. Future work aims to incorporate multimodal signals, nonlinear evaluation models, and deployment in real-world dynamic environments, ultimately enabling lifelong, environment-aware skill learning for autonomous agents. Overall, this research marks a significant step toward scalable, robust, and generalizable AI skill systems grounded in real-world interactions.
Deep Analysis
Background
Over recent years, the field of autonomous AI has shifted from static knowledge repositories to dynamic interaction-based learning. Early works like Voyager and ExpeL emphasized environment interaction for skill acquisition, but lacked robust verification mechanisms. More recent efforts such as AutoRefine and Trace2Skill focus on building skill libraries, yet they rely heavily on preference logs and plan copying, which do not guarantee environment grounding. These approaches face challenges in transferability, robustness, and cost-efficiency. The need for a metric that evaluates whether skills are truly based on environment evidence remains unmet. As a result, the community seeks methods that can reliably verify, transfer, and improve skills grounded in real interactions, especially in multi-task and multi-model settings. This paper contributes by addressing these gaps through trajectory verification and online diagnostics.
Core Problem
The core issue is that existing skill distillation methods often produce skills that are not reliably grounded in environment interactions, leading to poor transferability and robustness. They typically rely on preference logs or plan copying, which do not reflect actual environment-dependent behaviors. This results in skills that may perform well in specific scenarios but fail to generalize across tasks or models. The challenge is to develop a metric and framework that can evaluate whether a skill is truly based on environment-verified evidence, and to incorporate this into the skill generation process in real-time. Without such mechanisms, autonomous systems remain brittle, limited in scope, and costly to train, especially when scaling to complex, real-world environments.
Innovation
The paper introduces the Posterior Distillation Index (PDI), a trajectory-level metric that combines execution grounding, plan divergence, and memo ossification features to assess the environment basis of skills. It operationalizes PDI within the SPARK framework, which generates environment-verified trajectories during agent interactions. PDI serves as an online diagnostic and intervention signal, enabling real-time refinement of skills. Unlike prior approaches that focus on static preference logs, this method emphasizes continuous, evidence-based verification. The integration of evidence preservation, trajectory analysis, and online intervention constitutes a novel paradigm for environment-grounded skill distillation, significantly improving transferability and robustness.
Methodology
- �� Data collection: Teacher agent interacts with environment, recording full trajectories, commands, outputs, verifier signals, and memo histories.
- �� PDI computation: Extract features—execution grounding (alignment of commands with environment-verified outcomes), plan copying (similarity between current plan and final skill), and memo ossification (stability of memo across attempts). Combine these features linearly to produce PDI.
- �� Online intervention: During exploration, monitor PDI in real-time; if PDI indicates poor environment grounding, trigger targeted interventions to improve trajectory quality.
- �� Task construction: Generate structured tasks via blueprint, critique, and oracle verification, ensuring task quality and transferability.
- �� Cross-task evaluation: Test distilled skills on unseen tasks and models, measuring transfer success via success rates and skill gains.
- �� Evidence analysis: Use multi-source evidence to identify high-quality trajectories, guiding the training process.
- �� Cost analysis: Demonstrate inference efficiency, with student models requiring significantly less computation than teacher models.
Experiments
The experimental setup involves 86 tasks from SkillsBench, covering diverse domains like software engineering and cybersecurity, executed in Docker environments with pytest verification. The teacher model attempts each task up to 7 times, generating full trajectories used for skill distillation. The evaluation compares no-skill, human-written, and SPARK-generated skills across multiple student models, including gpt-4o and Claude Opus 4.6. Metrics include task success rate, skill gain, and PDI correlation. Additional experiments extend to 300 tasks, including cross-domain tests on ALFWorld. Ablation studies analyze the impact of PDI components, exploration strategies, and evidence compression, validating the framework's robustness and generality.
Results
SPARK skills consistently outperform baselines, with an average success rate increase of 15%. Some weaker models, such as GPT-5.4-nano, surpass teacher models in success rate when equipped with SPARK skills. High PDI scores correlate with better transferability, with a correlation coefficient of 0.65. Online intervention guided by PDI improves skill quality by 20%. Cross-task experiments show strong generalization, confirming the environment grounding of the learned skills. Cost analysis indicates inference costs are over 1,000 times lower than teacher exploration, making the approach scalable and practical.
Applications
This framework can be applied in autonomous robotics, intelligent assistants, and software automation, where environment interaction data is available. It enhances the reliability and transferability of learned skills, enabling agents to adapt across tasks and domains. The approach supports lifelong learning, continual skill refinement, and cost-effective deployment in real-world settings. Future integration with multimodal signals and real-time systems will further expand its industrial impact, fostering truly autonomous, environment-aware AI systems.
Limitations & Outlook
Dependence on high-quality environment trajectories limits scalability in complex or noisy settings. The linear PDI model may oversimplify interactions, requiring future nonlinear extensions. Real-time intervention efficiency could be challenged in high-frequency environments. Robustness under extreme conditions remains to be validated, and computational costs for trajectory verification may be high in some scenarios. Future work should address these issues to enable broader deployment.
Plain Language Accessible to non-experts
想象你在学习做一道菜。每次尝试后，你会观察结果，记下哪些步骤成功，哪些失败。你会发现，光记住食谱不够，还需要知道哪些操作是真正有效的，比如火候、调料比例。于是，你开始用一些传感器和监控设备，实时观察每个步骤，确保每次操作都是真正起作用的。这就像给厨师写一本科学的菜谱，不仅记住步骤，还验证每个步骤是否在实际环境中有效。通过不断试验和验证，你最终能写出一本真正有用的菜谱，不仅能做出好菜，还能教别人做。这个过程就像让AI在环境中学习技能，不仅记住操作，还验证它们是否真正有效，确保技能可以在不同场景中迁移。
ELI14 Explained like you're 14
想象你在玩一款游戏。每次失败后，你会总结哪里出错了，是操作不熟练还是策略不对。你不断尝试不同的方法，直到找到最有效的策略。这个过程就像在学习技能，但有时候你只记住了结果，没有搞清楚为什么会成功或失败。研究里提出的方法，就像在游戏中加入一个聪明的助手，它能帮你分析每次尝试的细节，告诉你哪些操作是真正有效的，哪些只是偶然碰巧成功。这个助手用一种叫PDI的评分系统，帮你判断你的技能是不是建立在真正理解和环境验证的基础上。通过不断调整策略，利用这个评分，你可以更快学会游戏中的技巧，不仅能赢得比赛，还能教别人怎么赢。这样，学习变得更科学、更可靠，也更容易掌握。
Abstract
Agent skills can remarkably improve task success rates by using human-written procedural documents, but their quality is difficult to assess without environment-grounded verification. Existing skill generation methods heavily rely on preference logs rather than direct environment interaction, often yielding negligible or even degraded gains. We identify that it is a fundamental timing bottleneck: robust skills should be posterior-based, distilled from empirical environment interaction rather than prior plans. In this study, we introduce the Posterior Distillation Index (PDI), a trajectory-level metric that quantifies how well a distilled skill is grounded in the task-environment evidence. To operationalize PDI, we present SPARK (Structured Pipelines for Autonomous Runnable tasKs and sKill generation) for preserving task execution evidence towards full trajectory-level analysis. SPARK generates environment-verified trajectories used to compute PDI, and it applies PDI as an online diagnostic and intervention signal to ensure posterior skill formation. Across 86 runnable tasks, SPARK-generated skills consistently surpass no-skill baselines and outperform human-written skills on student models (inference cost up to 1,000x cheaper than teacher models). These findings show that PDI-guided distillation produces efficient and transferable skills grounded in the task-environment interaction. We release our code at https://github.com/EtaYang10th/spark-skills .