SkillsVote: Lifecycle Governance of Agent Skills from Collection, Recommendation to Evolution

Recent advances in large language models (LLMs) like GPT-5 have enabled long-horizon agent systems that generate trajectories of decisions, tool interactions, and environmental feedback. Early efforts focused on storing and reusing experience via memory networks, few-shot trajectories, and heuristics. As agent complexity grew, skills—structured, executable knowledge units—became a focus, with systems like SkillNet and SkillsBench aiming to organize and verify skills. However, open-source ecosystems face scale issues: redundancy, quality variability, environment sensitivity, and pollution from indiscriminate updates. These challenges hinder reliable skill reuse and evolution, calling for systematic lifecycle management that integrates collection, recommendation, attribution, and controlled evolution to ensure sustainable growth.

The core challenge lies in managing massive, heterogeneous skill repositories to prevent quality degradation and environmental mismatch. Existing methods lack mechanisms for responsible updates, leading to pollution and instability. Long trajectories contain valuable reusable knowledge but are noisy and difficult to filter. Without fine-grained attribution and control, skill evolution risks incorporating unreliable or irrelevant information, undermining agent performance. Addressing these issues requires a structured, accountable approach that can scale with ecosystem growth, ensuring that only verified, beneficial knowledge influences future development.

SkillsVote introduces several innovations: 1) structured skill retrieval via filesystem-native search, enabling precise, scalable recommendation; 2) subtask-level trajectory decomposition with responsibility attribution, localizing knowledge and accountability; 3) evidence gating that filters only successful, reusable exploration for skill updates; 4) a closed-loop lifecycle integrating recommendation, attribution, and evolution, ensuring stability. These mechanisms collectively improve scalability, robustness, and trustworthiness of open-source skill ecosystems, surpassing prior approaches that lacked systematic control or relied on unverified experience.

• �� Collect and profile over one million open-source skills, analyzing environment fit, quality, and verifiability. • Before task execution, perform agentic search over structured skill folders to recommend a compact, relevant skill set, providing concise guidance. • During execution, decompose trajectories into subtask units based on changes in objectives or signals, each with a clear success/failure outcome. • Use attribution to assign responsibility for each subtask’s outcome to specific skills, exploration, or environment. • Extract reusable knowledge from successful subtasks, focusing on procedures, preconditions, and recovery patterns. • Apply evidence gating to filter only verified, beneficial updates. • Merge similar evidence units and route them to skill updates, fixing errors or creating new skills as justified. • Repeat this cycle iteratively, maintaining a high-quality, evolving skill library.

The evaluation uses Terminal-Bench 2.0 and SWE-Bench Pro, testing GPT-5.2, GPT-5.4, and GPT-5.5 models. Baselines include no skills, online evolution, offline transfer, and pre-task recommendation. Metrics are accuracy (avg@5) and success rate (avg@1). Experiments analyze the impact of recommendation, responsibility attribution, and evidence gating. Ablation studies compare different configurations, while scalability tests assess retrieval performance across skill pool sizes. Results show consistent improvements: accuracy gains of 2.6% on Terminal-Bench 2.0, 2.7% success rate increase on SWE-Bench, and better scalability in large pools, validating the system’s robustness and effectiveness.

SkillsVote outperforms baselines across all settings, with an average accuracy increase of 2.6 percentage points and success rate improvement of 2.7 points. It maintains higher routing accuracy (~70%) at full skill pool scale, with reduced costs. Offline transfer from historical trajectories enhances performance on unseen tasks, demonstrating strong generalization. The controlled evolution mechanism ensures only verified, reusable knowledge updates, preventing degradation of skill quality. These results confirm that systematic lifecycle management significantly boosts agent capabilities in complex, long-horizon tasks.

This framework is applicable in automated programming, robotic learning, and complex task planning. Its structured, accountable approach enables safer, more reliable autonomous systems. Future integration with reinforcement learning and multi-modal perception can further enhance autonomous skill acquisition, supporting long-term deployment in industrial automation, service robots, and intelligent assistants, ultimately fostering a self-improving AI ecosystem.

Dependence on large, heterogeneous open-source skill datasets may introduce quality issues; future work should incorporate automated quality assessment. In highly dynamic or multi-modal environments, attribution accuracy may decline; integrating richer sensory data could help. Computational costs for large-scale retrieval and gating remain non-trivial; optimizing indexing and search algorithms is necessary. Additionally, the current framework assumes static skill representations, which may limit adaptation in rapidly changing contexts.