SkillForge: Evolving Verifiable Skills for Reinforcement Learning Agents

Recent breakthroughs in large language models (Liu et al., 2024; Yang et al., 2025) have enabled RL agents to perform complex reasoning and decision-making tasks. Frameworks such as ReAct (Yao et al., 2023) and Reflection (Shinn et al., 2023) integrate reasoning and self-reflection, improving autonomy. However, these methods remain episodic, lacking mechanisms for knowledge accumulation across episodes. Skill extraction approaches like SkillRL (Xia et al., 2026) distill raw trajectories into decision-making primitives, improving generalization. Despite progress, static skill repositories suffer from outdated or low-quality skills, and memory-based methods struggle with noisy, redundant data. The challenge remains to develop a system that can continuously validate, refine, and expand skills during ongoing interactions, ensuring sustained performance and adaptability.

The core challenge is managing skills in RL agents to ensure their ongoing effectiveness. Static skill sets or unverified repositories lead to performance degradation over time, especially in open-ended environments. Existing methods lack explicit mechanisms for skill validation, resulting in low reliability and limited scalability. The problem is compounded by noisy environment feedback and the difficulty of distinguishing between effective and ineffective skills. Addressing these issues requires a framework that can dynamically evaluate, refine, and induce skills based on real-time interaction outcomes, enabling lifelong learning and adaptation.

SkillForge introduces several key innovations:

1) Explicit skill invocation with structured tags, making skill usage observable and optimizable.

2) Effectiveness tracking using Bayesian or exponential averages, enabling real-time validation of skill utility.

3) Multi-pathway skill induction—extracting, refining, and contrastively synthesizing new skills from successful and failed trajectories.

4) Trajectory abstraction via large models, summarizing key decisions and skill calls to guide ongoing skill evolution.

These innovations collectively enable a continuously evolving, high-quality skill bank, addressing the limitations of static repositories and static extraction methods, and significantly improving RL agent performance in complex environments.

• �� Skill Representation: Skills are structured with identifiers, intent, principles, applicability, category, and status flags, enabling self-contained management.
• �� Skill Initialization: Using environment rollouts, successful trajectories are distilled into general and task-specific skills via a teacher LLM, forming an initial skill bank.
• �� Skill Retrieval: Embedding-based semantic similarity retrieves relevant skills from the bank, forming a compact catalog for prompt augmentation.
• �� Explicit Skill Calling: During RL, the agent outputs environment actions and optional skill call tags, which are resolved to full skill descriptions, making usage explicit.
• �� Policy Optimization: Using GRPO, the policy jointly optimizes environment actions and skill invocation decisions based on trajectory rewards.
• �� Trajectory Abstraction: Large models summarize key decisions and skill calls, providing input for skill induction.
• �� Skill Induction: Periodically, successful, failed, and contrastive trajectories generate new skills via a teacher model, which are deduplicated and added to the bank.
• �� Effectiveness Verification: Success rates and usage counts are tracked, and skills are reviewed and refined via model reflexion, maintaining high quality.

The framework was tested on ALFWorld, WebShop, and AppWorld, measuring success rate, task completion, and scores. Models like Qwen2.5-7B, Qwen3-4B, and Qwen3-30B were used with hyperparameters: learning rate 1e-6, rollout temperature 0.9, and skill bank updates every 5 steps. Baselines included SkillRL, ReAct, and Reflection, with ablation studies removing modules such as explicit calling, effectiveness tracking, and multi-path induction. The experiments tracked skill bank growth, success rates, and transferability across model scales, validating the effectiveness of continuous skill validation and evolution.

SkillForge achieved success rates of 93.6% in ALFWorld, outperforming SkillRL by 3.7%. In WebShop, success rate increased from 72.7% to 83.0%, with score improvements of 10.5 points. AppWorld success rate rose from 19.0% to 23.8%, with scene goal completion from 3.57% to 14.3%. Ablation studies confirmed that explicit skill calling, effectiveness tracking, and multi-path induction are critical. The skill bank expanded steadily, with effective reflection removing outdated skills, and skill intent embeddings diversified, enabling robust transfer across model sizes.

This approach benefits autonomous robots, virtual assistants, and multi-task AI systems, enabling continuous learning, adaptation, and robustness in complex environments. It can be integrated into industrial automation, autonomous driving, and service robots, where ongoing skill validation ensures reliability and safety, reducing manual updates and improving long-term performance.

Current methods depend on environment feedback, which can be noisy, leading to potential misjudgments in skill effectiveness. The added computational overhead increases training time, especially for large models. Skill representations rely on predefined templates, limiting flexibility; future work should explore more adaptive, automatic skill generation. Additionally, robustness in highly dynamic or adversarial environments remains a challenge, requiring further research into noise-resistant validation mechanisms.