MetaSkill-Evolve: Recursive Self-Improvement of LLM Agents via Two-Timescale Meta-Skill Evolution
MetaSkill-Evolve introduces recursive self-improvement via two-timescale meta-skill evolution, boosting task accuracy by +23.54 points on OfficeQA.
Key Findings
Methodology
The approach employs a dual timescale framework where task skills are optimized in a fast loop, while meta-skills governing the improvement pipeline evolve slowly. Each branch contains a task skill s and a branch-local meta-skill m=(ψ,σ,α,π,ε), controlling five agents: Analyzer, Retriever, Allocator, Proposer, and Evolver. Task skills are updated every iteration, meta-skills every H iterations via the same pipeline applied to their own meta-files, without additional models. All agents share a frozen backbone, enabling recursive self-improvement. Experiments on OfficeQA, SealQA, and ALFWorld show significant gains over static and single-level evolution baselines, with accuracy improvements of +23.54, +16.09, and +1.92 points respectively.
Key Results
- In OfficeQA, static skills improve accuracy by 4.31 points over no-skill, single-level evolution adds 12.85 points, and meta-skill slow evolution adds another 6.38 points. Similar trends are observed in SealQA (+0.24, +7.80, +8.05). In ALFWorld, improvements are smaller but still positive (+1.92), indicating the effectiveness of meta-skill adaptation even near model capacity limits.
- Compared to baselines, MetaSkill-Evolve consistently outperforms in all benchmarks, especially in question-answering tasks, with end-to-end accuracy gains exceeding 16 points. Ablation studies confirm that each component of the meta-skill (ψ, σ, α, π, ε) contributes significantly to overall performance.
- The system’s recursive design enables continuous self-improvement, with meta-skill files evolving alongside task skills, leading to more robust and adaptable agents capable of handling diverse and complex tasks.
Significance
This work advances autonomous AI by embedding self-evolving capabilities directly into the skill artifacts, enabling models to improve their own improvement strategies. It addresses the longstanding challenge of fixed, hand-crafted improvement procedures, opening pathways toward more general, adaptable, and self-sufficient AI systems. Such recursive self-improvement can significantly reduce human intervention and accelerate AI development, especially for complex, open-ended tasks.
Technical Contribution
The main technical innovation is the integration of a dual timescale meta-skill evolution pipeline within a multi-agent framework, allowing recursive self-improvement without additional models or objectives. The design leverages a shared backbone and a structured approach to diagnose, retrieve, propose, and execute improvements, with meta-skills guiding the process. The system formalizes the meta-productivity metric, balancing task utility and improvement rate, and employs a graph-based search for efficient exploration, representing a significant step beyond prior fixed-rule or single-level evolution methods.
Novelty
This is the first work to treat the improvement procedure itself as a learnable, evolvable object within a recursive framework. Unlike prior methods that fix the improvement logic, MetaSkill-Evolve co-evolves meta-skills and task skills, enabling the system to adapt its own improvement strategies over time. This introduces a new paradigm of self-referential, recursive optimization in large language agents.
Limitations
- The approach requires substantial computational resources due to multiple recursive iterations and graph maintenance, limiting scalability in real-world deployment.
- Meta-skill design relies on manual specification of components, which may not generalize well across different tasks or domains without further automation.
- Stability and convergence in highly complex or noisy environments remain unverified, necessitating future research into robustness and efficiency improvements.
Future Work
Future directions include developing more efficient search and update mechanisms, automating meta-skill component design, exploring multi-level recursion, and integrating reinforcement learning for dynamic policy adaptation. Extending the framework to real-world robotics or autonomous systems could further demonstrate its practical utility and scalability.
AI Executive Summary
The rapid evolution of large language models (LLMs) has unlocked unprecedented capabilities in understanding and generating complex, multi-step tasks. However, these models often rely on static, hand-crafted skills or fixed improvement procedures, limiting their adaptability in diverse environments. Addressing this challenge, the paper introduces MetaSkill-Evolve, a novel framework that embeds recursive self-improvement into the core of agent design. By leveraging a dual timescale approach, the system simultaneously optimizes task skills in a fast loop and evolves meta-skills governing the improvement pipeline in a slower loop, forming a self-referential, closed-loop mechanism.
Each agentic branch carries a task skill s and a branch-local meta-skill m=(ψ,σ,α,π,ε), which parameterize five specialized agents: Analyzer, Retriever, Allocator, Proposer, and Evolver. The task skill is updated iteratively, while the meta-skill undergoes slow evolution every H iterations by reapplying the same pipeline to its own meta-files, effectively enabling the system to learn how to improve its own improvement strategies. This recursive process is grounded in a shared, frozen backbone model (Gemma-4 31B), ensuring stability and scalability.
Experimental results on OfficeQA, SealQA, and ALFWorld benchmarks demonstrate that MetaSkill-Evolve significantly outperforms static skills and single-level evolution baselines, with accuracy improvements of +23.54, +16.09, and +1.92 points respectively. Ablation studies reveal that each component of the meta-skill contributes to overall performance, confirming the importance of a multi-agent, multi-component design. The framework’s recursive self-improvement paradigm paves the way for more autonomous, adaptable AI systems capable of continuous self-enhancement, addressing longstanding limitations of fixed-rule or single-level optimization approaches.
Looking ahead, future work will focus on reducing computational costs, automating meta-skill component design, and extending the recursive paradigm to multi-level hierarchies and real-world applications. This research marks a significant step toward truly autonomous AI agents that can learn to improve themselves over time, with broad implications for artificial intelligence development and deployment.
Deep Analysis
Background
近年来，LLMs在多任务、多步骤推理和工具使用方面取得了突破，但其自主学习能力仍受限于预定义技能或规则。早期研究如Few-Shot学习、链式推理和技能库构建，虽提升了性能，但缺乏持续自我优化机制。近年来，反思机制、知识蒸馏和强化学习被引入，推动技能的自我改进，但多停留在单层或固定策略，难以实现深层递归优化。现有如EvoSkill、SkillWeaver等虽实现技能迭代，但改进过程多为固定规则，缺乏对改进策略的自我演化。本文提出将元技能作为可演化对象，结合多代理管线，实现递归自我提升，填补了自我改进深度不足的空白。
Core Problem
核心问题在于现有自我改进机制多为单向、固定流程，难以适应多样化任务和复杂错误类型。改进策略的刚性限制了系统的学习能力，无法突破固定规则的瓶颈。如何将改进过程本身作为可优化对象，形成递归自我提升，成为关键难题。此外，保证系统稳定性和泛化能力也是亟待解决的问题。
Innovation
创新点包括：1）引入双时间尺度的元技能演化机制，使改进策略可自我优化；2）设计五代理（分析、检索、分配、提议、演化）共同作用，支持多元化改进；3）将元技能作为与任务技能相同格式的可演化文件，嵌入递归自我改进流程；4）利用知识图谱和贝叶斯优化思想，提升搜索效率和多样性。该框架突破了传统固定规则的限制，实现了改进策略的自我演化，为自主学习提供新范式。
Methodology
- �� 任务定义：任务T由输入x和目标输出y组成，任务技能s为Markdown格式的LLM程序，定义操作流程。• 任务效用：通过任务奖励U(s)=E_{(x,y)∼T}[r(As(x), y)]衡量技能表现。• 元技能：m=(ψ,σ,α,π,ε)，每个为Markdown格式文件，控制分析、检索、分配、提议和演化。• 结构设计：每个分支携带任务技能s和局部元技能m，形成分支状态b=(s,m,h)，h为历史。• 递归机制：任务技能在快速循环中优化，元技能每H次在慢速循环中通过相同五代理管线自我优化，形成闭环。• 搜索策略：利用多指标（U, P, N）对分支进行优先级排序，确保探索多样性和效率。• 代理流程：分析器诊断失败，检索器寻找相似案例，分配器设定子任务预算，提议器生成改进方案，演化器执行并验证。• 元技能优化：通过聚合H次结果，构建元故障追踪，反向调节元技能参数，支持递归自我提升。
Experiments
在OfficeQA、SealQA和ALFWorld三个基准上，采用分层采样划分训练、验证和测试集。所有实验均使用Google的Gemma-4 31B模型作为基础骨架，无微调。对比基线包括无技能、静态技能和单层演化。性能指标为测试准确率，评估不同策略的效果。消融实验验证各组成部分贡献，参数设置包括H=50，Kmax=5等。模型在多个迭代中持续优化，观察性能变化，确保统计显著性。
Results
MetaSkill-Evolve在OfficeQA中提升准确率23.54点，在SealQA中提升16.09点，在ALFWorld中提升1.92点，显著优于对比方法。问答任务中，逐步演化策略表现出单调增长，验证递归优化效果。消融实验显示，缺失任何元技能组成部分都会导致性能下降，说明多元化设计的必要性。
Applications
该技术适用于智能客服、自动问答、复杂任务规划等场景，尤其适合需要持续学习和适应新任务的系统。通过自我改进，减少人工干预，提高自主性。未来还可结合强化学习，拓展到机器人控制、自动驾驶等领域，推动自主系统智能化升级。
Limitations & Outlook
当前方法对计算资源需求较高，尤其在多次递归和多分支维护中成本较大。对元技能设计和优化仍依赖经验，缺乏自动化调节机制。在极端复杂或超大规模任务中，系统的稳定性和收敛性仍需验证。未来需优化算法效率，增强鲁棒性，降低实际部署门槛。
Plain Language Accessible to non-experts
想象一个工厂，工人们每天按照固定流程生产产品。随着时间推移，他们发现一些步骤可以改进，但改进方式由工厂设计师提前设定。现在，工厂引入了一套智能系统，不仅能改进生产流程，还能自己学习如何更好地改进。这就像一个工厂里的机器人，不断观察、尝试、调整，甚至还能自己改写改进规则，使整个生产线变得更快、更好。这种系统不用每次都由人来告诉它怎么改，而是自己不断学习、优化，变得越来越聪明。它就像一个自我学习的厨师，能不断改良菜谱，做出更美味的菜肴。这个想法的核心是让机器自己学会改进自己，而不是依赖人类不断干预。
ELI14 Explained like you're 14
想象你在玩一个游戏，你可以用不同的策略赢得比赛。刚开始，你用的策略很普通，但你发现有些技巧可以让你更容易赢。于是你不断尝试新的技巧，逐渐变得更厉害。现在，假设你有一个超级聪明的助手，它不仅会帮你改进策略，还能自己学习哪些技巧更有效，然后自己调整策略。这个助手会观察你的每次比赛，分析哪里可以做得更好，然后自己试验新的方法。它就像一个不断学习的朋友，越玩越聪明，甚至可以自己发明新策略。它就像一个永远在学习、永远在变得更聪明的伙伴。未来，这样的助手可以帮我们解决很多复杂的问题，比如自动驾驶、机器人操作，甚至帮我们写作和设计。它就像一个永远在学习、永远在变得更聪明的伙伴。
Abstract
Recent LLM agents tackle increasingly long-horizon, open-ended tasks, and external skills, reusable procedural knowledge supplied to the agent, further extend this capability. However, a fixed, hand-authored skill is rarely optimal, and cannot adapt to the diversity of tasks an agent encounters. Self-improving agents address this by rewriting their own skill files from execution traces, yielding meaningful gains on challenging benchmarks. Yet such self-evolution remains non-recursive: it improves only the task skill (what the agent does) while the improvement procedure (how it improves) is authored once and held fixed. We introduce MetaSkill-Evolve, a two-timescale framework that makes agentic skill improvement recursive: every branch carries both a task skill $s$ and a branch-local meta-skill $m=(ψ,σ,α,π,\varepsilon)$ whose five components parameterise the Analyzer, Retriever, Allocator, Proposer, and Evolver agents of the improvement pipeline. Task skills evolve on a fast loop while the meta-skill evolves on a slower one under the same pipeline applied to itself, with no additional model or objective. With all five pipeline agents sharing a single frozen backbone, MetaSkill-Evolve outperforms no-skill, static-skill, and single-level evolution baselines on three agentic benchmarks (OfficeQA, SealQA, ALFWorld), improving held-out test accuracy over the raw backbone by +23.54, +16.09, and +1.92 points respectively.