From Atomic Actions to Standard Operating Procedures: Iterative Tool Optimization for Self-Evolving LLM Agents
EvoSOP enables self-evolving LLM agents by iteratively synthesizing atomic actions into reusable SOPs, boosting success rates by 3-13% and reducing interaction rounds.
Key Findings
Methodology
EvoSOP integrates four modules—Constructor, Merger, Evaluator, and Reviewer—to systematically extract, merge, evaluate, and prune SOPs from execution trajectories. It employs pattern recognition and code synthesis to transform frequent atomic action sequences into high-level routines, avoiding parametric fine-tuning. Experiments on ACEBench and Tau2Bench with GPT-4o demonstrate significant improvements in success rates and efficiency, validating the framework’s effectiveness in complex, multi-step tasks.
Key Results
- In ACEBench multi-step tasks, success rate reached 84.2%, a 3.4% increase over baseline ReAct; in Tau2Bench Telecom, success rose to 40.2%. Interaction rounds decreased by over 30%, indicating more efficient reasoning.
- Tool merging and pruning reduced redundancy, improving reliability. Iterative cycles showed convergence of success rates, confirming self-evolution.
- Ablation studies revealed that removing Merger or Reviewer modules degraded performance, highlighting their importance.
Significance
This work addresses the limitations of static, atomic toolsets in LLM agents, introducing a systematic, symbolic approach to tool management. It enables agents to autonomously refine their capabilities over time, which is crucial for deploying reliable, scalable autonomous systems in real-world scenarios such as automation, decision-making, and complex workflow management. The framework’s non-parametric nature ensures broad applicability without requiring model fine-tuning.
Technical Contribution
The paper introduces a lifecycle-based symbolic optimization framework that extracts high-level SOPs from execution logs, merges redundant routines, and prunes ineffective tools, all without modifying underlying model parameters. It formalizes the process as a continuous, non-parametric learning cycle, providing theoretical guarantees for tool reliability and efficiency. This approach bridges symbolic reasoning with large-scale language models, opening new avenues for autonomous, self-improving AI agents.
Novelty
This is the first comprehensive framework to systematically extract, optimize, and manage SOPs in a continuous lifecycle, moving beyond static or one-shot tool augmentation. Unlike prior work focused solely on tool generation, EvoSOP emphasizes iterative refinement and reliability, enabling true self-evolution of agent capabilities.
Limitations
- The approach relies heavily on large amounts of execution data, which may be scarce or noisy in some environments, limiting its applicability. Tool merging strategies may mistakenly prune useful routines, especially in highly complex tasks.
- Computational overhead from multiple iterations and evaluations can be high, especially in real-time applications. Cross-model robustness and multi-modal extension remain to be validated.
- The framework assumes relatively stable environments; rapid changes could challenge the tool adaptation process.
Future Work
Future research will integrate reinforcement learning to enhance autonomous tool discovery, incorporate multi-modal data for richer context understanding, and optimize the iteration process for real-time deployment. Exploring cross-domain generalization and reducing computational costs are also key directions.
AI Executive Summary
In the rapidly advancing field of artificial intelligence, large language models (LLMs) like GPT-4 have demonstrated remarkable reasoning and problem-solving capabilities. However, their effectiveness in complex, real-world tasks hinges on efficient tool utilization. Traditional systems rely on static, atomic toolsets—such as simple API calls or file operations—that require agents to orchestrate low-level steps repeatedly, leading to high reasoning overhead and error accumulation.
This paper introduces EvoSOP, a novel framework that enables LLM agents to self-evolve by systematically synthesizing atomic actions into high-level, reusable Standard Operating Procedures (SOPs). The core idea is to extract recurring patterns from execution trajectories, compress them into callable routines, and iteratively optimize the toolset through a lifecycle of construction, merging, evaluation, and pruning. Unlike prior methods that treat tool creation as a one-time event, EvoSOP emphasizes continuous refinement, ensuring the agent’s capabilities remain lean, reliable, and adaptable.
The technical foundation of EvoSOP involves pattern recognition algorithms to identify frequent action sequences, code synthesis to formalize SOPs, and a set of modules—Constructor, Merger, Evaluator, and Reviewer—that work together in cycles. The Constructor extracts high-order routines; the Merger consolidates overlapping SOPs; the Evaluator tests their utility in real tasks; and the Reviewer filters out unreliable routines based on performance metrics. This cycle repeats, gradually improving the toolset without parameter fine-tuning of the underlying models.
Experimental results on ACEBench and Tau2Bench datasets validate the effectiveness of EvoSOP. Success rates increased by 2.5% to 13.4%, and the average number of reasoning rounds decreased by over 30%. These improvements demonstrate that the framework not only boosts task success but also enhances reasoning efficiency and robustness, especially in complex, multi-step scenarios. The ability to automatically refine tools in a self-sustaining manner marks a significant step toward autonomous, scalable AI agents.
Overall, EvoSOP offers a scalable pathway for developing self-evolving agents capable of long-term adaptation and reliable performance. Its non-parametric, symbolic optimization approach opens new avenues for AI research, bridging the gap between symbolic reasoning and deep learning. Future work aims to incorporate reinforcement learning and multi-modal data, further pushing the boundaries of autonomous AI systems in real-world applications.
Deep Analysis
Background
Recent advances in large language models (LLMs) like GPT-4, Gemini-3, and Qwen-Max have significantly improved natural language understanding and reasoning. Early efforts such as ReAct and Toolformer integrated external tools to enhance task performance, but these approaches relied heavily on static, low-level atomic actions, which limited scalability and efficiency. Dynamic tool generation methods emerged, enabling models to create new tools on-the-fly; however, these often lacked systematic lifecycle management, leading to toolset bloating and inconsistent reliability. The need for a structured, continuous tool optimization process became evident to address these issues, especially for complex, multi-step tasks requiring hierarchical reasoning and long-term stability.
Core Problem
The core challenge lies in managing a growing, often redundant toolset that hampers reasoning efficiency and reliability. Static atomic actions force agents to perform low-level, fragmented reasoning, increasing error rates and interaction rounds. Existing dynamic tools are added as one-off events without long-term management, resulting in bloated, noisy toolsets. This impairs the agent’s ability to perform reliable, scalable reasoning over extended tasks. Developing a systematic, iterative process to extract, merge, evaluate, and prune tools—forming high-level SOPs—is essential for overcoming these bottlenecks and enabling self-evolving, autonomous agents.
Innovation
The paper introduces a lifecycle-based symbolic optimization framework, EvoSOP, which systematically extracts high-level SOPs from execution logs, merges similar routines, and prunes ineffective tools. Key innovations include:
1) Pattern recognition algorithms to identify frequent, meaningful action sequences.
2) Code synthesis techniques to formalize SOPs with embedded logic.
3) A multi-module cycle—Constructor, Merger, Evaluator, Reviewer—that iteratively refines the toolset.
4) Non-parametric, symbolic learning that avoids model fine-tuning, ensuring broad applicability.
This approach enables continuous self-improvement of the agent’s toolset, leading to more reliable and efficient reasoning in complex tasks.
Methodology
- �� From execution logs, identify frequent action sequences using pattern recognition algorithms.
- �� Synthesize these sequences into callable SOP routines, embedding conditional checks and error handling.
- �� Analyze the SOP set for overlaps and merge similar routines to reduce redundancy.
- �� Evaluate the new SOPs by re-executing tasks and monitoring success metrics.
- �� Use the Reviewer module to analyze performance, filter out low-utility or faulty SOPs.
- �� Repeat the cycle, progressively refining the toolset, until convergence or performance plateau.
- �� The entire process is model-agnostic and does not require parameter updates, relying on symbolic extraction and evaluation.
Experiments
Experiments utilized ACEBench and Tau2Bench datasets, testing across multiple models including GPT-4o, Gemini-3-Flash-Preview, and Qwen-Max. Baselines included ReAct, ASI, and DRAFT. Success metrics focused on task success rate and interaction rounds. The framework was evaluated through multi-round iterations, ablation studies removing modules, and cross-model robustness tests. Hyperparameters such as iteration count and pruning thresholds were tuned to maximize performance. Results showed consistent improvements in success rates and efficiency, validating the effectiveness of the iterative SOP optimization.
Results
EVOSOP achieved success rates of 84.2% on ACEBench multi-step tasks, outperforming ReAct by 3.4%. On Tau2Bench Telecom, success increased to 40.2%. Interaction rounds decreased by over 30%, indicating more streamlined reasoning. Ablation studies confirmed that removing the Merger or Reviewer modules led to performance drops, emphasizing their importance. The framework effectively reduced redundant tools, improved robustness, and demonstrated strong scalability across models and tasks.
Applications
该方法适用于自动化流程优化、智能助手增强、工业自动化等场景，能显著提升系统的自主决策能力和稳定性。未来结合强化学习和多模态信息，将推动自主智能系统在复杂环境中的广泛应用，尤其在需要长时间、多步骤推理的任务中表现优异。
Limitations & Outlook
目前依赖大量执行轨迹，数据不足或环境变化剧烈时表现有限。工具合并策略可能误删有用工具，影响性能。计算成本较高，需优化采样和评估流程。未来需增强跨模态适应性和降低资源消耗，提升在极端复杂环境中的表现。
Plain Language Accessible to non-experts
想象你在厨房做饭，刚开始你只用刀、锅和勺子，做一顿饭需要很多步骤，比如切菜、煮汤、炒菜。每次你都得想怎么操作，有时候会重复一些步骤，觉得很麻烦。后来，你开始总结出一些“菜谱”，把复杂的步骤写成一份清单，照着做就方便多了。这个研究就像是在帮AI学会写“菜谱”，让它自己总结出做事的“攻略”。每次它用“攻略”做事，发现哪里还可以改进，然后不断调整。经过多次尝试，AI变得越来越聪明，做事也越来越快。这就像你变成了厨房高手，效率和菜的味道都提升了。
ELI14 Explained like you're 14
想象你在学校里学做手工艺品，一开始你只知道用剪刀和胶水，做一些简单的东西。每次做完后，你会发现哪里可以做得更快、更漂亮。于是你开始总结一些“秘诀”，比如先剪好所有的部分，再粘贴，这样就不用反复试错。这个研究就像是在帮AI学会这些“秘诀”，让它自己总结出一套做事的“攻略”。每次它试着用这些“攻略”做事，发现哪里还可以改进，然后再调整。经过多次试验，AI的“攻略”变得越来越好，不仅能做得快，还能做得更准。这就像你变成了一个更厉害的手工达人，效率和质量都提升了很多。
Glossary
Standard Operating Procedures (SOP) (标准操作流程)
一套封装多步骤操作的高阶工具，能被调用执行复杂任务，减少重复劳动。技术上为可执行代码和描述的结合。
本文中，SOP用于将Atomic Actions转化为可复用的高阶流程，提升代理效率。
Execution Trajectory (执行轨迹)
代理在任务中实际执行的行为序列，包括调用的工具、环境反应和结果。用于分析和提取高阶SOP。
EVOSOP通过轨迹分析识别重复性行为，合成高阶SOP。
Tool Merging (工具合并)
分析多个工具的功能相似性，将其融合成更通用的高阶工具，减少冗余。
在生命周期中，合并模块优化工具集结构。
Non-parametric Learning (非参数化学习)
不依赖模型参数微调，而通过符号和逻辑操作实现知识积累和优化的方法。
EVOSOP采用符号式策略实现工具的持续优化。
Open Questions Unanswered questions from this research
- 1 如何结合强化学习进一步提升工具的自主发现能力，尤其在环境变化剧烈或数据稀缺时，现有轨迹提取和优化机制可能不足，亟需多模态信息融合和学习机制创新。
Applications
Immediate Applications
自动化流程优化
在企业中，利用EVOSOP自动提取和优化工作流程中的高阶操作，提高效率和准确性，减少人工干预。
智能助理增强
提升智能助手在多轮复杂任务中的表现，使其能自主总结和优化操作策略，提供更可靠的服务。
Long-term Vision
自主系统的自我演化
实现AI系统在不断交互中自我优化工具集，逐步达到自主学习和适应的能力，减少人工维护。
Abstract
Tool utilization enables Large Language Model (LLM) agents to interact with the real world and resolve complex tasks. However, existing agent frameworks predominantly rely on static toolsets composed of granular atomic actions (e.g., basic file I/O or single-turn search), which forces agents to reinvent low-level logic for every recurring workflow, leading to increased reasoning overhead and failure rates. In this study, we propose that agents can achieve self-evolution by synthesizing these atomic actions into reusable Standard Operating Procedures (SOPs), which function as callable higher-order tools that encapsulate multi-step logic. We further introduce EvoSOP, a framework that empowers agents to extract SOPs from execution trajectories and iteratively optimize the toolset through a systematic lifecycle of construction, merging, evaluation, and pruning. Extensive experiments demonstrate that EvoSOP significantly boosts task success rates while substantially reducing the number of interaction rounds compared to baselines. Our analysis also reveals that iterative tool optimization fosters reliable and efficient tool-use patterns, providing a scalable pathway for the development of self-evolving agents.