Executable Code Actions Elicit Better LLM Agents
Proposes CodeAct, a framework enabling LLMs to generate executable Python code for actions, significantly improving success rates (up to 20%) in complex multi-tool tasks via integrated Python interpreter and multi-turn interactions.
Key Findings
Methodology
This work introduces the CodeAct framework, where LLMs produce executable Python code as actions, integrated with a Python interpreter to facilitate dynamic adjustments and multi-turn interactions. The approach involves replacing traditional text or JSON-based actions with Python scripts that leverage control flow and data reuse capabilities. During interactions, the model generates code snippets, executes them in the interpreter, and receives feedback such as output or errors, which inform subsequent code revisions. The system is trained and fine-tuned using a large-scale instruction-tuning dataset, CodeActInstruct, containing 7,000 multi-turn interaction trajectories. Extensive evaluations on API-Bank and a newly curated benchmark, M3ToolEval, across 17 models demonstrate that CodeAct outperforms baseline methods, especially in complex, multi-step tasks requiring tool composition and control flow.
Key Results
- In API-Bank API call correctness tests, CodeAct achieves up to 20% higher success rates compared to JSON and text-based action formats, particularly benefiting open-source models with extensive code pretraining.
- On the M3ToolEval benchmark, which contains 82 complex multi-tool tasks, CodeAct attains a success rate of 74.4%, outperforming other formats, and reduces average interaction turns to 7.7, indicating higher efficiency and fewer steps needed.
- The fine-tuned CodeActAgent, based on LLaMA2 and Mistral, demonstrates autonomous debugging and complex task execution capabilities in real-world scenarios like model training and data visualization, confirming its practical utility.
Significance
This research addresses the core limitations of traditional action representations in LLMs by enabling code-based actions that support complex logical operations, control flow, and dynamic adjustments. The integration of executable Python code allows models to perform multi-step, multi-tool tasks more efficiently and reliably, paving the way for advanced autonomous agents in robotics, scientific research, and decision-making systems. The frameworkβs ability to self-debug and adapt in real-time significantly enhances the robustness and applicability of LLMs in real-world environments, marking a substantial step forward in AI autonomy.
Technical Contribution
Key technical innovations include: β’ Transitioning from static text/JSON actions to executable Python code, leveraging pretraining in programming languages; β’ Designing a multi-turn interaction mechanism where models generate, execute, and revise code based on environment feedback; β’ Creating the CodeActInstruct dataset for instruction tuning, which improves modelsβ code generation and debugging capabilities. These contributions enable models to handle complex, multi-step workflows with control flow and data reuse, surpassing traditional methods in flexibility and robustness.
Novelty
This work is the first to systematically embed executable Python code as the primary action format in LLM agents, combined with multi-turn feedback and self-debugging. Unlike prior approaches limited to static tool invocation or predefined commands, CodeAct exploits control flow and data flow features of programming languages, allowing for intricate tool composition and logical operations within a single action. This paradigm shift unlocks new possibilities for autonomous, flexible, and scalable AI agents.
Limitations
- While CodeAct demonstrates strong performance in controlled benchmarks, its effectiveness in highly unpredictable or resource-constrained environments remains to be tested. The reliance on Python execution may introduce latency and computational overhead, especially in real-time applications.
- The current implementation is primarily validated in Python; extending to other programming languages or heterogeneous systems poses challenges that require further research.
- Dependence on environment feedback accuracy is a potential vulnerability; noisy or delayed feedback could impair the modelβs debugging and adjustment processes, necessitating robustness enhancements.
Future Work
Future directions include expanding multi-language support, integrating reinforcement learning to optimize code generation policies, and applying the framework in real-world robotics, scientific experiments, and industrial automation. Additionally, developing more sophisticated feedback mechanisms and reducing computational costs will be key to deploying CodeAct in latency-sensitive applications. Exploring cross-modal interactions, such as integrating visual or sensor data into code-based actions, also offers promising avenues for advancing autonomous AI agents.
AI Executive Summary
The rapid advancement of large language models (LLMs) has revolutionized natural language understanding and generation, yet their application in complex, multi-step tasks involving tool use and environment interaction remains limited by the rigidity of traditional action representations. Conventional methods rely on predefined text or JSON formats to specify actions, which constrains the action space and hampers flexibility, especially when tasks require logical control, tool composition, or dynamic adjustment based on feedback.
This paper introduces CodeAct, a novel framework that empowers LLMs to generate executable Python code as their actions. By integrating with a Python interpreter, CodeAct enables models to perform complex operations, control flow, and data management within a single, flexible action format. This approach leverages the extensive programming knowledge embedded in pre-trained models, allowing for more sophisticated decision-making and task execution. Multi-turn interactions facilitate real-time feedback and self-debugging, significantly improving task success rates and reducing the number of interaction steps.
The authors conducted comprehensive experiments on API-Bank and a newly curated benchmark, M3ToolEval, involving 17 different models. Results show that CodeAct consistently outperforms traditional text and JSON-based methods, achieving up to 20% higher success rates and requiring fewer interaction turns. The frameworkβs ability to handle multi-tool, multi-step tasks demonstrates its potential in real-world applications such as robotic control, scientific experimentation, and autonomous decision-making.
Building upon this success, the authors created CodeActInstruct, a large-scale instruction-tuning dataset with 7,000 multi-turn interactions, to further enhance model performance. Fine-tuned models, such as CodeActAgent based on LLaMA2 and Mistral, exhibit autonomous debugging, complex task execution, and collaboration capabilities, showcasing practical deployment prospects.
This work marks a significant leap in making AI agents more autonomous, flexible, and capable of tackling real-world challenges. By transforming action representation from static commands to dynamic, executable code, it opens new horizons for intelligent systems in industry and research. Future efforts will focus on multi-language support, reinforcement learning integration, and expanding real-world applications, promising a new era of highly capable autonomous AI agents.
Deep Dive
Limitations & Outlook
What gaps remain?
Abstract
Large Language Model (LLM) agents, capable of performing a broad range of actions, such as invoking tools and controlling robots, show great potential in tackling real-world challenges. LLM agents are typically prompted to produce actions by generating JSON or text in a pre-defined format, which is usually limited by constrained action space (e.g., the scope of pre-defined tools) and restricted flexibility (e.g., inability to compose multiple tools). This work proposes to use executable Python code to consolidate LLM agents' actions into a unified action space (CodeAct). Integrated with a Python interpreter, CodeAct can execute code actions and dynamically revise prior actions or emit new actions upon new observations through multi-turn interactions. Our extensive analysis of 17 LLMs on API-Bank and a newly curated benchmark shows that CodeAct outperforms widely used alternatives (up to 20% higher success rate). The encouraging performance of CodeAct motivates us to build an open-source LLM agent that interacts with environments by executing interpretable code and collaborates with users using natural language. To this end, we collect an instruction-tuning dataset CodeActInstruct that consists of 7k multi-turn interactions using CodeAct. We show that it can be used with existing data to improve models in agent-oriented tasks without compromising their general capability. CodeActAgent, finetuned from Llama2 and Mistral, is integrated with Python interpreter and uniquely tailored to perform sophisticated tasks (e.g., model training) using existing libraries and autonomously self-debug.
References (20)
TaskWeaver: A Code-First Agent Framework
Bo Qiao, Liqun Li, Xu Zhang et al.
Measuring Coding Challenge Competence With APPS
Dan Hendrycks, Steven Basart, Saurav Kadavath et al.
Mistral 7B
Albert Qiaochu Jiang, Alexandre Sablayrolles, Arthur Mensch et al.
Llama 2: Open Foundation and Fine-Tuned Chat Models
Hugo Touvron, Louis Martin, Kevin R. Stone et al.
MINT: Evaluating LLMs in Multi-turn Interaction with Tools and Language Feedback
Xingyao Wang, Zihan Wang, Jiateng Liu et al.
ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs
Yujia Qin, Shi Liang, Yining Ye et al.
Describe, Explain, Plan and Select: Interactive Planning with Large Language Models Enables Open-World Multi-Task Agents
Zihao Wang, Shaofei Cai, Anji Liu et al.
WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agents
Shunyu Yao, Howard Chen, John Yang et al.
Code as Policies: Language Model Programs for Embodied Control
Jacky Liang, Wenlong Huang, F. Xia et al.
Scaling Instruction-Finetuned Language Models
Hyung Won Chung, Le Hou, S. Longpre et al.
Code4Struct: Code Generation for Few-Shot Event Structure Prediction
Xingyao Wang, Sha Li, Heng Ji
PAL: Program-aided Language Models
Luyu Gao, Aman Madaan, Shuyan Zhou et al.
ViperGPT: Visual Inference via Python Execution for Reasoning
D'idac Sur'is, Sachit Menon, Carl Vondrick
ReAct: Synergizing Reasoning and Acting in Language Models
Shunyu Yao, Jeffrey Zhao, Dian Yu et al.
Self-Consistency Improves Chain of Thought Reasoning in Language Models
Xuezhi Wang, Jason Wei, Dale Schuurmans et al.
ScienceWorld: Is your Agent Smarter than a 5th Grader?
Ruoyao Wang, Peter Alexander Jansen, Marc-Alexandre CΓ΄tΓ© et al.
Training language models to follow instructions with human feedback
Long Ouyang, Jeff Wu, Xu Jiang et al.
Chain of Thought Prompting Elicits Reasoning in Large Language Models
Jason Wei, Xuezhi Wang, Dale Schuurmans et al.
CoAuthor: Designing a Human-AI Collaborative Writing Dataset for Exploring Language Model Capabilities
Mina Lee, Percy Liang, Qian Yang
WebGPT: Browser-assisted question-answering with human feedback
Reiichiro Nakano, Jacob Hilton, S. Balaji et al.
Cited By (20)
The Path to Self-Evolving Clinical Systems: Scaling Medical Agents from Assistance to Autonomy
Evo-Bench: Can Language Models Improve Agent Harness?
Self-Evolving Coding Agents
DiDPO: Diff-in-Diff Policy Optimization for Coding Agent Training
A Systematic Evaluation of Trajectory Data Curation for LoRA Fine-Tuning of Code Agents
HiRA: Decoupling Planning and Execution with Hierarchical Reasoning in Deep Search
Multi-Agent LLM Collaboration for Unit Test Generation via Human-Testing-Inspired Workflows
Eluna: An Agentic LLM System for Automating Warehouse Operations with Reasoning and Task Execution
Harness VLA: Steering Frozen VLAs into Reliable Manipulation Primitives via Memory-Guided Agents
From Atomic Actions to Standard Operating Procedures: Iterative Tool Optimization for Self-Evolving LLM Agents
Agentic Service-Oriented Computing: A Manifesto for the Next Frontier of Service-Oriented Computing
Large Language Models for Quantum Software Engineering: Opportunities and Challenges in Modeling, Generation, Testing, and Repair
TRACE: Turn-level Reward Assignment via Credit Estimation for Long-Horizon Agents
MM-ToolSandBox: A Unified Framework for Evaluating Visual Tool-Calling Agents
RoboHarness: Memory-Driven Orchestration of Heterogeneous Robot Policies for Long-Horizon Planning
NVIDIA-labs OO Agents: Native Python Object-Oriented Agents
LLM-Supervised Point Cloud Processing: From Unsupervised 3D Scene-Graph Generation to Interactive Scene Manipulation
OpenForgeRL: Train Harness-native Agents in Any Environment
Agent Team Work Zone: An Automated, Persistent Workspace for Long-Lived Claude Code Agent Teams
StateAct: Program State, before Pixels, for Long-Horizon Computer-Use Agents