ART: Automatic multi-step reasoning and tool-use for large language models
ART enables automatic multi-step reasoning and tool use in LLMs, improving unseen task performance by over 22%.
Key Findings
Methodology
ART constructs structured programs for reasoning by retrieving related demonstrations from a task library, then generates programs with the model. During inference, the model pauses to invoke external tools (search, code execution), integrates outputs, and continues. This process allows the model to autonomously produce multi-step reasoning chains with tool calls, without fine-tuning. The approach leverages pre-trained models like InstructGPT and Codex, enabling cross-task transfer, extensibility, and minimal supervision, significantly enhancing performance on unseen tasks across benchmarks.
Key Results
- On BigBench and MMLU, ART outperforms few-shot prompts and auto-CoT, with an average accuracy increase of over 22%. Particularly in arithmetic and algorithmic tasks, improvements exceed 12 percentage points, matching or surpassing handcrafted prompts.
- Incorporating tool use boosts performance by an additional 12.3%, demonstrating the effectiveness of external tools in complex reasoning. The framework's ability to generalize and be refined via human feedback further enhances results.
- Compared to models with supervised decomposition, ART achieves comparable or better results with minimal human intervention, showing robustness and scalability.
Significance
This work addresses core limitations in large language models' multi-step reasoning, offering an automated, flexible framework that reduces manual prompt engineering. By enabling models to generate programs that call external tools, it pushes toward more autonomous, adaptable AI systems capable of complex reasoning across diverse domains. Its transferability and extensibility open pathways for applications in scientific research, automated coding, and knowledge-intensive tasks, marking a significant step toward general AI capabilities.
Technical Contribution
The paper introduces a novel framework combining structured program generation, task retrieval, and external tool invocation, all within a frozen pre-trained model. The program syntax supports flexible reasoning steps and tool calls, with a parser that manages pauses and resumes during inference. This design allows zero-shot transfer across tasks, minimal supervision, and easy extension by humans. The approach also demonstrates how to leverage existing models like Codex for code generation and search APIs for information retrieval, integrating these seamlessly into the reasoning process.
Novelty
This is the first framework to fully automate multi-step reasoning with external tool calls via program generation, without task-specific fine-tuning. Unlike prior work on chain-of-thought prompting or tool use, ART combines structured program synthesis, cross-task demonstration retrieval, and dynamic tool invocation, enabling scalable, interpretable, and extensible reasoning across diverse tasks. Its ability to operate zero-shot and incorporate human corrections marks a significant innovation.
Limitations
- The framework relies on a predefined task and tool library, limiting generalization to tasks outside these sets. Expanding the library requires manual effort or additional retrieval strategies.
- Program correctness depends on the model's ability to generate syntactically valid and semantically accurate programs; errors can lead to failure or incorrect answers.
- External tool invocation introduces latency and potential security concerns, especially in real-time or sensitive applications. Further robustness and safety measures are needed.
Future Work
Future directions include expanding the toolset to include more complex and domain-specific tools, integrating multi-modal data, and applying reinforcement learning to improve program accuracy. Additionally, developing automated methods for library expansion, error detection, and safety assurance will be crucial. Extending ART to multi-agent systems and real-world applications, such as scientific discovery or autonomous agents, also presents promising avenues.
AI Executive Summary
The rapid advancement of large language models (LLMs) like GPT-3 and InstructGPT has revolutionized NLP, yet their capacity for complex, multi-step reasoning remains limited. Traditional approaches, such as chain-of-thought prompting, have demonstrated improvements but rely heavily on manual prompt engineering and lack scalability. Moreover, models struggle with tasks requiring external computations, such as arithmetic, search, or code execution, which are essential for real-world problem solving.
To address these challenges, this paper introduces ART (Automatic Reasoning and Tool-use), a novel framework that enables LLMs to autonomously generate structured programs representing multi-step reasoning chains. By retrieving related demonstrations from a task library, ART constructs flexible prompts that guide the model to produce reasoning programs. During inference, the model pauses at tool call points, invokes external resources like search engines or code interpreters, and resumes reasoning with integrated outputs. This dynamic process allows the model to perform complex reasoning tasks without task-specific fine-tuning.
Experimental results on benchmarks such as BigBench and MMLU show that ART achieves over 22% accuracy improvements on unseen tasks, outperforming traditional few-shot prompts and auto-CoT methods. The inclusion of tool use further enhances performance, especially on arithmetic and algorithmic problems, with average gains exceeding 12%. Notably, human feedback can be incorporated to correct errors or add new tools, leading to even higher accuracy, surpassing previous state-of-the-art results.
The significance of ART lies in its ability to democratize multi-step reasoning, reducing reliance on manual prompt design, and enabling scalable, interpretable, and extensible AI systems. Its architecture supports cross-task transfer, easy integration of new tools, and minimal supervision, making it a promising step toward more autonomous and intelligent models. Future work aims to expand toolsets, incorporate multi-modal data, and develop safety mechanisms, paving the way for broader applications in scientific research, automation, and intelligent decision-making.
Deep Analysis
Background
近年来，预训练大模型（如GPT-3、InstructGPT）在自然语言处理中的表现持续突破，尤其在少样本和零样本学习方面展现出强大潜力。链式推理（CoT）作为一种引导模型逐步展开推理的技术，显著提升了模型在数学、逻辑推理等复杂任务中的表现。然而，传统的链式推理依赖手工设计提示，难以扩展到新任务或多工具场景。近年来，研究者尝试微调模型或引入工具（如Toolformer），以增强模型推理能力，但成本高、缺乏灵活性。本文提出的ART旨在通过自动生成程序化推理链，突破这些限制，推动模型自主推理和工具调用的能力。
Core Problem
当前大模型在多步推理任务中的表现受限，尤其在跨任务迁移和复杂推理链生成方面存在瓶颈。手工提示设计繁琐，难以适应多样任务和工具集。微调虽能改善性能，但成本高昂且缺乏灵活性。如何让模型自主生成推理程序，自动调用外部工具，成为提升复杂任务解决能力的关键。解决这一问题，不仅能降低人工成本，还能增强模型的适应性和扩展性，推动其在科学计算、知识检索等领域的应用。
Innovation
本研究的核心创新在于提出基于结构化程序的ART框架，结合任务库和工具库实现自动化多步推理。具体创新点包括：
- �� 设计支持多任务迁移和工具调用的程序语法，增强可解释性；
- �� 利用任务库检索示范，构建灵活的推理程序，避免人工提示；
- �� 在推理过程中暂停模型生成，调用外部工具（搜索、代码执行），整合输出后继续推理；
- �� 无需微调，模型自主生成程序，支持多任务迁移和扩展。这些创新突破了传统提示工程的局限，为模型赋予了更强的自主推理和工具利用能力。
Methodology
- �� 构建任务库：收集15个多样任务，定义程序语法，示范多任务推理程序；
- �� 检索示范：根据任务相似度，从库中选取示范，构建多任务prompt；
- �� 程序生成：模型在推理时自主写程序，调用工具（搜索、代码）暂停与继续；
- �� 工具调用：识别工具调用指令，暂停模型生成，调用工具，整合输出；
- �� 人类反馈：用户可编辑程序或工具库，改进性能；
- �� 评估：在BigBench、MMLU等基准上测试，比较不同策略和基线，验证效果。
Experiments
采用InstructGPT作为主模型，使用预定义任务库和工具库，设置不同提示策略，评估准确率。对比few-shot、自动CoT、ART（有无工具），在未见任务上进行测试。指标包括准确率提升、任务迁移能力和工具调用效率。还进行人类微调实验，验证可扩展性和性能提升。实验结果显示，ART在大部分任务中优于基线，尤其在算术和算法推理方面表现突出。
Results
ART在BigBench和MMLU测试中，平均提升准确率达22%以上，算术任务提升超过12个百分点。工具调用后，性能平均提升12.3个百分点，验证了工具的有效性。与手工提示相比，自动生成的推理链表现相当甚至更优，且能通过人类微调进一步提升。整体结果表明，ART在多任务迁移和复杂推理中具有显著优势。
Applications
该框架适用于科学计算、知识问答、自动编程等场景，能帮助模型自主生成推理流程，调用搜索引擎、代码执行器等工具，提升复杂任务的解决能力。企业可利用ART实现智能客服、自动诊断等应用，减少人工干预。未来，结合多模态数据和强化学习，ART有望实现更高层次的自主推理与学习。
Limitations & Outlook
目前依赖预定义任务和工具库，泛化到未覆盖任务仍有限。程序生成的正确性受模型能力影响，错误程序可能导致推理失败。调用外部工具存在延迟和安全风险。未来需增强程序验证、扩展工具集、提升鲁棒性，以实现更广泛应用。
Plain Language Accessible to non-experts
想象你在厨房做饭，所有的步骤都需要按照一定顺序，比如先洗菜、再切菜、然后炒菜。每一步都可以用不同的工具，比如洗菜池、刀、锅。现在，假设你有一个聪明的助手，它可以帮你自动规划这些步骤，还能调用不同的工具，比如帮你查菜谱、计算调料用量。这个助手不需要你手工写每个步骤，而是根据你提供的食谱自动生成详细的操作流程，并在需要时调用工具帮忙。这样一来，做饭变得更快、更智能，也更容易应对不同的菜谱和工具。ART的思想也是类似的，它让大模型像这个聪明的助手一样，自动规划推理步骤，调用各种工具，完成复杂任务。
ELI14 Explained like you're 14
想象你在学校里做科学实验，你需要按照步骤操作，比如先准备材料，然后测量、计算、最后得出结论。现在，如果你有一个超级聪明的朋友，他可以帮你规划每一步，还能帮你查资料、做计算，甚至帮你写代码。你只需要告诉他任务，他就会自动想出详细的步骤，调用不同的工具帮忙，最后帮你完成整个实验。这个朋友不用你教他每个细节，他自己会学会怎么做，还能根据需要改进。ART的想法也是一样，它让大模型像这个聪明的朋友一样，自己规划推理步骤，调用工具，完成复杂的任务。这样，模型就变得更聪明、更自主了！
Glossary
Chain of Thought (思维链)
一种引导模型逐步推理的方法，通过生成中间步骤提升复杂任务的表现。技术上，模型在提示中逐步展开推理链。
在论文中，CoT用于提升模型在多步推理任务中的准确性，但手工设计提示繁琐，扩展性有限。
程序化推理 (Program-based Reasoning)
将推理过程表示为结构化程序，包含步骤和工具调用，模型自动生成执行流程。技术上，类似编程语言的表达。
ART利用程序化推理，将推理拆解成可调用工具的程序，增强可解释性和迁移能力。
工具调用 (Tool Use)
模型在推理中调用外部工具（如搜索、代码执行）支持计算或信息检索，提升推理能力。
在ART中，工具调用通过暂停模型生成，调用工具后再继续，增强复杂推理表现。
任务库 (Task Library)
存放多任务示范程序和示例，用于检索相关示范指导新任务推理。
ART通过任务库检索示范，构建推理程序，实现跨任务迁移。
工具库 (Tool Library)
存放各种工具（搜索、代码、查词等），供模型调用支持推理。
模型在推理时识别工具调用指令，暂停生成调用工具，整合输出后继续。
Open Questions Unanswered questions from this research
- 1 如何进一步扩展工具库以支持更复杂的推理场景，尤其是在多模态和动态环境中，仍是未解难题。
- 2 模型自动生成程序的准确性和鲁棒性仍需提升，避免错误程序导致推理失败。
- 3 在实际应用中，调用外部工具的安全性和效率问题亟待解决，尤其在敏感或实时场景。
Applications
Immediate Applications
科学计算与教育辅助
利用ART自动生成解题推理程序，帮助学生理解复杂问题，支持自动批改和辅导，提升学习效率。
智能问答与知识检索
结合搜索工具，提升大模型在开放域问答中的准确性，应用于客服、信息检索等场景。
Long-term Vision
自主科研与自动化推理系统
未来结合多模态和强化学习，打造具备自主推理、工具扩展和学习能力的AI，推动科学研究和工业自动化。
Abstract
Large language models (LLMs) can perform complex reasoning in few- and zero-shot settings by generating intermediate chain of thought (CoT) reasoning steps. Further, each reasoning step can rely on external tools to support computation beyond the core LLM capabilities (e.g. search/running code). Prior work on CoT prompting and tool use typically requires hand-crafting task-specific demonstrations and carefully scripted interleaving of model generations with tool use. We introduce Automatic Reasoning and Tool-use (ART), a framework that uses frozen LLMs to automatically generate intermediate reasoning steps as a program. Given a new task to solve, ART selects demonstrations of multi-step reasoning and tool use from a task library. At test time, ART seamlessly pauses generation whenever external tools are called, and integrates their output before resuming generation. ART achieves a substantial improvement over few-shot prompting and automatic CoT on unseen tasks in the BigBench and MMLU benchmarks, and matches performance of hand-crafted CoT prompts on a majority of these tasks. ART is also extensible, and makes it easy for humans to improve performance by correcting errors in task-specific programs or incorporating new tools, which we demonstrate by drastically improving performance on select tasks with minimal human intervention.