Planner Matters! An Efficient and Unbalanced Multi-agent Collaboration Framework for Long-horizon Planning
Proposed a planner-centric multi-agent framework, significantly improving long-horizon task success rates.
Key Findings
Methodology
The paper proposes a multi-agent framework decomposing tasks into planner, actor, and memory manager roles. The planner handles high-level decisions, the actor executes tasks, and the memory manager provides contextual reasoning. Reinforcement learning optimizes the planner using trajectory-level rewards from a VLM-as-judge, while freezing other components.
Key Results
- Experiments show that with the Qwen2.5-VL-7B model, optimizing the planner increases success rates by 28%, outperforming GPT-4o and Gemini-2.5-Pro on the WebVoyager benchmark.
- In multi-agent configurations, Qwen2.5-VL-7B's overall success rate on four WebVoyager domains increased from 12.3% to 40.6%.
- Compute allocation analysis reveals the planner as the primary bottleneck for performance improvement.
Significance
This research significantly improves the efficiency and effectiveness of long-horizon planning tasks by concentrating computational resources on the planner. It holds important implications for academia and industry, particularly in tasks requiring complex decision-making and long-term reasoning.
Technical Contribution
The technical contribution lies in proposing an unbalanced multi-agent framework emphasizing the planner's core role, optimized through reinforcement learning. This contrasts sharply with existing single-model approaches, offering new engineering possibilities.
Novelty
This is the first systematic analysis of compute allocation importance in multi-agent frameworks, proposing a planner-centric optimization strategy that significantly innovates over traditional balanced compute allocation methods.
Limitations
- In dynamic environments, the planner may still make errors, such as overly aggressive decisions or premature termination.
- The memory manager's scale has a minor impact on performance, potentially leading to insufficient information in complex tasks.
Future Work
Future research can explore finer-grained reward mechanisms, incorporating step-level rewards to further enhance the planner's decision-making capabilities. Additionally, optimizing multi-agent collaboration across different task types can be investigated.
AI Executive Summary
In the automation of complex tasks, existing language model agents struggle with long-horizon planning and reasoning. To address this, the paper proposes an enhanced multi-agent framework, decomposing tasks into planner, actor, and memory manager roles. Reinforcement learning optimizes the planner while freezing other components, significantly improving task success rates.
Experimental results show robust and compute-efficient improvements in benchmarks spanning web navigation, OS control, and tool use. The Qwen2.5-VL-7B model in multi-agent configuration outperforms GPT-4o and Gemini-2.5-Pro.
Despite significant progress, limitations remain, such as potential errors in dynamic environments. Future research directions include exploring finer-grained reward mechanisms and optimizing multi-agent collaboration across different task types.
Deep Analysis
Background
With the development of vision-language models (VLMs), automated agents have demonstrated strong capabilities in operating complex systems. However, existing single-model approaches still struggle with long-horizon planning and reasoning. Researchers have begun exploring multi-agent systems to improve task completion efficiency through role specialization.
Core Problem
Long-horizon planning tasks require balancing high-level goal management and low-level action execution in dynamic environments. Single-model approaches struggle to excel in both areas simultaneously, leading to a loss of overall goals in complex tasks.
Innovation
The innovation lies in proposing an unbalanced multi-agent framework emphasizing the planner's core role. Through compute allocation analysis, the planner is identified as the primary bottleneck for performance improvement, optimized through reinforcement learning.
Methodology
- �� Adopt a multi-agent framework decomposing tasks into planner, actor, and memory manager roles.
- �� The planner handles high-level decisions, the actor executes tasks, and the memory manager provides contextual reasoning.
- �� Optimize the planner using reinforcement learning, freezing other components, and utilizing VLM to evaluate trajectory rewards.
Experiments
Experiments are conducted on benchmarks like WebVoyager, OS control, and tool use, using the Qwen2.5-VL-7B model in multi-agent configurations. Compute allocation analysis validates the planner's key role in performance improvement.
Results
Results show that optimizing the planner increases Qwen2.5-VL-7B's success rate by 28% on the WebVoyager benchmark, outperforming GPT-4o and Gemini-2.5-Pro.
Applications
The framework can be applied to tasks requiring complex decision-making and long-term reasoning, such as web navigation, software control, and API tool use. Its efficient compute allocation strategy holds significant industrial value.
Limitations & Outlook
Despite significant progress, the planner may still make errors in dynamic environments, such as overly aggressive decisions or premature termination. Future research can explore finer-grained reward mechanisms to further enhance the planner's decision-making capabilities.
Plain Language Accessible to non-experts
Imagine a factory with three workers: a planner, an actor, and a memory manager. The planner is like the factory manager, responsible for making production plans and goals. The actor is the worker, executing tasks according to the manager's plan. The memory manager is the archivist, recording and providing past production information. This system improves factory efficiency by allowing the manager to focus on high-level decisions, the worker on specific tasks, and the archivist on necessary information.
ELI14 Explained like you're 14
Imagine you're playing a super complex game where you have to manage many characters. You have a super smart captain who makes strategies and plans. Then there's an actor who follows the captain's plan to fight monsters. There's also a memory master who tells the captain about past battle experiences. This method makes it easier to win the game because the captain can focus on strategy, not every little detail. Isn't that cool?
Glossary
Planner
The agent role responsible for high-level decision-making and goal decomposition.
In the multi-agent framework, the planner is the core decision-maker.
Actor
The agent role responsible for executing specific tasks.
The actor operates based on the planner's instructions.
Memory Manager
The agent role responsible for contextual reasoning and information management.
The memory manager provides historical information needed by the planner.
Reinforcement Learning
A learning method that optimizes agent decisions through reward mechanisms.
Used to optimize the planner's performance.
Vision-Language Model
A model that combines visual and language information for reasoning.
Used to evaluate trajectory rewards.
Open Questions Unanswered questions from this research
- 1 How to further optimize the planner's decision-making in dynamic environments?
- 2 What are the optimization strategies for multi-agent frameworks across different task types?
Applications
Immediate Applications
Web Navigation
Improve automation efficiency in web navigation tasks using the multi-agent framework.
Long-term Vision
Complex System Automation
Apply the multi-agent framework in industry to achieve efficient automation of complex systems.
Abstract
Language model (LM)-based agents have demonstrated promising capabilities in automating complex tasks from natural language instructions, yet they continue to struggle with long-horizon planning and reasoning. To address this, we propose an enhanced multi-agent framework that decomposes automation into three roles: a planner for high-level decision-making, an actor for task execution, and a memory manager for contextual reasoning. While this modular decomposition aligns with established design patterns, our core contribution lies in a systematic compute-allocation analysis, revealing that planning is the dominant factor influencing task performance. Execution and memory management require significantly less compute and model capacity to achieve competitive results. Building on these insights, we introduce a planner-centric reinforcement learning approach, which exclusively optimizes the planner using trajectory-level rewards from a VLM-as-judge, while freezing the other components. Extensive experiments on benchmarks spanning web navigation, OS control, and tool use demonstrate that concentrating model capacity and learning on high-level planning yields robust and compute-efficient improvements in long-horizon agent automation. Our code is publicly released.