Multi-Agent Computer Use
Multi-Agent Computer Use (MACU) improves performance by 3.4-25.5% on desktop and web benchmarks.
Key Findings
Methodology
The paper introduces a Multi-Agent Computer Use (MACU) system where a manager model decomposes tasks into a Directed Acyclic Graph (DAG) and dispatches parallel subagents. The manager continuously revises the DAG based on feedback, ensuring information flow and task optimization. This approach is particularly suited for partially observable environments.
Key Results
- MACU improved success rate by 4.7% on OSWorld, reducing average task completion time by 5.2 minutes.
- On the Odysseys long-horizon web navigation benchmark, MACU improved task completion time by 1.5x and success rate by 25.5%.
- On WebTailBench-v2, MACU increased success rate from 20.8% to 29.5%, significantly enhancing complex task execution.
Significance
MACU addresses the bottlenecks of single-agent systems in long-horizon tasks by leveraging multi-agent collaboration, significantly enhancing the efficiency of complex task execution. This approach offers new directions for expanding computer use agents, especially in scenarios requiring rapid response and dynamic adjustment.
Technical Contribution
MACU provides a new technical framework for task decomposition and parallel execution, significantly improving task handling efficiency compared to existing single-agent methods. Its DAG structure allows flexible task management and information flow, enhancing system adaptability and scalability.
Novelty
MACU is the first to apply multi-agent systems to computer use tasks, using DAGs for task decomposition and parallel execution, setting it apart from traditional single-agent methods.
Limitations
- MACU may not fully leverage its advantages in fully observable environments.
- The complexity of the system increases the computational overhead of the manager model.
- Further optimization is needed to reduce resource contention during parallel execution.
Future Work
Future research could explore MACU's performance in more application scenarios and optimize its efficiency in resource-constrained environments. Additionally, integrating more advanced agent models could further enhance system performance.
AI Executive Summary
Current Computer Use Agents (CUAs) are primarily deployed as single agents, which are inefficient for complex long-horizon tasks. This paper proposes a Multi-Agent Computer Use (MACU) system, where a manager model decomposes tasks into a Directed Acyclic Graph (DAG) and dispatches parallel subagents. The manager continuously revises the DAG based on feedback, ensuring information flow and task optimization. Experiments show that MACU consistently outperforms strong single-agent baselines, particularly excelling in long-horizon tasks. MACU's design allows it to better address the challenges of partially observable environments, significantly improving task completion efficiency. Future research could further optimize MACU's performance in resource-constrained environments and explore its potential in more application scenarios.
Deep Analysis
Background
The study of Computer Use Agents (CUAs) aims to interact with software through Graphical User Interfaces (GUIs). Most CUAs are deployed as single agents, which perform well on certain tasks but are limited in handling complex long-horizon tasks. Recently, multi-agent systems have shown advantages in task decomposition and parallel execution across various fields.
Core Problem
Single-agent CUAs are inefficient for complex long-horizon tasks, primarily due to their inability to effectively decompose tasks and execute them in parallel. Additionally, single-agent systems have limited adaptability in dynamic environments, making it difficult to quickly adjust plans.
Innovation
MACU uses a manager model to decompose tasks into a Directed Acyclic Graph (DAG), enabling parallel execution. The manager continuously adjusts the DAG based on feedback from subagents, ensuring information flow and task optimization. This innovation allows the system to better address the challenges of partially observable environments.
Methodology
- �� The manager model decomposes tasks into a DAG, identifying dependencies and goals for subagents.
- �� In each iteration, the manager dispatches parallel subagents to execute nodes on the ready frontier of the DAG.
- �� The manager dynamically adjusts the DAG based on feedback from subagents, adding, canceling, or rewriting nodes.
- �� The DAG structure retains and passes forward environment information through the manager.
Experiments
Experiments were conducted on multiple benchmarks, including OSWorld, Online-Mind2Web, WebTailBench-v2, and Odysseys. Qwen3.6-27B was used as the subagent, and Claude Opus 4.6 as the manager. The experimental setup included varying the number of parallel subagents and replanning budgets to evaluate MACU's performance.
Results
MACU improved success rate by 4.7% on OSWorld, reducing average task completion time by 5.2 minutes. On the Odysseys long-horizon web navigation benchmark, MACU improved task completion time by 1.5x and success rate by 25.5%. On WebTailBench-v2, MACU increased success rate from 20.8% to 29.5%, significantly enhancing complex task execution.
Applications
MACU is suitable for complex task scenarios requiring rapid response and dynamic adjustment, such as desktop application automation and web navigation. Its parallel execution capability offers potential value in industries requiring efficient task processing.
Limitations & Outlook
MACU may not fully leverage its advantages in fully observable environments. The complexity of the system increases the computational overhead of the manager model. Further optimization is needed to reduce resource contention during parallel execution. Future research could explore MACU's performance in more application scenarios and optimize its efficiency in resource-constrained environments.
Plain Language Accessible to non-experts
Imagine a large factory where a single worker needs to complete all tasks, similar to a single-agent system. In contrast, a multi-agent system is like a team, where each worker handles different parts. The MACU system uses a manager to decompose tasks and assign them to different workers, adjusting tasks as needed. This greatly increases the factory's efficiency as each worker focuses on their task and can adjust based on new information.
ELI14 Explained like you're 14
Imagine playing a complex game with friends, where everyone has their own tasks. MACU is like the team captain in the game, assigning tasks to everyone and adjusting plans based on the game's progress. This way, you can complete tasks faster because everyone knows what to do and can change strategies as needed. Isn't that cool?
Glossary
Multi-Agent System
A system composed of multiple autonomous agents that can collaborate to complete complex tasks.
In MACU, multiple subagents execute tasks in parallel.
Directed Acyclic Graph (DAG)
A graph structure with directed edges and no cycles.
Used to represent task decomposition and dependencies.
Task Decomposition
Breaking down complex tasks into smaller sub-tasks for parallel execution.
MACU uses DAGs to achieve task decomposition.
Partially Observable Environment
An environment where the state is not fully visible, requiring agents to make decisions based on incomplete information.
MACU is designed to address challenges in partially observable environments.
Replanning
Adjusting the original plan based on new information to optimize task execution.
The MACU manager model replans based on subagent feedback.
Open Questions Unanswered questions from this research
- 1 How to optimize MACU's performance in fully observable environments?
- 2 How to reduce computational overhead in resource-constrained environments?
- 3 How to further improve task decomposition efficiency in MACU?
Applications
Immediate Applications
Desktop Application Automation
MACU can be used to automate complex desktop application tasks, improving work efficiency.
Long-term Vision
Intelligent Web Navigation
Implementing smarter web navigation through MACU, enhancing user experience and information retrieval efficiency.
Abstract
Computer use agents (CUAs) today are primarily deployed as single serial agents. This setup is suboptimal for complex long-horizon tasks that benefit from task decomposition, parallel execution, and consistent re-planning based on new information. In this paper, we argue that we should instead move towards evaluating and building multi-agent computer use (MACU) systems. These systems, which emphasize planning and parallel execution, alleviate many of the shortcomings of single-agent CUAs. We propose a general multi-agent setup in which a manager model decomposes computer use tasks as a directed acyclic graph (DAG), encoding relevant dependencies and goals for subagents. At each iteration, the manager dispatches parallel CUA subagents to carry out nodes on the ready frontier of the DAG, and continuously revises the DAG (adding, canceling, or rewriting nodes) as new findings arrive from subagents. This design treats the partially observable environment of computer use as a first class challenge: information that downstream agents may not be able to re-observe are retained and passed forward through the manager and DAG structure. We demonstrate that MACU consistently improves over strong single-agent baselines by $3.4-25.5\%$ on desktop (OSWorld) and web navigation (Online-Mind2Web, WebTailBench, Odysseys) benchmarks, exhibits more favorable test-time scaling, and solves complex long-horizon tasks where single-agent CUAs get stuck. On Odysseys, a long-horizon web navigation benchmark, MACU improves average task completion wall-clock time by ${\sim} 1.5 \times$, demonstrating its efficacy in speeding up traditionally slow CUA pipelines. Our findings highlight that multi-agent coordination is a promising axis for scaling computer use agents to work productively for longer and more effectively. We release all code and interactive visualizations at https://jykoh.com/multi-agent-computer-use.