Beyond Browsing: API-Based Web Agents
Proposed API-calling and Hybrid Agents, API Agent improved by 24.0% on WebArena, achieving 38.9% success rate.
Key Findings
Methodology
The study introduces two agents: API-calling and Hybrid Agents. The API-calling agent performs tasks solely through APIs, while the Hybrid Agent combines browsing with API calls. Experiments were conducted on WebArena, demonstrating the superiority of API agents.
Key Results
- API Agent outperformed Browsing Agent by approximately 15% on WebArena, achieving a success rate of 29.2%.
- Hybrid Agent outperformed both others across tasks, achieving a 38.9% success rate.
- API Agent performed better on sites with good API support, like Gitlab.
Significance
The study shows that APIs can serve as an alternative or complement to traditional browsing methods, especially when APIs are well-supported. It provides a more efficient way for machines to interact with the web, advancing task automation.
Technical Contribution
Introduced a Hybrid Agent combining API calls and browsing, demonstrating flexibility in scenarios with limited API support. This method sets a new performance benchmark for task-agnostic agents.
Novelty
First systematic combination of API calls with traditional browsing, showcasing the potential of APIs in task automation, especially in well-supported environments.
Limitations
- Hybrid Agent still relies on browsing for sites with limited API support.
- Performance is affected by incomplete API documentation on some sites.
Future Work
Future work includes expanding API support, improving Hybrid Agent performance on sites with limited APIs, and researching automatic API documentation generation.
AI Executive Summary
The internet is a crucial portal for human activities, yet traditional web agents often struggle with complex tasks due to browser interface limitations. This paper proposes API-calling and Hybrid Agents to interact directly with web services via APIs, enhancing task automation.
In WebArena benchmarks, the API-calling agent excelled without relying on browsing, achieving a success rate of 29.2%. The Hybrid Agent further improved task success rates to 38.9% by dynamically combining API calls and browsing. This indicates that APIs, when available, can significantly enhance agent efficiency and accuracy.
However, limited API support remains a challenge. Future research directions include expanding API coverage, improving API documentation usability, and exploring more application scenarios for Hybrid Agents.
Deep Analysis
Background
With the proliferation of the internet, web agents have become essential tools for task automation. Traditional agents rely on graphical user interfaces for interaction, which is not ideal for machines. Recently, APIs have gained attention as interfaces for machine interaction with web services.
Core Problem
Traditional web agents face limitations in handling complex tasks due to browser interface interaction methods, leading to inefficiency and errors. The challenge is to leverage APIs to enhance agent efficiency and accuracy.
Innovation
This paper introduces two innovative agents: API-calling and Hybrid Agents. The API-calling agent interacts directly with web services via APIs, while the Hybrid Agent combines browsing and API calls to flexibly handle different tasks.
Methodology
- �� API-calling Agent: Completes tasks solely through APIs, avoiding browser interaction.
- �� Hybrid Agent: Combines API calls and browsing, dynamically choosing the optimal strategy.
- �� Experiments conducted on WebArena benchmark to evaluate agent performance.
Experiments
Experiments were conducted on WebArena, covering multiple sites like Gitlab and Map. Evaluation metrics included success rate and task completion time. The baseline was traditional browsing agents.
Results
API-calling Agent achieved a 29.2% success rate on WebArena, while Hybrid Agent achieved 38.9%. API Agent excelled on sites with good API support.
Applications
API Agents are suitable for sites with good API support, like Gitlab. Hybrid Agents offer flexible solutions for sites with limited API support.
Limitations & Outlook
API support limitations pose a major challenge, affecting agent performance. Future work should expand API coverage and enhance agent adaptability in different environments.
Plain Language Accessible to non-experts
Imagine shopping in a large mall. Traditional web agents are like you walking to each store to find items, which is inefficient. API agents are like having a shopping assistant who directly fetches items from the warehouse for you. Hybrid agents combine the best of both, using the assistant when available and walking to stores when necessary.
ELI14 Explained like you're 14
Imagine playing a game where your task is to find a treasure. Traditional web agents are like searching the map, which is slow. API agents are like having a compass in the game that points directly to the treasure. Hybrid agents have both the compass and the ability to explore, ensuring you find the treasure fastest!
Glossary
API (Application Programming Interface)
An API is an interface that allows different software systems to communicate, typically used for machine interaction with web services.
In this paper, APIs are used for agents to interact directly with web services, bypassing browser interfaces.
Web Agent
A web agent is an automated tool that simulates human user actions on web pages.
Traditional web agents interact with web pages through browser interfaces, which is less efficient.
Hybrid Agent
A hybrid agent combines API calls with traditional browsing methods, dynamically choosing the optimal task completion strategy.
Hybrid agents provide flexible solutions for sites with limited API support.
WebArena
WebArena is a benchmark platform for evaluating web agent performance across multiple real-world websites.
Experiments in this paper were conducted on WebArena to evaluate API and Hybrid Agents.
Success Rate
Success rate measures an agent's accuracy and efficiency in completing tasks.
In this paper, API and Hybrid Agents achieved success rates of 29.2% and 38.9%, respectively.
Open Questions Unanswered questions from this research
- 1 How to improve agent performance on sites with limited API support?
- 2 How will automatic API documentation generation technology develop?
Applications
Immediate Applications
Online Shopping
On shopping sites with API support, API agents can quickly complete shopping tasks, improving efficiency.
Long-term Vision
Fully Automated Web Interaction
In the future, Hybrid Agents can be applied in more scenarios, achieving more efficient web task automation.
Abstract
Web browsers are a portal to the internet, where much of human activity is undertaken. Thus, there has been significant research work in AI agents that interact with the internet through web browsing. However, there is also another interface designed specifically for machine interaction with online content: application programming interfaces (APIs). In this paper we ask -- what if we were to take tasks traditionally tackled by Browsing Agents, and give AI agents access to APIs? To do so, we propose two varieties of agents: (1) an API-calling agent that attempts to perform online tasks through APIs only, similar to traditional coding agents, and (2) a Hybrid Agent that can interact with online data through both web browsing and APIs. In experiments on WebArena, a widely-used and realistic benchmark for web navigation tasks, we find that API-Based Agents outperform web Browsing Agents. Hybrid Agents out-perform both others nearly uniformly across tasks, resulting in a more than 24.0% absolute improvement over web browsing alone, achieving a success rate of 38.9%, the SOTA performance among task-agnostic agents. These results strongly suggest that when APIs are available, they present an attractive alternative to relying on web browsing alone.