Are Online Skill and Memory Modules Always Worth Their Tokens? A Budget-Constrained Study of Web Agents
Under fixed inference budgets, vanilla actors often outperform or match augmented methods in web agent tasks, with multi-run variance highlighting evaluation challenges.
Key Findings
Methodology
This study compares three online augmentation strategies—AWM, ASI, and ReasoningBank—against a budget-matched baseline (Vanilla-IB) across WebArena and WorkArena benchmarks. Using three models (Gemini 3 Flash, gpt-4o, Qwen 3.6-27B), the experiments control total token usage by adjusting actor step limits (e.g., 15 steps). Multiple independent runs (≥3) per task quantify success rates and standard deviations, emphasizing stochastic variability. The core algorithms involve LLM-based workflow induction, skill synthesis, and reasoning memory retrieval, all evaluated under strict token constraints to assess cost-benefit trade-offs. Detailed token accounting distinguishes between environment interaction tokens and auxiliary module tokens, revealing cost structures. The approach rigorously tests whether external modules justify their token costs in fixed-budget scenarios.
Key Results
- Across WebArena domains, vanilla baseline (Vanilla-IB) achieves the highest or comparable success rates with fewer tokens. For example, Gemini 3 Flash's Vanilla-IB success rate is 47.77%, using approximately 45.7K tokens, outperforming augmented methods that often consume more tokens without proportional success gains.
- In enterprise tasks (WorkArena-L1), Vanilla-IB's success rate is 55.56%, on par with ReasoningBank, but with lower total token consumption, demonstrating competitiveness in real-world scenarios.
- Multiple runs reveal high task-level variance; success in one run does not guarantee success in another, highlighting the importance of multi-run evaluation for reliable performance measurement.
Significance
This research questions the assumed benefits of external skills and memory modules in web agents under budget constraints. Results demonstrate that simple, extended interaction strategies often outperform complex augmentation, especially as models become more capable. The findings urge a reevaluation of cost-benefit assumptions in designing efficient, scalable web agents. Additionally, emphasizing multi-run variance introduces a more rigorous standard for agent evaluation, aligning performance metrics with real-world stochastic environments. The work contributes to a more nuanced understanding of resource allocation, encouraging the development of leaner, more effective web automation systems.
Technical Contribution
The paper introduces a systematic, budget-aware comparison framework for online web agents, integrating multi-run statistical analysis to account for stochastic variability. It dissects token consumption into environment interaction and auxiliary module costs, revealing that most overhead stems from explicit module calls and context inflation. The study demonstrates that a simple, extended-horizon vanilla actor can match or surpass the performance of more complex, module-based systems within the same token budget, challenging existing assumptions about augmentation benefits. This approach provides a new benchmark for evaluating cost-effectiveness in web agent design, emphasizing the importance of multi-run reliability metrics.
Novelty
This is the first comprehensive comparison of online augmentation methods against a budget-matched baseline across multiple models and real-world web tasks. It highlights the critical role of multi-run variance in performance evaluation, a factor often overlooked in prior studies. The research shifts focus from single-run success metrics to a more robust, stochastic assessment, providing new insights into cost-performance trade-offs. Its innovative framework for token accounting and multi-run analysis sets a new standard for future evaluations of web agents and reinforcement learning systems operating under resource constraints.
Limitations
- The primary limitation is reliance on actor step count as a proxy for token budget, which may not precisely match total token consumption across different methods, potentially biasing comparisons.
- The experiments focus on specific models and environments; broader validation across more complex, dynamic, or multi-modal scenarios is needed to generalize conclusions.
- While multi-run variance is emphasized, environmental stochasticity and web page variability are not deeply analyzed, which could influence success rate fluctuations.
Future Work
Future research should develop adaptive budget management strategies, possibly leveraging reinforcement learning to dynamically allocate tokens between environment interaction and auxiliary modules. Extending evaluations to more diverse, complex environments and larger models will test the robustness of current findings. Incorporating environment stochasticity analysis and multi-modal data can further refine performance assessment. Additionally, exploring hybrid architectures that combine simple interaction strategies with selective augmentation may yield optimal cost-performance balances for real-world deployment.
AI Executive Summary
The rapid advancement of large language models (LLMs) has driven significant interest in developing web agents capable of complex multi-step interactions. Traditionally, augmenting these agents with external skills, memory modules, or workflows was believed to enhance performance, especially on challenging tasks. However, these modules incur token costs, which are often overlooked in performance evaluations. This study critically examines whether such augmentation strategies are justified under fixed inference budgets.
Using the WebArena and WorkArena benchmarks, the authors compare three online augmentation methods—AWM, ASI, and ReasoningBank—against a simple, budget-matched baseline called Vanilla-IB. The core idea is to control total token consumption by adjusting the maximum number of interaction steps, ensuring a fair comparison. Experiments across three models reveal that Vanilla-IB, which extends the interaction horizon without external modules, often outperforms or matches the augmented methods in success rate while using fewer tokens.
A key insight from the study is the high variability in success rates across multiple runs, driven by stochasticity in web environments and model outputs. This highlights the importance of multi-run evaluation, as single-run metrics can be misleading. The findings suggest that, especially as models improve, the marginal gains from external skills diminish, and simple, prolonged interaction can be more cost-effective.
Overall, the research challenges the conventional wisdom that external modules always justify their token costs. It advocates for a more nuanced, budget-aware approach to web agent design, emphasizing the importance of multi-run reliability and cost-efficiency. These insights have broad implications for deploying scalable, resource-conscious AI systems in real-world web automation and enterprise workflows. Despite some limitations in budget control precision and environment scope, the study sets a new standard for rigorous, stochastic evaluation of web agents, guiding future development towards leaner, more effective architectures.
Deep Analysis
Background
Web智能体在自动化任务中的应用不断扩大，早期研究主要集中在利用大规模预训练模型（如GPT-3、GPT-4）实现端到端任务完成。随着任务复杂度增加，研究引入技能、记忆和工作流模块（如Wang et al., 2025a,b; Ouyang et al., 2026）以提升性能。这些方法通过外部知识存储和流程引导，试图弥补模型在长序列推理中的不足。然而，模块调用和上下文膨胀带来的代币消耗未被充分量化，导致成本-收益分析不完整。近年来，WebArena和Mind2Web等基准推动了多步交互环境的建立，为评估Web智能体提供了更真实的场景。模型能力的快速提升（如GPT-5.4、Gemini 3）也使得外部模块的必要性受到质疑，促使学界重新审视增强策略的有效性。
Core Problem
在有限推理预算下，是否应优先投入技能和记忆模块，还是通过增加交互次数实现性能提升？传统观点认为外部模块能显著改善任务成功率，但其代币成本未被充分考虑。实际应用中，模型能力不断增强，外部模块的边际收益逐渐递减，甚至可能被成本所抵消。此外，环境随机性和多任务场景增加了评估难度，单次成功率难以反映真实性能。如何在预算限制下科学衡量不同策略的效果，成为关键问题。
Innovation
本研究提出了基于预算匹配的比较框架，系统评估多种增强策略在不同模型和任务中的表现差异。引入多次随机运行统计，揭示了单次成功率的局限性。通过分析模块调用的代币结构，发现大部分成本来自显式调用和上下文膨胀，而非模型本身能力提升。这一方法创新性地结合了成本控制与性能评估，为Web智能体的设计提供了新的理论基础。
Methodology
- �� 设计多任务、多模型环境（WebArena和WorkArena）进行评估。
- �� 比较三种增强方法（AWM、ASI、ReasoningBank）与匹配预算的基础Actor（Vanilla-IB）。
- �� 控制Actor最大步数（如15步）作为预算控制手段，确保总代币消耗相近。
- �� 在每个任务中多次随机运行（至少三次），统计成功率和标准差。
- �� 详细分析模块调用的代币结构（包括提示和调用成本）与环境交互的代币消耗。
- �� 通过扩展Actor步数（如Reddit中的25步）验证预算和性能关系。
- �� 比较不同模型（Gemini 3、gpt-4o、Qwen 3.6-27B）上的表现差异。
- �� 重点关注多次运行的变异性和任务级别的随机性对评估的影响。
Experiments
采用WebArena和WorkArena两个环境，涵盖购物、Reddit、管理后台和企业软件任务。模型包括Gemini 3 Flash、gpt-4o和Qwen 3.6-27B。每个配置进行三次独立随机运行，统计成功率和总代币消耗。比较方法包括增强策略（AWM、ASI、ReasoningBank）与基础Actor（Vanilla-IB），后者通过延长Actor步数（如15步）实现预算匹配。实验中还分析了模块调用的具体代币结构，验证了成本主要来自显式调用和上下文膨胀。
Results
Vanilla-IB在WebArena中成功率最高，且总代币消耗最低。例如，Gemini 3模型中，成功率为47.77%，总代币约45.7K，而增强方法虽有提升，但成本更高。在WorkArena-L1中，Vanilla-IB成功率55.56%，与ReasoningBank持平，且用币更少。多次运行显示，单次成功率存在显著波动，强调多次统计的重要性。扩展Actor步数（如Reddit中的25步）后，Vanilla-IB表现优于增强方法，验证了预算和交互次数的关系。
Applications
该研究为Web智能体设计提供了成本效益分析工具，适用于需要在预算限制下优化性能的场景，如企业自动化、客户服务和内容管理。通过合理配置Actor步数，减少不必要的模块调用，提升系统整体效率。未来，结合强化学习优化预算分配，将进一步提升Web Agent在复杂环境中的表现。
Limitations & Outlook
实验主要依赖Actor最大步数作为预算控制手段，未能完全匹配总代币数，可能影响结果的绝对比较。多次运行虽揭示随机性，但未深入分析环境不确定性对不同方法的影响。模型和任务范围有限，未来需扩展到更复杂环境和更大模型，以验证结论的普适性。
Plain Language Accessible to non-experts
想象你在厨房做饭，手边有各种调料和工具。有些厨师喜欢提前准备好所有调料（就像技能和记忆模块），这样可以快速做出菜肴，但每次准备都要花时间和材料（代币）。而另一些厨师则喜欢边做边试（增加交互次数），用有限的材料不断尝试，最终也能做出美味菜肴。研究发现，在预算有限的情况下，直接多试几次（多交互）往往比提前准备好所有调料更省钱又有效。就像厨房里，简单的厨师用有限材料多试几次，反而比那些提前准备好所有调料的厨师更快做出好菜。这说明，复杂的外部工具未必总是值得花费更多资源，直接多尝试可能更划算。
ELI14 Explained like you're 14
想象你在玩一个游戏，你可以用点数（像代币）来买装备或者多试几次。有人说，花点点买装备（技能和记忆）可以帮你更快赢，但其实在点数有限的情况下，多试几次（多交互）反而更有效。比如，你用点数多试几次，可能比用点数买很多装备还更容易赢。这个研究就告诉我们，在有限的点数里，直接多试几次，可能比买很多装备更聪明。它还发现，很多时候，单次试几次的成功率会有很大波动，所以要多试几次才能知道自己到底行不行。就像你打游戏，偶尔会遇到运气不好，但多玩几次就能看出自己真正的水平。这个发现对设计智能机器人也很重要：不要总想着用复杂的工具，简单多试试，反而更划算！
Abstract
Online web agents often augment a base actor with memory, workflow, or skill modules. These modules can improve performance, but they also consume test-time tokens, a cost rarely reported alongside the actor's inference cost. We study online augmentation, where this overhead is paid on every task, and re-evaluate its benefits under a fixed total inference budget. We compare AWM, ASI, and ReasoningBank with a token-matched vanilla baseline that uses the same budget for additional actor steps. Across four WebArena domains and three models, Gemini 3 Flash, GPT-5.4-mini, and Qwen 3.6-27B, the vanilla baseline matches or surpasses all three augmentation methods in aggregate success rate while often using fewer total tokens. We observe a similar trend on WorkArena-L1 with Qwen 3.6-27B, indicating that the effect extends to enterprise knowledge-work tasks. Our results suggest that skills and workflow memory can be useful in specific domains, but their apparent gains often vanish against a budget-matched actor. We further show that run-to-run variance materially affects outcomes and should be reported as a core evaluation criterion for online web agents.