HASHIRU: Hierarchical Agent System for Hybrid Intelligent Resource Utilization
HASHIRU enhances resource utilization via CEO-agent hierarchy, achieving 96% on GSM8K.
Key Findings
Methodology
HASHIRU employs a CEO-agent hierarchical structure where the CEO manages task allocation and resource management, dynamically creating employee agents based on task needs. Its hybrid intelligence prioritizes smaller local LLMs while flexibly using external APIs when necessary. An economic model with hiring/firing costs promotes team stability.
Key Results
- On the GSM8K dataset, HASHIRU achieved 96% in complex reasoning, significantly outperforming Gemini 2.0 Flash's 61%.
- On JEEBench, HASHIRU achieved 80% accuracy, surpassing Gemini 2.0 Flash's 68.3%.
- On the SVAMP dataset, HASHIRU scored 92%, compared to Gemini 2.0 Flash's 84%.
Significance
HASHIRU offers a robust, efficient, and adaptable MAS through dynamic hierarchical control and resource-aware hybrid intelligence. It addresses key limitations in flexibility, resource awareness, and model diversity, impacting both academia and industry.
Technical Contribution
HASHIRU introduces dynamic management and an economic model in MAS, prioritizing local LLMs and reducing reliance on external APIs, offering new engineering possibilities and theoretical guarantees.
Novelty
HASHIRU is the first to apply a CEO-agent hierarchical structure in MAS, combining economic models and hybrid intelligence strategies to significantly enhance system flexibility and resource efficiency.
Limitations
- In high-resource tasks, HASHIRU may face limitations in memory and computational resources.
- The system's dynamic nature may lead to increased response times in certain scenarios.
Future Work
Future work includes optimizing resource management strategies, enhancing performance in large-scale tasks, and exploring more automated tool generation methods.
AI Executive Summary
HASHIRU is a novel multi-agent system framework designed to address the lack of flexibility and resource awareness in existing frameworks. By introducing a CEO-agent hierarchical structure, HASHIRU dynamically manages tasks and resources, enhancing system adaptability and efficiency.
The core technologies include a hybrid intelligence strategy that prioritizes smaller local LLMs while flexibly utilizing external APIs when necessary. An economic model with hiring and firing costs promotes team stability and efficient resource allocation.
Experimental results show that HASHIRU excels in various benchmarks, notably outperforming existing methods in complex reasoning tasks. Its dynamic hierarchical control and resource-aware hybrid intelligence offer new directions for the development of multi-agent systems.
Deep Dive
Abstract
Rapid Large Language Model (LLM) advancements are fueling autonomous Multi-Agent System (MAS) development. However, current frameworks often lack flexibility, resource awareness, model diversity, and autonomous tool creation. This paper introduces HASHIRU (Hierarchical Agent System for Hybrid Intelligent Resource Utilization), a novel MAS framework enhancing flexibility, resource efficiency, and adaptability. HASHIRU features a "CEO" agent dynamically managing specialized "employee" agents, instantiated based on task needs and resource constraints (cost, memory). Its hybrid intelligence prioritizes smaller, local LLMs (via Ollama) while flexibly using external APIs and larger models when necessary. An economic model with hiring/firing costs promotes team stability and efficient resource allocation. The system also includes autonomous API tool creation and a memory function. Evaluations on tasks like academic paper review (58% success), safety assessments (100% on a JailbreakBench subset), and complex reasoning (outperforming Gemini 2.0 Flash on GSM8K: 96% vs. 61%; JEEBench: 80% vs. 68.3%; SVAMP: 92% vs. 84%) demonstrate HASHIRU's capabilities. Case studies illustrate its self-improvement via autonomous cost model generation, tool integration, and budget management. HASHIRU offers a promising approach for more robust, efficient, and adaptable MAS through dynamic hierarchical control, resource-aware hybrid intelligence, and autonomous functional extension. Source code and benchmarks are available at https://github.com/HASHIRU-AI/HASHIRU and https://github.com/HASHIRU-AI/HASHIRUBench respectively, and a live demo is available at https://hashiruagentx-hashiruai.hf.space upon request.