Efficient and Interpretable Multi-Agent LLM Routing via Ant Colony Optimization
AMRO-S uses Ant Colony Optimization for efficient, interpretable multi-agent LLM routing, achieving 4.7x speedup.
Key Findings
Methodology
AMRO-S models multi-agent system routing as a semantic-conditioned path selection problem. It uses a supervised fine-tuned small language model for intent inference, decomposes routing memory into task-specific pheromone specialists, and employs a quality-gated asynchronous update mechanism. This framework leverages the biological logic of Ant Colony Optimization, providing a low-overhead semantic interface to optimize path selection under mixed workloads.
Key Results
- AMRO-S achieved an average improvement of 1.90 points over the strongest multi-agent routing baseline and up to 4.7x speedup under 1000 concurrent processes, maintaining stable latency.
- AMRO-S outperformed existing strong routing baselines across MMLU, GSM8K, MATH, HumanEval, and MBPP datasets.
- Structured pheromone patterns provide traceable routing evidence, supporting transparent diagnosis and continual optimization.
Significance
AMRO-S is significant in both academia and industry, addressing high inference costs, latency, and limited transparency in multi-agent systems, especially under high concurrency. By combining small language models and ant colony optimization, AMRO-S significantly reduces inference costs while maintaining high accuracy, providing semantically meaningful routing evidence for high-stakes applications.
Technical Contribution
AMRO-S differs significantly from existing methods by introducing task-specific pheromone specialists and query-conditioned fusion to reduce cross-task interference, and a quality-gated asynchronous update mechanism for controllable online optimization. These innovations offer new theoretical guarantees and engineering possibilities, especially under strict serving constraints.
Novelty
AMRO-S is the first to combine ant colony optimization with large language models for semantic routing in multi-agent systems. Compared to existing methods, it achieves more efficient and interpretable routing strategies through task decomposition and asynchronous update mechanisms.
Limitations
- AMRO-S may experience performance degradation under extreme high-load conditions, as the asynchronous update mechanism may not adapt quickly to rapid system dynamics.
- Training costs can be high when large-scale annotated data is required.
- Further fine-tuning may be needed to optimize performance on specific tasks.
Future Work
Future research directions include further optimizing pheromone update strategies to improve performance under extreme high-load conditions. Additionally, exploring applications in different domains such as healthcare and finance to validate its generality and adaptability.
AI Executive Summary
Multi-Agent Systems (MAS) excel in complex reasoning and tool use but are often limited by high inference costs, latency, and transparency issues in real-world deployment. Existing routing strategies typically rely on expensive LLM selectors or static policies, making semantic-aware routing under dynamic loads and mixed intents challenging. To address these issues, this paper introduces AMRO-S, an efficient and interpretable MAS routing framework. AMRO-S models MAS routing as a semantic-conditioned path selection problem, leveraging a supervised fine-tuned small language model for intent inference and employing a quality-gated asynchronous update mechanism to optimize path selection without increasing latency. Experimental results demonstrate that AMRO-S consistently improves the quality-cost trade-off over strong routing baselines across five public benchmarks and high-concurrency stress tests, while providing traceable routing evidence through structured pheromone patterns. The introduction of AMRO-S offers new possibilities for efficient routing in multi-agent systems, particularly in high-stakes applications such as healthcare and finance. However, further optimization is needed for performance under extreme high-load conditions, and future research can explore more advanced pheromone update strategies and applications in different domains.
Deep Analysis
Background
In recent years, large language models (LLMs) have made significant advances in natural language understanding, multi-step reasoning, and code generation, driving the rapid development of LLM-driven Multi-Agent Systems (MAS). MAS are viewed as distributed systems composed of multiple LLM-driven agents that communicate, collaborate, and coordinate to accomplish complex tasks. However, as MAS scale and task distributions diversify, MAS routing has emerged as a key bottleneck in dynamic and resource-constrained environments.
Core Problem
MAS routing requires selecting an appropriate execution path from a heterogeneous agent pool while balancing output quality and serving overhead, including latency, token usage, and load. Under high-concurrency and low-latency constraints, existing routing strategies often lead to limited throughput, degraded latency, and escalating costs.
Innovation
AMRO-S introduces three core innovations by modeling MAS routing as a semantic-conditioned path selection problem: 1) leveraging a supervised fine-tuned small language model for intent inference, providing a low-overhead semantic interface; 2) decomposing routing memory into task-specific pheromone specialists to reduce cross-task interference; 3) employing a quality-gated asynchronous update mechanism to optimize routing without increasing latency.
Methodology
- �� Use a small language model for intent inference, providing a semantic interface.
- �� Decompose routing memory into task-specific pheromone specialists to reduce interference.
- �� Employ a quality-gated asynchronous update mechanism to optimize routing performance.
Experiments
Experiments were conducted on five public benchmarks and high-concurrency stress tests to validate the effectiveness of AMRO-S. The benchmarks include GSM8K, MMLU, MATH, HumanEval, and MBPP, covering mathematical reasoning, domain-specific knowledge, code generation, and problem-solving capabilities.
Results
AMRO-S outperformed existing strong routing baselines across all benchmarks, achieving up to 4.7x speedup under high concurrency and maintaining stable latency under 1000 concurrent processes.
Applications
AMRO-S can be applied to multi-agent systems requiring efficient routing, such as automated programming, mathematical reasoning, and collaborative decision-making, particularly in high-stakes applications like healthcare and finance.
Limitations & Outlook
While AMRO-S performs well in many aspects, further optimization is needed for performance under extreme high-load conditions. Additionally, training costs can be high, especially when large-scale annotated data is required.
Plain Language Accessible to non-experts
Imagine a large factory with many different machines, each specializing in a particular task. Our goal is to get each product through the most suitable combination of machines in the shortest time. Traditional methods might send each product through all machines, which is simple but inefficient. AMRO-S acts like a smart scheduler, choosing the best machine path for each product based on its characteristics, saving time and resources. It uses a method similar to how ants find food, marking and optimizing the best paths with pheromones. This way, the factory's production efficiency is greatly improved, and each product is completed in the shortest time possible.
ELI14 Explained like you're 14
Imagine you're playing a big online multiplayer game, and you and your friends need to complete a mission together. Everyone has different skills; some are great at fighting, others at solving puzzles. To finish the mission quickly, you need to assign roles wisely. AMRO-S is like a smart team leader, quickly figuring out what skills are needed for each task and assigning the best person for the job. This way, your team can complete the mission in no time, and everyone gets to shine with their skills. Isn't that cool?
Glossary
Ant Colony Optimization
An optimization algorithm inspired by the foraging behavior of ants, using pheromones to guide path selection.
Used to optimize routing strategies in multi-agent systems.
Large Language Model
A large-scale neural network model capable of understanding and generating natural language.
Core technology driving multi-agent systems.
Multi-Agent System
A distributed system composed of multiple agents that collaborate to complete complex tasks.
Application scenario for AMRO-S.
Semantic Routing
A routing strategy that selects paths based on task semantics.
One of the core innovations of AMRO-S.
Pheromone Specialist
A matrix used to store and optimize path selection for specific tasks.
Key mechanism to reduce cross-task interference.
Open Questions Unanswered questions from this research
- 1 How can AMRO-S's performance be further optimized under extreme high-load conditions?
- 2 What is the generality and adaptability of AMRO-S in different domains?
- 3 How can the training cost of AMRO-S be reduced, especially when large-scale annotated data is required?
Applications
Immediate Applications
Automated Programming
Improves programming efficiency by optimizing code generation paths, suitable for software development companies.
Mathematical Reasoning
Applied in education to help students solve complex math problems faster.
Long-term Vision
Medical Diagnosis
Enhances medical efficiency and accuracy by optimizing diagnostic paths, with data privacy issues to be addressed.
Abstract
Large Language Model (LLM)-driven Multi-Agent Systems (MAS) have demonstrated strong capability in complex reasoning and tool use, and heterogeneous agent pools further broaden the quality--cost trade-off space. Despite these advances, real-world deployment is often constrained by high inference cost, latency, and limited transparency, which hinders scalable and efficient routing. Existing routing strategies typically rely on expensive LLM-based selectors or static policies, and offer limited controllability for semantic-aware routing under dynamic loads and mixed intents, often resulting in unstable performance and inefficient resource utilization. To address these limitations, we propose AMRO-S, an efficient and interpretable routing framework for Multi-Agent Systems (MAS). AMRO-S models MAS routing as a semantic-conditioned path selection problem, enhancing routing performance through three key mechanisms: First, it leverages a supervised fine-tuned (SFT) small language model for intent inference, providing a low-overhead semantic interface for each query; second, it decomposes routing memory into task-specific pheromone specialists, reducing cross-task interference and optimizing path selection under mixed workloads; finally, it employs a quality-gated asynchronous update mechanism to decouple inference from learning, optimizing routing without increasing latency. Extensive experiments on five public benchmarks and high-concurrency stress tests demonstrate that AMRO-S consistently improves the quality--cost trade-off over strong routing baselines, while providing traceable routing evidence through structured pheromone patterns.