QueenBee Planner: Skill-Evolving Communication Topologies for Token-Efficient LLM Multi-Agent Systems
QueenBee Planner self-evolves communication topologies using evidence-driven rules, reducing RMSE from 12.53 to 7.87 and communication costs in multi-agent LLM systems.
Key Findings
Methodology
This approach models inter-agent communication topology as a learnable skill, with a frozen pool of worker agents, a task adapter, and a scoring function. The outer LLM planner generates temporal DAGs that specify message passing sequences, merging, and output nodes. It distills execution traces into evidence-backed rules with three actions: Preserve, Modify, Avoid. To ensure robustness, it employs mechanisms like held-out acceptance gates, variance-aware credit, motif-level attribution, transfer trust, insight falsification, and structural deduplication. Experiments on Count-Frequency and Silo-Bench tasks show that self-evolved graphs outperform fixed topologies, reducing RMSE from 12.53 to 7.87, while decreasing message exchanges, model calls, and token costs.
Key Results
- In the Count-Frequency full test, the best generated communication graph reduced RMSE from 12.53 (fixed topology) to 7.87, with fewer messages, model calls, and tokens. Similar improvements were observed in Silo-style tasks, confirming the effectiveness of learned communication structures. The system's ability to learn reusable architectural knowledge rather than mere task solutions demonstrates a significant step forward in multi-agent system design.
Significance
This work addresses the static nature of existing multi-agent communication topologies, introducing a self-improving mechanism that learns from experience. It enables systems to autonomously optimize their organizational structure, leading to more accurate, cost-effective, and robust multi-agent interactions. Such adaptive architecture learning is crucial for scaling multi-agent systems in complex, real-world scenarios, bridging the gap between fixed design and dynamic adaptation, and paving the way for more intelligent distributed AI systems.
Technical Contribution
The key innovation lies in treating communication topology as a retrievable skill, distilled from execution evidence, and continuously refined via evidence-backed rules. The method employs a time-unrolled DAG generation process, structural motif-level credit assignment, and rigorous validation gates to prevent noise-driven drift. Unlike static optimization, this approach emphasizes knowledge reuse and transferability, enabling the system to improve its organizational design over time, thus offering a new paradigm for multi-agent architecture learning.
Novelty
This study is the first to formalize multi-agent communication topology as a self-evolving design skill, integrating evidence-driven rule distillation with structural motif crediting. It moves beyond static or hand-crafted topologies, demonstrating that a system can learn, validate, and transfer architectural knowledge across tasks, significantly advancing the field of adaptive multi-agent system design.
Limitations
- The current framework relies on tasks with explicit, checkable answers, limiting applicability to open-ended or ambiguous problems. The self-evolution process can be sensitive to noisy or high-variance signals, potentially leading to suboptimal updates. Computational overhead remains significant, requiring further optimization for large-scale deployment.
Future Work
Future research will focus on integrating reinforcement learning and meta-learning techniques to enhance adaptive capabilities, extending the framework to more complex, multi-modal, and real-world tasks. Efforts will also be made to improve computational efficiency and robustness, enabling broader applicability in large-scale distributed AI systems.
AI Executive Summary
In recent years, the development of large language models (LLMs) has revolutionized AI, especially in multi-agent systems where multiple models or agents collaborate to solve complex tasks. Traditionally, the focus has been on improving individual reasoning capabilities through prompt engineering or reasoning chains. However, the organization and communication among agents—namely, the topology—have emerged as critical factors influencing overall system performance. Static, hand-designed communication structures often limit adaptability and scalability. Recognizing this, the paper introduces QueenBee Planner, a novel framework that treats communication topology as a self-improving, retrievable skill.
The core idea is to freeze the worker agents and task adapter, while an outer LLM-based planner learns to generate temporal communication DAGs that specify who communicates with whom, in which round, and how messages are merged or finalized. This generator is conditioned on a skill bank distilled from prior execution traces, which contain evidence-backed rules categorized as Preserve, Modify, or Avoid. These rules guide the planner to produce more effective communication structures over time, with mechanisms like held-out acceptance gates, variance-aware credit, motif-level attribution, transfer trust, insight falsification, and structural deduplication ensuring robustness against noise and false signals.
Experimental results on Count-Frequency and Silo-Bench tasks demonstrate that self-evolved communication graphs outperform fixed topologies, reducing RMSE from 12.53 to 7.87 in the full test setting. Additionally, the learned structures lead to fewer messages, model calls, and token costs, confirming improved efficiency. These findings suggest that the system not only solves specific tasks but also learns reusable architectural design knowledge, marking a significant step toward autonomous, adaptive multi-agent AI systems.
This approach opens new avenues for scalable, robust distributed AI, where communication strategies are not static but evolve with experience. It addresses longstanding challenges in multi-agent coordination, offering a framework that balances exploration and validation to prevent noise-driven drift. The potential applications span distributed data processing, multi-robot coordination, and large-scale AI orchestration, promising more intelligent, resource-efficient systems in the future. Future work will aim to enhance robustness, extend to more complex scenarios, and integrate reinforcement learning for even greater adaptability.
Deep Dive
Plain Language Accessible to non-experts
想象一个厨房里有很多厨师，每个人负责不同的菜肴。以前，厨师们都按照固定的流程合作，比如每个人都知道自己该做什么，合作方式一成不变。这种方式虽然简单，但遇到新菜或者变化时就不灵活。现在，有个聪明的厨师长，他会观察每次合作的效果，记住哪些合作方式做得好，哪些不好，然后自己设计出更聪明的合作方案。每次做完菜后，他都会总结经验，遇到类似的任务就用以前学到的好方法，避免犯错。随着时间推移，他变得越来越聪明，能帮厨师们安排出最合适的合作方式，让厨房的效率和菜肴质量都提升。这就像一个不断学习、不断改进的厨房管理系统，能让整个厨房变得更快、更好吃。
ELI14 Explained like you're 14
想象你在学校里有一群朋友，每个人负责不同的任务。有时候，你们会合作完成一个大项目。以前，你们的合作方式都是固定的，比如每个人都知道自己该做什么，大家按照固定的流程来合作。可是，有时候这种固定的方式不太好用，遇到新任务就会出错。现在，想象有个聪明的哥哥，他会观察你们的合作，发现哪些方法效果好，哪些不好，然后自己设计出更聪明的合作方式。每次完成任务后，他都会记下来，遇到类似的任务就用以前学到的好方法，避免出错。慢慢地，他变得越来越聪明，能帮你们设计出最适合的合作流程，让你们的任务都能更快更好地完成。这就像一个会自己学习、不断改进的团队领导，帮大家变得更厉害！
Abstract
Large language model (LLM) multi-agent systems increasingly depend not only on how individual agents reason, but also on how agents are connected. This paper introduces QueenBee Planner, a framework that treats inter-agent communication topology as a retrievable and self-improving design skill. A pool of worker agents, the task adapter, and the scoring function are frozen; only an outer LLM planner learns to generate temporal communication DAGs specifying who sends information to whom, in which round, who merges messages, and who emits the final answer. Execution traces are distilled into evidence-backed design rules with three actions: \emph{Preserve}, \emph{Modify}, and \emph{Avoid}. To prevent self-evolution from turning lucky runs or plausible but false explanations into policy, QueenBee uses held-out acceptance gates, variance-aware credit, motif-level attribution, transfer trust, insight falsification, and structural deduplication. We evaluate the method on Count-Frequency aggregation and Silo-Bench-style distributed coordination tasks. With fixed workers, self-evolved graph generation produces communication structures that improve over fixed topologies and cold generation. In the CF fulltest setting, the best generated graph reduces RMSE from 12.53 for the strongest fixed topology to 7.87 while also reducing messages, model calls, and token cost; Silo-style results show the same direction of improvement over cold and fixed-topology baselines. These results suggest that multi-agent systems can learn reusable architectural design knowledge rather than merely memorizing task answers.