MAO-ARAG: Multi-Agent Orchestration for Adaptive Retrieval-Augmented Generation

The evolution of large language models (LLMs) such as GPT and BERT has revolutionized question answering (QA) and information retrieval. Early methods like BM25 and dense retrievers (Contriever, ColBERT) laid the foundation for integrating external knowledge. Modular RAG architectures emerged to combine retrieval and generation, with components like query rewriting, document selection, and self-reflection. Despite progress, fixed workflows limit adaptability, especially as query complexity varies. Recent efforts incorporate reinforcement learning to optimize process sequences, but challenges remain in balancing accuracy, cost, and scalability across diverse scenarios.

Existing RAG systems rely on static, hand-designed workflows that cannot adapt dynamically to different query complexities. Simple questions are over-resourced, while complex ones may lack sufficient processing, leading to inefficiency and suboptimal performance. Furthermore, balancing answer quality with resource consumption, such as token usage and latency, remains difficult. The core challenge is designing a flexible, trainable system capable of selecting and orchestrating modules in real-time, tailored to each question’s demands. This involves modeling the process as a multi-objective optimization problem with multiple constraints and uncertainties.

The paper introduces a multi-agent framework based on MSMDP, where a trainable planner dynamically orchestrates modules (executors) like query reformulation, retrieval, and answer generation. Key innovations include:

1) Modeling QA as an MSMDP, capturing variable action durations and multi-agent coordination.

2) Using PPO reinforcement learning to optimize a multi-objective reward balancing answer accuracy and costs.

3) Designing a flexible, modular workflow system that adapts per query, outperforming fixed pipelines.

This approach enables real-time, personalized workflow construction, significantly improving efficiency and accuracy.

• �� Model QA as an MSMDP with states representing question features, actions as module selections, and rewards based on answer quality and costs.
• �� The planner receives observations (question prompts) and outputs a sequence of modules (actions) to execute.
• �� The reward function combines F1 score, token cost, latency penalty, and format penalty, scaled between 0 and 1.
• �� The training employs PPO, updating the policy via advantage estimation, with the loss combining actor and critic components.
• �� During training, the system interacts over multiple turns, refining its scheduling policy based on feedback.
• �� The final system can adaptively generate workflows for diverse questions, optimizing overall QA performance.

Experiments involve seven QA datasets (NQ, PopQA, HotpotQA, etc.), using Wikipedia as the knowledge base and E5 for retrieval. The models include Qwen2.5-7B-Instruct as the planner and GPT-4o-Mini as executor. Evaluation metrics cover F1 scores, token costs, retrieval calls, and workflow turns. Baselines include fixed-process RAG, iterative RAG, and RL-based methods. Results show MAO-ARAG achieves an average F1 of 52.91, outperforming baselines by over 3 points, with lower resource consumption. Ablation studies confirm the importance of adaptive scheduling and reward design.

The adaptive framework significantly improves QA accuracy, with the trained model surpassing static methods. The F1 score increases by 3.08 points over the best baseline, while costs such as token usage and retrieval calls remain controlled. Hyperparameter tuning of α demonstrates a flexible trade-off between performance and efficiency. Even untrained models effectively organize workflows, indicating robustness. These results validate the effectiveness of dynamic, reinforcement-learned scheduling in complex QA tasks.

This framework can be deployed in enterprise customer service, academic research, and knowledge management, where question complexity varies widely. Its ability to adapt workflows reduces costs and improves response accuracy, making AI assistants more intelligent and resource-efficient. Future integration with multi-modal data and knowledge graphs will further expand its real-world impact, enabling scalable, personalized knowledge services across industries.

The approach depends heavily on high-quality training data and computational resources, limiting scalability in resource-constrained environments. Its effectiveness on unstructured or highly novel questions remains to be validated. The training process is computationally intensive, and the current architecture is primarily designed for structured QA, requiring further adaptation for open-ended tasks. Future work should focus on improving generalization, reducing training costs, and extending applicability to broader domains.