Multi-agent Application System in Office Collaboration Scenarios
Proposes a Plan+Solver multi-agent system with multi-turn query rewriting, achieving 95% query understanding accuracy and 92% task success in office scenarios.
Key Findings
Methodology
The system employs a decoupled Plan and Solver architecture, enhanced by multi-turn query rewriting and business tool retrieval. It integrates deep fine-tuning of Qwen2.5 models for multi-task learning, with modules for intent recognition, context management, and multi-tool invocation. Data is generated via a multi-agent workflow framework, covering 100K samples across office scenarios. The architecture supports multi-intent, multi-step tasks, with a master node coordinating specialized worker agents. Key algorithms include multi-turn query rewriting, context-aware intent classification, and dynamic tool retrieval, ensuring high accuracy and robustness in complex interactions.
Key Results
- In real enterprise environments, the system achieved over 95% accuracy in query understanding and 92% success in task completion, outperforming rule-based systems. Multi-turn dialogue experiments showed intent recognition accuracy of 97% and tool invocation success of 94%. Response times improved by 20%, with a 15% efficiency gain over traditional methods. The multi-agent framework effectively handled multi-step, multi-intent tasks, demonstrating scalability and stability across scenarios like email, scheduling, and meetings.
- Ablation studies indicated that multi-turn query rewriting improved understanding accuracy by 8%, while multi-tool retrieval increased invocation success by 7%. The separation of planning and execution modules reduced error propagation, leading to a 3% lower error rate. Cross-scenario tests confirmed the system’s adaptability, with performance metrics remaining stable across different enterprise settings. The results validate the architecture’s effectiveness in complex, multi-turn, multi-tool scenarios.
- Further analysis revealed that multi-agent collaboration reduced individual model load, enabling better scalability. The multi-turn rewriting module contributed significantly to maintaining context coherence, especially in colloquial queries. The system’s ability to handle ambiguous requests and correct errors dynamically was confirmed through user feedback and controlled experiments.
Significance
This work addresses longstanding challenges in enterprise natural language understanding, particularly in multi-turn, multi-intent scenarios. By integrating a novel Plan+Solver architecture with multi-turn query rewriting, it significantly enhances the flexibility and accuracy of intelligent assistants. The system bridges the gap between theoretical NLP advances and practical office automation needs, enabling more natural, efficient human-computer interaction. Its scalable multi-agent framework offers a blueprint for future large-scale intelligent systems, fostering smarter workplaces and reducing manual workload. The approach also paves the way for broader applications in complex multi-task environments, such as customer service, knowledge management, and decision support.
Technical Contribution
The core innovation lies in the decoupled Plan and Solver modules, enabling specialized optimization for task decomposition and parameter extraction. The multi-turn query rewriting mechanism effectively handles colloquial and ambiguous language, maintaining context coherence. The multi-tool retrieval component dynamically selects relevant tools based on query relevance, improving invocation accuracy. The multi-agent collaboration framework distributes workload, enhances scalability, and improves robustness. Fine-tuning of Qwen models with multi-task data, combined with a comprehensive multi-turn data annotation scheme, ensures high performance and generalization. These innovations collectively advance the state-of-the-art in multi-turn, multi-task NLP systems for enterprise use.
Novelty
This research is the first to implement a decoupled Plan+Solver architecture within a multi-agent framework for office automation, integrating multi-turn query rewriting and dynamic tool retrieval. Unlike prior works that rely on monolithic models or predefined workflows, this approach enables flexible, multi-step, multi-intent task handling with high accuracy and robustness. The combination of multi-agent collaboration with advanced NLP techniques represents a significant step forward in scalable, intelligent enterprise systems, setting a new benchmark for multi-turn multi-task dialogue management.
Limitations
- Despite high accuracy, the system can still misinterpret colloquial or highly ambiguous queries, especially in noisy or domain-specific contexts, due to inherent model limitations.
- Data annotation for multi-turn dialogues is resource-intensive, limiting rapid deployment across new scenarios or languages.
- Model inference speed, while improved, still requires optimization for real-time applications in large-scale enterprise environments.
Future Work
Future directions include integrating reinforcement learning to optimize task planning and tool invocation strategies, expanding multi-agent scalability, and reducing annotation costs via semi-supervised techniques. Enhancing model robustness to domain shifts and colloquial language, as well as deploying in multilingual settings, are also key goals. Additionally, exploring self-supervised pretraining tailored for multi-turn dialogues could further improve understanding and response quality, pushing the system toward fully autonomous enterprise assistants.
AI Executive Summary
In the rapidly evolving landscape of enterprise automation, traditional rule-based systems struggle to handle the complexity and variability of multi-task, multi-intent interactions. Recognizing this challenge, the authors propose a novel multi-agent system grounded in a decoupled Plan and Solver architecture. This system leverages advanced natural language processing techniques, including multi-turn query rewriting and dynamic tool retrieval, to significantly improve understanding accuracy and task success rates.
At its core, the architecture employs a master node that orchestrates specialized worker agents, each responsible for specific subtasks such as intent recognition, parameter extraction, and tool invocation. The system is built upon fine-tuned Qwen2.5 models, trained on extensive datasets covering diverse office scenarios like email, scheduling, and meetings. Multi-turn dialogue management ensures context coherence, enabling the system to interpret colloquial and ambiguous requests effectively.
Experimental results demonstrate that the system achieves over 95% query understanding accuracy and 92% task completion success in real enterprise settings. The multi-turn query rewriting module alone improves understanding accuracy by 8%, while the multi-tool retrieval mechanism boosts invocation success by 7%. The multi-agent collaboration framework reduces model load and enhances scalability, making it suitable for large-scale deployment.
This work addresses critical bottlenecks in enterprise NLP, providing a scalable, accurate, and flexible solution for complex multi-step workflows. Its implications extend beyond office automation, offering a blueprint for future intelligent systems capable of managing intricate multi-task interactions across various domains. While promising, challenges remain in reducing annotation costs, optimizing inference speed, and expanding multilingual capabilities. Nonetheless, this research marks a significant step toward truly intelligent, autonomous enterprise assistants, promising to transform workplace productivity and decision-making processes in the coming years.
Deep Dive
Abstract
This paper introduces a multi-agent application system designed to enhance office collaboration efficiency and work quality. The system integrates artificial intelligence, machine learning, and natural language processing technologies, achieving functionalities such as task allocation, progress monitoring, and information sharing. The agents within the system are capable of providing personalized collaboration support based on team members' needs and incorporate data analysis tools to improve decision-making quality. The paper also proposes an intelligent agent architecture that separates Plan and Solver, and through techniques such as multi-turn query rewriting and business tool retrieval, it enhances the agent's multi-intent and multi-turn dialogue capabilities. Furthermore, the paper details the design of tools and multi-turn dialogue in the context of office collaboration scenarios, and validates the system's effectiveness through experiments and evaluations. Ultimately, the system has demonstrated outstanding performance in real business applications, particularly in query understanding, task planning, and tool calling. Looking forward, the system is expected to play a more significant role in addressing complex interaction issues within dynamic environments and large-scale multi-agent systems.