Multi-Agent Medical Decision Consensus Matrix System: An Intelligent Collaborative Framework for Oncology MDT Consultations
Multi-agent system with role-specific LLMs uses Kendall's W for consensus, enhanced by reinforcement learning for optimal oncology treatment recommendations.
Key Findings
Methodology
The system comprises seven role-specific LLM agents simulating MDT workflows. It introduces a structured consensus matrix based on Kendall's W to quantify agreement among agents. Reinforcement learning algorithms (Q-Learning, PPO, DQN) are integrated to optimize multi-round interactions, guiding agents toward consensus efficiently. Each agent generates structured opinions supported by evidence chains citing clinical guidelines and literature, ensuring transparency. The process involves evidence retrieval, opinion generation, consensus matrix updating, and policy optimization, culminating in treatment recommendations. Evaluations on five benchmarks (MedQA, PubMedQA, DDXPlus, MedBullets, SymCat) show significant improvements, with an average accuracy of 87.5%, consensus rate of 89.3%, and Kendall's W of 0.823, rated 8.9/10 by clinicians.
Key Results
- The system outperformed baselines with 87.5% accuracy across benchmarks, surpassing 83.8%. It achieved 89.3% consensus rate and a Kendall's W of 0.823, indicating high agreement.
- Reinforcement learning strategies accelerated convergence and enhanced decision quality, demonstrating the effectiveness of policy optimization in multi-agent interactions.
- All recommendations are linked to evidence chains following GRADE principles, ensuring traceability and clinical credibility.
Significance
This work addresses the critical gap in structured, quantifiable consensus mechanisms in clinical MDTs. By integrating role-specific LLM agents, a mathematically grounded consensus metric, and reinforcement learning, it elevates decision transparency, consistency, and efficiency. The approach provides a scalable, explainable AI framework adaptable to various complex medical scenarios, advancing AI-assisted personalized oncology care. Its emphasis on evidence traceability and multi-agent collaboration paves the way for broader adoption of AI in clinical practice, ultimately improving patient outcomes and healthcare quality.
Technical Contribution
The core innovation lies in embedding Kendall's W within a multi-agent framework, enabling quantitative measurement of agreement. The system combines role-specific LLMs, a structured consensus matrix, and reinforcement learning-based policy optimization, ensuring adaptive, efficient, and transparent decision-making. This architecture surpasses traditional voting or weighted averaging methods, offering theoretical convergence guarantees and practical improvements in decision accuracy and speed. The integration of evidence chains aligned with clinical guidelines further enhances trust and interpretability, setting a new standard for AI-driven clinical decision support.
Novelty
This is the first work to incorporate Kendall's W as a core metric within a multi-agent medical decision system, coupling it with reinforcement learning to dynamically optimize consensus formation. Unlike prior approaches relying solely on majority voting or unstructured dialogue, this framework emphasizes role-specific expertise, evidence traceability, and multi-round adaptive strategies, representing a significant leap forward in AI-assisted clinical decision-making.
Limitations
- The system depends heavily on high-quality, up-to-date clinical guidelines and literature; its performance may decline with emerging or rare conditions lacking sufficient evidence.
- Training reinforcement learning models requires substantial computational resources, which may limit real-time deployment in resource-constrained settings.
- Potential biases in role-specific agents or evidence sources could influence recommendations; further work is needed to ensure fairness and robustness across diverse populations.
Future Work
Future research will focus on integrating multimodal data such as imaging and genomics to enhance personalized recommendations. Developing more efficient reinforcement learning algorithms and online adaptation mechanisms will improve real-time clinical applicability. Multi-center clinical trials are planned to validate the system's effectiveness in diverse settings. Additionally, efforts will be made to address fairness, reduce biases, and extend the framework to other complex decision domains beyond oncology.
AI Executive Summary
Multidisciplinary team (MDT) meetings are central to cancer treatment planning, yet they often lack structured mechanisms for quantifying consensus and ensuring decision traceability. Traditional discussions rely heavily on subjective judgment, which can lead to inconsistencies and difficulties in auditing decisions. To address these issues, this work introduces a novel multi-agent system that models each MDT role as a specialized large language model (LLM) agent. These agents simulate real-world clinical roles such as oncologists, radiologists, nurses, and others, each generating structured opinions supported by evidence chains citing clinical guidelines and peer-reviewed literature.
The core innovation is the development of a consensus matrix based on Kendall's coefficient of concordance (W), which quantifies the agreement level among agents. This matrix dynamically updates as agents revise their opinions through multiple rounds of interaction, guided by reinforcement learning algorithms including Q-Learning, PPO, and DQN. These algorithms optimize the interaction policies, accelerating convergence toward high-quality, clinically appropriate decisions.
Experimental validation across five benchmark datasets demonstrated the system's superior performance, achieving an average accuracy of 87.5%, a consensus rate of 89.3%, and a Kendall's W of 0.823. Clinicians rated the clinical appropriateness of outputs at 8.9/10, confirming the system's practical relevance. The evidence chains adhere to GRADE principles, ensuring full traceability and transparency.
This framework significantly advances AI-assisted clinical decision-making by providing a structured, explainable, and adaptive approach to consensus formation. It addresses longstanding challenges in MDT decision processes, offering a scalable solution that enhances decision quality, efficiency, and accountability. Future directions include integrating multimodal data, refining online learning capabilities, and expanding validation in diverse clinical settings, aiming to embed AI more deeply into routine oncology care and beyond.
Deep Dive
Plain Language Accessible to non-experts
想象一个厨房里有七个厨师，每个人都擅长不同的菜系，比如中餐、西餐、甜点、汤品等等。他们需要合作做出一道完美的菜单。每个厨师会提出自己的建议，比如用什么食材、怎么做、味道如何。为了确保大家都满意，他们会不断交流、投票，看看哪些建议最受欢迎。系统就像这个厨房，七个厨师代表不同专业，每个人都用自己的“专业知识”给出建议，然后通过一种特别的评分方法，衡量大家的意见有多一致。如果意见差异太大，厨师们会讨论、调整，直到达成一致。最终，他们会一起做出一道既美味又受大家喜欢的菜肴。这种合作方式确保每个人的想法都被考虑，菜肴也更好吃。
ELI14 Explained like you're 14
想象你和你的朋友们在决定玩什么游戏。每个人都喜欢不同的游戏，有的喜欢冒险，有的喜欢运动，有的喜欢拼图。你们每个人都说出自己喜欢的，然后开始讨论。可是，有时候大家的意见差得很远，难以达成一致。这个系统就像你们的朋友团，有七个“朋友”，每个都用自己擅长的“专业”给建议。每个“朋友”会根据自己的经验和偏好，提出建议，并用一种特别的评分方法，衡量大家的意见有多一致。如果意见差异太大，朋友们会互相讨论、调整，直到找到一个大家都能接受的方案。这样，大家既能合作，又能玩得开心，最后的决定也更靠谱。这就像一个聪明的团队，大家一起合作，做出最棒的决定！
Abstract
Multidisciplinary team (MDT) consultations are the gold standard for cancer care decision-making, yet current practice lacks structured mechanisms for quantifying consensus and ensuring decision traceability. We introduce a Multi-Agent Medical Decision Consensus Matrix System that deploys seven specialized large language model agents, including an oncologist, a radiologist, a nurse, a psychologist, a patient advocate, a nutritionist and a rehabilitation therapist, to simulate realistic MDT workflows. The framework incorporates a mathematically grounded consensus matrix that uses Kendall's coefficient of concordance to objectively assess agreement. To further enhance treatment recommendation quality and consensus efficiency, the system integrates reinforcement learning methods, including Q-Learning, PPO and DQN. Evaluation across five medical benchmarks (MedQA, PubMedQA, DDXPlus, MedBullets and SymCat) shows substantial gains over existing approaches, achieving an average accuracy of 87.5% compared with 83.8% for the strongest baseline, a consensus achievement rate of 89.3% and a mean Kendall's W of 0.823. Expert reviewers rated the clinical appropriateness of system outputs at 8.9/10. The system guarantees full evidence traceability through mandatory citations of clinical guidelines and peer-reviewed literature, following GRADE principles. This work advances medical AI by providing structured consensus measurement, role-specialized multi-agent collaboration and evidence-based explainability to improve the quality and efficiency of clinical decision-making.