DeALOG: Decentralized Multi-Agents Log-Mediated Reasoning Framework
DeALOG employs a decentralized multi-agent framework with shared natural language logs, achieving robustness and transparency in multimodal QA.
Key Findings
Methodology
DeALOG introduces five specialized agents (Table, Context, Visual, Summarizing, Verification) that communicate solely via a shared, typed natural language log, eliminating the need for a central planner. A lightweight scheduler manages turn-taking, ensuring robustness and transparency. Each agent appends ordered entries to the log, which serves as persistent memory and provenance trail. The verification agent cross-checks answers against the log, enhancing reliability. The framework leverages algorithms like BM25+miniLM for retrieval and LLMs such as LLaMA-3 8B for reasoning. Experiments across six datasets (FinQA, TAT-QA, WikiTableQuestions, FeTaQA, CRT-QA, MultiModalQA) demonstrate superior performance in accuracy, robustness, and interpretability compared to baseline methods.
Key Results
- On FinQA, DeALOG with LLaMA-3 8B achieved 76.5% EM, outperforming multi-agent baselines, and showed only a 6.4% increase in error rate under evidence corruption, indicating enhanced robustness.
- In MultiModalQA, DeALOG maintained the highest accuracy (75.3%) across different models, demonstrating resilience to noise and long reasoning chains, validating the shared log's effectiveness.
- Ablation studies reveal that removing the shared log or verification mechanism significantly reduces performance, confirming the critical role of logging, provenance, and validation in robustness and transparency.
Significance
This work advances multi-agent reasoning by replacing the traditional central planner with a decentralized log-mediated approach, significantly improving robustness and interpretability. It addresses long-standing issues of error propagation and opaque reasoning in complex multimodal tasks. The framework’s modularity and transparency make it suitable for real-world applications like automated question answering, knowledge inference, and human-AI collaboration, fostering trust and explainability in AI systems. Its design principles can inspire future research on scalable, robust, and interpretable AI architectures.
Technical Contribution
DeALOG's main innovation is the use of a shared, typed natural language log as the communication backbone among specialized agents, replacing the central planner. This design ensures provenance, traceability, and robustness. The framework combines multi-agent modularity with cross-validation and gating mechanisms, enabling scalable and transparent reasoning. It introduces formal definitions of the scheduler–planner boundary and demonstrates improved robustness against evidence corruption. The approach also facilitates flexible integration of multimodal inputs and reasoning modules, opening pathways for future enhancements in AI system design.
Novelty
This is the first framework to utilize a shared, typed natural language log for decentralized multi-agent reasoning in multimodal QA, avoiding single points of failure and error propagation typical of planner-based architectures. Its provenance-anchored log provides transparency and traceability, setting a new standard for explainability in AI reasoning systems. Unlike prior work relying on explicit plans or code, DeALOG’s log-centric approach offers a lightweight, scalable, and robust alternative that can adapt to diverse modalities and tasks.
Limitations
- The system relies on large pre-trained models, which entail high computational costs and latency, especially for long reasoning chains.
- While the log mechanism enhances robustness, it may still be vulnerable under extreme noise or missing evidence scenarios, requiring further robustness improvements.
- Current scheduling is fixed and question-agnostic; adaptive, learning-based scheduling could further optimize performance and efficiency.
Future Work
Future directions include developing adaptive, learning-based scheduling strategies, integrating dynamic evidence validation, and exploring reinforcement learning to optimize agent interactions. Extending the framework to handle more complex multimodal inputs and reasoning tasks, such as real-time video understanding or multi-turn dialogues, is also planned. Additionally, efforts will focus on reducing computational overhead and improving scalability for deployment in real-world applications.
AI Executive Summary
DeALOG introduces a novel decentralized multi-agent framework for multimodal question answering, fundamentally shifting away from traditional centralized planning architectures. Instead of relying on a single planner to orchestrate reasoning, DeALOG employs five specialized agents—Table, Context, Visual, Summarizing, and Verification—that communicate exclusively through a shared, typed natural language log. This log acts as both persistent memory and a provenance trail, enabling transparent and traceable reasoning processes. The lightweight scheduler manages the turn-taking among agents, ensuring a simple yet effective coordination mechanism that avoids the pitfalls of early error propagation common in planner-based systems.
The core technical innovation lies in replacing the central planner with a log-mediated communication protocol. Each agent appends evidence, intermediate reasoning steps, and final answers to the log, which can be cross-verified by the Verification agent. This design significantly enhances robustness, as errors are isolated and do not cascade downstream, and transparency, since every step is recorded with source attribution. The framework leverages algorithms like BM25+miniLM for information retrieval and large language models such as LLaMA-3 8B for reasoning, tested across six benchmark datasets. Results demonstrate that DeALOG achieves competitive accuracy (over 75%) while exhibiting superior robustness under evidence corruption and long-horizon reasoning scenarios.
Experimental ablations confirm that the shared log and verification mechanisms are critical for performance gains. The system’s modularity allows easy integration of new modalities and reasoning modules, making it scalable and adaptable. Its transparent provenance trail fosters explainability, addressing key concerns in AI deployment. Looking ahead, the authors plan to incorporate adaptive scheduling, reinforcement learning, and multimodal knowledge fusion to further enhance system efficiency and applicability. Overall, DeALOG paves the way for more reliable, interpretable, and scalable AI reasoning systems, with broad implications for automated knowledge inference and human-AI collaboration.
Deep Analysis
Background
多模态问答技术经历了从单一模态到多模态信息融合的演变，早期依赖规则和模板方法，逐步引入深度学习模型（如BERT、GPT系列）实现信息抽取和推理。代表性工作包括WIKITABLEQUESTIONS、TAPEX、TAPAS等，解决了表格理解和推理问题。近年来，结合大规模预训练模型的链式推理（Chain-of-Thought）和多智能体架构逐渐兴起，旨在提升复杂推理能力。然而，现有方法多依赖中央规划器，容易在推理链长或证据噪声时出现错误传播，缺乏透明性和鲁棒性。多模态任务如MultiModalQA引入图像和文本的融合，进一步增加了系统复杂性。尽管取得一定进展，但在推理可解释性、抗干扰能力和系统扩展性方面仍有待突破。
Core Problem
现有多模态问答系统多依赖集中式架构，中央规划器在多步骤推理中成为单点故障源，错误易在推理链中传播，导致系统不稳定。此外，缺乏有效的推理过程追溯机制，难以解释推理路径，限制了模型的可信度和调试能力。面对长推理链和多模态信息融合，系统的鲁棒性不足，容易受到噪声和信息缺失影响。如何设计一个既具备模块化、可扩展性，又能保证推理过程透明和鲁棒的系统，成为亟待解决的核心问题。
Innovation
DeALOG的创新点在于引入类型化、持久的自然语言日志作为多智能体间的通信媒介，取代传统的中央计划器。其核心创新包括：• 多智能体架构（表格、文本、视觉、总结、验证）实现任务模块化，便于扩展；• 通过日志的append-only特性，确保推理路径的可追溯性和透明性；• 验证机制在日志中进行交叉验证，提升鲁棒性，减少错误传播；• 采用轻量级调度器管理轮次，避免复杂的计划推导。该设计极大改善了系统的抗干扰能力和可解释性，提供了新颖的多模态推理原语。
Methodology
- �� 智能体：包括表格、文本、视觉、总结、验证，分别负责信息抽取、推理和验证。• 日志机制：所有智能体在共享日志中追加类型化条目（如LOOKUP、ANSWER、FLAG），确保推理路径可追溯。• 调度策略：固定轮次调度，轮流激活智能体，无需中央计划器。• 交叉验证：验证器对总结结果进行交叉验证，确保答案支持。• 证据检索：利用BM25+miniLM检索相关文本和表格片段。• 任务流程：从检索开始，逐步由不同智能体补充信息、总结和验证，直到达成一致或触发重检。• 训练：门控策略在WikiTQ上训练，应用于所有任务。• 评估：在六个数据集上进行准确率、鲁棒性和透明性测试，结合消融实验验证机制重要性。
Experiments
采用六个公开多模态问答数据集（FinQA、TAT-QA、WikiTableQuestions、FeTaQA、CRT-QA、MultiModalQA），统一检索和提取流程，评估准确率、鲁棒性和效率。对比多智能体和单模型基线，进行腐败噪声干扰、长链推理和机制消融。模型使用LLaMA-3 8B、Qwen-3 8B等，调研不同配置下的性能表现。指标包括EM、F1、准确率，统计95%置信区间。还测试了不同的日志去重、门控策略和并行检索对性能的影响。实验设计确保公平性和可重复性，验证DeALOG在复杂推理和多模态融合中的优越性。
Results
DeALOG在六个数据集中的平均准确率超过75%，在抗噪声和长推理链条件下表现出更强的稳定性。腐败干扰实验显示，去中心化架构的误差传播率明显低于集中式方案（误差增长仅6.4%），验证了其鲁棒性。消融实验表明，日志类型化和验证机制对性能提升至关重要，未使用日志或验证时，准确率下降超过10%。此外，系统在多模态融合任务中保持了较高的准确率，显示出良好的扩展性和适应性。
Applications
该框架适用于自动问答、知识推理、智能客服、教育辅导等场景，尤其在需要多模态信息融合和高可解释性的应用中表现优越。系统可部署于企业知识库、智能助手和科研工具中，提升信息检索和推理的效率与可信度。未来还可结合强化学习优化智能体调度，扩展到更复杂的环境和任务。
Limitations & Outlook
目前系统依赖大模型，计算成本较高，推理速度受限。日志验证机制在极端噪声环境下仍可能失效，鲁棒性有限。调度策略较为简单，未来需引入学习型调度以提升效率。除此之外，系统在极端信息缺失或多模态信息不一致时表现尚待优化。未来工作将关注模型压缩、动态调度和多模态知识融合，以增强实用性和适应性。
Plain Language Accessible to non-experts
想象一支由不同专家组成的团队共同完成一项复杂任务，比如策划一场盛大的派对。每个人负责不同的部分：有人负责食物，有人负责装饰，还有人负责邀请客人。大家在一块大白板上写下每一步的计划（日志），每个人都可以看到所有人的想法和进展。每完成一项，就把结果写在白板上，其他人可以检查和确认。如果有人发现问题，可以在白板上指出，大家一起修正。没有一个人是唯一的指挥官，而是轮流行动，确保每一步都清楚透明。这样，即使有人犯错，也不会影响整个计划，因为每个步骤都被记录下来，大家都可以追溯到源头。这就像DeALOG的多智能体系统，通过写日志合作，保证任务既稳健又透明。
ELI14 Explained like you're 14
想象你和朋友们一起做一个大项目，比如做一个科学实验。每个人负责不同的部分，比如准备材料、设计实验、记录数据。你们用一块白板写下每一步的内容，大家都可以看到。每次完成一部分，就把结果写在白板上，其他人可以检查和确认。如果有人发现问题，可以在白板上指出，大家一起改正。没有一个人是唯一的指挥官，而是轮流做事，确保每一步都清楚透明。这样，即使有人犯错，也不会搞砸整个实验，因为每个步骤都被记录下来，大家都能追溯到源头。这就像DeALOG的系统，多个“智能体”合作，通过写日志保证任务的正确和透明。
Abstract
Complex question answering across text, tables and images requires integrating diverse information sources. A framework supporting specialized processing with coordination and interpretability is needed. We introduce DeALOG, a decentralized multi-agent framework for multimodal question answering. It uses specialized agents: Table, Context, Visual, Summarizing and Verification, that communicate through a shared natural-language log as persistent memory. This log-based approach enables collaborative error detection and verification without central control, improving robustness. Evaluations on FinQA, TAT-QA, CRT-QA, WikiTableQuestions, FeTaQA, and MultiModalQA show competitive performance. Analysis confirms the importance of the shared log, agent specialization, and verification for accuracy. DeALOG, provides a scalable approach through modular components using natural-language communication.