Wrong but Useful: Trajectory Value Beyond Answer Correctness in Multi-Agent Messages
Introduces DHD protocol to measure trajectory value of multi-agent messages, revealing that wrong messages can be helpful; over 40% of such messages aid reasoning in benchmarks.
Key Findings
Methodology
This paper proposes the Diverse Hypothesis Deliberation (DHD) protocol as a controlled measurement framework. It involves caching five independently generated messages per problem, with roles assigned dynamically to generate diverse hypotheses. The core process includes fixing the message pool before inference, then performing leave-one-out (LOO) replay: selectively hiding each message and rerunning the downstream solver (integrator) to observe the impact on final reasoning outcomes. This setup allows for a direct comparison between the full message pool and partial pools, isolating each message’s contribution to the reasoning process. The method does not rely on correctness scores for filtering but instead measures the actual influence of messages on subsequent reasoning, capturing the trajectory value. Multiple repetitions ensure statistical robustness, and experiments span five scientific benchmarks across two model families (gpt-oss-120b and gemma-4-31B-it). This approach emphasizes the contextual, dynamic contribution of messages rather than their static correctness, providing a nuanced understanding of information utility in multi-agent systems.
Key Results
- Across five benchmarks, including Omni-MATH-2, JEEBench, SciBench, LAB-Bench, and MaScQA, all tested models (gpt-oss-120b and gemma-4-31B-it) revealed the presence of error messages that nonetheless contributed positively to reasoning. Specifically, more than 40% of messages that changed the final correctness outcome were helpful, demonstrating that incorrect messages can contain valuable reasoning cues. Repeated experiments confirmed these effects are statistically significant (p=0.0002), indicating they are not due to random replay variation. Notably, complete messages—containing both reasoning and answers—outperformed only answer-based retention, leading to higher success rates. These findings challenge the conventional reliance on correctness as the sole metric for message filtering, highlighting the importance of contextual, trajectory-based evaluation.
- Further analysis showed that in all tested scenarios, the presence of wrong but helpful messages improved the success probability more than 40% of the time when they altered the outcome. The full message, which includes reasoning steps, was more effective than only keeping the answer, emphasizing the importance of reasoning content. The experiments also demonstrated that the trajectory value could be reliably estimated through repeated leave-one-out comparisons, providing a practical tool for message selection. Overall, the results underscore that message utility depends on its influence on subsequent reasoning, not just on correctness, and that identifying helpful errors can significantly enhance multi-agent reasoning performance.
- The broad implication is that traditional filtering strategies based solely on correctness scores are insufficient. Messages that are initially wrong can still carry critical scientific principles or reasoning pathways that, if properly leveraged, can improve overall system robustness and interpretability. The study’s methodology and findings suggest a paradigm shift toward context-aware, trajectory-based message evaluation, which can be integrated into future multi-agent architectures to improve decision-making, scientific discovery, and educational tools. This approach opens new avenues for research into dynamic information valuation and adaptive reasoning systems.
Significance
This research fundamentally redefines how information utility is assessed in multi-agent reasoning systems. By demonstrating that a significant portion of wrong messages are nonetheless beneficial, it challenges the prevailing paradigm that correctness equates to value. The introduction of trajectory value as a metric provides a nuanced, context-dependent perspective, enabling systems to retain and leverage messages previously deemed unhelpful. This has profound implications for AI robustness, transparency, and scientific reasoning, especially in domains where incomplete or noisy data is common. The methodology bridges the gap between static correctness and dynamic contribution, offering a scalable, quantifiable approach to improve message filtering and integration. Ultimately, this work advances the theoretical understanding of information flow and paves the way for more resilient, interpretable AI systems capable of harnessing the full spectrum of available knowledge, including imperfect or erroneous data.
Technical Contribution
The core technical innovation lies in the development of the DHD protocol, which employs a fixed message pool and controlled leave-one-out (LOO) replay to quantify each message’s trajectory value. Unlike traditional approaches that rely solely on correctness scores, this method assesses the contextual influence of messages on downstream reasoning outcomes. It integrates role-based hypothesis generation, fixed message caching, and systematic replay comparisons, enabling precise attribution of helpful or harmful effects. The statistical rigor is ensured through multiple repetitions, which validate the robustness of the observed effects. The framework is model-agnostic, demonstrated on two open-source models (gpt-oss-120b and gemma-4-31B-it), and across diverse scientific benchmarks, establishing broad applicability. This approach provides a new quantitative tool for message evaluation, facilitating more intelligent filtering strategies that go beyond correctness, thereby enhancing the interpretability and robustness of multi-agent reasoning systems.
Novelty
This work introduces the novel concept of trajectory value for message assessment, which captures the contextual, dynamic contribution of messages to reasoning pathways. Unlike prior methods that focus solely on final answer correctness or confidence scores, the proposed protocol systematically measures the influence of individual messages via leave-one-out replay, isolating their impact in a fixed context. The idea that incorrect messages can be helpful is a significant departure from conventional filtering paradigms, supported by extensive empirical evidence across multiple benchmarks and models. The combination of role-based hypothesis generation, fixed message pools, and controlled replay constitutes a unique methodological contribution, establishing a new standard for evaluating information utility in multi-agent AI systems.
Limitations
- While the DHD protocol effectively measures trajectory value in static, fixed pools, its applicability to dynamic, multi-turn, or multi-round reasoning scenarios remains limited. The current setup assumes a single-shot message generation, which may not capture evolving message utility over multiple interactions.
- The reliance on role-based hypothesis generation and fixed message pools may restrict adaptability to tasks requiring more flexible or context-dependent message creation. Automatic role assignment and message diversification are needed for broader applicability.
- Computational costs are significant due to multiple repeated replays, especially for large-scale models or real-time applications. Optimization of replay efficiency and scalability is necessary for practical deployment.
- The method’s effectiveness depends on the quality of role design and message generation; poorly designed roles may lead to biased or uninformative message pools. Further research is needed to automate and optimize role assignment.
- The current experiments focus on open-answer benchmarks; extending the approach to multiple-choice, multi-modal, or real-world scenarios requires additional validation.
Future Work
Future directions include developing adaptive role assignment mechanisms to generate more diverse hypotheses, extending the framework to multi-turn and multi-round reasoning tasks, and integrating reinforcement learning to optimize message selection based on trajectory value. Additionally, scaling the approach to larger models and multi-modal data will be crucial for real-world applications. Exploring automated role design and message diversification strategies can further enhance robustness. Applying the methodology to scientific discovery, education, and decision-making systems will validate its practical utility. Lastly, combining trajectory value with other interpretability metrics could lead to comprehensive frameworks for trustworthy AI, fostering systems that not only perform well but also explain their reasoning process transparently.
AI Executive Summary
In the rapidly evolving landscape of artificial intelligence, multi-agent reasoning systems have become essential for tackling complex scientific, educational, and decision-making tasks. Traditional approaches predominantly rely on message filtering strategies that prioritize answers based on correctness or confidence scores. However, this paradigm overlooks the nuanced contribution of individual messages within the reasoning process. Recognizing this gap, the present study introduces the concept of trajectory value—a measure of a message’s actual influence on subsequent reasoning—challenging the assumption that only correct messages are valuable.
The core innovation is the Diverse Hypothesis Deliberation (DHD) protocol, a systematic framework designed to quantify the contribution of each message in a fixed message pool. DHD employs role-based hypothesis generation, where five distinct roles produce diverse, structured messages independently. These messages are then cached, and a downstream integrator synthesizes a final answer by selectively including or excluding each message through a leave-one-out (LOO) replay mechanism. This process isolates the impact of each message on the reasoning pathway, enabling a direct assessment of its helpful or harmful influence.
Extensive experiments across five scientific benchmarks—such as Omni-MATH-2, JEEBench, SciBench, LAB-Bench, and MaScQA—demonstrate that a significant proportion of error messages (over 40%) are actually helpful in changing the final outcome. The results, validated on two open-source models (gpt-oss-120b and gemma-4-31B-it), reveal that complete messages containing both reasoning steps and answers outperform partial messages, emphasizing the importance of content-rich information. Notably, the findings show that message utility is context-dependent and cannot be solely predicted by correctness. This insight underscores the limitations of existing filtering strategies and advocates for a trajectory-based evaluation.
The implications are profound: by moving beyond correctness as the sole criterion, AI systems can leverage a broader spectrum of information, including errors that contain valuable scientific principles or reasoning cues. This paradigm shift enhances the robustness, interpretability, and scientific fidelity of multi-agent systems, paving the way for more trustworthy AI in scientific discovery, education, and automated reasoning. Despite its strengths, the approach faces challenges in scalability, dynamic reasoning scenarios, and automated role design, which future research aims to address. Overall, this work marks a significant step toward more nuanced, context-aware evaluation of information utility in AI reasoning systems, promising a future where errors are not just mistakes but potential sources of insight.
Deep Analysis
Background
随着人工智能在科学推理、教育和自动化决策中的不断应用，构建具有高度鲁棒性和解释性的多智能体系统成为研究热点。早期工作主要集中在提升模型的整体性能，如GPT-3、PaLM等大型预训练模型在数学和科学任务中的突破（如GSM8K、DROP、ARC等数据集）。然而，这些方法多依赖于最终答案的正确率作为衡量标准，忽视了推理路径中的中间信息价值。近年来，诸如Chain-of-Thought（CoT）推理、Self-Consistency、多路径生成和多智能体协作机制（如Debate、ReAct）等技术，试图增强推理的透明性和多样性，但仍存在筛选偏差和信息丢失的问题。特别是在科学推理中，错误消息可能包含关键的科学原理或推理线索，传统筛选方法难以捕捉其潜在价值。因此，如何系统性地评估每条消息在推理路径中的实际贡献，成为当前亟待解决的核心问题。
Core Problem
核心问题在于：现有多智能体推理系统中，筛选机制过度依赖答案的正确性或置信度，导致潜在有用的错误消息被排除，限制了系统的推理深度和解释能力。具体表现为：• 筛选标准单一，忽略推理内容的潜在价值；• 错误消息可能包含关键科学原则或推理线索，误判为无用；• 现有方法缺乏对消息在推理路径中实际贡献的量化手段。这不仅影响系统的鲁棒性，也限制了其在科学研究、教育和自动化决策中的应用潜力。解决这一问题，要求引入更细粒度、更上下文敏感的消息评估指标，从而实现信息的最大化利用。
Innovation
本研究的创新点主要体现在：1）提出轨迹价值（Trajectory Value）概念，强调消息在推理路径中的动态贡献，突破以往仅关注答案正确率的限制；2）设计多角色、多消息的固定池机制，确保消息内容不变，避免偏差；3）采用留一重放（LOO）机制，通过逐一隐藏消息，定量评估其对推理结果的帮助或阻碍作用；4）结合多次重复实验，确保轨迹价值的统计稳健性；5）在五个科学基准和两个模型家族上进行广泛验证，验证其普适性和有效性。这些创新为多智能体系统中的信息筛选提供了全新的理论基础和技术工具。
Methodology
- �� 角色分配：系统动态为每个问题分配五个不同角色（如化学家、物理学家、工程师等），每个角色独立生成一条结构化消息，内容涵盖推理过程和最终答案；• 固定消息池：在推理开始前，确保五个消息内容固定不变，避免生成偏差；• 留一重放（LOO）：逐一隐藏每条消息，重新运行下游整合器（integrator），观察其对最终推理结果的影响；• 比较两种重放：单条消息重放（只显示一条消息）与全池重放（显示全部消息），以评估每条消息的帮助或阻碍作用；• 多次重复：对每个问题多次进行留一重放，统计消息的帮助率和影响方向，确保结果的统计显著性；• 评估指标：定义帮助、阻碍和中性三类效果，结合模型输出的最终正确性，量化每条消息的轨迹价值。
Experiments
实验采用五个科学推理基准：Omni-MATH-2、JEEBench、SciBench、LAB-Bench和MaScQA，涵盖数学、物理、生物等多个领域。模型方面，选择开源模型gpt-oss-120b和gemma-4-31B-it，确保方法的可复现性。每个任务中，系统根据任务特性动态分配五个角色，生成五条结构化消息，内容包括推理步骤和答案。利用留一重放机制，逐一隐藏消息，观察对最终推理结果的影响，统计帮助、阻碍和中性效果。每个问题重复多次，确保统计稳健。还对不同模型和任务类型进行验证，分析完整消息与只保留答案的效果差异，验证内容丰富的完整消息在提升成功率方面的优势。
Results
实验结果显示：• 在所有基准中，超过40%的错误消息在改变最终正确性时，表现出积极的轨迹价值，说明错误消息中潜藏有用信息；• 多次重复实验确认，这些效果具有统计显著性（p=0.0002），非随机现象；• 完整消息（推理+答案）比只保留答案更能提升成功率，尤其在复杂推理任务中表现突出；• 轨迹价值的估算通过留一重放实现，验证了其在不同模型和任务中的稳健性。这些发现挑战了传统筛选策略，强调应关注消息在推理路径中的实际贡献，而非单一的答案正确性指标。
Applications
该方法在科学研究、教育、自动化决策等领域具有广泛应用潜力。具体应用包括：• 科学数据分析：帮助科研人员筛选潜在有用的推理线索，即使它们暂时未能得出正确答案；• 智能教育系统：提升AI辅导的推理能力，识别学生错误中潜藏的理解误区；• 自动化科研：在复杂推理任务中筛选出具有潜在价值的中间推理，辅助科研创新。未来，结合自动角色分配、多轮推理和多模态数据，将进一步增强系统的适应性和智能水平。
Limitations & Outlook
当前方法在极端或新颖任务中的适应性仍有限，角色设计和消息生成策略可能影响测量效果。模型规模和架构多样性未充分验证，需在更大规模和多模态场景中进行扩展。留一重放机制在多轮交互中可能无法捕捉消息随时间变化的价值，未来应结合连续推理场景进行改进。此外，计算成本较高，尤其在大模型和多次重复实验中，需优化效率。最后，自动化角色设计和多样化消息生成仍是未来研究的重要方向。
Plain Language Accessible to non-experts
想象你在厨房里准备一道复杂的菜肴，你有很多不同的食谱（消息），每个食谱都告诉你怎么做。传统的方法只看最后做出来的菜是不是好吃（答案是否正确），如果不好吃就扔掉。但其实，有些菜谱虽然做出来的菜不完美（错误答案），但里面的步骤可能藏着一些特别的技巧，比如用了一种特别的调料或烹饪方法，这些技巧以后能帮你做出更棒的菜。就像论文里说的，错误的消息也可能包含有用的线索，只要我们知道怎么去发现它们。研究人员设计了一种方法，就像反复试验，把每个食谱放进去，看看它对最终菜肴的帮助有多大。结果发现，很多错误的食谱其实都能帮你做出更好的菜，只要你善于发现它们的价值。这告诉我们，错误不一定是坏事，有时候，错误的想法里藏着宝藏！
ELI14 Explained like you're 14
想象你在学校里参加科学比赛，你和朋友们都提出了自己的想法（消息），有的想法虽然不完全正确（错误答案），但里面可能藏着一些有用的线索，比如某个科学原理或者解决问题的技巧。传统的方法只看最终答案是不是对的，如果错了就扔掉，但其实，有些错的想法还能帮你找到正确的路，就像拼图游戏中，某块拼图虽然不完美，但能帮你拼出正确的图案。这篇论文告诉我们，要学会识别这些“错误但有用”的想法，而不是只看答案是不是正确。科学家们设计了一种特别的测试方法，像反复试验一样，把每个想法放进去，看看它到底帮了还是拖了后腿。结果发现，很多错误的想法其实都很有价值，只要用对了，它们就能帮你更快找到正确答案。这就像在玩游戏时，失败的策略有时候反而能教会你怎么赢，关键是要知道哪些策略是有用的。这个发现让我们明白，错误不一定是坏事，有时候，错误的想法里藏着宝藏！
Abstract
Multi-agent reasoning systems often use agreement, confidence, or automated scores to decide which messages should shape a final answer. Such filtering assumes that a message likely to be correct is also worth keeping. Yet a wrong answer can contain a useful decomposition, constraint, or scientific principle. We test this distinction with Diverse Hypothesis Deliberation (DHD), a controlled measurement protocol that caches five independently generated messages and replays the same downstream solver, called the integrator, with each message available or hidden. The replay comparison measures a message's trajectory value: whether making the message available helps or harms subsequent reasoning. Across five mathematics and science benchmarks and two openly available model families, gpt-oss-120b and gemma-4-31B-it, wrong-helpful messages appear in every benchmark-model combination. Among wrong-answer messages that change final correctness, more than four in ten changes are helpful in each model. Controlled repeats show that the number of repeatable message effects is unlikely to arise from replay variation alone (p=0.0002). A focused intervention on repeatable wrong-helpful messages finds that the complete message works best, while retaining its reasoning preserves more success than retaining only its answer; the source of the complete-message advantage remains open. Within the same problem, repeated trajectory-value evidence also identifies a better keep-or-remove choice than answer correctness alone. Answer correctness is therefore informative but does not determine trajectory value. DHD measures this missing property and produces reusable labels for learning when agents should listen.
References (20)
ReConcile: Round-Table Conference Improves Reasoning via Consensus among Diverse LLMs
J. Chen, Swarnadeep Saha, Mohit Bansal
Can Large Language Models Really Improve by Self-critiquing Their Own Plans?
Karthik Valmeekam, Matthew Marquez, Subbarao Kambhampati
ReST-MCTS*: LLM Self-Training via Process Reward Guided Tree Search
Dan Zhang, Sining Zhoubian, Yisong Yue et al.
Let's Verify Step by Step
H. Lightman, Vineet Kosaraju, Yura Burda et al.
MARS: toward more efficient multi-agent collaboration for LLM reasoning
Xiao Wang, Jia Wang, Yijie Wang et al.
Learning From Failure: Integrating Negative Examples when Fine-tuning Large Language Models as Agents
Renxi Wang, Haonan Li, Xudong Han et al.
Agents that Matter: Optimizing Multi-Agent LLMs via Removal-Based Attribution
Mingyu Lu, Yushan Huang, Chris Lin et al.
Information Value Theory
R. Howard
A Coefficient of Agreement for Nominal Scales
Jacob Cohen
Benchmarks Saturate When The Model Gets Smarter Than The Judge
Marthe Ballon, A. Algaba, Brecht Verbeken et al.
CriticBench: Benchmarking LLMs for Critique-Correct Reasoning
Zicheng Lin, Zhibin Gou, Tian Liang et al.
Solving math word problems with process- and outcome-based feedback
Jonathan Uesato, Nate Kushman, Ramana Kumar et al.
Reflexion: language agents with verbal reinforcement learning
Noah Shinn, Federico Cassano, Beck Labash et al.
A Dynamic LLM-Powered Agent Network for Task-Oriented Agent Collaboration
Zijun Liu, Yanzhe Zhang, Peng Li et al.
Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotations
Peiyi Wang, Lei Li, Zhihong Shao et al.
Improving Factuality and Reasoning in Language Models through Multiagent Debate
Yilun Du, Shuang Li, A. Torralba et al.
MedAgents: Large Language Models as Collaborators for Zero-shot Medical Reasoning
Xiangru Tang, Anni Zou, Zhuosheng Zhang et al.
Unleashing the Emergent Cognitive Synergy in Large Language Models: A Task-Solving Agent through Multi-Persona Self-Collaboration
Zhenhailong Wang, Shaoguang Mao, Wenshan Wu et al.
Self-Refine: Iterative Refinement with Self-Feedback
Aman Madaan, Niket Tandon, Prakhar Gupta et al.
LAB-Bench: Measuring Capabilities of Language Models for Biology Research
Jon M. Laurent, Joseph D. Janizek, Michael Ruzo et al.