CoBa: Cost-Effective Test-Time Scaling via Compute-Balanced Routing
Proposes CoBa, a compute-balanced routing method reducing 49.1% tokens, achieving 85.13% accuracy in test-time reasoning.
Key Findings
Methodology
This work formalizes test-time reasoning as a compute allocation problem, designing a multi-tier verification framework. CoBa first generates a small diverse candidate set, applies inexpensive verification, and then routes uncertain or high-value candidates to stronger evaluators based on uncertainty and answer agreement metrics. The approach employs models like Qwen3-14B, Phi-4, and Qwen3-8B across math and symbolic reasoning tasks. The core algorithm dynamically balances generation, lightweight verification, and deep verification actions within a fixed computational budget, optimizing accuracy and cost. The system's routing policy is learned and reproducible, enabling significant reductions in parameter-weighted tokens while maintaining high accuracy.
Key Results
- On 3,129 samples, CoBa-Routed-Strong achieves 85.13% macro accuracy, with a 49.1% reduction in parameter-weighted tokens compared to baseline methods. It closely matches the self-evaluation weighted voting proxy at 85.20%.
- In the best-of-16 majority voting scenario, accuracy is within 0.01 percentage points, with 58.9% fewer parameter tokens, demonstrating efficient cost-performance trade-offs.
- Paired bootstrap tests confirm significant improvements over single-sample decoding, with remaining gaps indicating potential for further routing refinement.
Significance
This research advances test-time reasoning by integrating dynamic compute scheduling, addressing the longstanding challenge of balancing inference quality and resource consumption. The multi-layered verification and adaptive routing framework significantly improve efficiency, making large language models more practical for real-world applications such as education, scientific research, and decision support systems. It provides a scalable, reproducible approach to optimize inference under fixed budgets, contributing both theoretical insights and practical algorithms for cost-effective AI deployment.
Technical Contribution
The paper introduces a formal framework that models test-time reasoning as a compute resource allocation problem, with a multi-tier verification architecture and a routing policy based on uncertainty and answer consensus. It employs a parameter-weighted token cost metric, enabling precise cost control. The routing policy is designed to maximize accuracy within a fixed compute budget, dynamically deciding whether to generate, verify lightly, verify strongly, or stop. This approach is validated through extensive offline replay experiments, demonstrating superior cost-efficiency over traditional sampling and verification strategies, and establishing a foundation for future adaptive inference systems.
Novelty
This work is the first to systematically formulate test-time reasoning as a compute resource scheduling problem, introducing a multi-layered verification and routing mechanism that adaptively allocates compute based on sample uncertainty and answer agreement. Unlike prior static or monolithic approaches, CoBa dynamically balances generation and verification, achieving significant cost reductions while maintaining high accuracy, thus representing a fundamental shift in test-time inference methodology.
Limitations
- The routing policy relies on predefined thresholds and heuristics, which may not generalize well to unseen tasks or domains without further adaptation. Its performance in highly novel or adversarial scenarios remains untested.
- While effective in reducing tokens and calls, the approach still requires multiple model evaluations, which may be costly in resource-constrained environments. Further optimization is needed for real-time deployment.
- The current evaluation is limited to specific models and datasets; broader validation across diverse tasks and hardware setups is necessary to confirm generalizability.
Future Work
Future research will focus on integrating reinforcement learning to optimize routing policies dynamically, enabling self-adaptive compute allocation. Extending the framework to multi-modal inputs and real-time systems, as well as exploring more sophisticated uncertainty estimation techniques, will further enhance efficiency. Additionally, developing domain-specific tuning methods could improve robustness across tasks, making the approach more versatile for industrial deployment.
AI Executive Summary
In the rapidly evolving field of natural language processing, large language models have demonstrated remarkable capabilities in reasoning tasks. However, their high computational costs pose significant barriers to practical deployment, especially when balancing inference accuracy with resource constraints. Traditional methods often rely on fixed strategies, such as increasing sampling or invoking stronger evaluators, which can lead to inefficient use of computational resources. This paper introduces CoBa, a novel compute-balanced routing framework that dynamically allocates compute actions during test-time inference.
CoBa models the reasoning process as a resource allocation problem, employing a multi-tier verification architecture. It begins by generating a small, diverse set of candidate solutions, then applies inexpensive verification to evaluate their answer consistency. Based on uncertainty metrics and answer agreement, the system decides whether to stop, generate additional candidates, or route certain candidates to more expensive, stronger verifiers. This adaptive approach ensures that computational effort is concentrated where it is most impactful, significantly reducing token usage while maintaining high accuracy.
Extensive experiments across multiple datasets, including math competitions and symbolic reasoning tasks, demonstrate the effectiveness of CoBa. The results show that CoBa-Routed-Strong achieves 85.13% macro accuracy, with nearly half the parameter-weighted tokens compared to baseline sampling methods. It matches the accuracy of much more costly sampling strategies and outperforms fixed strategies, especially on challenging tasks. The approach’s ability to adaptively allocate resources based on sample uncertainty and answer consensus offers a promising pathway toward scalable, cost-effective AI systems.
This work's broader impact lies in enabling large models to operate efficiently within fixed budgets, making advanced reasoning accessible in real-world applications such as education, scientific research, and decision support. The framework's flexibility allows integration with various models and tasks, paving the way for future innovations in adaptive inference and resource-aware AI. Despite current limitations in threshold tuning and hardware dependency, ongoing research aims to incorporate reinforcement learning and multi-modal inputs, further enhancing the system’s robustness and versatility. Overall, CoBa represents a significant step toward practical, efficient, and intelligent AI reasoning systems.
Deep Analysis
Background
近年来，随着大规模语言模型（如GPT-3、PaLM等）在自然语言理解和推理任务中的表现不断提升，推理能力成为研究焦点。早期工作如Chain-of-Thought（Wei et al., 2022）通过引导模型生成推理链，显著改善复杂问题的解答效果。随后，自我一致性（Wang et al., 2022）等方法通过多样采样增强鲁棒性，但带来计算成本的增加。验证器（Cobbe et al., 2021）引入，为模型提供了更可靠的答案确认机制，但其高成本限制了应用范围。近年来，调度与资源管理策略逐渐成为研究重点（Zhang et al., 2025），旨在在有限预算内实现推理性能最大化。本文在此基础上，提出了多层次验证与动态调度框架，结合多模型、多验证层次，推动推理效率和成本控制的创新。
Core Problem
现有测试时推理方法多采用固定策略，如多次采样或调用强验证器，难以在不同任务和样例中实现成本与性能的平衡。尤其是在资源有限的情况下，如何动态调配生成、验证和停止操作，最大化准确率成为核心难题。传统方法缺乏灵活性，导致资源浪费或推理效果不足。解决这一问题需要一种能根据样例复杂度和不确定性自适应调节操作的机制，以实现高效且精确的推理。
Innovation
本文的主要创新包括：1）将测试时推理问题形式化为计算资源调度任务，明确生成、验证和停止的操作边界；2）设计多层次验证体系，结合规则、轻量级和深度验证，提升候选筛选效率；3）引入不确定性和答案一致性指标，动态调节验证强度，实现资源的最优配置；4）提出多模型、多验证层次的调度算法，兼顾推理质量与成本控制。这些创新突破了传统单一操作的局限，为大规模模型在实际场景中的应用提供了新思路。
Methodology
- �� 生成候选：利用生成模型（如Qwen3-14B）在有限预算内产生多个候选答案。• 轻量验证：对所有候选应用低成本验证（如规则或轻量判别模型），评估答案一致性。• 不确定性评估：基于候选的得分差和答案一致性指标，判断是否需要更多候选或更强验证。• 路由决策：根据指标，将部分候选路由到深度验证（如Qwen3-14B深度验证器），或提前停止。• 终极选择：在预算耗尽或满足停止条件后，依据候选的综合得分选择最终答案。• 预算管理：实时监控计算成本（参数加权令牌、调用次数），确保在预算范围内优化性能。• 训练与调优：通过模拟不同场景，优化调度策略参数，确保在多任务环境下的泛化能力。
Experiments
实验采用数学竞赛（MATH-500、AIME、AMC）和符号推理任务，使用Qwen3-14B、Phi-4和Qwen3-8B模型。对比基线包括贪婪解码、固定长推理、最佳N多数投票等。评估指标涵盖准确率、参数加权令牌数、模型调用次数和延迟。采用16个候选样本，调度策略在不同预算下进行重放，验证其在成本控制和准确率提升上的效果。通过配对自助检验，确认调度策略的统计显著性。参数调节包括候选数、验证层数和阈值，确保策略的鲁棒性。
Results
调度策略在3,129个样例中实现85.13%的宏观准确率，参数加权令牌减少49.1%，优于传统采样和单一验证方法。最佳16多数投票中，准确率仅差0.01个百分点，参数令牌减少58.9%。调度在硬任务（如AIME 2025）表现尤为优越，有效集中强验证资源于难题，显著提升边际性能。统计检验确认其优越性，显示调度机制在实际推理场景中的潜力。
Applications
该方法适用于需要高效推理的教育、科研和工业场景，尤其在资源有限或实时性要求高的应用中。通过动态调度，减少不必要的模型调用，降低成本，同时保证推理质量。未来可结合多模态信息和强化学习，进一步优化调度策略，实现更广泛的智能系统成本控制。
Limitations & Outlook
当前调度策略依赖预定义阈值和验证层次，缺乏自适应调整机制，可能在极端或新颖任务中表现不足。模型在复杂推理中仍存在候选筛选不充分的问题，未来需引入更强的不确定性估计和候选生成优化。此外，实验环境受限于特定模型和硬件，泛化能力和实际部署效果仍待验证。
Plain Language Accessible to non-experts
想象你在厨房做饭，手里有很多食材（候选答案），你需要用最少的时间和材料做出最美味的菜。你先快速试几种调料（生成候选），用简单的味道测试（轻量验证）判断哪几种可能更好。如果味道差不多，你就不用再试了；如果味道还可以，就用更贵的调料（强验证）仔细品尝。这样，你既保证了菜的味道，又节省了调料（计算资源）。这个过程就像模型在推理时，先用低成本的方法筛选，再用高成本验证，最后做出决定。
ELI14 Explained like you're 14
你知道在厨房做饭时，有时候你会先试一点点调料，看看味道怎么样，然后决定要不要用更贵的调料，或者直接做完？如果味道已经很好了，你就不用再试了；如果还差一点，就多试几次。这种做法可以节省时间和材料，还能做出好吃的菜。模型推理也一样，它会先用简单的方法筛选答案，然后用更复杂、更贵的验证器确认，最后决定哪个答案是最好的。这样既省钱又快，还能保证答案的质量。
Glossary
Compute-Balanced Routing (计算平衡调度)
一种在模型推理中动态分配计算资源的方法，旨在在保证准确率的同时降低成本。
论文提出的核心调度策略。
Parameter-Weighted Tokens (参数加权令牌)
根据模型参数规模调整的令牌成本度量，用于衡量不同模型调用的相对成本。
评估模型推理成本的重要指标。
Lightweight Verification (轻量验证)
成本较低的验证方法，用于快速评估候选答案的合理性。
在调度策略中作为筛选层使用。
Deep Verifier (深度验证器)
成本较高但准确性更高的验证模型，用于最终确认候选答案。
在候选路由中被优先调度到此验证层。
Answer Diversity (答案多样性)
生成多个不同候选答案的能力，用于提高模型推理的全面性。
候选生成阶段的重要指标。
Open Questions Unanswered questions from this research
- 1 如何在极端任务中自适应调节验证层次和阈值，确保模型在不同场景下都能高效工作。
- 2 未来调度策略如何结合强化学习实现更智能的资源分配。
- 3 在多模态推理中，调度机制如何扩展以支持图像、视频等多模态输入。
Applications
Immediate Applications
教育智能答题系统
利用调度策略优化推理过程，提升答题准确率同时降低计算成本，适合在线教育平台。
科研辅助推理工具
在复杂数学和符号推理中实现高效推理，帮助科研人员快速验证假设，节省资源。
Long-term Vision
智能决策系统
未来可在自动驾驶、金融决策等领域实现动态资源调度，提升系统效率和安全性。
Abstract
Test-time scaling is often implemented by spending more compute along one axis: sampling more solutions, extending a chain of thought, or applying a stronger evaluator. Under a fixed inference budget, these choices compete. This paper formulates test-time reasoning as a compute-allocation problem in which a system must decide whether the next unit of compute should be spent on generation, verification, or stopping. We introduce CoBa, a compute-balanced routing policy that first obtains a small set of candidates, applies cheap verification broadly, and routes uncertain or high-value candidates to stronger verification. On 3,129 example-generator evaluations spanning MATH-500, AIME 2024/2025, AMC 2023, and procedural symbolic reasoning, CoBa-Routed-Strong reaches 85.13% macro accuracy, statistically matching a self-evaluation weighted-voting proxy at 85.20% while using 49.1% fewer parameter-weighted tokens. It also matches best-of-16 majority voting within 0.01 macro-accuracy points while using 58.9% fewer parameter-weighted tokens; paired tests retain a small best-of-16 edge at substantially higher cost. Paired bootstrap tests show significant gains over single-sample decoding, while the remaining gap to the pool oracle exposes headroom for sharper routing. For local reasoning systems, test-time scaling becomes a question of where the next computation is most valuable.