Recursive Harness Self-Improvement

The evolution of artificial intelligence has been marked by the scaling of model parameters, increased computational resources, and sophisticated post-training techniques such as fine-tuning and reinforcement learning. Landmark models like GPT-3, BERT, and T5 have demonstrated that larger models trained on vast datasets can achieve remarkable performance across NLP tasks. However, scaling alone faces diminishing returns and escalating costs, prompting researchers to explore more efficient pathways. One promising direction is the co-evolution of models and their harnesses—the structures that guide inference, such as prompts, workflows, and communication protocols. Recent works by Anthropic, OpenAI, and Google have shown that dynamically evolving harnesses can significantly enhance model performance, especially in multi-agent and task-specific settings. Nonetheless, optimizing these harnesses remains challenging due to the high costs of global search and the static nature of many existing designs. The need for lightweight, iterative, and task-adaptive harness optimization methods has become increasingly urgent, especially for real-world applications requiring continual adaptation and personalization.

The core challenge addressed in this paper is how to efficiently optimize user-constructed harnesses for specific tasks, using minimal computational resources and only a few update iterations. Traditional approaches rely on exhaustive search or large candidate pools, which are computationally expensive and impractical for frequent updates. This bottleneck hampers the ability of AI systems to perform continual learning and self-improvement, especially in open-ended, real-world scenarios where tasks vary widely and data is scarce. The fundamental problem is to develop an optimization framework that is both lightweight and effective, capable of guiding harnesses toward higher-quality execution traces without incurring prohibitive costs. Achieving this requires rethinking the optimization paradigm from global search to local, preference-guided refinement, leveraging the inherent structure of the harness as a prompt-level specification rather than executable code.

The paper introduces several key innovations: • Defining harness as a prompt-level specification, enabling lightweight, flexible optimization without code execution; • Developing a trajectory-local objective function that compares the current harness only with its immediate predecessor, drastically reducing computational costs; • Utilizing pairwise preference feedback derived from the agent's own outputs to iteratively refine the harness, forming a recursive self-improvement loop; • Formalizing an information-theoretic hypothesis that the implicit optimization goal is to maximize mutual information between harness components and the task, while minimizing internal redundancy; • Demonstrating through extensive experiments that only a few iterations of RHI can surpass the performance of models with maximal reasoning effort, while significantly reducing inference costs. These innovations collectively enable scalable, task-specific harness optimization suitable for continual learning scenarios.

• �� Define the harness as a prompt-level description including roles, instructions, communication contracts, and workflow hops; • Formulate a trajectory-local objective function that compares the current harness with the previous one, using pairwise preference feedback; • At each iteration, generate outputs using the current harness, then evaluate pairwise preferences between current and previous outputs; • Store preference feedback in a history buffer, which guides the next harness revision; • Use an LLM-based harness optimizer (Lharness) to update the harness based on accumulated preference data, without requiring explicit evaluation prompts; • Incorporate an information-theoretic perspective, aiming to maximize the mutual information between harness components and the task, while reducing redundancy; • Repeat this process for a few iterations until performance plateaus, leveraging cached outputs to minimize computational costs.

The experimental setup involves 30 synthetic, open-ended machine learning tasks spanning finance, robotics, and pharmacy. Each task requires generating a complete code repository with standardized deliverables, evaluated via pairwise comparisons by LLM judges. The models tested include Sonnet-4.6, Opus-4.7, and Opus-4.8, representing different scales of foundation models. Baselines include test-time scaling strategies such as xhigh, max, and ultracode. The RHI process involves 2-3 iterations per task, with performance measured by win rates in pairwise judgments and inference costs in terms of token usage and cache operations. Ablation studies examine the impact of preference feedback, caching, and the information-theoretic objective. The experiments demonstrate that RHI consistently improves performance across tasks and models, with notable reductions in inference costs, validating the efficiency and effectiveness of the approach.

The key results show that with just 2-3 iterations, RHI-enhanced agents outperform all baseline test-time scaling strategies, achieving higher win rates in pairwise comparisons across all 30 tasks. For example, in the Opus-4.8 model, RHI improved win counts by approximately 20% over ultracode and max baselines, with inference costs reduced by 50-60%. Output token usage remained stable, indicating that performance gains are primarily due to better context management and information flow rather than longer reasoning chains. Ablation experiments confirmed that preference feedback and cached outputs are critical for success, while the information-theoretic objective aligns with observed improvements. These results demonstrate that lightweight, preference-guided harness optimization can effectively push the performance ceiling of low-reasoning agents, making continual self-improvement feasible with minimal computational overhead.

RHI can be directly applied to tasks such as automated code generation, multi-agent coordination, and adaptive dialogue systems, where task-specific tuning is essential. Its lightweight nature makes it suitable for resource-constrained environments, enabling rapid customization and continual learning without extensive retraining. In industry, RHI can facilitate personalized AI assistants, autonomous robots, and intelligent automation platforms that adapt to new tasks with minimal overhead. Long-term, this approach could underpin self-improving AI ecosystems capable of autonomous evolution, reducing reliance on manual updates and enabling scalable deployment across diverse domains. Its ability to optimize information flow and context management also opens avenues for enhancing multi-modal and multi-task systems, pushing the boundaries of adaptive intelligence.

While RHI demonstrates impressive performance on synthetic tasks, its robustness and scalability in real-world, multi-modal, and multi-task environments require further validation. The reliance on high-quality preference feedback may introduce biases or noise, affecting convergence. Computational costs, though minimized, could still be significant in very large models or highly complex tasks, especially when frequent caching and comparisons are needed. The theoretical framework based on mutual information, while compelling, needs empirical validation across broader scenarios. Future work should address these limitations by integrating more robust feedback mechanisms, exploring multi-modal data, and developing more efficient mutual information estimation techniques. Additionally, extending RHI to multi-agent systems and real-world applications remains an important direction for research.