Prime Agent: A Self-Improving RLM Harness

The evolution of large language models (LLMs) like GPT-4, Claude, and PaLM has revolutionized NLP and code synthesis. However, their capacity for long-horizon reasoning remains limited by context window sizes, fragmented information management, and lack of persistent memory. Prior efforts, including OpenAI Codex and DeepMind’s Gato, have introduced multi-task learning and multi-agent frameworks, yet none fully address the integration of recursive invocation, persistent state, and dynamic multi-agent orchestration for autonomous long-term tasks. Recent research emphasizes recursive language models (RLM) and continual learning, but practical implementations often lack standardized evaluation platforms or scalable architectures. Prime Agent responds to these gaps by proposing a unified, extensible system that combines RLM-based programmatic control with persistent storage and multi-agent communication, enabling models to perform complex, multi-stage tasks over extended periods with minimal human intervention.

Current language models struggle with long-horizon tasks due to limited context windows, fragmented information management, and inability to sustain multi-step reasoning over days or weeks. These limitations hinder autonomous systems from performing complex, multi-phase activities such as scientific research, software development, or strategic planning. Existing solutions often rely on manual intervention, static workflows, or isolated modules, which restrict scalability and adaptability. The core challenge is to develop a system that can dynamically manage information, coordinate multiple sub-agents, and self-improve through trajectory refinement, all while maintaining resource efficiency and robustness in real-world environments.

Prime Agent’s key innovations include: 1) a hierarchical state organization integrating model weights, active context, persistent disk storage, and trajectory histories, enabling continuous information flow; 2) recursive language models (RLM) that facilitate programmatic invocation, subagent management, and dynamic scheduling; 3) a persistent IPython REPL supporting test-time compute and programmatic control; 4) multi-agent communication channels allowing direct, asynchronous interaction among subagents and with human operators; 5) trajectory versioning and continual refinement mechanisms that support self-improvement and knowledge accumulation. These innovations collectively enable a flexible, scalable, and autonomous AI system capable of long-term reasoning and adaptation, surpassing prior approaches limited to static workflows or single-pass inference.

• �� Implement RLM primitives for programmatic invocation, enabling recursive subagent calls and dynamic task decomposition.
• �� Organize system state into multiple levels: L0 (model weights), L1 (active context), L2 (runtime session with Python and tools), L3 (disk-backed history, memories, skills).
• �� Use a persistent IPython REPL for test-time compute, allowing model inference, code execution, and tool calls with retained intermediate values.
• �� Facilitate recursive sessions with direct asynchronous communication via daemon-mediated queues, supporting multi-agent collaboration.
• �� Incorporate trajectory refinement, version control, and self-improvement by converting execution evidence into persistent, reusable state.
• �� Standardize execution, recovery, verification, and resource accounting to ensure robustness and maximal utilization of model capabilities.
• �� Provide visualization tools (Agents View) for human oversight and intervention.

The system was evaluated on ARC-AGI-3, Factorio, EmulatorBench, and MazeBench, using standard benchmarks for reasoning, long-context code generation, emulator reconstruction, and spatial reasoning. Baselines included traditional harnesses and open-source tools. Metrics measured included task success rate, resource consumption, trajectory quality, and solution rate. Experiments involved multi-day autonomous runs, ablation studies removing key components, and cross-scenario comparisons. Hyperparameters such as model size, subagent count, and session depth were tuned for optimal performance. Results demonstrated significant improvements over baselines, with high scores in complex, multi-step tasks and robust recovery from disruptions.

Prime Agent achieved a 95.5% RHAE score on ARC-AGI-3, far exceeding previous bests at 30%. It maintained continuous progress in Factorio over 85.5 hours, completing 24 technologies and reaching 71% on advanced circuits. Emulator reconstruction success rates exceeded 60%, with accurate replication of SEGA Genesis and Game Boy Color systems. In long-context reasoning tasks, Prime Agent outperformed native and alternative harnesses, demonstrating superior information management and multi-agent coordination. These results validate the system’s capacity for sustained, autonomous, long-horizon reasoning and self-improvement.

Prime Agent can be applied in autonomous scientific research, complex software development, long-term strategic planning, and robotic control. Its ability to manage persistent knowledge, coordinate multiple agents, and self-refine makes it suitable for industries requiring continuous adaptation and optimization. It can facilitate automated system debugging, multi-stage project management, and real-world environment exploration, provided sufficient computational resources and security measures are in place.

High computational and storage demands limit scalability, especially with many subagents and long trajectories. Security concerns arise from persistent data storage and communication channels, necessitating robust safeguards. The reliance on pre-trained models constrains adaptability to unforeseen environments or novel tasks. Additionally, the system’s efficiency in extremely complex or unpredictable scenarios needs further validation. Future work should focus on resource optimization, security enhancements, and broader applicability to real-world autonomous systems.