Knowledge-Centric Self-Improvement
Knowledge-centric self-improvement uses a curated knowledge base to enhance task solving and transferability, reducing costs by 30%.
Key Findings
Methodology
This paper introduces a knowledge-centric framework where a shared knowledge base is the core of self-improvement. It employs a three-stage protocol: task forum, cross-task discussion, and knowledge distillation. Agents, kept generic and stateless, perform tasks, then contribute evidence-based claims to the knowledge base. These claims are discussed and refined across tasks, with only supported, actionable insights distilled into reusable bundles. Experiments across abstract reasoning, coding, and terminal tasks demonstrate improved solve rates and cost efficiency, with knowledge transfer across models and tasks confirming robustness.
Key Results
- Across benchmarks like ARC-AGI-1/2, Polyglot, and Terminal-Bench 2, the protocol improved solve rates by over 15%, e.g., ARC-AGI-1 from 68% to 86.7%, and reduced costs by approximately 30%. The knowledge bundles transferred effectively across different models, such as gpt-4o, maintaining performance on unseen tasks.
- Compared to agent-centric and prompt optimization baselines, the knowledge curation approach achieved higher accuracy and lower costs, especially in complex tasks, with solution rates surpassing 43.8% in Terminal-Bench 2.
- Knowledge transfer experiments confirmed that curated knowledge remains effective across different models and task types, demonstrating its generalizability and practical utility.
Significance
This work shifts the paradigm from agent-centric to knowledge-centric self-improvement, emphasizing external knowledge bases as the primary driver of progress. It enhances system interpretability, transferability, and maintainability, addressing key challenges in scaling AI systems for diverse tasks. The approach offers a scalable, explainable, and cost-effective pathway for future AI development, with broad implications for multi-task learning and autonomous systems.
Technical Contribution
The paper proposes a structured protocol combining discussion forums and knowledge distillation, ensuring evidence-backed, verifiable knowledge accumulation. It maintains agent simplicity and statelessness, externalizing improvements into a shared knowledge base. Empirical results show superior performance over traditional agent-based methods, establishing the importance of externalized knowledge in self-improvement. The framework supports transfer across models and tasks, opening new avenues for scalable AI systems.
Novelty
This is the first systematic implementation of a knowledge-centric self-improvement paradigm, contrasting with prior agent-focused or memory-based methods. It emphasizes evidence-based discussion and distillation, externalizing learning into a reusable knowledge artifact. This shift from internal model updates to external knowledge curation fundamentally changes how AI systems can improve and transfer skills.
Limitations
- The protocol relies on high-quality evidence and active discussion, which may be challenging in ambiguous or less-structured tasks. Maintaining and updating the knowledge base incurs costs, especially at scale.
- Transferability across very different models or modalities remains to be fully validated. The approach’s effectiveness in highly dynamic or real-time environments needs further exploration.
- Scalability of knowledge curation and distillation processes could pose challenges as task complexity and data volume grow. Future work should focus on automating and optimizing these processes.
Future Work
Future research will explore automated knowledge generation, validation, and dynamic updating mechanisms. Integrating reinforcement learning to refine knowledge distillation and expanding to multi-modal data will further enhance system robustness. Additionally, scaling the framework to real-world applications like robotics and autonomous agents remains a promising direction.
AI Executive Summary
This study introduces a novel paradigm shift in AI self-improvement—moving from agent-centric optimization to a knowledge-centric approach. Traditional methods focus on continuously refining the agent’s prompts, code, or internal parameters, often leading to complex, tightly coupled systems that are expensive to maintain and difficult to transfer. In contrast, the proposed framework externalizes the improvement process into a shared, curated knowledge base, which acts as the core asset for future tasks.
The protocol involves three stages: agents attempt tasks with fresh contexts, contribute evidence-grounded claims to task-specific forums, engage in cross-task discussions to identify recurring principles, and finally distill these insights into compact, reusable knowledge bundles. This process ensures that improvements are evidence-based, verifiable, and transferable, independent of the agent’s internal architecture.
Empirical evaluations across diverse benchmarks—abstract reasoning (ARC-AGI-1/2), coding (Polyglot), and real-world terminal tasks (Terminal-Bench 2)—demonstrate that this approach outperforms traditional agent-centric methods in solve rates and cost efficiency. For instance, solution rates on ARC-AGI-1 improved from 68% to 86.7%, with a 30% reduction in costs. Knowledge bundles curated on one model (e.g., gpt-4o) transferred effectively to others, confirming the robustness and generality of the approach.
These results underscore the potential of externalized, evidence-based knowledge as the primary driver of AI progress. By decoupling improvement from agent design, the framework offers a scalable, interpretable, and cost-effective pathway for building autonomous, adaptable AI systems. Future directions include automating knowledge generation, expanding to multi-modal data, and integrating reinforcement learning to enhance knowledge quality and update mechanisms, paving the way for truly self-sustaining intelligent systems.
Deep Dive
Abstract
Self-improving AI systems typically treat the agent as the object that improves, by optimizing prompts, workflows, harnesses, or even the agent's own code. This agent-centric view can make improvements expensive to maintain and difficult to transfer, because gains become tied to a particular agent design, task distribution, or adaptation run. We study a complementary paradigm: knowledge-centric self-improvement, in which agents remain generic and disposable while the persistent object is a curated knowledge base that agents can leverage for future tasks. We conduct controlled case studies to operationalize this idea via a simple protocol. Agents attempt one task, then contribute evidence-grounded insights to a shared knowledge base via task-level and cross-task forums, followed by knowledge distillation. Because self-improvement is contained in the knowledge rather than the agent, improvement can be more inspectable, transferable, and portable. Across abstract reasoning, coding, and terminal benchmarks, this protocol improves solve rates while reducing dollar cost relative to agent-centric baselines. The resulting distilled knowledge also transfers to held-out tasks and across LLM families, indicating that the improvement is not merely an LLM- or run-specific behavior. These results support a new view of self-improving agentic systems: progress can be driven primarily by the curated persistent knowledge. Code is available at https://github.com/recursive-knowledge/KSI.