Test-Time Learning with an Evolving Library
EvoLib constructs an evolving knowledge library via self-supervised abstraction extraction, boosting performance in mathematical reasoning and code generation without parameter updates.
Key Findings
Methodology
EvoLib maintains a structured knowledge library, automatically extracting modular skills and reflective insights from model inference trajectories. It employs information gain and future information gain to dynamically weight and consolidate abstractions, enabling continuous evolution without gradient updates or external supervision. The framework iteratively samples abstractions, generates solutions, evaluates them via self-assessment, and updates the library through a process of abstraction extraction, consolidation, and weight adjustment. This self-supervised approach facilitates knowledge accumulation, reuse, and generalization across diverse tasks, including mathematical reasoning, code synthesis, and multi-turn agentic environments.
Key Results
- EvoLib achieves 15-20% accuracy improvements over baseline test-time scaling and learning methods across benchmarks. For example, on HMMT math questions, accuracy increased from 57% to 77.4%. In code tasks like BigCodeBench, pass rate rose from 29.7% to 40.8%. The method demonstrates robustness to task order in continual learning and outperforms existing TTL approaches like ExpRAG and DC, especially under limited compute budgets, showing higher cost-efficiency and knowledge reuse effectiveness.
Significance
This work addresses fundamental limitations in current large language models regarding knowledge retention and transfer during inference. By enabling models to autonomously extract, consolidate, and evolve knowledge without parameter updates, it paves the way for more autonomous, adaptable AI systems capable of continual learning in real-world scenarios. The approach reduces reliance on external supervision, making it applicable in privacy-sensitive or resource-constrained environments, and offers a scalable path toward lifelong learning in AI systems, with broad implications for automation, reasoning, and intelligent interaction.
Technical Contribution
The paper introduces a novel self-supervised framework that constructs a dynamic, structured knowledge library through abstraction extraction and evolution. It leverages information gain metrics to guide the addition and consolidation of knowledge, enabling nonlinear, continuous evolution of reusable modules and insights. The approach departs from traditional gradient-based TTL methods, offering a parameter-free, scalable solution that enhances knowledge transfer, generalization, and robustness across tasks. The theoretical formulation of the weighting and consolidation mechanisms, combined with empirical validation, constitutes a significant advancement in test-time learning research.
Novelty
This is the first framework to achieve knowledge evolution purely through self-supervised abstraction induction without gradient updates or external labels. It uniquely combines structured knowledge representations with dynamic, utility-driven evolution, addressing issues like knowledge interference, overfitting, and misgrouping present in prior memory-based methods. Its emphasis on nonlinear, continual knowledge evolution distinguishes it from linear or static memory approaches, marking a new paradigm in test-time learning and knowledge management for large models.
Limitations
- The effectiveness depends on the accuracy of self-assessment; biased or inaccurate evaluations may misguide knowledge extraction. The knowledge base may grow large over time, complicating management and retrieval. In highly complex or highly diverse tasks, the current abstraction mechanisms may struggle to generalize effectively, requiring external signals or domain-specific tuning. Computational costs for sampling and updating may increase with library size, necessitating efficient compression or pruning strategies.
Future Work
Future directions include integrating multi-modal knowledge sources, developing more scalable knowledge compression techniques, and enhancing interpretability. Exploring external supervision signals to guide abstraction refinement, applying the framework to real-world applications like robotics or healthcare, and extending the approach to multi-agent systems are promising avenues. Additionally, research into formal guarantees of knowledge evolution stability and robustness will further solidify its theoretical foundations.
AI Executive Summary
The rapid advancement of large language models (LLMs) has revolutionized natural language processing, enabling remarkable achievements in tasks like reasoning, coding, and dialogue. However, these models traditionally rely on static parameters, limiting their ability to adapt and learn from ongoing interactions without costly retraining. Existing test-time scaling techniques, such as sampling and iterative refinement, treat each problem independently, discarding insights after inference and hindering knowledge transfer across tasks. This gap restricts models from building a cumulative, generalized understanding akin to human learning.
Addressing this challenge, the paper introduces EvoLib, a novel framework that facilitates test-time learning through a dynamically evolving knowledge library. Instead of updating model weights, EvoLib extracts modular skills and reflective insights directly from the model’s inference trajectories. These abstractions are stored in a structured library, which is continuously refined using information gain metrics that evaluate their immediate utility and potential for future knowledge generation. This process enables the library to evolve nonlinearly, promoting the emergence of more general, reusable knowledge units over time.
Empirical evaluations across benchmarks like Harvard–MIT Mathematics Tournament, BigCodeBench, and multi-turn agentic tasks demonstrate that EvoLib significantly outperforms existing methods, achieving 15-20% accuracy improvements and better cost-efficiency. Ablation studies confirm that combining skills and insights yields the best results, with the knowledge evolution mechanism providing robustness against task order sensitivity. The approach's self-supervised nature makes it applicable in real-world scenarios where external supervision is unavailable, marking a substantial step toward autonomous, lifelong learning in AI.
Overall, EvoLib offers a scalable, effective solution for continuous knowledge accumulation and reuse, pushing the boundaries of what large models can achieve without traditional retraining. Its innovative use of abstraction and evolution mechanisms opens new avenues for developing more intelligent, adaptable AI systems capable of lifelong learning and generalization.
Deep Dive
Plain Language Accessible to non-experts
想象你在厨房里做饭，每次尝试一道新菜，你会记住用的材料、火候和调味料。随着不断尝试，你会总结出一些“菜谱”，这些菜谱可以帮你更快做出好菜。EvoLib就像这个厨房的厨师，它会在每次做菜时，把学到的技巧和错误总结成“菜谱”，存到一个“知识库”里。每次遇到新菜，它会从这个“知识库”里抽取合适的“菜谱”来帮忙做。随着时间推移，它会不断改进这些“菜谱”，让厨艺变得越来越好。这种方法不用重新学习所有内容，而是通过总结和改进已有的经验，变得越来越聪明。就像我们在生活中不断总结经验，变得更厉害一样，EvoLib让模型也能自己学习、变得更聪明。
Abstract
We introduce EvoLib, a test-time learning framework that enables large language models to accumulate, reuse, and evolve knowledge across problem instances without parameter updates or external supervision. Instead of adapting model parameters, our approach maintains a shared library of knowledge abstractions, including modular skills and reflective insights, automatically extracted from the model's own inference trajectories. To support continual improvement, we introduce a principled weighting and consolidation mechanism that jointly optimizes for immediate utility and long-term value. This allows simple, instance-specific abstractions to evolve into more general and reusable ones over time. Across challenging benchmarks in mathematical reasoning, code generation, and multi-turn agentic environments, EvoLib improves substantially over the top test-time scaling and learning methods without ground-truth feedback.