Toward Generalist Autonomous Research via Hypothesis-Tree Refinement

Scientific research has evolved from manual experimentation to automated AI-assisted discovery, with systems like AutoML, AI Scientist, and MARS pushing automation boundaries. However, these systems often lack long-term memory, multi-path exploration, and evidence integration, limiting their ability to conduct sustained, cumulative research. Recent advances in large language models enable code editing, tool invocation, and extended experimentation, but integrating these capabilities into a coherent, long-horizon research framework remains challenging. Existing systems tend to focus on isolated tasks or short-term optimization, unable to maintain a persistent research state that captures hypotheses, evidence, and insights over time. This gap motivates the development of a structured, evidence-based approach to autonomous research.

The core challenge is to design an AI system capable of managing multiple research hypotheses simultaneously, integrating evidence over long periods, and making strategic decisions about exploration, merging, or pruning. Traditional methods lack mechanisms for persistent memory and structured hypothesis management, leading to fragmented knowledge and inefficient search. Achieving reliable, verifiable progress requires a system that can maintain a long-term, auditable record of experiments, learn from failures, and transfer knowledge across tasks. Balancing exploration and exploitation while ensuring evidence validity and transferability is a fundamental difficulty that this work aims to address.

The key innovations include: 1) Hypothesis Tree as a persistent, layered structure that encodes hypotheses, evidence, and insights, enabling multi-path exploration; 2) A long-term coordinator that manages the tree, guiding exploration, merging, and pruning based on validation metrics; 3) Evidence abstraction and insight backpropagation, which turn experimental results into reusable knowledge; 4) An open-source system that supports multi-task, long-horizon research. These elements collectively enable continuous knowledge accumulation, verifiable improvements, and scalable exploration, setting a new standard for autonomous scientific systems.

• �� Construct a hypothesis tree where each node contains a hypothesis, evidence, and insights, supporting layered exploration. • The coordinator observes the current tree, proposes new hypotheses, selects promising leaves, and dispatches executors. • Executors implement hypotheses in isolated environments, returning structured evidence, scores, and insights. • Evidence is written back to nodes, and insights are abstracted and propagated upward, updating the tree. • The coordinator evaluates candidate improvements using held-out metrics, merging or pruning branches accordingly. • The process iterates, expanding the tree, accumulating knowledge, and refining artifacts. • Experiments are conducted across six tasks, with metrics like loss reduction, accuracy, and validation scores, demonstrating effectiveness.

The experimental setup involves six real-world research tasks: model training, hardware tuning, and data synthesis, with defined initial artifacts, objectives, and evaluation metrics. Baselines include Codex and Claude Code, with resource budgets matched. Hyperparameters such as tree depth, exploration strategies, and evidence abstraction methods are tuned. Ablation studies isolate the impact of hypothesis tree structure, evidence backpropagation, and strategic decision gates. Results show consistent performance improvements, with average gains exceeding 2.5×, validating the hypothesis tree's role in long-term knowledge retention and strategic exploration. The experiments also demonstrate robustness across diverse tasks, confirming the framework's generality.

Arbor reduces model training loss by 6.32%, improves architecture scores by 7.55%, and enhances data synthesis efficiency by 13%. In math reasoning, scores increase by 19.79%. On MLE-Bench Lite, Arbor with GPT-5.5 achieves 86.36% Medal, outperforming all baselines. Ablation results confirm that hypothesis tree structure, evidence backpropagation, and strategic pruning are critical for performance. The system's ability to accumulate and transfer knowledge over long horizons results in sustained improvements, validating the core design principles. These results demonstrate that evidence-structured, multi-path exploration significantly outperforms traditional methods in complex scientific tasks.

Arbor can be applied to drug discovery, materials science, automated data analysis, and other research-intensive fields requiring multi-stage hypothesis testing. Its ability to manage long-term knowledge and verify artifact improvements makes it suitable for complex, multi-faceted problems. The framework can be integrated into existing AI-assisted research pipelines, enhancing their capacity for sustained, verifiable progress. Long-term, it envisions AI systems acting as persistent scientific partners, accelerating innovation cycles, and reducing human workload in experimental design and analysis.

Current limitations include scalability issues in extremely high-dimensional hypothesis spaces, where the tree may grow exponentially. Computational costs remain high, especially for large models and datasets, limiting real-world deployment. The system's effectiveness depends on well-designed hypotheses and evidence abstraction, which may require domain expertise. Future work should focus on adaptive pruning, resource-efficient algorithms, and multi-modal data integration to handle more complex scientific problems and improve scalability.