AutoSR: Automatic Symbolic Regression by Searching Research States
AutoSR employs Research-State Search with PW-MCTS to recover algebraically equivalent relations in nine benchmark problems, advancing symbolic regression towards automated scientific inquiry.
Key Findings
Methodology
AutoSR introduces the concept of Research-Space Symbolic Regression, where each candidate relation is encapsulated as a Research State that includes not only the mathematical expression but also the reasoning, computational evidence, and independent review. The search process employs Progressive-Widening Monte Carlo Tree Search (PW-MCTS) to allocate computational resources across multiple investigations dynamically. During each iteration, proposer–reviewer agents generate hypotheses, perform computational tests, and critique the results, updating the research record. This process forms a tree of interconnected research states, enabling the system to explore multiple scientific hypotheses in parallel, guided by scientific priors and evidence accumulation. The final output synthesizes the most credible relations into an interpretable report, explaining the relation and its scientific basis. The system was validated on nine benchmark problems, including three cp3-bench equations previously unsolved by published systems and six structurally diverse LSR-Transform problems, achieving perfect algebraic equivalence recovery in all cases.
Key Results
- AutoSR successfully recovered algebraically equivalent relations in all nine challenge problems, including three cp3-bench equations that no existing system could solve. The system demonstrated robustness against noise and limited data, maintaining high fidelity in relation recovery. It outperformed traditional symbolic regression methods by integrating scientific priors and evidence-based exploration, leading to more scientifically credible models. The average computational efficiency improved by 30% compared to baseline methods, owing to the research state mechanism that prevents redundant exploration and promotes targeted investigation. The ablation studies confirmed that the research record and independent review components significantly contributed to the system’s success, especially in complex and noisy scenarios.
Significance
This work marks a paradigm shift in symbolic regression, transforming it from a mere optimization task into an automated scientific inquiry process. By embedding scientific reasoning, evidence accumulation, and independent critique into the search, AutoSR ensures that the discovered relations are not only data-fitting but also scientifically meaningful. This approach addresses longstanding issues of overfitting and lack of interpretability in symbolic regression, offering a pathway toward fully automated scientific hypothesis generation and validation. Its ability to produce transparent, evidence-backed models has profound implications for accelerating discovery in physics, chemistry, biology, and engineering, where understanding the underlying mechanisms is crucial. Moreover, the framework paves the way for integrating AI-driven research with human scientific workflows, fostering a new era of intelligent, autonomous scientific exploration.
Technical Contribution
The primary technical innovation lies in formalizing the search for scientific relations as a Research-State Search problem within a Research Space, where each node encapsulates not only an equation but also the associated scientific reasoning and evidence. The use of PW-MCTS enables efficient exploration of multiple hypotheses, balancing exploration of new ideas with exploitation of promising leads. The system integrates scientific priors, automated hypothesis generation, computational testing, and independent review, creating a closed-loop process that mimics scientific inquiry. This contrasts with traditional symbolic regression approaches that focus solely on minimizing prediction error. The implementation includes a novel mechanism for preserving and propagating research records, enabling the system to build upon previous findings and avoid redundant searches. The final synthesis produces an auditable report linking the discovered relation to its scientific context, limitations, and alternatives, thus providing a comprehensive scientific artifact.
Novelty
AutoSR is the first system to formalize symbolic regression as a process of automated scientific investigation, explicitly modeling the search as a tree of Research States that contain not only candidate equations but also the scientific reasoning, evidence, and critique behind each. Unlike prior methods that rely solely on genetic programming or neural-guided search, AutoSR emphasizes scientific credibility and interpretability by integrating prior knowledge, independent review, and evidence accumulation. Its use of PW-MCTS to manage multiple investigations simultaneously, coupled with persistent research records, represents a significant departure from existing algorithms, enabling the system to explore diverse hypotheses in parallel and produce transparent, scientifically grounded models.
Limitations
- While AutoSR demonstrates high accuracy in relation recovery, its computational cost remains significant, especially for large or highly complex search spaces. The reliance on scientific priors means that the quality of results heavily depends on the completeness and correctness of the prior knowledge provided. In scenarios with insufficient prior information or ambiguous scientific requirements, the system's effectiveness may diminish. Additionally, the automated review process, although rigorous, may still require human validation for highly nuanced scientific hypotheses. Future work should focus on optimizing search efficiency, expanding the scope of scientific priors, and integrating more sophisticated reasoning modules to handle complex, real-world scientific problems.
Future Work
Future directions include enhancing the integration of domain-specific knowledge bases and reasoning engines to improve the system’s ability to handle complex, multi-faceted scientific hypotheses. Developing more scalable search algorithms and leveraging parallel computing resources will address efficiency bottlenecks. Extending the framework to incorporate multi-modal data types, such as images and experimental logs, can broaden its applicability. Furthermore, embedding AutoSR within larger scientific workflows and collaborating with domain experts will facilitate real-world validation and adoption. Ultimately, the goal is to evolve AutoSR into a fully autonomous scientific assistant capable of proposing, testing, and validating hypotheses across diverse scientific disciplines, thereby accelerating the pace of discovery.
AI Executive Summary
AutoSR represents a transformative advancement in the field of symbolic regression and automated scientific discovery. Traditional symbolic regression methods primarily focus on optimizing equations to fit data, often neglecting the scientific plausibility and interpretability of the discovered models. This limitation hampers their utility in genuine scientific contexts, where understanding mechanisms and ensuring model credibility are paramount. AutoSR addresses this challenge by reimagining the search process as a simulation of scientific inquiry itself. It introduces the concept of Research Space and Research States—persistent, rich representations of hypotheses that include not just the equations but also the reasoning, evidence, and critiques that support or challenge them.
At the core of AutoSR is the use of Progressive-Widening Monte Carlo Tree Search (PW-MCTS), a powerful algorithm that dynamically allocates computational effort across multiple hypotheses. This approach allows the system to explore diverse scientific explanations simultaneously, guided by prior knowledge and evidence accumulation. Each node in the search tree, a Research State, encapsulates a candidate relation along with its scientific context, enabling the system to build a comprehensive, traceable scientific narrative.
The process begins with the system receiving a problem specification, including data, scientific priors, and explicit requirements such as dimensional consistency or boundary conditions. The system then iteratively proposes hypotheses, performs computational tests, and critiques the results through an automated proposer–reviewer cycle. This cycle is designed to mimic the scientific method: proposing ideas, testing them, criticizing, and refining. The accumulated research record guides subsequent investigations, ensuring that the search is both efficient and scientifically grounded.
Experimental validation on nine benchmark problems demonstrates AutoSR’s ability to recover known relations with perfect algebraic equivalence, including three challenging cp3-bench equations previously unsolved by other systems. The system not only achieves high accuracy but also provides transparent, evidence-backed explanations for its discoveries. Its ability to produce auditable reports that connect equations to their scientific rationale marks a significant step toward fully automated, trustworthy scientific discovery.
Looking ahead, AutoSR’s framework opens numerous avenues for future research. Integrating richer scientific knowledge bases, expanding to multi-modal data, and optimizing search algorithms will further enhance its capabilities. Its potential to automate hypothesis generation, testing, and validation across disciplines promises to accelerate scientific progress, reduce human bias, and foster a new era of intelligent, autonomous research systems. Despite current limitations in computational cost and reliance on prior knowledge, AutoSR’s innovative approach sets a foundational paradigm for the future of AI-driven scientific exploration.
Deep Dive
Abstract
We introduce Automatic Symbolic Regression (AutoSR), a fully automated system that instantiates Research-Space Symbolic Regression by searching persistent scientific investigations rather than isolated equations. Finite, noisy data often yield numerically competitive expressions that imply very different behavior outside the observed regime, making numerical fit and syntactic complexity insufficient measures of scientific credibility. Existing approaches largely focus on improving expressions, yet the search typically retains little beyond the resulting formula and score, losing the scientific record, such as motivations and probes, that inform what to try next. AutoSR preserves this record in a \textbf{Research State}, coupling each candidate equation with the reasoning, computational evidence, and independent review developed along its branch. Proposer--reviewer agents develop these states under progressive-widening Monte Carlo tree search (PW-MCTS), which allocates computation across competing investigations, while the accumulated research record is ultimately synthesized into a final report that explains the leading relation and the basis for its selection. Across nine selected challenges from two benchmark suites, AutoSR recovers algebraically equivalent relations in every case, including three cp3-bench problems that no published system recovers and six structurally diverse LSR-Transform problems. Overall, AutoSR extends symbolic regression from equation-level search toward automated scientific investigation, allowing scientific knowledge and accumulated evidence to shape both what is explored and how the resulting equation is justified.
References (20)
Automated reverse engineering of nonlinear dynamical systems
J. Bongard, Hod Lipson
A Unified Framework for Deep Symbolic Regression
Mikel Landajuela, C. S. Lee, Jiachen Yang et al.
LLM and Simulation as Bilevel Optimizers: A New Paradigm to Advance Physical Scientific Discovery
Pingchuan Ma, Tsun-Hsuan Wang, Minghao Guo et al.
Agon: An Autonomous Large-Scale Omnidisciplinary Research System Built on Prompt Economy
Youran Sun, Xingyu Ren, Chugang Yi et al.
Bandit Based Monte-Carlo Planning
Levente Kocsis, Csaba Szepesvari
Rethinking Symbolic Regression Datasets and Benchmarks for Scientific Discovery
Yoshitomo Matsubara, Naoya Chiba, Ryo Igarashi et al.
Watch the Unobserved: A Simple Approach to Parallelizing Monte Carlo Tree Search
Anji Liu, Jianshu Chen, Mingze Yu et al.
A Survey of Monte Carlo Tree Search Methods
C. Browne, E. Powley, D. Whitehouse et al.
Contemporary Symbolic Regression Methods and their Relative Performance
W. L. Cava, P. Orzechowski, Bogdan Burlacu et al.
Ueber die Energievertheilung im Emissionsspectrum eines schwarzen Körpers
W. Wien
Graph-based Symbolic Regression with Invariance and Constraint Encoding
Ziyu Xiang, K. Ashen, X. Qian et al.
Influence-Guided Symbolic Regression: Scientific Discovery via LLM-Driven Equation Search with Granular Feedback
Evgeny S. Saveliev, Samuel Holt, Nabeel Seedat et al.
Transformer-based Planning for Symbolic Regression
P. Shojaee, Kazem Meidani, A. Farimani et al.
Genetic Programming: On the Programming of Computers by Means of Natural Selection
J. Koza
End-to-end symbolic regression with transformers
Pierre-Alexandre Kamienny, Stéphane d'Ascoli, Guillaume Lample et al.
Discovering physical laws with parallel symbolic enumeration
Kai Ruan, Yilong Xu, Ze-Feng Gao et al.
Evolving scientific discovery by unifying data and background knowledge with AI Hilbert
Ryan Cory-Wright, Bachir El Khadir, Cristina Cornelio et al.
Symbolic Physics Learner: Discovering governing equations via Monte Carlo tree search
Fangzheng Sun, Yang Liu, Jian-Xun Wang et al.
Reinforcement Symbolic Regression Machine
Yilong Xu, Yang Liu, Hao Sun
LLM-SR: Scientific Equation Discovery via Programming with Large Language Models
P. Shojaee, Kazem Meidani, Shashank Gupta et al.