ScienceWorld: Is your Agent Smarter than a 5th Grader?

The evolution of AI in scientific reasoning has transitioned from rule-based systems to large-scale pre-trained language models like GPT-3 and BERT, which excel in knowledge retrieval and surface-level understanding. However, these models lack the ability to perform physical manipulations or procedural experiments essential for true scientific reasoning. Interactive environments such as TextWorld and KitchenWorld have provided platforms for training models through simulated tasks, but their scope remains limited to simple object manipulation or basic puzzles. Recent efforts aim to incorporate physics simulations, yet most are constrained by low fidelity or narrow task sets. The challenge remains to develop environments that can support multi-domain physics, complex experiment design, and explainable reasoning, bridging the gap between static knowledge and dynamic understanding.

Existing models struggle with generalizing scientific reasoning to unseen scenarios, especially when procedural actions are required. The core bottleneck is the lack of environment-grounded training that can teach models to understand cause-effect relationships, manipulate objects, and generate explanations for their actions. Without such capabilities, AI cannot autonomously perform multi-step experiments or adapt to novel materials and conditions, limiting their utility in real-world scientific applications. This gap hampers progress in AI-assisted research, education, and automated experimentation, necessitating new frameworks that integrate physics simulation, procedural reasoning, and explainability.

The key innovations include: 1) Developing a physics-rich interactive environment (ScienceWorld) that models thermodynamics, circuits, chemistry, and biology, enabling realistic experiment simulations; 2) Transforming scientific reasoning into an action-based task where models perform step-by-step procedures, with action sequences serving as explicit explanations; 3) Combining reinforcement learning with imitation learning from expert trajectories to train small, parameter-efficient models that outperform large static models. These innovations address the limitations of prior static QA models by embedding reasoning within a physical context, fostering better generalization and interpretability.

• �� Build a physics-based simulation engine supporting thermodynamics, electrical circuits, chemistry, and biological processes.
• �� Design 30 diverse science tasks with multiple variations to test generalization.
• �� Collect expert demonstrations for imitation learning, guiding models on canonical procedures.
• �� Train models using reinforcement learning to optimize task completion, with action sequences serving as explanations.
• �� Evaluate models on unseen variations, measuring task success scores.
• �� Compare small (150k params) and large (11B params) models, analyzing the impact of environment interaction.
• �� Use automatic validation of action sequences within the simulator to assess correctness.

The experiments involve training multiple models—DRRN, CALM, T5-based agents—on the ScienceWorld environment with 30 tasks across variations. Metrics include average task scores, success rates in experiments like electrical conductivity and phase change. Hyperparameters include 8 environment threads, 100k training steps, and evaluation on unseen task variants. Ablation studies compare environment-grounded training versus static models, demonstrating the benefits of interactive learning. Results show small models trained interactively outperform large models trained statically, especially in novel scenarios, validating the environment's role in enhancing reasoning.

The 150k-parameter model trained for 100k steps achieves an average score of 0.17, outperforming the 11B static model (score 0.05). In conductivity tasks, models correctly identify conductors and successfully build circuits, with success rates exceeding 70%. The models show limited ability to generalize to unseen materials, indicating the need for richer physics modeling. Ablation results confirm environment interaction significantly boosts reasoning robustness, especially in complex, novel tasks. Overall, the findings underscore the importance of physical simulation and procedural training in advancing AI scientific reasoning.

This framework can be used to develop virtual laboratories for education, enabling students to perform experiments safely and interactively. It also supports AI-assisted research, automating experimental procedures and hypothesis testing. Industry applications include automated process design, quality control, and virtual prototyping, where AI models can simulate and optimize physical systems before real-world deployment. The environment's flexibility allows customization for diverse scientific domains, fostering innovation in training, testing, and deploying intelligent systems.

Current physics simulation simplifies real-world phenomena, limiting fidelity. Models exhibit poor generalization to entirely new materials or complex multi-step experiments. Training costs are high due to environment complexity, and the physics engine requires further refinement for realism. Additionally, reliance on expert demonstrations constrains autonomous learning. Future work should focus on improving physics fidelity, reducing training costs, and enabling models to generate experiments autonomously, moving toward more human-like scientific reasoning.