EnterpriseBench Corecraft: Training Generalizable Agents on High-Fidelity RL Environments
Training AI agents in Corecraft high-fidelity environment, GLM 4.6 task pass rate improved from 25.37% to 36.76%.
Key Findings
Methodology
The study employs the high-fidelity Corecraft environment, using Group Relative Policy Optimization (GRPO) and adaptive clipping to train the GLM 4.6 model. Corecraft simulates a complex customer support enterprise with over 2,500 entities and 23 tools, designed to evaluate AI agents' performance in multi-step, domain-specific tasks.
Key Results
- GLM 4.6 improved task pass rate from 25.37% to 36.76% after one epoch of training in the Corecraft environment.
- Achieved +4.5%, +7.4%, and +6.8% improvements on BFCL Parallel, Tau2-Bench Retail, and Tool Decathlon benchmarks, respectively.
- Demonstrated learning improvements in multi-step workflow execution, constraint handling, and response quality.
Significance
The study shows that environment quality, diversity, and realism are key to enabling generalizable AI agent capabilities. Training in a complex enterprise simulation environment not only enhances task-specific performance but also demonstrates excellent transferability to out-of-distribution tasks.
Technical Contribution
This research applies GRPO and adaptive clipping in a high-fidelity environment, showcasing the feasibility of training AI agents in complex enterprise settings. Detailed rubric-based evaluations provide reliable reward signals, enhancing model learning.
Novelty
First to train AI agents in an enterprise-level high-fidelity environment, demonstrating transferability to out-of-distribution tasks. Unlike simplified simulation environments, Corecraft offers more realistic enterprise workflows.
Limitations
- Models still show poor search strategy in some tasks, failing to effectively utilize tools.
- Failure to recognize pagination signals when handling incomplete results.
- Limited tool exploration, failing to fully utilize all available tools.
Future Work
Future research could explore applying high-fidelity environments to more domains and further optimize model performance in complex tasks.
AI Executive Summary
In AI agent training, traditional simplified simulation environments often fail to capture real-world complexity, leading to poor model performance in practical applications. To address this, the research team developed Corecraft, a high-fidelity enterprise simulation environment that mimics a complex customer support organization. By training the GLM 4.6 model in this environment, the study demonstrated significant improvements in multi-step task performance and achieved excellent transferability to out-of-distribution benchmarks.
Corecraft's design is based on three core principles: task-centric design, expert-authored evaluation rubrics, and realistic enterprise workflows. These designs make the environment suitable not only for training but also for effectively evaluating model generalization capabilities. Using Group Relative Policy Optimization (GRPO) and adaptive clipping, the model showed remarkable performance improvements, significantly increasing task pass rates.
Despite significant progress, the model still shows limitations in tool usage and result handling. Future research will continue to optimize model learning strategies and explore the application of high-fidelity environments in more domains.
Deep Analysis
Background
In recent years, AI agents have made significant progress in reinforcement learning. However, many existing training environments are overly simplified and fail to reflect real-world complexity, leading to poor model performance in practical applications and limited generalization capabilities.
Core Problem
Existing AI agent training environments are often overly simplified, failing to capture the complexity of real enterprise workflows. This leads to poor model performance in practical applications and difficulty adapting to out-of-distribution tasks.
Innovation
Corecraft's innovation lies in its high-fidelity simulation, providing a complex enterprise environment with over 2,500 entities and 23 tools. Expert-authored evaluation rubrics enable effective assessment of model performance in multi-step tasks.
Methodology
- �� Train in Corecraft environment
- �� Use GRPO and adaptive clipping for optimization
- �� Provide reward signals through expert-authored evaluation rubrics
- �� Conduct out-of-distribution benchmark tests post-training
Experiments
Experiments used 1,000 training tasks and 150 evaluation tasks from the Corecraft environment. Post-training, the model underwent out-of-distribution benchmark tests, including BFCL Parallel, Tau2-Bench Retail, and Tool Decathlon.
Results
GLM 4.6 improved task pass rate from 25.37% to 36.76% after one epoch of training in the Corecraft environment. On out-of-distribution benchmarks, the model achieved +4.5%, +7.4%, and +6.8% improvements on BFCL Parallel, Tau2-Bench Retail, and Tool Decathlon, respectively.
Applications
The study's results can be applied to enterprise environments requiring complex task handling, such as customer support, order management, and data analysis. Training in high-fidelity environments enables models to better adapt to practical application scenarios.
Limitations & Outlook
Despite the model's excellent performance post-training, there are still limitations in tool usage and result handling. Future research needs to continue optimizing model learning strategies and explore applications in more domains.
Plain Language Accessible to non-experts
Imagine you're working in a complex company, like an online computer parts retailer. Your job is to help customers solve problems, such as order inquiries or product compatibility checks. Corecraft is like a virtual company that simulates all these complex workflows. By training AI agents in this virtual company, we can teach them how to handle these complex tasks. It's like learning to cook in a kitchen, where AI agents need to learn how to use different tools and ingredients to complete a dish. In Corecraft, AI agents learn how to use different tools to solve customer problems and perform well in other similar tasks after training.
ELI14 Explained like you're 14
Imagine you're playing a super complex simulation game that mimics a computer parts store. You need to help customers solve various problems, like finding orders or checking product compatibility. Corecraft is this kind of virtual world, helping AI agents learn how to work in such complex environments. By training in this virtual world, AI agents can learn how to handle complex tasks, just like leveling up in a game. After training, AI agents not only perform well in this virtual world but also show excellent abilities in other similar tasks.
Glossary
Corecraft
A high-fidelity enterprise simulation environment used to train and evaluate AI agents in complex tasks.
Used to simulate complex workflows of a customer support organization.
GRPO
Group Relative Policy Optimization, a reinforcement learning algorithm used to optimize AI agent training.
Used to train the GLM 4.6 model in the Corecraft environment.
Adaptive Clipping
A technique to prevent entropy collapse during training, enhancing model stability.
Used in conjunction with GRPO to optimize the training process.
High-Fidelity Environment
An environment that simulates realistic enterprise workflows, providing diverse and challenging tasks.
Used to train AI agents to improve their generalization capabilities.
Task Pass Rate
A metric measuring model performance in specific tasks, requiring all rubric criteria to be satisfied.
Used to evaluate GLM 4.6 performance in the Corecraft environment.
Open Questions Unanswered questions from this research
- 1 How to apply high-fidelity environment training to more domains to enhance generalization capabilities.
- 2 How to optimize model tool usage strategies in complex tasks.
- 3 How to further improve model transferability to out-of-distribution tasks.
Applications
Immediate Applications
Customer Support
Apply AI agents in complex customer support environments to help solve multi-step problems and improve efficiency.
Long-term Vision
Enterprise Automation
Achieve comprehensive automation of enterprise processes by training AI agents in high-fidelity environments, enhancing productivity.
Abstract
We show that training AI agents on high-fidelity reinforcement learning environments produces capabilities that generalize beyond the training distribution. We introduce CoreCraft, the first environment in EnterpriseBench, Surge AI's suite of agentic RL environments. CoreCraft is a fully operational enterprise simulation of a customer support organization, comprising over 2,500 entities across 14 entity types with 23 unique tools, designed to measure whether AI agents can perform the multi-step, domain-specific work that real jobs demand. Frontier models such as GPT-5.2 and Claude Opus 4.6 solve fewer than 30% of tasks when all expert-authored rubric criteria must be satisfied. Using this environment, we train GLM 4.6 with Group Relative Policy Optimization (GRPO) and adaptive clipping. After a single epoch of training, the model improves from 25.37% to 36.76% task pass rate on held-out evaluation tasks. More importantly, these gains transfer to out-of-distribution benchmarks: +4.5% on BFCL Parallel, +7.4% on Tau2-Bench Retail, and +6.8% on Tool Decathlon (Pass@1). We believe three environment properties are consistent with the observed transfer: task-centric world building that optimizes for diverse, challenging tasks; expert-authored rubrics enabling reliable reward computation; and enterprise workflows that reflect realistic professional patterns. Our results suggest that environment quality, diversity, and realism are key factors enabling generalizable agent capabilities.