CME-CAD: Heterogeneous Collaborative Multi-Expert Reinforcement Learning for CAD Code Generation
CME-CAD introduces a multi-expert reinforcement learning framework for high-precision CAD code generation, trained on 17,300+ industrial-grade instances.
Key Findings
Methodology
CME-CAD employs a two-stage training process: Multi-Expert Fine-Tuning (MEFT) leverages diverse expert models to generate varied reasoning paths, enhancing model expressiveness; followed by Multi-Expert Reinforcement Learning (MERL), which optimizes outputs via multi-dimensional rewards—format, executability, geometric accuracy—and incorporates a hard negative sample buffer to improve robustness. The dataset CADExpert contains 17,299 instances with orthographic projections, expert reasoning chains, executable CADQuery code, and 3D models, aligning with industrial requirements. The framework exploits heterogeneous pre-trained models’ complementary strengths through collaborative learning, significantly improving CAD code accuracy and editability.
Key Results
- On the CADExpert dataset, CME-CAD achieves over 85% code correctness and 0.78 IoU, outperforming state-of-the-art methods by over 10%.
- In complex industrial operations such as drilling and chamfering, the model demonstrates superior robustness and diversity, generating constraint-compliant high-quality CAD models.
- The multi-expert collaboration mechanism effectively mitigates bias, enhancing exploration and accuracy; ablation studies confirm the contribution of each component to overall performance.
Significance
This work advances the automation of industrial CAD design, addressing longstanding challenges of precision, editability, and robustness. By integrating multi-expert reinforcement learning with a high-quality dataset, it bridges the gap between academic research and industrial application, enabling faster, more reliable design workflows. The approach reduces reliance on expert engineers, lowers costs, and accelerates product development cycles, fostering the shift toward intelligent manufacturing and digital twin paradigms.
Technical Contribution
The paper introduces a novel heterogeneous multi-expert reinforcement learning framework, combining collaborative learning, reward design, and knowledge transfer mechanisms. It innovates with a multi-dimensional reward system ensuring format correctness, executable code, and geometric fidelity, while the hard negative sample buffer enhances training on challenging cases. The creation of CADExpert provides a high-fidelity, industrial-grade dataset for training and benchmarking, setting a new standard for CAD code generation research. The architecture enables scalable, high-precision, editable CAD generation from 2D projections, pushing the frontier of AI-driven design automation.
Novelty
This is the first work to integrate heterogeneous pre-trained models within a collaborative reinforcement learning framework specifically tailored for CAD code generation. It uniquely combines multi-dimensional reward functions with a hard negative sample buffer, addressing exploration and bias issues prevalent in prior methods. The CADExpert dataset fills a critical gap by providing complex, industrial-relevant examples, including operations like chamfering and drilling, which are rarely covered in existing datasets. Overall, the framework achieves a new level of accuracy, robustness, and industrial applicability.
Limitations
- Despite significant improvements, the model still struggles with extremely complex geometries or sparse training data, leading to occasional inaccuracies. The training process is computationally intensive, requiring substantial resources, which may limit real-time deployment. Additionally, the model’s performance depends on input data quality; noisy or imprecise inputs can degrade output quality. Future work should focus on enhancing generalization, reducing computational costs, and expanding dataset diversity to cover more industrial scenarios.
Future Work
Future directions include integrating multi-modal inputs such as point clouds or images for richer scene understanding, developing self-supervised learning strategies to reduce annotation dependency, and optimizing inference efficiency for real-time industrial applications. Expanding the dataset to include more complex operations and real-world scenarios will further improve robustness. Additionally, exploring transfer learning across different manufacturing domains could accelerate industrial adoption of AI-driven CAD design.
AI Executive Summary
In modern industrial design, the automation of CAD model generation remains a significant challenge. Traditional methods rely heavily on manual expertise, requiring years of training to produce precise, editable models. Existing AI approaches, often based on vision-language models or rule-based systems, struggle with complex operations and lack industrial-grade datasets. This gap hampers scalability and limits the deployment of intelligent design tools in real-world manufacturing.
To address these issues, the authors propose CME-CAD, a novel framework that combines heterogeneous pre-trained models within a multi-expert reinforcement learning paradigm. The approach involves two key stages: Multi-Expert Fine-Tuning (MEFT), where diverse experts generate varied reasoning paths from orthographic projections, and Multi-Expert Reinforcement Learning (MERL), which optimizes code generation using multi-dimensional rewards—covering format correctness, executable code, and geometric fidelity. A hard negative sample buffer mechanism further enhances robustness by focusing on difficult cases.
The authors also introduce CADExpert, a comprehensive dataset with over 17,000 instances, including complex industrial operations like drilling, chamfering, and difference operations, along with precise dimensional annotations and expert reasoning chains. Extensive experiments demonstrate that CME-CAD surpasses existing methods, achieving over 85% code correctness and 0.78 IoU, significantly advancing the state-of-the-art in CAD automation.
This work bridges the gap between research and industry, enabling faster, more accurate, and editable CAD generation from simple 2D inputs. Its implications extend to rapid prototyping, digital manufacturing, and smart factory automation. Despite current limitations in handling extremely complex geometries and computational costs, the framework sets a new benchmark, opening avenues for future research in multi-modal, scalable, and real-time industrial design automation.
Deep Dive
Plain Language Accessible to non-experts
想象你在一家工厂里，工人们用各种工具制作复杂的机械零件。有的工人擅长钻孔，有的擅长倒角，每个人都用不同的方法做出不同的零件。现在，假设你有一个机器人助手，想让它学会用这些工具自己设计零件。这个机器人会观察不同工人的工作方式，学习他们的技巧，然后自己尝试设计。通过让多个“专家”合作，机器人可以学到更多不同的技巧，设计出更复杂、更符合要求的零件。这样一来，只要告诉它需要什么样的零件，它就能自动帮你设计出来，不仅快，还很准确。这个方法就像让多个工匠一起合作，打造出最完美的作品一样，未来工厂的设计会变得更智能、更高效。
ELI14 Explained like you're 14
想象你在学校的手工课上，要用剪刀、胶水和纸张做模型。每次用不同的工具和方法，做出来的模型都不一样。有时候你试着用不同的颜色或形状，得到的作品也会不同。这个研究就像教一个机器人用各种“工具”和“技巧”设计模型。它会学习不同工匠的做法，合作完成一件漂亮的模型。比如，有的工匠擅长钻孔，有的擅长倒角，机器人学会结合这些技能，就能设计出更复杂、更漂亮的模型。这样，未来只要告诉它设计需求，它就能自动帮你做出符合要求的模型，既快又省力！
Abstract
Computer-Aided Design (CAD) is essential in industrial design, but the complexity of traditional CAD modeling and workflows presents significant challenges for automating the generation of high-precision, editable CAD models. Existing methods that reconstruct 3D models from sketches often produce non-editable and approximate models that fall short of meeting the stringent requirements for precision and editability in industrial design. Moreover, the reliance on text or image-based inputs often requires significant manual annotation, limiting their scalability and applicability in industrial settings. To overcome these challenges, we propose the Heterogeneous Collaborative Multi-Expert Reinforcement Learning (CME-CAD) paradigm, a novel training paradigm for CAD code generation. Our approach integrates the complementary strengths of these models, facilitating collaborative learning and improving the model's ability to generate accurate, constraint-compatible, and fully editable CAD models. We introduce a two-stage training process: Multi-Expert Fine-Tuning (MEFT), and Multi-Expert Reinforcement Learning (MERL). Additionally, we present CADExpert, an open-source benchmark consisting of 17,299 instances, including orthographic projections with precise dimension annotations, expert-generated Chain-of-Thought (CoT) processes, executable CADQuery code, and rendered 3D models.