CADmium: Fine-Tuning Code Language Models for Text-Driven Sequential CAD Design
This work fine-tunes GPT-4.1 and Qwen2.5-Coder for text-to-CAD sequence generation, achieving significant improvements in automation and structural accuracy.
Key Findings
Methodology
The authors assembled a dataset of over 170,000 CAD models annotated with high-quality descriptions generated by GPT-4.1, combining multimodal inputs. They used GPT-4.1 to produce geometrically precise natural language descriptions, then fine-tuned Qwen2.5-Coder with LoRA to translate these descriptions into JSON-based CAD sequences. The approach reformulates CAD generation as a text-to-text task, avoiding complex embeddings. Structural and topological metrics like sphericity, mean curvature, and Euler characteristic evaluate model outputs. Training involved AdamW optimizer, mixed precision, and distributed computing, ensuring efficiency.
Key Results
- Fine-tuned models outperform baselines on Text2CAD and Fusion360 datasets, with invalidity ratios below 10%, and improvements in Chamfer Distance and F1 scores (e.g., 15% reduction in CD). Structural metrics such as SD and DMCD show higher shape similarity, with SD approaching 0.05, indicating precise shape capture. Descriptions generated by GPT-4.1 are more natural and diverse, leading to better generalization.
- The new geometric and topological metrics effectively reflect the structural quality of generated models, demonstrating the model's ability to produce geometrically consistent objects even in complex scenarios.
- Ablation studies reveal that GPT-4.1-generated descriptions, being concise and varied, significantly enhance model performance compared to traditional automatic annotations, highlighting the importance of high-quality textual supervision.
Significance
This research advances the application of large pre-trained language models in automated CAD design, addressing long-standing bottlenecks in annotation quality and structural evaluation. It enables faster, more accessible 3D modeling, reducing reliance on expert intervention. The proposed dataset, annotation pipeline, and structural metrics set new standards for the field, fostering progress in intelligent manufacturing, rapid prototyping, and digital twin development. The approach demonstrates the potential of language models to revolutionize engineering workflows, making complex design tasks more efficient and democratized.
Technical Contribution
The key technical innovation lies in transforming CAD generation into a text-to-text prediction task, leveraging GPT-4.1 for high-quality annotation and LoRA fine-tuning of Qwen2.5-Coder for efficient code generation. The introduction of geometric and topological evaluation metrics provides a comprehensive assessment framework, surpassing traditional point-cloud or mesh-based metrics. This approach maximizes the use of pre-trained models without requiring complex embedding layers, enabling scalable and flexible CAD generation from natural language prompts.
Novelty
This is the first work to utilize GPT-4.1 for high-quality CAD description generation and to fine-tune Qwen2.5-Coder for end-to-end text-to-sequence CAD modeling. The structural metrics introduced, such as sphericity discrepancy and Euler characteristic match, offer novel quantitative tools for evaluating CAD model quality, significantly advancing the state-of-the-art in text-conditioned 3D design.
Limitations
- The models still struggle with highly intricate or non-standard geometries, often producing topological errors or incomplete meshes, especially in complex designs.
- Training requires extensive high-quality annotated data, which is costly to produce, limiting scalability.
- Inference speed remains a challenge for real-time applications; further optimization is needed for industrial deployment.
Future Work
Future directions include integrating multimodal inputs such as images and tactile data to enrich descriptions, developing interactive interfaces for user-guided refinement, and exploring zero-shot generalization to unseen geometries. Enhancing model robustness and reducing computational costs are also priorities to facilitate real-world industrial applications.
AI Executive Summary
This work addresses the challenge of automating CAD design through natural language prompts. Traditional CAD modeling is manual, time-consuming, and requires expert skills. Recent efforts using small transformer models have improved automation but still face limitations in annotation quality and structural evaluation. The authors propose a novel approach leveraging GPT-4.1 to generate high-quality, geometrically precise descriptions of over 170,000 CAD models, creating a rich dataset that bridges the gap between natural language and complex 3D structures.
Building on this, they fine-tune the instruction-tuned code model Qwen2.5-Coder with LoRA, enabling it to translate natural language prompts into JSON-based CAD sequences. This reformulation of CAD generation as a text-to-text task simplifies the pipeline and leverages the full potential of pre-trained language models. The authors introduce innovative geometric and topological metrics—such as sphericity, mean curvature, and Euler characteristic—to evaluate the structural fidelity of generated models, providing a more comprehensive assessment than traditional point-cloud metrics.
Experimental results demonstrate that their models outperform existing methods like Text2CAD, achieving lower invalidity ratios and higher shape similarity scores across multiple datasets. The generated models exhibit high structural integrity, with a significant portion being watertight and topologically consistent. The approach significantly accelerates the design process, making CAD accessible to non-experts and enabling rapid prototyping.
Overall, this research paves the way for intelligent, automated CAD systems driven by natural language, with broad implications for industrial design, manufacturing, and digital twin applications. Future work will focus on multimodal integration, interactive design, and scalability, aiming to realize fully autonomous design workflows.
Deep Dive
Limitations & Outlook
What gaps remain?
Abstract
Computer-aided design (CAD) is the digital construction of 2D and 3D objects, and is central to a wide range of engineering and manufacturing applications like automobile and aviation. Despite its importance, CAD modeling remains largely a time-intensive, manual task. Recent works have attempted to automate this process with small transformer-based models and handcrafted CAD sequence representations. However, there has been little effort to leverage the potential of large language models (LLMs) for sequential CAD design. In this work, we introduce a new large-scale dataset of more than 170k CAD models annotated with high-quality, human-like descriptions generated with our pipeline based on GPT-4.1. Using this dataset, we fine-tune powerful code-LLMs to generate CAD sequences represented in a JSON-based format from natural language descriptions, demonstrating the viability and effectiveness of this approach for text-conditioned CAD generation. Because simple metrics often fail to reflect the quality of generated objects, we introduce geometric and topological metrics based on sphericity, mean curvature, and Euler characteristic to provide richer structural insights. Our experiments and ablation studies on both synthetic and human-annotated data demonstrate that CADmium is able to automate CAD design, drastically speeding up the design of new objects. The dataset, code, and fine-tuned models are available online.