SldprtNet: A Large-Scale Multimodal Dataset for CAD Generation in Language-Driven 3D Design

Computer-Aided Design (CAD) plays a critical role in mechanical design and manufacturing, offering significant advantages over traditional paper-based drafting. Unlike manual drawings, CAD allows intuitive visualization of a part’s shape and dimensions and simplifies modifications. SolidWorks, a powerful CAD platform, has become a default choice for many mechanical designers. Its native .sldprt format records the feature operations and parameters used in model creation, enabling quick iteration and flexible editing of designs. This parametric, feature-based representation ensures higher precision and editability than discrete 3D formats like point clouds or meshes. However, compared to other categories of 3D model datasets, CAD datasets are unique in that each sample must be manually created using specialized software. The high skill and time required for quality CAD modeling result in datasets that are much smaller in scale than image or text datasets. Furthermore, limited data quantity and quality, annotation difficulties, and the lack of a standardized parametric representation format for 3D models have constrained progress in this area. As a result, despite the surge of interest in LLMs, research on semantic-driven CAD modeling tasks remains in its early stages.

Existing CAD datasets are small in scale and cannot meet the needs of semantic-driven CAD modeling tasks. CAD datasets are unique in that each sample must be manually created using specialized software, which requires high skill and time, resulting in datasets that are much smaller in scale than image or text datasets. Furthermore, limited data quantity and quality, annotation difficulties, and the lack of a standardized parametric representation format for 3D models have constrained progress in this area. These issues have made it difficult for research on semantic-driven CAD modeling tasks to progress, despite the surge of interest in large language models (LLMs).

The core innovations of SldprtNet lie in its construction of a multimodal dataset that combines precise geometry (3D CAD models), rendered images (multi-view projections), structured modeling sequences (parametric CAD commands), and natural language descriptions. This dataset provides rich supervision signals for understanding and generating models, significantly enhancing the model's capabilities in semantic-driven CAD modeling tasks. Additionally, the developed encoder and decoder tools enable closed-loop transformation between models and instructions, supporting data augmentation and future application expansion. SldprtNet is the first study to apply a multimodal dataset to semantic-driven CAD modeling. Unlike existing 3D model datasets, SldprtNet not only provides geometric information but also includes multi-view images and natural language descriptions, filling the gap in CAD automation and parametric modeling.

The construction and application methodology of the SldprtNet dataset includes the following key steps:

• �� Dataset Construction: Collect over 242,000 industrial parts in .sldprt and .step formats to support diverse training and testing needs.

• �� Encoder and Decoder Tools: Develop encoder and decoder tools supporting 13 CAD commands, enabling lossless transformation between 3D models and structured text representation.

• �� Multimodal Input: Each sample is paired with a composite image created from seven different viewpoints, combined with parameterized text output from the encoder, using the lightweight multimodal language model Qwen2.5-VL-7B to generate natural language descriptions.

• �� Experimental Design: Compare the fine-tuning of baseline models with image-plus-text inputs against text-only inputs to evaluate the effectiveness of multimodal supervision in enhancing model performance.

To evaluate the effectiveness of the SldprtNet dataset in CAD generation tasks, researchers fine-tuned two baseline models on a 50K-sample subset of SldprtNet. The experimental design includes:

• �� Dataset: Use a 50,000-sample subset of the SldprtNet dataset for experiments.

• �� Baseline Models: Compare Qwen2.5-7B (trained with Encodertext only) and Qwen2.5-7B-VL (trained with both image and Encodertext).

• �� Evaluation Metrics: Use metrics such as exact match score, command-level F1 score, parameter tolerance accuracy, and partial match rate to evaluate model performance.

• �� Ablation Studies: Analyze the impact of multimodal input on model performance, validating the importance of multimodal supervision in enhancing model performance.

The experimental results show that the Qwen2.5-7B-VL model with multimodal input outperforms the Qwen2.5-7B model with text-only input across several key metrics. Specifically:

• �� Exact Match Score: The Qwen2.5-7B-VL model achieved a score of 0.0099, compared to 0.0058 for the Qwen2.5-7B model, showing significant advantages.

• �� Command-Level F1 Score: The Qwen2.5-7B-VL model achieved 0.3670, compared to 0.3247 for the Qwen2.5-7B, indicating improved geometric semantic understanding.

• �� Partial Match Rate: The multimodal model scored 0.6162, surpassing the text-only model's 0.5554, further validating the effectiveness of multimodal supervision.

• �� Parameter Tolerance Accuracy: Although the text-only model slightly outperformed in parameter tolerance accuracy (0.5016 vs. 0.4630), this may suggest a tendency toward overfitting on numeric values.

The SldprtNet dataset has potential value in multiple application scenarios:

• �� Semantic-Driven CAD Modeling: Generate CAD models through natural language descriptions, supporting automated design and rapid prototyping.

• �� Cross-Modal Learning: Combine geometric information, images, and natural language descriptions to enhance cross-modal learning and reasoning capabilities.

• �� Industrial Design Optimization: Support the design and optimization of complex industrial parts, improving design efficiency and accuracy.

• �� Education and Training: As a teaching tool, help students and engineers learn and master CAD modeling techniques.

Despite its advantages in multimodal alignment and data scale, SldprtNet also has some limitations:

• �� The generated natural language descriptions still require manual verification to ensure accuracy, which may impact large-scale automation applications.

• �� The dataset is primarily focused on industrial parts, which may limit its generalizability to other fields.

• �� Due to the complexity of the dataset, the training and inference process may require high computational resources.

Future research directions include further optimizing the automatic generation of natural language descriptions to reduce the need for manual verification and exploring the application of SldprtNet to CAD modeling tasks in other fields.