CMT: A Cascade MAR with Topology Predictor for Multimodal Conditional CAD Generation
CMT: Boundary-representation-based multimodal cascade MAR, improves coverage by 10.68%, supports complex CAD generation.
Key Findings
Methodology
CMT integrates cascade autoregressive networks (Cascade MAR) with a topology predictor, leveraging boundary representation (B-Rep) for detailed 3D CAD modeling. It employs edge MAR and surface MAR to generate boundary contours and surfaces sequentially, conditioned on multimodal inputs encoded via a unified encoder. The topology predictor estimates adjacency relations between edges and surfaces, enabling accurate complex geometry and topology synthesis. Trained on the large-scale mmABC dataset with over 1.3 million models, it outperforms state-of-the-art methods in both unconditional and conditional CAD generation, with significant improvements in coverage, validity, and Chamfer distance metrics.
Key Results
- On the ABC dataset, CMT achieves a 10.68% higher coverage and 10.3% better validity ratio than BrepGen in unconditional generation, demonstrating superior modeling fidelity.
- In conditioned generation, CMT reduces Chamfer distance by 4.01 when using point clouds, and shows robust multi-view image conditioned results, with more complex and topologically accurate models.
- The topology predictor accelerates adjacency estimation by 4200 times, taking only 0.038 seconds, vastly improving efficiency over traditional post-processing methods.
Significance
This work advances CAD automation by enabling high-fidelity, multimodal conditioned generation of complex models with accurate topology. It addresses longstanding challenges in representing detailed geometry and intricate topological relationships simultaneously, facilitating applications in industrial design, rapid prototyping, and personalized manufacturing. The integration of large-scale multimodal datasets and innovative network architectures paves the way for intelligent, autonomous design tools, reducing manual effort and improving productivity across manufacturing industries.
Technical Contribution
The core innovation lies in combining cascade MAR with a topology predictor, supported by continuous tokenization of B-Rep primitives. This allows the model to generate detailed geometry and topology in a unified framework, overcoming the limitations of discrete command sequences. The large-scale mmABC dataset provides extensive multimodal annotations, fostering robust training. The approach also introduces a fast, accurate topology estimation module, significantly enhancing the practicality of CAD generation pipelines.
Novelty
This is the first framework to support multimodal conditioned CAD generation directly based on boundary representation, integrating a cascade autoregressive approach with topology prediction. Unlike prior works limited to command sequences or diffusion models, CMT achieves detailed, complex geometry with accurate topological relationships in a single, end-to-end trainable system, marking a significant leap forward in the field.
Limitations
- Despite high accuracy, the model struggles with extremely complex or highly detailed models, especially where boundary ambiguity or occlusion occurs, leading to occasional topology errors.
- Heavy reliance on large-scale multimodal datasets increases data collection and annotation costs, potentially limiting generalization in niche domains.
- While inference speed is improved, real-time applications with very large models still face computational challenges and require further optimization.
Future Work
Future directions include enhancing the topology predictor’s robustness, reducing data dependency through semi-supervised learning, and extending multimodal inputs to include more diverse data types like tactile or thermal data. Additionally, optimizing inference speed and integrating the system into end-to-end industrial design platforms will be key to broader adoption.
AI Executive Summary
The demand for intelligent, automated CAD tools in industrial design continues to grow, yet existing methods face significant limitations in generating complex, high-fidelity models conditioned on diverse inputs. Traditional CAD workflows rely heavily on manual sketching and multi-step operations, which are time-consuming and costly. Recent deep learning approaches, such as command sequence models and diffusion-based methods, have made progress but lack the ability to support multimodal inputs and complex topologies simultaneously. This gap hinders the realization of fully autonomous, user-friendly CAD systems suitable for modern manufacturing needs.
In response, this paper introduces CMT, a novel framework that combines boundary representation (B-Rep) with a cascade autoregressive network (Cascade MAR) and a topology predictor. The architecture enables the progressive generation of boundary contours and surfaces, conditioned on multimodal inputs like text, point clouds, and multi-view images. The key innovation is the integration of a topology predictor that rapidly estimates adjacency relationships, ensuring the generated models maintain accurate and complex topological structures. Continuous tokenization of B-Rep primitives further enhances the model’s ability to learn detailed geometry.
Training on the newly constructed mmABC dataset, which contains over 1.3 million multimodal annotated models, CMT demonstrates significant improvements over existing methods. Quantitative metrics show a 10.68% increase in coverage and a 10.3% increase in valid models in unconditional generation. In conditioned tasks, Chamfer distance reduces by 4.01, indicating higher geometric fidelity. The topology predictor’s speed surpasses traditional post-processing by 4200 times, making the approach practical for industrial applications.
This work marks a substantial step toward fully automated, multimodal CAD generation, with broad implications for rapid prototyping, personalized manufacturing, and intelligent design automation. Future efforts will focus on improving model robustness, reducing data costs, and integrating into real-time industrial workflows, ultimately transforming how complex 3D models are created and utilized in industry.
Deep Dive
Limitations & Outlook
What gaps remain?
Abstract
While accurate and user-friendly Computer-Aided Design (CAD) is crucial for industrial design and manufacturing, existing methods still struggle to achieve this due to their over-simplified representations or architectures incapable of supporting multimodal design requirements. In this paper, we attempt to tackle this problem from both methods and datasets aspects. First, we propose a cascade MAR with topology predictor (CMT), the first multimodal framework for CAD generation based on Boundary Representation (B-Rep). Specifically, the cascade MAR can effectively capture the ``edge-counters-surface'' priors that are essential in B-Reps, while the topology predictor directly estimates topology in B-Reps from the compact tokens in MAR. Second, to facilitate large-scale training, we develop a large-scale multimodal CAD dataset, mmABC, which includes over 1.3 million B-Rep models with multimodal annotations, including point clouds, text descriptions, and multi-view images. Extensive experiments show the superior of CMT in both conditional and unconditional CAD generation tasks. For example, we improve Coverage and Valid ratio by +10.68% and +10.3%, respectively, compared to state-of-the-art methods on ABC in unconditional generation. CMT also improves +4.01 Chamfer on image conditioned CAD generation on mmABC.