DeepJEB++: Foundation Model-Driven Large-Scale 3D Engineering Dataset via 2D Latent Space Augmentation
DeepJEB++ leverages 2D latent space interpolation and foundation models to expand a small seed set into 15,360 labeled 3D jet engine brackets, with minimal resources.
Key Findings
Methodology
DeepJEB++ employs a three-stage pipeline: first, fine-tuning a pretrained 2D latent diffusion model on multi-view renders, generating new views via latent interpolation, and filtering with a vision-language model (VLM); second, lifting validated images into 3D meshes using a domain-adapted generative foundation model (TRELLIS); third, automatically recognizing load and bolt interfaces on each mesh and assigning finite element labels—mass, stress, displacement—via an automated CAE pipeline. This approach effectively leverages large-scale pretrained models for cross-modal transfer, enabling the expansion from fewer than 400 seed designs to over 15,000 labeled 3D brackets with high fidelity and diversity, all within limited computational resources.
Key Results
- Starting from fewer than 400 seed designs, the pipeline produced 15,360 simulation-labeled 3D brackets, a 40-fold increase, using only a single GPU per stage, demonstrating high efficiency and scalability.
- The generated dataset exhibits high manufacturability, label fidelity, and distributional consistency, with over 22,000 validated high-quality designs after filtering, surpassing traditional augmentation methods.
- The domain-adapted TRELLIS model reliably reconstructs 3D geometries from single-view images, ensuring accurate FE analysis and performance prediction, validating the automatic labeling process.
Significance
This work addresses the critical bottleneck of limited large-scale engineering datasets by harnessing pretrained foundation models for cross-modal transfer, significantly reducing data collection and annotation costs. It enables rapid, automated generation of diverse, high-fidelity 3D structural data, facilitating advances in structural optimization, generative design, and digital twin applications. The approach democratizes data-driven engineering by making large datasets accessible with modest hardware, paving the way for broader adoption of AI in industrial design and manufacturing. Its scalable, automated pipeline sets a new standard for engineering data synthesis, with potential extensions to complex multi-physics simulations and multi-domain applications.
Technical Contribution
The core innovation lies in combining latent space interpolation of a pretrained 2D diffusion model with a domain-adapted 3D generative foundation (TRELLIS), enabling high-diversity, geometrically faithful 3D reconstruction from minimal seed data. The integration of a VLM-based quality filter effectively prevents the generation of defective designs, ensuring dataset reliability. This cross-modal transfer approach, leveraging external priors from billions of images and shapes, reduces reliance on domain-specific training and accelerates large-scale data expansion. The automated boundary recognition and FE label assignment further streamline the pipeline, making it feasible to generate thousands of labeled samples with limited computational resources.
Novelty
This research is the first to systematically utilize 2D latent space interpolation combined with vision-language filtering for large-scale engineering dataset augmentation. Unlike prior work relying on scratch-trained implicit models, it exploits the rich priors embedded in pretrained foundation models, enabling rapid, diverse, and high-quality 3D data generation. The integration of cross-modal transfer, automated interface recognition, and physics-based labeling within a resource-efficient pipeline marks a significant departure from traditional manual or domain-specific augmentation methods, setting a new paradigm for engineering data synthesis.
Limitations
- The current pipeline relies on multi-view rendering and may struggle with highly complex or detailed geometries, affecting reconstruction fidelity.
- Automated interface detection can be less accurate for geometries with ambiguous features, potentially impacting boundary condition assignment.
- The models' generalization to non-standard or highly deformed designs remains limited; robustness needs enhancement for broader applicability.
Future Work
Future efforts will focus on integrating multi-physics simulations, improving boundary detection robustness, and extending the pipeline to other structural components and materials. Incorporating active learning and human-in-the-loop strategies could further refine data quality and diversity. Additionally, exploring unsupervised domain adaptation techniques may enable the pipeline to handle more complex geometries and boundary conditions, broadening its industrial relevance.
AI Executive Summary
Addressing the scarcity of large-scale, high-fidelity 3D engineering datasets remains a fundamental challenge in advancing data-driven design and structural optimization. Traditional methods, reliant on manual CAD modeling and costly simulations, are prohibitively expensive and slow, limiting the scope of AI applications in engineering. In response, this study introduces DeepJEB++, a novel framework that leverages the power of pretrained foundation models to automate and scale the generation of complex 3D structural data. The approach hinges on a three-stage pipeline: first, in the 2D domain, a fine-tuned latent diffusion model generates diverse, manufacturable bracket designs through latent space interpolation, with a vision-language model filtering out defective or unrealistic samples; second, these validated images are lifted into 3D meshes using a domain-adapted generative model (TRELLIS), capable of reconstructing geometries from single views; third, an automated boundary recognition and finite element analysis pipeline assigns physical labels such as mass, stress, and displacement without manual intervention. Remarkably, starting from fewer than 400 seed designs, the pipeline produces over 15,000 high-quality, simulation-labeled 3D brackets, a 40-fold expansion, with only a single GPU per stage. This demonstrates that large-scale, diverse engineering datasets can be generated efficiently and cost-effectively, democratizing access to data for academia and industry. The integration of cross-modal transfer, automatic quality filtering, and physics-based labeling represents a significant step forward in AI-driven engineering. The dataset generated by DeepJEB++ will be openly released, fostering reproducible research and enabling new avenues in generative design, structural analysis, and digital twin development. Future work will aim to incorporate multi-physics simulations and enhance robustness, further broadening the impact of this innovative approach in complex engineering applications.
Deep Dive
Key Concepts
Latent Space Interpolation
A technique to generate new designs by blending features in the model's internal representation, enabling smooth variation between existing designs.
Foundation Models
Large pretrained neural networks that encode broad prior knowledge, adaptable to specific tasks via fine-tuning.
Vision-Language Model (VLM)
A model that understands visual content and associates it with textual descriptions, used here for quality filtering.
TRELLIS
A domain-adapted 3D generative foundation model capable of reconstructing geometries from images in multiple formats.
Automated Boundary Recognition
A process to identify load and bolt interfaces on 3D meshes for automatic physical labeling.
Open Questions Unanswered questions from this research
- 1 How to further improve the fidelity of single-view 3D reconstructions in highly complex geometries remains an open challenge, requiring better priors or multi-view integration.
- 2 The robustness of automated boundary detection under diverse boundary conditions and geometries needs further validation, especially for non-standard designs.
- 3 Extending the pipeline to handle multi-physics simulations and materials with complex behaviors is an important future direction to enhance practical applicability.
Applications
Immediate Applications
Structural Design Automation
Rapidly generate large, labeled 3D datasets for training surrogate models in structural optimization and failure prediction.
Engineering Data Augmentation
Enhance existing datasets with diverse, high-fidelity models, reducing manual modeling and simulation costs.
Long-term Vision
Digital Twin Development
Create comprehensive, real-time simulatable models for complex machinery, enabling predictive maintenance and adaptive control.
Abstract
Data-driven engineering design is constrained by the lack of large-scale 3D datasets that pair geometry with physics-based performance labels. In particular, existing 3D data augmentation techniques have limitations in preserving subtle and diverse geometric variations, and it remains difficult to automate the subsequent simulation-labeling process, where boundary conditions vary depending on the generated geometry. We present DeepJEB++, a foundation-model-driven data-augmentation framework that expands a small seed set of jet engine brackets into a large, simulation-labeled 3D dataset under constrained resources. Our key idea is to augment in the data-rich 2D latent space, then transfer to 3D. In Stage 1, we fine-tune a pretrained 2D latent diffusion model on multi-view renders and synthesize novel views by latent interpolation, retaining manufacturable designs through a vision-language-model (VLM) quality filter. In Stage 2, the validated images are lifted to 3D meshes by a domain-adapted generative foundation model. In Stage 3, an automated pipeline recognizes the load and bolt interfaces on each mesh and assigns finite-element labels -- mass, stress, and displacement -- without manual intervention. We assess augmentation quality along three intrinsic axes: manufacturability, label fidelity against the SimJEB ground truth, and distributional consistency. Starting from fewer than 400 seed designs, DeepJEB++ yields 15,360 simulation-labeled 3D brackets -- a 40x expansion -- using a single GPU per stage. The dataset will be made publicly available to support reproducible engineering-AI research.