ISAP-3D: Identity-Slot Aligned Part-Aware 3D Generation

The rapid growth of 3D content creation in gaming, virtual reality, and industrial design demands models that produce detailed, controllable, and structurally stable objects. Traditional holistic 3D generators excel at producing realistic shapes but lack explicit part-level control, limiting their utility in editing and reuse. Part-aware methods introduced semantic segmentation and part decomposition, yet often suffer from structural instability, with parts swapping or merging unpredictably across different views or edits. This instability stems from the core issue of implicit identity-layout entanglement, where semantic parts are not explicitly bound to dedicated spatial slots. As a result, the correspondence between semantic identity and spatial position remains ambiguous during training, leading to inconsistent decompositions and unreliable editing capabilities.

To address this, the authors propose ISAP-3D, a novel framework grounded in the principle of explicit identity-slot alignment. The key idea is to assign each semantic part a unique identity token, which serves as an anchor throughout the generation process. This explicit binding ensures a one-to-one correspondence between semantic identity and generation slots, effectively eliminating the permutation ambiguity. The framework employs multimodal encodings—textual descriptions, visual cues, and masks—to construct robust identity conditions. These are fed into a transformer-based layout predictor, which infers spatial arrangements in a manner conditioned on semantic identities, ensuring consistent part placement across views.

Following layout prediction, a geometry synthesis module conditioned on the predicted layouts generates detailed part geometries. This stage uses a geometry flow transformer with local-global attention mechanisms, preserving identity consistency and spatial coherence. The entire pipeline is supported by a newly constructed part-level dataset with a unified semantic protocol, enabling stable supervision and training. Extensive experiments demonstrate that ISAP-3D outperforms existing methods in structural stability, controllability, and robustness, achieving superior metrics such as PartIoU, Chamfer distance, and NMI. The approach significantly advances the field by providing a reliable, interpretable, and editable 3D part generation framework, with promising applications in VR, CAD, and content creation.

Despite its strengths, the method faces challenges in highly occluded or sparse data scenarios, and relies on large annotated datasets. Future work aims to incorporate self-supervised learning, extend to dynamic scenes, and improve robustness under adverse conditions, paving the way for more intelligent and flexible 3D content synthesis.