GptGet

LATO: 3D Mesh Flow Matching with Structured TOpology Preserving LAtents

TL;DR

LATO uses topology-preserving sparse voxel VAE to efficiently generate structured 3D meshes.

cs.CV 🔴 Advanced 2026-03-06 32 views
Tianhao Zhao Youjia Zhang Hang Long Jinshen Zhang Wenbing Li Yang Yang Gongbo Zhang Jozef Hladký Matthias Nießner Wei Yang
AI Reader Arxiv Page Download PDF
3D generation topology preservation sparse voxel VAE flow matching

Key Findings

Methodology

LATO employs a sparse voxel VAE to encode meshes as Vertex Displacement Fields (VDF) and uses a two-stage flow matching strategy to generate topology. The decoder progressively refines voxels and predicts vertex connectivity to reconstruct explicit mesh topology.

Key Results

  • Result 1: Achieved lowest Chamfer Distance of 0.043/0.044 on dense/artistic meshes, improving by ~20% over MeshGPT and MeshCraft.
  • Result 2: Best Normal Consistency (NC) at 0.832/0.835 on dense/artistic meshes, outperforming all baselines.
  • Result 3: Significant inference efficiency, generating meshes in 3-10 seconds compared to minutes for autoregressive methods.

Significance

LATO addresses the loss of topology in existing methods, enabling the generation of artist-friendly meshes suitable for downstream tasks like rigging and game engines. This represents a major advancement in 3D generation, especially for applications in VR and industrial design.

Technical Contribution

LATO introduces topology-preserving features into 3D mesh generation for the first time, proposing T-Voxels as a sparse voxel representation. It avoids the complexity of isosurface extraction and heuristic meshing while leveraging flow matching for efficient topology generation.

Novelty

LATO's novelty lies in directly generating explicit mesh topology from sparse voxels, avoiding dense triangulation issues in implicit methods and significantly improving efficiency with flow matching.

Limitations

  • Limitation 1: Limited validation on non-manifold geometries; performance may degrade in complex scenarios.
  • Limitation 2: High computational resource requirements for high-resolution meshes.
  • Limitation 3: Dependence on specific datasets may limit generalization.

Future Work

Future work includes extending support for non-manifold geometries, optimizing computational efficiency, and exploring broader applications like medical imaging and architectural design.

AI Executive Summary

LATO is a novel 3D mesh generation framework designed to address the loss of topology in existing methods. Traditional approaches rely on implicit field representations like Signed Distance Functions (SDF) or occupancy fields, extracting meshes via isosurfacing. However, these methods often produce dense, irregular triangulations unsuitable for tasks like rigging or game engines.

LATO encodes meshes as Vertex Displacement Fields (VDF) using a sparse voxel VAE and employs a two-stage flow matching strategy to generate topology. The decoder progressively refines voxels and predicts vertex connectivity, directly reconstructing explicit mesh topology. Compared to existing methods, LATO achieves superior geometric complexity and topology fidelity while significantly improving inference efficiency.

Experiments demonstrate that LATO achieves state-of-the-art performance in Chamfer Distance and Normal Consistency, with inference times of just 3-10 seconds. While limitations remain in handling non-manifold geometries and computational demands, LATO sets a new standard for scalable, topology-preserving 3D mesh generation, with promising applications in VR, gaming, and industrial design.

Deep Analysis

Background

Recent advances in 3D generation have enabled applications in VR, gaming, and industrial design. Traditional methods often use implicit fields like Signed Distance Functions (SDF) or occupancy grids, extracting meshes via algorithms like Marching Cubes. However, these approaches fail to preserve explicit topology, leading to dense, irregular meshes unsuitable for downstream tasks.

Core Problem

Existing methods overlook explicit topology, resulting in meshes that are unsuitable for tasks like rigging and deformation. Implicit field methods also require watertight data, limiting their ability to handle open surfaces and non-manifold geometries.

Innovation

Key innovations of LATO include:

  • �� Introduction of T-Voxels, a sparse voxel representation encoding vertex distribution and connectivity.
  • �� Two-stage flow matching strategy for generating geometry and topology features.
  • �� A decoder that progressively refines voxels and predicts vertex connectivity to reconstruct explicit topology.

Methodology

LATO's implementation involves:

  • �� Representing meshes as Vertex Displacement Fields (VDF) and compressing them into T-Voxels via a sparse voxel VAE.
  • �� A decoder that progressively subdivides voxels and prunes them to generate precise vertex locations.
  • �� A connection head that directly predicts vertex pair connectivity.
  • �� A two-stage flow matching strategy for generating coarse geometry and refining topology features.

Experiments

Experiments were conducted on artistic mesh datasets (G-Objaverse, Toys4K, ShapeNet) and dense mesh datasets (generated by TRELLIS). Metrics included Chamfer Distance, Hausdorff Distance, and Normal Consistency. Baselines included MeshGPT, MeshCraft, and others.

Results

LATO achieved the lowest Chamfer Distance (0.043/0.044) and highest Normal Consistency (0.832/0.835) on dense/artistic meshes. It also demonstrated significant inference speed, generating meshes in 3-10 seconds compared to minutes for autoregressive methods.

Applications

LATO is ideal for VR, gaming, and industrial design, particularly in scenarios requiring high-quality mesh topology, such as rigging and non-manifold geometry modeling.

Limitations & Outlook

LATO's limitations include challenges with non-manifold geometries, high computational demands for high-resolution meshes, and reliance on specific datasets, which may limit generalization.

Plain Language Accessible to non-experts

Imagine building a LEGO model. Traditional methods are like molding a single block of clay into a shape—you can't see the internal structure. LATO, on the other hand, is like having a smart assistant that first builds a rough frame and then carefully places each LEGO piece in the right spot, ensuring everything is connected properly.

ELI14 Explained like you're 14

Imagine you're playing Minecraft and want to build a castle. Regular methods are like piling up dirt blocks to make a rough shape, but it might have holes or weird edges. LATO is like having a super-smart builder who first sketches the outline, then places each block perfectly, and even checks if everything is connected. Cool, right?

Glossary

Vertex Displacement Field (VDF)

A field representing mesh vertices' displacements relative to surface points, encoding both geometry and topology.

Used to generate T-Voxels in LATO.

T-Voxels

A sparse voxel representation encoding vertex distribution and connectivity.

Core topology representation in LATO.

Sparse Voxel VAE

A generative model for compressing and decoding topology features into T-Voxels.

Used in LATO to encode and decode topology.

Flow Matching

A generation strategy that optimizes sparse voxel and topology features sequentially.

Used in LATO's two-stage generation process.

Chamfer Distance

A metric for evaluating geometric similarity between generated and ground truth meshes.

Used to assess LATO's performance.

Open Questions Unanswered questions from this research

  • 1 How can LATO be optimized for higher-resolution mesh generation?
  • 2 Can the algorithm be improved to reduce computational requirements?
  • 3 How can LATO be extended to support diverse non-manifold geometries?

Applications

Immediate Applications

VR Content Creation

LATO can generate high-quality VR scenes, especially for environments requiring complex geometry and topology.

Game Development

LATO's meshes are ready for game engines, reducing post-processing needs.

Long-term Vision

Medical Imaging

LATO could generate complex 3D models for surgical planning and medical research.

Abstract

In this paper, we introduce LATO, a novel topology-preserving latent representation that enables scalable, flow matching-based synthesis of explicit 3D meshes. LATO represents a mesh as a Vertex Displacement Field (VDF) anchored on surface, incorporating a sparse voxel Variational Autoencoder (VAE) to compress this explicit signal into a structured, topology-aware voxel latent. To decapsulate the mesh, the VAE decoder progressively subdivides and prunes latent voxels to instantiate precise vertex locations. In the end, a dedicated connection head queries the voxel latent to predict edge connectivity between vertex pairs directly, allowing mesh topology to be recovered without isosurface extraction or heuristic meshing. For generative modeling, LATO adopts a two-stage flow matching process, first synthesizing the structure voxels and subsequently refining the voxel-wise topology features. Compared to prior isosurface/triangle-based diffusion models and autoregressive generation approaches, LATO generates meshes with complex geometry, well-formed topology while being highly efficient in inference.

cs.CV