Discrete Diffusion Bridges for Spatiotemporally Aligned Image Translation and Generation
Proposes Discrete Diffusion Bridges (DDB) to resolve spatiotemporal misalignment in image generation using hybrid absorption and information-guided noise scheduling.
Key Findings
Methodology
DDB introduces hybrid absorption to inject source image as spatial anchors into the latent space and designs an information-guided noise schedule to prioritize high-information regions, ensuring training and inference alignment.
Key Results
- On OmniEdit dataset, DDB achieves an Edit Score of 0.779, outperforming baselines while maintaining background consistency (DINO score 0.790).
- On SynthRAD dataset for CT-to-MRI translation, DDB achieves PSNR of 28.43, SSIM of 0.91, and LPIPS reduced to 0.12, surpassing existing baselines.
- On MIMIC-CXR dataset for text-to-image generation, DDB achieves FID of 16.8 and CLIP score of 0.87, demonstrating strong text-image alignment.
Significance
DDB addresses the spatiotemporal misalignment in discrete diffusion models for image translation and generation, significantly improving edit alignment and structural fidelity, with applications in medical imaging and complex image editing.
Technical Contribution
Introduces hybrid absorption and information-guided noise scheduling, redefining the noise state in diffusion models to optimize alignment between training and inference trajectories, enhancing generation quality and efficiency.
Novelty
DDB is the first to inject source image information directly into the latent space via hybrid absorption and align training and inference paths through information-guided scheduling, pioneering a new diffusion model design paradigm.
Limitations
- Generation quality decreases in extremely low sampling steps for complex scenarios.
- Requires substantial computational resources for large-scale model training.
- Applicability of hybrid absorption in source-free tasks needs further validation.
Future Work
Future work could explore DDB's extension to multimodal generation tasks, optimize architecture for lower computational costs, and study its performance in real-time generation scenarios.
AI Executive Summary
Discrete diffusion models have recently gained traction in image generation but face spatiotemporal misalignment issues in translation tasks: random masking during training contradicts the “easy-first, hard-last” inference mechanism, and pure mask states discard structural priors from source images.
This paper proposes Discrete Diffusion Bridges (DDB), introducing hybrid absorption to inject source image information as spatial anchors into the latent space and designing an information-guided noise schedule to prioritize high-information regions, ensuring alignment between training and inference paths.
Experiments demonstrate DDB's superior performance across OmniEdit, SynthRAD, and MIMIC-CXR datasets, significantly improving edit alignment, structural fidelity, and generation quality, showcasing its broad applicability in complex image translation and text-to-image generation tasks.
Deep Analysis
Background
Discrete diffusion models use masking and prediction mechanisms for high-fidelity image generation, widely applied in editing and generation tasks. However, standard models face spatiotemporal misalignment issues, affecting edit alignment and structural fidelity.
Core Problem
Standard discrete diffusion models apply random masking during training, ignoring the need to prioritize complex regions, and pure mask states fail to retain structural priors from source images, impacting generation quality.
Innovation
DDB introduces hybrid absorption to inject source image information as spatial anchors into the latent space and designs an information-guided noise schedule to prioritize high-information regions, ensuring training and inference alignment.
Methodology
- �� Hybrid Absorption: Defines absorbing state as a mixture of source image and mask, preserving spatial anchors.
- �� Information-Guided Noise Scheduling: Prioritizes masking high-information regions based on density metrics, creating a “hard-first” training trajectory.
- �� Inference Process: Starts from hybrid absorption state, dynamically decodes low-information regions first, progressively handling complex edits.
Experiments
Evaluates image editing performance on OmniEdit dataset, modality translation on SynthRAD dataset, and text-to-image generation quality on MIMIC-CXR dataset.
Results
DDB achieves an Edit Score of 0.779 on OmniEdit, PSNR of 28.43 and SSIM of 0.91 on SynthRAD, and FID of 16.8 with CLIP score of 0.87 on MIMIC-CXR.
Applications
DDB is applicable to medical imaging generation, complex image editing, and cross-modal image translation, suitable for scenarios requiring high fidelity and precise edits.
Limitations & Outlook
DDB's generation quality decreases in low sampling steps and requires substantial computational resources for training. Future work could optimize architecture to reduce costs.
Plain Language Accessible to non-experts
Imagine DDB as a smart painter. Traditional painters start from a blank canvas, but DDB starts with a partial sketch (source image information). It first tackles the hardest parts, like drawing a face, then fills in the background and details. This keeps the original structure intact while quickly completing complex edits.
ELI14 Explained like you're 14
Think of it like playing a puzzle game. Traditional methods shuffle all the pieces before starting, but DDB fixes some key pieces first, like the edges and corners, and then builds the rest of the puzzle around them. It's faster and more accurate — pretty clever, right?
Glossary
Discrete Diffusion Model
A model that generates images using masking and prediction mechanisms within a discrete state space.
Used for image translation and generation tasks.
Hybrid Absorption Mechanism
Injects source image information into the absorbing state as spatial anchors.
Addresses spatial misalignment issues.
Information-Guided Noise Schedule
Prioritizes masking high-information regions based on density metrics.
Optimizes training and inference alignment.
FID (Fréchet Inception Distance)
Measures distribution differences between generated and real images.
Used to evaluate generation quality.
CLIP Score
Evaluates text-image alignment quality.
Used in text-to-image generation tasks.
Open Questions Unanswered questions from this research
- 1 How can hybrid absorption be optimized for source-free tasks?
- 2 Can DDB's computational cost be reduced for real-time generation?
Applications
Immediate Applications
Medical Imaging Generation
Supports high-quality medical image generation from text, enhancing diagnostic efficiency.
Complex Image Editing
Enables precise local edits and style transformations, suitable for design applications.
Long-term Vision
Real-Time Generation Systems
Develop real-time image generation technologies for gaming and virtual reality.
Abstract
We propose Discrete Diffusion Bridges (DDB), a novel framework designed to resolve the fundamental spatiotemporal misalignment of standard discrete diffusion in image translation and generation. By corrupting data into a pure mask state via a random schedule, the conventional forward process induces a twofold misalignment: spatially, this pure-mask destination entirely discards the rich structural priors of the source image; temporally, the random masking order inherently contradicts the ``easy-first, hard-last'' decoding mechanism used during inference. To address this, DDB constructs a direct and efficient trajectory between domains. Spatially, we introduce a hybrid absorption mechanism that redefines the absorbing state to a stochastic mixture of mask and source tokens, effectively injecting source prior as spatial anchors into the latent space. Temporally, we design an information-guided noise schedule that quantifies semantic variation to prioritize the corruption of high-information regions at earlier timesteps. This ensures the model learns to resolve difficult semantic changes using robust context from invariant regions. Extensive experiments validate the versatility and robustness of our framework across diverse generative paradigms. DDB effectively balances edit alignment with structural fidelity across both text-guided semantic manipulation and pure structural image translation, while inherently complementing text-to-image generation and guaranteeing robust high-quality decoding under extremely low sampling steps. Code and models are available at \href{https://github.com/HKU-HealthAI/DDB}{https://github.com/HKU-HealthAI/DDB}.