LCG: Long-Context Consistent Image Generation with Sparse Relational Attention
LCG uses Sparse Relational Attention for consistent long-context image generation, improving consistency and scalability.
Key Findings
Methodology
LCG employs Sparse Relational Attention (SRA) to selectively attend to core features across extended visual contexts, ensuring semantic and layout information propagation remains computationally feasible. It introduces the Routing Consistency Constraint (RCC) to align structural patterns across generation branches using identity-aware masks, effectively reducing appearance drift.
Key Results
- LCG outperforms baselines in prompt alignment and character consistency for long-context image generation, especially in multi-character scenes.
- On the LCCD test set, LCG improved character consistency by 15% and visual quality by 10%.
- Ablation studies confirm the critical role of SRA and RCC in performance enhancement.
Significance
This research offers a novel solution for long-context image generation, particularly in applications requiring consistency across multiple images, such as comics and storyboards. LCG demonstrates significant advantages in maintaining character identity and visual continuity.
Technical Contribution
By introducing Sparse Relational Attention and Routing Consistency Constraint, LCG provides a new framework for long-context image generation, significantly enhancing cross-panel consistency and scalability compared to existing methods.
Novelty
LCG is the first to introduce Sparse Relational Attention in long-context image generation, effectively addressing computational bottlenecks faced by existing methods in handling long sequences.
Limitations
- Handling extremely complex multi-character scenes may still result in character identity drift.
- High computational resource requirements may limit its application in resource-constrained environments.
Future Work
Future research could explore optimizing the computational efficiency of SRA and RCC and their application in larger-scale and more complex scenarios.
AI Executive Summary
Recent advancements in image generation models have achieved impressive quality in single-image synthesis, but maintaining consistency across sequential outputs remains challenging in applications like comics and storyboards. To address this, the paper introduces the Long-Context Generation (LCG) framework, which enhances consistency and scalability in long-context multi-image generation through Sparse Relational Attention (SRA) and Routing Consistency Constraint (RCC).
LCG selectively attends to core features across extended visual contexts, ensuring semantic and layout information propagation remains computationally feasible. RCC leverages identity-aware masks to align structural patterns across generation branches, effectively mitigating appearance drift. To support training and evaluation in this setting, the researchers constructed the Long-Context Consistency Dataset (LCCD), comprising 600K training sequences and a separate 1K test set.
Experiments demonstrate that LCG outperforms compared baselines in prompt alignment and character consistency, particularly in multi-character scenes. This research provides a novel solution for long-context image generation, offering significant academic and practical value.
Deep Dive
Abstract
Recent image generation models achieve impressive quality in single-image synthesis, but often fail to maintain consistency across sequential outputs, as required in comics, storyboards, and visual narratives. We propose Long-Context Generation (LCG), a framework for long-context multi-image text-to-image generation, to improve consistency and scalability in long-context multi-image generation. LCG employs the Sparse Relational Attention (SRA) mechanism to selectively attend to core features across extended visual contexts, ensuring that the propagation of semantic and layout information remains computationally tractable. To enforce semantic alignment, we introduce the Routing Consistency Constraint (RCC), which leverages identity-aware masks to align structural patterns across generation branches, effectively mitigating drift in appearance even in complex multi-character scenes. To support training and evaluation in this setting, we construct the Long-Context Consistency Dataset (LCCD), a large-scale synthetic dataset comprising character-centric multi-image sequences spanning varied situational contexts. LCCD contains 600K training sequences and a separate 1K test set, with each sequence containing 6 to 20 images. The experiments demonstrate that LCG outperforms the compared baselines in prompt alignment and character consistency for long-context image generation, including multi-character scenes.