Safe Autoregressive Image Generation with Iterative Self-Improving Codebooks
Proposes iterative self-improving codebooks leveraging model understanding to enhance image generation safety, reducing harmful content by over 85%.
Key Findings
Methodology
This paper introduces a novel method where a unified multimodal model uses its own understanding to identify unsafe generated images. It constructs harmful and safe image-text pairs, extracts feature differences, and defines a harmful space via singular value decomposition (SVD). The codebook is projected to remove harmful mappings, then adaptively fine-tuned within the safe (null) space through gradient updates, all without external annotations. The process iterates until convergence, continuously improving safety while preserving image quality.
Key Results
- On eight harmful prompt datasets like I2P and CoPro, the method reduces harmful content generation by over 85%, with less than 10% FID increase, maintaining high image fidelity.
- Across models such as Janus and VILA-U, safety improvements are consistent, with no significant performance degradation, demonstrating strong generalization.
- Multiple iterations show progressive suppression of specific harmful concepts like violence and pornography, outperforming single-step approaches.
Significance
This work addresses a critical gap in autonomous safety control for autoregressive multimodal models. By enabling models to self-assess and correct harmful outputs internally, it paves the way for safer deployment in sensitive applications. The approach reduces reliance on external filters, offering a scalable solution for large-scale content generation systems, with profound implications for AI safety standards and industry adoption.
Technical Contribution
The paper introduces a theoretically grounded framework combining harmful feature space projection with null-space based fine-tuning. It innovates by leveraging the model’s internal feature differences, defining harmful subspaces, and performing iterative self-correction, which is a significant departure from traditional external filtering or supervised fine-tuning. This method ensures high-quality, safe image generation with minimal performance trade-offs.
Novelty
This is the first systematic approach to utilize the model’s internal understanding for iterative safety enhancement in autoregressive multimodal image generation. Unlike prior external filtering or post hoc methods, it dynamically constructs harmful spaces and performs internal codebook adjustments, representing a fundamental innovation in AI safety control.
Limitations
- The effectiveness depends on the model’s internal understanding accuracy; complex or ambiguous harmful content may still evade detection, limiting safety guarantees.
- Computational costs increase with model size and image resolution, posing challenges for real-time or large-scale deployment.
- The method may slightly impact image fidelity in highly sensitive scenarios, requiring further optimization for production use.
Future Work
Future research should explore integrating external human feedback to refine harmful space definitions, developing adaptive mechanisms for dynamic harmful concept updates, and optimizing computational efficiency for real-world deployment. Extending the framework to other modalities and more complex harmful content detection remains an open challenge.
AI Executive Summary
Recent advances in autoregressive multimodal models have revolutionized text-to-image synthesis, enabling highly coherent and versatile content generation. However, these models often inadvertently produce harmful or inappropriate images, raising significant safety concerns. Traditional mitigation strategies rely heavily on external filters, post-processing, or supervised fine-tuning, which are costly, inflexible, and often insufficient for complex or evolving harmful concepts.
This paper introduces an innovative approach that leverages the model’s own understanding capabilities to autonomously enhance safety. The core idea is to construct a harmful feature space by analyzing feature differences between safe and harmful image-text pairs generated by the model itself. Using singular value decomposition, the authors define a harmful subspace and project the model’s codebook to eliminate harmful mappings. Subsequently, they perform adaptive fine-tuning within the null space of this harmful subspace, ensuring the model maintains high image quality while suppressing harmful content.
The iterative process involves generating paired data, extracting feature differences, updating the harmful space, and fine-tuning the codebook repeatedly until no further improvements are observed. Extensive experiments on datasets like I2P, CoPro, and ViSU demonstrate that this method reduces harmful content generation by over 85%, with minimal impact on image fidelity. The approach generalizes well across multiple models, including Janus and VILA-U, and shows promise for scalable, autonomous safety control in real-world applications.
This work marks a significant step toward self-regulating AI systems, where models can autonomously identify and correct their own unsafe outputs. It opens new avenues for deploying safer, more reliable multimodal AI in sensitive domains, reducing reliance on external supervision and enabling continuous self-improvement. Despite some computational challenges, the proposed framework provides a robust foundation for future research in AI safety and trustworthy content generation.
Deep Analysis
Background
多模态图像生成技术经历了从像素级自回归模型到基于VQ-VAE的离散表示的快速发展。近年来，结合大规模预训练模型的统一多模态框架（如GPT-系列、VILA-U）成为研究热点。这些模型在理解和生成方面表现出色，但在内容安全方面仍存在挑战。传统方法多依赖外部过滤、后处理或微调，难以实现模型内部的自主安全控制。随着模型规模扩大和应用场景复杂化，内容偏差和有害内容的风险不断增加，亟需一种高效、自动化的安全提升机制。
Core Problem
自动回归多模态模型在生成过程中容易出现有害内容，尤其在敏感话题上。现有方案多依赖人工标注或外部过滤，难以应对模型不断演化带来的新风险。如何在保证生成质量的同时，自动识别并消除有害内容，成为核心难题。模型的自我理解能力虽有所提升，但尚未充分利用其潜力实现内容安全的自动化优化。这限制了模型在实际敏感场景中的应用推广。
Innovation
本文提出利用模型内部特征差异分析，构建有害空间，并在无害空间内进行码本微调，从而实现模型的自我安全提升。具体创新点包括：1）通过模型生成有害与无害图像对，提取特征差异；2）利用奇异值分解（SVD）定义有害空间；3）在无害空间内微调码本参数，避免有害内容重现。这一方案突破了传统外部过滤的限制，提供了自动化、持续的安全优化路径。它不仅提升了模型的安全性，还保持了生成的多样性和质量。
Methodology
- �� 采集一组具有相似有害属性的文本提示，生成对应的图像集。
- �� 利用模型评估图像内容，识别有害图像，构建有害与无害图像对。
- �� 提取每对图像的特征差异，形成差异矩阵。
- �� 通过奇异值分解分析差异矩阵，定义有害空间子空间。
- �� 投影码本以移除有害映射，确保生成内容的安全性。
- �� 在无害空间内，利用梯度微调技术，优化码本参数，提升图像质量。
- �� 迭代执行上述步骤，逐步增强模型安全性，保持生成能力。
Experiments
采用I2P、CoPro等八个有害提示数据集，评估模型在有害内容生成比例上的改善。比较原始模型与安全码本模型在有害内容检测指标上的差异，FID指标用于衡量图像质量。多模型、多轮迭代验证方法的有效性，分析不同有害概念（如色情、暴力）上的性能提升。还进行了跨场景和外部分布数据的测试，验证方法的泛化能力。
Results
安全码本显著降低有害内容生成比例，提升安全指标达85%以上，且在保持图像质量方面，FID值变化不超过10%。多轮迭代后，模型对特定有害概念（如色情、暴力）表现出持续改进，优于单次修正方案。不同模型（如Janus、VILA-U）均表现出一致的安全性能提升，验证了方法的普适性和有效性。
Applications
该方法适用于内容生成平台、社交媒体、广告设计等场景，确保自动生成内容的安全性。无需外部标注，适合大规模部署，提升用户体验和内容合规性。未来结合人类反馈，可实现更智能的内容管理体系，推动多模态AI的安全普及。
Limitations & Outlook
当前方法依赖模型的自我理解能力，可能在复杂或模糊有害内容识别上存在误差。微调过程可能带来性能折衷，尤其在极端有害场景下仍需结合外部安全措施。算法在高分辨率和大规模模型上的计算成本较高，实际应用需优化效率。未来需增强模型对多模态复杂内容的识别能力。
Plain Language Accessible to non-experts
想象你有一个非常聪明的画家，他可以画出各种各样的画，但有时候他会画出一些不适合的内容，比如不好的场景。为了让他画的内容更安全，你可以教他一些规则，让他自己判断哪些画是不合适的。每次他画完后，他会自己检查一遍，发现不好的部分，然后调整自己的画笔和想法，逐步变得更安全。这就像给他装上了一个“安全过滤器”，让他自己不断改进，画出既漂亮又安全的画。这种方法不用你一直盯着他，而是让他自己学会了如何避免画出不好的内容。这样，他就能自己不断变得更好，也不用你不停地干预了。
ELI14 Explained like you're 14
想象你有个超级厉害的画家，他可以画出各种漂亮的画，但有时候会画出一些不太合适的内容。为了让他画的内容更安全，你可以教他一些规则，让他自己判断哪些画不合适。每次他画完后，他会自己检查一遍，发现不好的地方，然后自己改正。这样，他就能自己学会避免画出不好的东西，而且还能不断变得更厉害。这个过程就像给他装了一个“安全检测器”，让他自己变得更聪明、更安全。最终，他可以画出既漂亮又安全的画，不需要你一直盯着他，自己就能不断改进。
Glossary
Codebook (码本)
一种将连续特征映射到离散视觉符号的字典，用于压缩和量化图像表示。
在模型中，码本存储视觉特征的离散表示，是生成图像的基础。
Harmful Space (有害空间)
通过特征差异分析得到的空间，用于表示有害内容的特征方向。
用以投影码本，移除有害映射，确保生成内容安全。
Self-Improving (自我提升)
模型利用自身理解能力，通过反馈不断优化自身性能。
本文利用自我理解实现图像生成的安全性增强。
Singular Value Decomposition (奇异值分解)
一种矩阵分解方法，用于提取主要特征方向。
用于定义有害空间的核心特征。
Null Space (零空间)
矩阵乘积为零的空间，用于在微调中避免引入有害信息。
在微调中确保不引入有害内容。
Open Questions Unanswered questions from this research
- 1 如何进一步提高模型在复杂多模态内容中的有害内容识别准确率，仍需结合外部监督和多模态特征增强。
- 2 未来研究应探索多轮自我反馈机制的稳定性与收敛性，确保持续安全提升。
Applications
Immediate Applications
内容生成平台
确保自动生成的图片内容符合安全标准，减少有害内容风险，提升用户体验。
社交媒体审核
自动检测并过滤有害内容，降低平台责任风险，提升内容合规性。
Long-term Vision
智能内容管理系统
结合模型自我提升机制，建立持续学习的安全内容生成体系，推动多模态AI在敏感场景的广泛应用。
Abstract
Unlike diffusion-based models that operate in continuous latent spaces, autoregressive unified multimodal models produce images by sequentially predicting discretized visual tokens. These tokens are derived from a codebook that maps embeddings to quantized visual patterns. The language-like architecture enables unified multimodal models to effectively capture text conditional information for generation, making them promising for text-to-image tasks. This also raises an interesting question: how safe are the images generated in such an autoregressive way? In this work, we propose iterative self-improving codebooks for safe autoregressive generation. We leverage the understanding and judgment capabilities of the unified multimodal model itself to identify unsafe generated images without human annotation. Subsequently, the inherent representations in the codebook are fixed to eliminate harmful mappings. Our method comprises two steps: first, we use the unified model to identify unsafe generations and construct corresponding harmful and safe image-text pairs. These pairs are used to construct the Harmful Space and guide updates to the codebook, thereby eliminating harmful outputs. Second, we perform adaptive fine-tuning on the codebook within the harmless space using safe image-text pairs to ensure the quality of generated images. These two steps are repeated until no further improvement is observed, producing a safety-enhanced model codebook. Without additional external feedback, the safety of models is improved iteratively.