CreatiPoster: Towards Editable and Controllable Multi-Layer Graphic Design Generation
CreatiPoster employs a multi-layer protocol model to generate editable graphic compositions, with background synthesis ensuring visual harmony and flexibility.
Key Findings
Methodology
The approach features a two-stage architecture: first, a large multimodal protocol generator based on RGBA models converts user prompts, assets, and constraints into a detailed JSON specification describing each layer’s content, position, style, and order, along with a background caption. Second, a foreground-aware background synthesizer (MM-DiT backbone) generates a background conditioned on the rendered foreground, maintaining element integrity and visual coherence. The protocol generator is trained via a self-supervised protocol completion task supporting multiple modes: prompt-only, asset-conditioned, canvas editing, and re-layout. This unified framework enables flexible, multi-task graphic design, with the background synthesis ensuring contextual harmony. The entire system preserves element editability and asset fidelity, addressing limitations of template-based and pixel-based methods.
Key Results
- On a newly proposed multi-layer benchmark, CreatiPoster outperforms state-of-the-art open-source and commercial systems, with average scores exceeding competitors by approximately 0.5 points (out of 5). The model demonstrates strong generalization across tasks such as prompt completion, asset placement, and layout reconfiguration. Using a dataset of 100,000 copyright-free multi-layer designs, it exhibits high diversity and robustness. In practical applications, it achieves high-quality, editable outputs for canvas editing, text overlay, multilingual generation, and animated posters, with significant improvements in visual coherence and user control.
Significance
This research introduces a novel paradigm shift in AI-assisted graphic design, transforming pixel-based generation into structured, editable protocol programs. It addresses core challenges in asset preservation, multi-task control, and aesthetic quality, enabling more flexible, professional, and user-friendly design tools. The approach fosters democratization of design, reducing reliance on templates and expert skills, and opens new avenues for personalized content creation in industry and academia. Its ability to generate hierarchically organized, editable layers with contextual background synthesis marks a significant step forward in intelligent design automation.
Technical Contribution
Key innovations include: (1) a layered protocol model that encodes content, layout, and style as an executable JSON specification supporting multi-task completion; (2) a foreground-aware background synthesizer based on MM-DiT that generates backgrounds conditioned on foreground elements, ensuring visual harmony; (3) a unified training strategy for multi-modal protocol completion supporting prompt, asset, canvas, and re-layout modes; (4) a flexible application framework supporting canvas resizing, text overlay, multilingual, and animated outputs. These contributions collectively advance the controllability, editability, and diversity of AI-generated graphic designs, surpassing prior pixel-centric or template-dependent methods.
Novelty
This work is the first to formalize graphic design as an editable protocol program, integrating multimodal models for multi-task control. Unlike previous methods that rely on fixed templates or pixel-level synthesis, CreatiPoster encodes design elements hierarchically, enabling flexible editing and scene reconfiguration. The foreground-aware background synthesis further distinguishes this approach by maintaining element fidelity while generating contextually consistent backgrounds. These innovations collectively push the boundary of AI-driven design, offering a unified, controllable, and extensible framework.
Limitations
- The current model struggles with complex dynamic scenes involving multiple interacting elements and animated backgrounds, due to limited training data and model capacity.
- Background synthesis is primarily static, lacking temporal consistency for animations or light effects, which requires future integration of temporal modeling.
- High computational costs for training and inference limit large-scale deployment and real-time applications.
Future Work
Future directions include integrating temporal modeling for dynamic scene and animation generation, optimizing model architectures for efficiency, expanding multilingual and style adaptation capabilities, and developing interactive interfaces for real-time user control. Additionally, incorporating user feedback loops and reinforcement learning could further enhance customization and usability, pushing AI design tools toward more intelligent, adaptive, and accessible systems.
AI Executive Summary
CreatiPoster represents a significant advancement in AI-assisted graphic design, shifting from traditional pixel-based generation to a structured, editable protocol framework. By employing a layered protocol generator based on a large multimodal RGBA model, it converts user prompts, assets, and constraints into a detailed JSON specification that describes each design element's content, position, style, and hierarchy. This protocol supports multiple modes, including prompt-only creation, asset-conditioned placement, canvas editing, and re-layout, all within a unified, end-to-end trained system. Complementing this, a foreground-aware background synthesizer based on MM-DiT architecture generates backgrounds conditioned on the rendered foreground, ensuring visual harmony and element integrity. Extensive experiments on a new benchmark demonstrate that CreatiPoster outperforms existing open-source and commercial systems in layout quality, aesthetic appeal, and adherence to prompts, with scores surpassing competitors by approximately 0.5 points on average. Its ability to produce fully editable, multi-layer compositions enables versatile applications such as canvas resizing, text overlay, multilingual support, and animated posters, facilitating a democratization of AI-driven design. The release of a dataset of 100,000 multi-layer designs further accelerates research in this domain. Overall, CreatiPoster offers a flexible, scalable, and high-quality solution that addresses core challenges in AI graphic design, paving the way for more intelligent, user-centric creative tools.
Deep Analysis
Background
随着深度学习和多模态模型的发展，AI在图形设计中的应用逐渐成熟。早期主要依赖规则和模板，如Adobe Spark和Canva，解决了部分自动化需求。近年来，GAN和Transformer模型推动了高质量图像生成，但多任务、多场景的可控设计仍受限于资产保护和编辑灵活性。多模态大模型（如Stable Diffusion、MM-DiT）实现了多层次、多任务的生成，但资产融合和多场景控制仍未完全解决。CreatiPoster结合协议表达和前景感知背景合成，突破了多场景、多任务设计的瓶颈，推动了AI在个性化和专业化设计中的应用。
Core Problem
现有AI设计工具难以同时满足高质量、可编辑、资产完整和专业审美的需求。模板依赖限制创新，像素生成缺乏层级和结构，难以进行后续编辑。多资产融合和多任务支持不足，限制了设计的多样性和实用性。如何在保证资产完整的基础上，实现多场景、多任务的灵活控制，是当前亟待解决的核心问题。
Innovation
提出多层协议模型，将设计内容、布局、样式等信息以JSON协议表达，支持多任务端到端补全。引入前景感知背景合成器，利用MM-DiT架构在保持前景不变的同时，生成协调背景，确保整体一致性。采用协议补全训练策略，支持prompt-only、资产布局和重布局等多场景操作。构建多应用场景支持体系，实现画布重排、多语言和动画，突破传统像素限制，推动设计的可控性和可编辑性。这些创新共同推动了AI在图形设计中的变革。
Methodology
- �� 输入：用户指令、资产、画布尺寸、约束。• 协议生成：利用RGBA多模态模型，将内容、位置、样式转化为JSON协议。• 协议补全：训练模型在部分信息缺失时补全完整协议，支持多任务。• 背景合成：在保持前景元素不变的基础上，利用MM-DiT生成协调背景。• 渲染：将协议转化为可编辑图层，支持后续修改。• 训练：端到端多任务训练，结合多模态条件，优化协议补全和背景合成。• 应用：支持画布调整、文字叠加、多语言和动画等多场景操作。
Experiments
采用自制的多层设计基准，包含45个prompt-only、39个资产条件和6个多资产任务。指标包括布局、色彩、风格和符合度，由GPT-4和人类评审评分。模型在自动指标和主观评价中优于OpenCOLE、Microsoft Designer和Canva。 Ablation研究验证协议模型的多任务能力和背景合成的协调性。模型在不同场景下表现出良好的泛化能力和稳定性。
Results
CreatiPoster在布局、色彩、风格和符合度方面均优于对比系统，平均评分提升0.4-0.6分（满分5）。在多场景应用中，支持多语言、动画和响应式调整，效果自然流畅。100,000样本验证了模型的多样性和稳定性，显示出强大的适应能力。用户和评审均认可其设计质量和编辑便利性，显著推动AI设计的实用化。
Applications
支持画布重排、文字叠加、多语言生成和动画海报，适用于广告、社交媒体、电子商务等场景。用户只需提供基本指令和资产，即可快速生成高质量设计。系统可集成到设计软件中，提升设计效率和个性化水平。未来，结合实时交互和智能推荐，将实现更智能、更个性化的设计体验。
Limitations & Outlook
模型对复杂动态场景和多元素交互仍有局限，背景合成在光影和动画方面表现不足。训练成本较高，需大量GPU资源，限制大规模推广。未来需引入时序建模和优化算法，提升动态效果和效率。
Plain Language Accessible to non-experts
想象你在厨房做菜，准备各种食材、调料和厨具。传统的做菜方法可能需要你逐一准备每样东西，花费时间。而现在，有个智能厨师可以根据你的菜单建议，自动准备好所有食材和调料，把它们按照最佳顺序摆放好，还能根据你的偏好调整味道和摆盘。这就像CreatiPoster一样，它用一种特别的“菜谱”——协议程序，把设计的每个元素、位置、样式都写下来。你只需告诉它想要什么风格，它就能自动帮你安排好所有内容，背景也会根据前景元素自动生成，整体协调又漂亮。而且，你还可以随时修改某个菜品或摆放方式，它都能快速调整，整个过程既高效又灵活。这种智能“厨师”让设计变得像做菜一样简单、快捷、个性化。
ELI14 Explained like you're 14
想象你在玩一个超级酷的拼图游戏，你有很多不同的拼图块，比如图片、文字和背景。以前，要拼出一张漂亮的海报，你得自己一块一块拼，还要保证颜色搭配、排版合理，非常难。而现在，有个智能帮手，它可以根据你的想法，自动告诉你每个拼图块应该放在哪里，还能帮你生成漂亮的背景，让整个海报看起来很专业。你只要说出你的主题，比如“科技未来”，它就会帮你安排好所有元素，还能让背景和前景协调一致。你可以随时改变某个元素的位置或内容，它会立刻帮你调整。就像你有了一个超级聪明的拼图助手，让你轻松做出漂亮的海报，不用担心复杂的排版问题。这就是CreatiPoster带来的魔法！
Abstract
Graphic design plays a crucial role in both commercial and personal contexts, yet creating high-quality, editable, and aesthetically pleasing graphic compositions remains a time-consuming and skill-intensive task, especially for beginners. Current AI tools automate parts of the workflow, but struggle to accurately incorporate user-supplied assets, maintain editability, and achieve professional visual appeal. Commercial systems, like Canva Magic Design, rely on vast template libraries, which are impractical for replicate. In this paper, we introduce CreatiPoster, a framework that generates editable, multi-layer compositions from optional natural-language instructions or assets. A protocol model, an RGBA large multimodal model, first produces a JSON specification detailing every layer (text or asset) with precise layout, hierarchy, content and style, plus a concise background prompt. A conditional background model then synthesizes a coherent background conditioned on this rendered foreground layers. We construct a benchmark with automated metrics for graphic-design generation and show that CreatiPoster surpasses leading open-source approaches and proprietary commercial systems. To catalyze further research, we release a copyright-free corpus of 100,000 multi-layer designs. CreatiPoster supports diverse applications such as canvas editing, text overlay, responsive resizing, multilingual adaptation, and animated posters, advancing the democratization of AI-assisted graphic design. Project homepage: https://github.com/graphic-design-ai/creatiposter