DesignAsCode: Bridging Structural Editability and Visual Fidelity in Graphic Design Generation
Proposes DesignAsCode framework using HTML/CSS for editable and high-fidelity graphic design, outperforming state-of-the-art baselines.
Key Findings
Methodology
The proposed DesignAsCode framework adopts a Plan-Implement-Reflect pipeline, integrating a Semantic Planner and a Visual-Aware Reflection mechanism to achieve high-quality, editable graphic design. The Semantic Planner, trained via knowledge distillation from professional design datasets like Crello, interprets natural language prompts to generate a hierarchical, multi-layered element structure, avoiding fixed templates. The Implementation module converts this abstract plan into executable HTML/CSS code, utilizing multimodal large models such as GPT-5 for layout composition and image retrieval or generation. The Visual-Aware Reflection component iteratively renders the HTML, detects visual inconsistencies (e.g., text-background conflicts) using models like CLIP, and refines the code to improve aesthetic harmony. This closed-loop process leverages the expressive power of web standards, supporting complex styles, effects, and dynamic content, thus addressing the limitations of prior pixel-based or rigid layout methods.
Key Results
- On the Crello benchmark with 546 samples, our method achieved a structural validity score (Val) of 0.9521 and a visual consistency score (Clip) of 0.6287, surpassing recent models like DeepSeek-R1 and Qwen-8B. The designs generated received the highest subjective aesthetic scores, demonstrating superior visual fidelity and structural coherence.
- On the broader dataset of 500 high-quality samples, DesignAsCode outperformed baselines in metrics such as Validity (0.9905), Readability (0.0911), and Color harmony (67.63). The model effectively handled complex layouts, irregular hierarchies, and diverse styles, confirming its robustness across scenarios.
- Ablation studies confirmed that removing components like HTML representation, semantic planning, or visual reflection significantly degraded performance, validating the importance of each module. The framework's multi-component synergy is essential for achieving high-quality, editable designs.
Significance
This work advances the field of automated graphic design by bridging the gap between structural editability and visual fidelity. Traditional methods either produce static images, which lack post-editing flexibility, or generate abstract layouts with poor visual quality. By leveraging web standards and deep learning, DesignAsCode offers a unified solution that supports professional-grade styling, complex layouts, and automatic visual correction. This innovation has profound implications for industries like advertising, publishing, and UI design, enabling rapid, customizable, and high-quality content creation. It also opens new avenues for integrating dynamic effects and interactivity, pushing the boundaries of automated creative tools.
Technical Contribution
The core technical innovations include formalizing graphic design as an HTML/CSS code synthesis problem, introducing a semantic reasoning module trained via knowledge distillation, and designing a multi-round visual-aware reflection mechanism that iteratively refines designs based on visual feedback. The framework effectively combines structured code generation with perceptual quality control, enabling the creation of complex, visually rich, and fully editable designs. This approach surpasses prior end-to-end image generation or layout prediction models by providing explicit control over style, hierarchy, and visual effects, while maintaining high fidelity and flexibility.
Novelty
This research is the first to systematically utilize HTML/CSS as a programmatic medium for graphic design synthesis, integrating semantic reasoning and visual feedback into a unified, closed-loop pipeline. Unlike previous works limited to pixel-based or fixed-schema approaches, it leverages web standards' expressiveness to support rich styles, complex hierarchies, and dynamic effects. The combination of knowledge-distilled semantic planning with iterative visual correction represents a novel paradigm, enabling professional-grade, editable designs directly from natural language prompts. This holistic approach sets a new benchmark in automated graphic design.
Limitations
- Current models have limited support for advanced visual effects such as complex gradients, filters, and animations, which are common in professional design. Enhancing CSS style expressiveness remains a challenge.
- The visual reflection mechanism depends on pre-trained models like CLIP, which may not perform well in stylized or highly unconventional visual scenarios, potentially leading to suboptimal corrections.
- The multi-round rendering and optimization process is computationally intensive, affecting real-time applicability in industrial workflows. Future work should focus on efficiency improvements and faster inference.
Future Work
未来将重点扩展模型的视觉表达能力，支持更丰富的动画和交互效果，提升生成效率，探索端到端训练策略以降低计算成本。此外，将引入用户交互反馈机制，实现人机协同优化，增强设计的个性化和多样性。还计划结合生成对抗网络（GAN）和强化学习，进一步提升设计的创新性和适应性，推动自动化设计走向工业化应用。
AI Executive Summary
In an era where visual content dominates digital communication, automating high-quality graphic design has become a critical challenge. Traditional approaches, such as pixel-based image synthesis (e.g., Rombach et al., 2022's Stable Diffusion), excel in visual fidelity but lack editability, making post-production adjustments difficult. Conversely, layout prediction models (e.g., Gupta et al., 2021) can generate structural arrangements but produce abstract, pixel-free representations that require manual visual realization. Recent hybrid efforts combining these paradigms (e.g., Zhang et al., 2025b) attempt to bridge the gap but often fall short in producing professional-grade, editable designs due to rigid schemas and visual inconsistencies.
This paper introduces DesignAsCode, a groundbreaking framework that reimagines graphic design as a programmatic synthesis task grounded in HTML/CSS code. By leveraging the expressive power of web standards, the framework supports complex styles, layered structures, and visual effects, all while maintaining full editability. The core idea is to treat design as code generation, where a natural language prompt is translated into a structured, executable HTML/CSS document. This process is orchestrated through a three-stage pipeline: semantic reasoning, code implementation, and visual correction.
The semantic reasoning component, called the Semantic Planner, uses knowledge distillation from professional design datasets to interpret user instructions and generate a hierarchical layout plan. This plan captures content semantics, spatial relationships, and stylistic attributes, enabling flexible and diverse layout structures beyond fixed templates. The implementation module then converts this plan into HTML/CSS code, utilizing multimodal large models such as GPT-5 for layout composition and image retrieval or synthesis, ensuring the generated design aligns with the semantic intent.
To address visual inconsistencies, the Visual-Aware Reflection mechanism performs multi-round rendering and detection. It employs models like CLIP to identify visual conflicts—such as text-background color clashes or occlusions—and iteratively refines the code to improve aesthetic harmony. This closed-loop process ensures the final output is both structurally sound and visually appealing.
Extensive experiments on datasets like Crello and Broad demonstrate that DesignAsCode outperforms existing state-of-the-art models in metrics such as validity, visual coherence, and aesthetic quality. The generated designs exhibit rich styles, complex hierarchies, and high visual fidelity, validated through quantitative scores and human evaluations. The framework also unlocks advanced capabilities like automatic layout retargeting, complex document generation (e.g., resumes), and CSS-based animations.
This work marks a significant step toward intelligent, flexible, and professional graphic design automation. Its ability to produce editable, high-fidelity designs directly from natural language instructions opens new horizons for creative industries, digital marketing, and personalized content creation. Future directions include enhancing visual style support, improving computational efficiency, and integrating user feedback for interactive design refinement. Overall, DesignAsCode sets a new benchmark for programmatic, standards-based graphic synthesis, bridging the gap between structural control and visual excellence.
Deep Analysis
Background
随着深度学习技术的不断发展，图像生成模型（如Rombach et al., 2022的Stable Diffusion）在视觉内容生产中取得了巨大成功，但其输出多为非编辑的像素图，限制了后续内容调整的灵活性。同时，布局预测模型（如Gupta et al., 2021）能够提供抽象的空间结构，但缺乏细节和视觉效果，难以直接用于高质量设计。近年来，结合图像生成与布局模型的研究（如Zhang et al., 2025b）试图弥合二者差距，但仍存在表达力不足、结构刚性和视觉不协调的问题。传统的设计自动化工具多依赖模板和手工调节，效率低、缺乏灵活性。网页技术的发展为解决方案提供了新思路，HTML/CSS作为网页标准，具有丰富的样式表达能力和复杂布局支持，逐渐成为实现自动化设计的理想媒介。
Core Problem
现有的图形设计自动化方法在结构可编辑性和视觉效果之间难以兼顾。像素级的图像生成虽然具有高保真，但难以后续编辑，限制了设计的灵活性。布局预测模型虽能提供结构框架，但多为抽象表示，难以直接用于视觉呈现。结合两者的尝试（如文本到图像+布局预测）存在表达力不足、模板限制和视觉冲突等问题，导致生成的设计难以达到专业水平。如何在保证设计结构可调的同时，提升视觉效果，成为行业亟待解决的核心难题。
Innovation
本研究的创新点主要体现在：1）提出基于HTML/CSS的程序化设计合成，将设计任务转化为代码生成问题，突破传统像素和模板限制，支持丰富的样式和复杂布局；2）引入语义规划器，结合专业设计语料库，动态构建多层次元素树，避免模板刚性，增强结构多样性；3）设计多轮视觉反思机制，利用视觉模型检测渲染中的视觉不协调（如文字背景色冲突），并自动优化HTML/CSS代码，确保设计的视觉一致性和审美品质；4）实现支持复杂层级、多样样式和丰富视觉效果的设计生成，极大丰富了表达空间。这一体系将结构可编辑性与视觉保真度融合，为自动化设计提供了全新技术路径。
Methodology
- �� 将设计生成问题形式化为HTML/CSS代码合成任务，输入用户自然语言指令，输出可执行的网页代码，支持复杂布局和丰富样式。
- �� 语义规划器（Semantic Planner）基于知识蒸馏，从专业设计数据（如Crello）中提取布局推理序列、元素分组、图像层提示和文本规格，生成多层次元素树结构，避免模板刚性。
- �� 实现模块根据规划输出，将抽象布局转化为HTML/CSS代码，结合图像检索或生成模型（如GPT-5）填充视觉资产，形成初步设计。
- �� 视觉反思机制在多轮迭代中，渲染HTML，利用视觉模型（如CLIP）检测不协调（如文字背景色冲突），并自动优化代码，逐步提升设计质量。
- �� 最终生成支持丰富样式、复杂层级和高视觉质量的设计，支持动画和交互扩展。
Experiments
采用Crello（546例）和Broad（500例）两个公开数据集进行评估，比较对象包括DeepSeek-R1、Qwen-8B、GPT-5等最先进模型。指标涵盖结构有效性（Val）、布局一致性（Ali）、可读性（Rea）、语义匹配（CLIP）等，结合主观评分（由gpt-4o评估）。在Crello上，DesignAsCode在Val、Clip、色彩和布局指标上均优于对比模型，平均排名第一。在Broad数据集上，表现出更强的适应性和复杂场景处理能力。通过消融实验验证各模块的重要性，显示HTML表示、语义规划和视觉反思机制对性能提升至关重要。
Results
实验结果显示，DesignAsCode在结构有效性（Val=0.9521）和视觉一致性（Clip=0.6287）方面显著优于基线模型，整体性能优越。其生成的设计在用户主观评分中排名第一，特别是在复杂布局和多样内容场景中表现出色。消融实验表明，去除HTML表示或视觉反思机制都导致性能下降，验证了框架的协同作用。此外，模型在动画和复杂样式支持方面也表现出潜力，为未来扩展提供基础。
Applications
该方法适用于广告海报、UI界面、出版物封面等场景，用户只需提供简洁的文本指令，即可自动生成高质量设计稿。其优势在于支持快速迭代、个性化定制和多样化布局，极大提升设计效率。未来可结合交互式界面，让设计师与模型协同优化，实现个性化定制和风格迁移。长远来看，该技术有望推动自动化设计普及，降低专业门槛，赋能中小企业和个人创作者，推动创意产业的数字化转型。
Limitations & Outlook
目前模型在支持极端视觉效果（如高级渐变、滤镜、动画）方面仍有限，需进一步增强CSS样式表达能力。视觉反思机制依赖预训练模型（如CLIP），在特殊风格或极端视觉冲突场景中表现不足，可能导致优化效果不理想。此外，生成过程较为复杂，计算成本较高，影响实时性和工业应用的推广。未来需优化算法效率，增强模型的泛化能力，并探索端到端训练策略，降低硬件依赖。
Plain Language Accessible to non-experts
想象你在一家非常先进的工厂里工作，这个工厂可以自动帮你设计出漂亮的海报或网页。以前，你需要用画图软件一遍遍调整每个元素，比如文字、图片、背景，既费时又难保证每次都完美。而现在，这个工厂用一种特殊的“语言”——就像写程序一样，把设计的想法写成代码，然后工厂就能自动生成你想要的样子。这个“语言”其实是网页用的HTML和CSS，工厂通过理解你的描述，自动排版、上色、加动画，甚至还能自己检查哪里不协调，比如文字看不清或颜色不搭配，然后自动修正。这样一来，设计变得既灵活又高效，就像有一个聪明的助手帮你打理一切，让你专注于创意，而不用担心技术细节。
ELI14 Explained like you're 14
想象你在玩一个超级厉害的游戏，你可以告诉游戏里的角色你想要的场景，比如“一个热闹的派对，有很多人跳舞，背景是彩色的灯光”。游戏会根据你的描述，自动帮你搭建出这个场景。以前，你得用复杂的工具一遍遍调整每个角色的位置、灯光和颜色，特别麻烦。而现在，这个游戏用一种特别的“写作方式”——就像写作文一样，把你的想法写成代码，游戏就能理解并自动生成场景。它还能自己检查场景是否合理，比如灯光太暗或人物重叠，然后自动调整。这样一来，你只需要专注你的想法，游戏帮你把它变成了真实的画面。这个技术就像有一个聪明的助手，帮你快速创造出漂亮的场景，省时又有趣！
Glossary
HTML/CSS (HyperText Markup Language / Cascading Style Sheets)
网页的基础技术，用于定义网页的结构和样式。HTML描述内容结构，CSS控制外观效果。
作为设计的程序化表达媒介，本文利用HTML/CSS实现复杂布局和丰富视觉效果。
Semantic Planner (语义规划器)
基于自然语言理解和设计知识库，生成设计布局的推理序列和内容结构。
用于指导布局树的构建，确保设计符合内容语义和专业规范。
Visual-Aware Reflection (视觉感知反思)
利用视觉模型检测渲染效果中的不协调，并自动优化HTML/CSS代码。
确保设计在视觉上的一致性和审美品质。
Closed-loop Optimization (闭环优化)
通过多轮渲染、检测和修正，逐步提升设计质量的过程。
实现结构与视觉的同步优化，避免单向流程带来的不协调。
Multimodal Large Model (多模态大模型)
结合文本、图像等多模态信息的深度学习模型，如GPT-5。
用于生成和填充设计中的视觉资产，支持复杂布局的实现。
Knowledge Distillation (知识蒸馏)
从大模型中提取高层次推理能力，训练轻量模型。
用于提升语义规划器的专业性和多样性。
Multi-round Rendering (多轮渲染)
多次将HTML/CSS代码转化为图像，检测视觉效果，逐步优化。
实现设计的视觉一致性和美学提升。
Visual Model (视觉模型)
如CLIP，用于理解和评估图像与文本的语义一致性。
在反思机制中检测设计中的视觉不协调。
Complex Layout (复杂布局)
多层次、多区域、多样式的排版结构。
支持丰富的设计表达，超越简单模板。
CSS Animation (CSS动画)
利用CSS实现网页元素的动态效果。
作为设计的扩展能力，支持动画和交互。
Abstract
Graphic design generation demands a delicate balance between high visual fidelity and fine-grained structural editability. However, existing approaches typically bifurcate into either non-editable raster image synthesis or abstract layout generation devoid of visual content. Recent combinations of these two approaches attempt to bridge this gap but often suffer from rigid composition schemas and unresolvable visual dissonances (e.g., text-background conflicts) due to their inexpressive representation and open-loop nature. To address these challenges, we propose DesignAsCode, a novel framework that reimagines graphic design as a programmatic synthesis task using HTML/CSS. Specifically, we introduce a Plan-Implement-Reflect pipeline, incorporating a Semantic Planner to construct dynamic, variable-depth element hierarchies and a Visual-Aware Reflection mechanism that iteratively optimizes the code to rectify rendering artifacts. Extensive experiments demonstrate that DesignAsCode significantly outperforms state-of-the-art baselines in both structural validity and aesthetic quality. Furthermore, our code-native representation unlocks advanced capabilities, including automatic layout retargeting, complex document generation (e.g., resumes), and CSS-based animation. Our project page is available at https://liuziyuan1109.github.io/design-as-code/.
References (20)
Igd: Instructional Graphic Design With Multimodal Layer Generatio
Yadong Qu, Hongtao Xie, Yongdong Zhang et al.
High-Resolution Image Synthesis with Latent Diffusion Models
Robin Rombach, A. Blattmann, Dominik Lorenz et al.
LayoutVAE: Stochastic Scene Layout Generation From a Label Set
Akash Abdu Jyothi, Thibaut Durand, Jiawei He et al.
Decoupled Weight Decay Regularization
I. Loshchilov, F. Hutter
Improving
N. Loader
LayoutGAN: Generating Graphic Layouts with Wireframe Discriminators
Jianan Li, Jimei Yang, Aaron Hertzmann et al.
Billion-Scale Similarity Search with GPUs
Jeff Johnson, Matthijs Douze, H. Jégou
Improved Denoising Diffusion Probabilistic Models
Alex Nichol, Prafulla Dhariwal
Learning Transferable Visual Models From Natural Language Supervision
Alec Radford, Jong Wook Kim, Chris Hallacy et al.
Variational Transformer Networks for Layout Generation
Diego Martín Arroyo, Janis Postels, Federico Tombari
CanvasVAE: Learning to Generate Vector Graphic Documents
Kota Yamaguchi
Denoising Diffusion Implicit Models
Jiaming Song, Chenlin Meng, S. Ermon
Chain of Thought Prompting Elicits Reasoning in Large Language Models
Jason Wei, Xuezhi Wang, Dale Schuurmans et al.
Composition-aware Graphic Layout GAN for Visual-textual Presentation Designs
Min Zhou, Chenchen Xu, Ye Ma et al.
Adding Conditional Control to Text-to-Image Diffusion Models
Lvmin Zhang, Anyi Rao, Maneesh Agrawala
PosterLayout: A New Benchmark and Approach for Content-Aware Visual-Textual Presentation Layout
Hsiao-An Hsu, Xiangteng He, Yuxin Peng et al.
SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis
Dustin Podell, Zion English, Kyle Lacey et al.
LayoutPrompter: Awaken the Design Ability of Large Language Models
Jiawei Lin, Jiaqi Guo, Shizhao Sun et al.
COLE: A Hierarchical Generation Framework for Multi-Layered and Editable Graphic Design
Peidong Jia, Chen-Xuan Li, Yuhui Yuan et al.
SVGDreamer: Text Guided SVG Generation with Diffusion Model
Ximing Xing, Haitao Zhou, Chuang Wang et al.
Cited By (4)
CommerceVibe: Learning to Design E-Commerce Creatives as Executable Visual Code via Dual-Feedback Reinforcement Learning
TransAnyText: Translating Arbitrary Text in E-commerce Images via Structured Visual Generation
AutoDesign: Meta-Harness Optimization for Long-Horizon Agentic Design
Research on the Visual Rhythm and Emotional Communication Mechanism of Dynamic Graphic Design in the Digital Media Era