PosterCopilot: Toward Layout Reasoning and Controllable Editing for Professional Graphic Design
PosterCopilot employs a three-stage training framework—PSFT, RL-VRA, RLAF—to improve layout accuracy and aesthetics, enabling layer-controlled iterative editing with high fidelity.
Key Findings
Methodology
The approach integrates Perturbed Supervised Fine-Tuning (PSFT) to model continuous spatial relationships by introducing Gaussian perturbations to ground-truth coordinates. Reinforcement Learning for Visual-Reality Alignment (RL-VRA) employs geometric rewards like DIoU within a policy optimization framework (GRPO) to align generated layouts with visual and geometric principles. Reinforcement Learning from Aesthetic Feedback (RLAF) incorporates a learned aesthetic reward model to promote diverse, visually appealing layouts. The trained layout reasoning model is coupled with generative models, supporting multi-round, layer-specific editing, thus fulfilling professional design workflows.
Key Results
- PosterCopilot achieved a layout IoU of 0.347 and aspect ratio distortion of 0.061, outperforming baselines such as Nano-Banana. Human preference tests showed over 74% favorability, with user satisfaction exceeding 87%. The model supports multi-round, fine-grained editing with element preservation rates over 95%, demonstrating robustness in complex multi-asset scenarios.
- In extensive experiments, the model generalized well across a large dataset of 160,000 posters, significantly improving layout rationality and aesthetic diversity. Ablation studies confirmed the effectiveness of each training stage, especially the geometric and aesthetic rewards, which contributed to the model’s superior performance. The integration with generative agents enabled asset synthesis and theme switching, further validating practical utility.
- Results indicate that the combined training strategy effectively balances geometric accuracy and aesthetic quality, setting new standards in automated poster design. The model’s ability to handle incomplete assets and perform iterative refinements makes it suitable for real-world professional workflows.
Significance
This work advances the field of AI-assisted graphic design by bridging the gap between geometric reasoning and aesthetic judgment. The three-stage training paradigm enhances the interpretability and controllability of large multimodal models, making them more suitable for complex, multi-layered design tasks. Its practical implications include reducing manual effort, accelerating creative processes, and enabling personalized, high-quality visual content generation. The approach paves the way for intelligent design tools that can seamlessly integrate into professional workflows, transforming industries such as advertising, branding, and digital media.
Technical Contribution
The paper introduces a novel combination of distribution-based coordinate learning (PSFT) and geometric reward-driven reinforcement learning (RL-VRA), complemented by aesthetic feedback (RLAF). This framework addresses the limitations of symbolic coordinate quantization and enhances the model’s spatial reasoning and aesthetic capabilities. The coupling with generative models for layer-specific, multi-round editing extends the traditional static generation paradigm, enabling flexible, high-fidelity refinements. These innovations collectively push the boundaries of multimodal layout reasoning and controllable editing in AI-driven graphic design.
Novelty
This is the first comprehensive framework integrating three distinct training stages—perturbed supervision, geometric reinforcement, and aesthetic feedback—to enhance layout reasoning and controllability in large multimodal models. Unlike prior work that focuses solely on image synthesis or rule-based layout generation, PosterCopilot emphasizes continuous spatial reasoning, visual-real alignment, and aesthetic diversity. Its layered, iterative editing capability further distinguishes it from static generation methods, marking a significant leap in AI-assisted professional graphic design.
Limitations
- Despite improvements, the model still faces challenges in extremely complex scenes with dense overlapping elements, where layout accuracy may degrade due to data scarcity and model capacity limits.
- High-quality, annotated multi-layer poster datasets are costly to produce, limiting scalability and domain adaptation.
- Real-time multi-round editing remains computationally intensive, requiring further optimization for practical deployment.
Future Work
Future research will explore more efficient training strategies, larger and more diverse datasets, and enhanced user interaction interfaces. Incorporating user feedback directly into training loops and expanding aesthetic evaluation metrics could further improve output quality. Additionally, integrating the system into commercial design software and enabling real-time collaborative editing are promising directions.
AI Executive Summary
PosterCopilot signifies a major advancement in AI-powered graphic design, addressing longstanding challenges in layout accuracy and aesthetic control. Traditional automated methods often produce layouts with geometric inaccuracies and lack the flexibility for iterative, layer-specific refinement. To overcome these issues, the authors introduce a sophisticated three-stage training framework comprising Perturbed Supervised Fine-Tuning (PSFT), Reinforcement Learning for Visual-Reality Alignment (RL-VRA), and Reinforcement Learning from Aesthetic Feedback (RLAF). PSFT leverages Gaussian perturbations to ground-truth coordinates, enabling the model to learn continuous spatial relationships rather than discrete token mappings. This significantly improves geometric coherence, as evidenced by the IoU of 0.347 and AR of 0.061, outperforming baselines like Nano-Banana. RL-VRA employs geometric rewards such as DIoU within a policy optimization scheme (GRPO) to align generated layouts with visual principles, further refining geometric accuracy. RLAF introduces a learned aesthetic reward, encouraging diverse and visually appealing compositions, thus broadening the design space beyond ground-truth constraints. The integrated system couples these trained models with generative agents that support multi-round, layer-specific editing, allowing precise refinements and asset synthesis. Extensive experiments on a large-scale dataset of 160,000 posters demonstrate that PosterCopilot not only produces layouts with high geometric fidelity but also achieves superior aesthetic quality and user preference, with over 74% favorability in human evaluations. Its ability to handle incomplete assets, perform theme switching, and support iterative refinements makes it highly applicable to real-world professional workflows. This work opens new avenues for intelligent design tools, promising to reduce manual effort, enhance creativity, and accelerate the digital transformation of visual communication industries. Future directions include optimizing real-time interaction, expanding dataset diversity, and integrating user feedback for personalized design assistance.
Deep Analysis
Background
随着深度学习和多模态模型的发展，自动化图形设计逐渐成为研究热点。早期方法多依赖规则或模板，缺乏灵活性。近年来，Diffusion模型和大型多模态模型（如DALL·E、Stable Diffusion、Qwen-VL）在图像生成和布局推理方面取得突破，但仍存在几何失真和交互不足的问题。尤其在复杂多资产、多层次场景中，传统模型难以实现精确布局和多轮细节调整，限制了其在专业设计中的应用潜力。
Core Problem
当前LMM在布局推理中存在几何偏差，尤其在多资产、多层次场景下容易出现错位、比例失调等问题。符号化坐标的离散化导致连续空间推理能力不足，缺乏视觉反馈机制使模型难以学习符合美学原则的布局。专业设计强调多轮细节调整和层级控制，但现有模型难以实现交互式、可控的编辑，限制了其实际应用。解决这些瓶颈，成为推动智能设计工具普及的关键。
Innovation
本研究的核心创新在于引入三阶段训练策略：• PSFT通过扰动ground-truth坐标，将离散符号转化为分布学习，改善连续空间表达；• RL-VRA利用几何奖励（如DIoU）实现布局的视觉与现实对齐，确保几何准确性；• RLAF引入审美奖励，拓展布局审美空间，提升美学品质。结合生成模型，支持多轮、层级可控的细粒度编辑，突破传统静态生成限制。这些创新共同推动模型在复杂场景中的布局推理和交互能力。
Methodology
- �� 输入：用户提供元素（图片、文本、形状）和画布尺寸，编码为多模态提示。• PSFT：在ground-truth布局基础上加入高斯扰动，训练模型学习连续空间的分布表达，减少符号化偏差。• RL-VRA：定义MDP，利用几何奖励（如DIoU）优化布局空间一致性，通过策略梯度（GRPO）实现布局几何对齐。• RLAF：引入审美奖励模型，鼓励生成多样且美观的布局。• 结合生成模型，支持缺失资产补全和多轮细粒度编辑，满足专业设计需求。
Experiments
在160K多层海报数据集上训练，采用IoU、AR、元素保持率等指标评估布局质量。对比基线包括Nano-Banana、LaDeCo等，模型在几何和审美指标上显著优越。人类评估显示，PosterCopilot偏好率超过74%，用户满意度高。 Ablation研究验证了每个训练阶段的贡献，特别是RL阶段对布局精度和多样性提升明显。模型还在缺失资产补全和主题切换任务中表现出色，验证其实用性。
Results
模型在布局IoU达0.347，AR降至0.061，布局几何误差显著降低。人类偏好测试中，PosterCopilot偏好率超过87%。支持多轮细粒度编辑，元素保持率超过95%。在复杂多资产场景中，模型实现了高质量、多样化的设计输出，验证了其在实际应用中的潜力。
Applications
广泛应用于广告、文化宣传、商业海报等场景。设计师可利用模型快速生成高质量布局，进行多轮细节调整，提升效率。企业可借助模型实现自动化内容生成和个性化定制，降低成本，缩短设计周期。未来，结合交互界面，模型有望成为专业设计软件的核心组件，推动行业数字化升级。
Limitations & Outlook
模型在极端复杂场景下仍存在布局偏差，部分元素重叠或比例失调。高质量训练数据的获取成本较高，泛化能力有限。实时交互和多轮编辑的效率有待提升，需优化推理速度和用户体验。未来需解决模型在特定风格和行业中的适应性问题。
Plain Language Accessible to non-experts
想象你在做一份海报，就像在搭建一个拼图。每个元素（图片、文字、形状）都像拼图块，你需要把它们放在合适的位置，让整个画面既漂亮又有逻辑。传统的方法就像用手指一块一块拼，容易出错，还要反复调整。现在，这个系统像是有一个聪明的助手，能理解每个拼图块的空间关系，帮你自动安排好位置，还能根据你的喜好调整风格。它学会了如何让拼图既整齐又美观，还能帮你补全缺失的部分，甚至换个主题。这样一来，设计变得快多了，也更容易做出专业的效果，就像有了一个超级助手帮你搞定所有难题。
ELI14 Explained like you're 14
想象你在学校里做一个海报，想让它看起来既漂亮又有趣。以前，你得自己慢慢调整每个元素的位置，比如图片、文字和装饰物，花很多时间，而且还不一定能做到完美。现在，有个特别聪明的机器人助手，它能理解每个元素应该放在哪里，让整个海报看起来很协调。它还能帮你补全缺少的部分，比如没有背景或者装饰物，还能根据你的想法换个主题，比如从“生日派对”变成“毕业庆典”。你只需要告诉它你的想法，它就会帮你安排好所有细节，最后你只要做一些微调就可以了。这样，你的海报既专业又漂亮，还省了很多时间，真是太棒了！
Abstract
Graphic design forms the cornerstone of modern visual communication, serving as a vital medium for promoting cultural and commercial events. Recent advances have explored automating this process using Large Multimodal Models (LMMs), yet existing methods often produce geometrically inaccurate layouts and lack the iterative, layer-specific editing required in professional workflows. To address these limitations, we present PosterCopilot, a framework that advances layout reasoning and controllable editing for professional graphic design. Specifically, we introduce a progressive three-stage training strategy that equips LMMs with geometric understanding and aesthetic reasoning for layout design, consisting of Perturbed Supervised Fine-Tuning, Reinforcement Learning for Visual-Reality Alignment, and Reinforcement Learning from Aesthetic Feedback. Furthermore, we develop a complete workflow that couples the trained LMM-based design model with generative models, enabling layer-controllable, iterative editing for precise element refinement while maintaining global visual consistency. Extensive experiments demonstrate that PosterCopilot achieves geometrically accurate and aesthetically superior layouts, offering unprecedented controllability for professional iterative design.