PLACID: Identity-Preserving Multi-Object Compositing via Video Diffusion with Synthetic Trajectories
PLACID leverages pre-trained video diffusion models with synthetic trajectories for multi-object compositing, achieving high fidelity and identity preservation.
Key Findings
Methodology
This approach builds upon a pre-trained image-to-video (I2V) diffusion model, integrating text control and synthetic trajectories to ensure object identity consistency during compositing. The process begins with randomly placing object images on a background, then generating a sequence of frames where objects move smoothly along synthetic trajectories towards target positions. During training, short synthetic videos are created by interpolating object movements along linear paths, guided by text prompts and multi-modal encodings (CLIP, SAM). The model, based on a modified DiT architecture, incorporates high-resolution object image encodings and special textual tokens to explicitly reference each object and background. During inference, objects initialized randomly converge into coherent layouts under text guidance, with the final frame serving as the composite image.
Key Results
- On public datasets ABO and DreamBench++, PLACID achieved a CLIP-I score of 0.705, Chamfer distance of 4.641, and a missing object rate of 0.044, outperforming state-of-the-art methods. Quantitative metrics show superior identity preservation, background fidelity, and fewer omissions. User studies also favored PLACID’s outputs for visual coherence and detail accuracy.
- Synthetic trajectory training ensures smooth object motion and layout consistency, avoiding artifacts common in naive interpolation. Ablation studies confirm that high-resolution encoding and special textual tokens significantly improve compositional quality, especially in complex scenes.
- Results demonstrate that leveraging video priors and synthetic trajectories enhances multi-object scene generation, with notable improvements in detail, color fidelity, and layout control over previous static or single-object models.
Significance
This work advances the field of generative AI by integrating video priors into multi-object scene synthesis, addressing longstanding challenges in identity preservation, layout control, and detail retention. Its practical impact spans advertising, virtual try-on, and content creation, enabling automated, high-quality scene generation that previously required manual editing. The method bridges the gap between static image synthesis and dynamic scene understanding, opening new avenues for realistic virtual environments and interactive content. By effectively combining synthetic data generation with pre-trained video models, it sets a new standard for scalable, controllable multi-object generation.
Technical Contribution
Key innovations include: 1) introducing synthetic, motion-based trajectories for training, ensuring temporal and spatial consistency; 2) enhancing the DiT architecture with high-resolution object encodings and explicit textual tokens for multi-object guidance; 3) designing a synthetic data pipeline that simulates realistic object movements, compensating for the scarcity of real dynamic datasets. These contributions enable stable, detailed, and identity-preserving multi-object scene synthesis, surpassing existing static image diffusion models and single-object video methods.
Novelty
This is the first work to incorporate synthetic motion trajectories into a video diffusion framework for multi-object compositing, explicitly ensuring object identity and layout coherence. Unlike prior static or single-object models, PLACID leverages video priors for dynamic, multi-object scene generation guided by text, representing a paradigm shift in controllable scene synthesis.
Limitations
- The model struggles with scenes involving heavy occlusion or complex interactions, where object details may be lost or misaligned. Its reliance on linear trajectories limits handling non-linear or highly dynamic motions. Computational costs remain high due to synthetic data generation and model fine-tuning. Future work should explore more flexible motion models and multi-modal scene understanding to address these issues.
Future Work
Future directions include developing non-linear and more realistic motion trajectories, integrating depth and lighting cues for enhanced scene realism, and expanding training datasets with real dynamic scenes. Additionally, exploring interactive editing and real-time synthesis could broaden practical applications in AR/VR and virtual production.
AI Executive Summary
The rapid progress of generative AI has revolutionized image and video synthesis, yet creating professional-grade multi-object scenes remains challenging. Traditional workflows rely heavily on manual editing, which is time-consuming and requires expertise. Existing diffusion models excel at single-object generation but struggle with complex multi-object layouts, especially in maintaining object identities, colors, and spatial relationships. This gap limits their deployment in commercial and creative industries.
The paper introduces PLACID, a novel framework that leverages a pre-trained image-to-video diffusion model, enhanced with synthetic trajectories and multi-modal encodings, to produce high-fidelity multi-object composites guided by text prompts. The core innovation lies in synthesizing short videos where objects follow smooth, physically plausible trajectories from random initial positions to target layouts. This approach exploits the strong temporal priors of video models, ensuring objects retain their identities and details throughout the transformation.
Training involves generating synthetic data with linear trajectories, which aligns with the video model’s priors, enabling the model to learn realistic object motion and layout control. During inference, objects initialized randomly converge into coherent scenes under text guidance, with the final frame serving as the composite image. Extensive experiments on datasets like ABO and DreamBench++ demonstrate that PLACID significantly outperforms existing methods, achieving higher scores in identity preservation, background fidelity, and completeness, with fewer omissions and more natural layouts.
This work marks a significant step toward automating complex scene synthesis, with broad implications for advertising, virtual try-on, content creation, and beyond. By integrating motion-based training and multi-modal guidance, it opens new avenues for scalable, controllable, and realistic multi-object scene generation. Future research will focus on handling more complex motions, reducing computational costs, and expanding real-world applications in immersive media.
Deep Analysis
Background
多物体合成技术经历了从传统拼接、色彩融合到深度学习的演变。早期方法如色彩匹配和边缘融合解决了简单场景，但在复杂布局和细节保持方面仍有限。近年来，扩散模型如DALL·E、Stable Diffusion在图像生成中取得突破，但多目标场景布局和身份保持仍是难点。现有方法如VACE、NanoBanana在单目标或有限目标场景表现良好，但多目标一致性和细节复原仍需改进。传统模型多依赖静态图像，难以实现自然运动和连续布局。随着视频理解和生成技术的发展，结合视频先验的多目标合成成为新趋势，但如何在保证细节和身份的同时实现复杂布局，仍是研究难点。
Core Problem
多目标场景合成的核心难题在于同时满足身份保持、布局控制、细节还原和视觉自然性。现有模型在多目标合成中常出现物体变形、遗漏或错位，布局不合理，色彩失真等问题。这些问题限制了其在专业场景中的应用。尤其是在复杂背景和多物体交互场景中，保持每个物体的唯一性和细节一致性尤为困难。传统方法缺乏有效的运动连续性保证机制，导致合成结果不自然，难以满足工业级需求。
Innovation
本研究的创新点包括：1）引入基于视频先验的合成轨迹，确保多物体运动的连续性和身份一致性；2）利用高分辨率编码增强细节保留，避免模糊；3）设计特殊文本标记，明确指示每个目标物体和背景，提升文本引导的准确性；4）采用合成轨迹生成策略，弥补真实视频数据不足的问题。这些创新使得模型在多目标合成中表现出更高的稳定性和细节还原能力，超越传统静态图像扩散模型。
Methodology
- �� 输入：背景图像（可选）、未分割的目标物体图像、描述性文本。
- �� 初始布局：随机放置目标物体在背景上，形成初始帧。
- �� 轨迹合成：通过线性合成轨迹，将目标物体逐步引导到目标位置，生成连续运动的视频序列。
- �� 训练阶段：利用合成的短视频，结合CLIP和SAM编码，微调预训练的DiT视频扩散模型，确保运动连续性和细节保留。
- �� 模型架构：在原有DiT基础上，加入高分辨率物体图像编码和特殊文本标记，增强多目标指示能力。
- �� 推理阶段：随机初始化目标物体，利用文本引导逐步收敛到合理布局，最后一帧作为合成输出。
Experiments
采用ABO和DreamBench++数据集，比较多目标合成效果。指标包括身份保持（CLIP-I得分）、背景色彩（Chamfer距离）、遗漏比例（Missing）等。模型参数调优，进行消融实验验证轨迹合成和高分辨率编码的贡献。通过用户偏好调查，评估视觉效果和布局合理性。结果显示，PLACID在多目标合成中优于SOTA，特别是在细节还原和色彩保持方面表现突出。
Results
在公开数据集上，PLACID的CLIP-I得分达0.705，Chamfer距离为4.641，遗漏比例仅为0.044，明显优于对比模型。多目标场景中，细节一致性和色彩还原显著提升，用户偏好调查中获得最高评价。消融实验验证了轨迹合成和高分辨率编码的关键作用，模型在复杂背景和多目标交互中表现出更强的稳定性和自然感。
Applications
该方法适用于广告、虚拟试衣、内容创作等行业，能自动生成高质量多物体场景，减少人工操作。只需提供目标图像和文本描述，即可实现快速布局和细节调整，极大提升生产效率。未来结合虚拟现实和增强现实，将推动虚拟场景的自动化生成，拓展多模态交互应用。
Limitations & Outlook
模型在极端复杂场景（如大量重叠或遮挡）下仍存在细节丢失和错位问题。训练成本较高，合成轨迹假设线性，难以应对非线性复杂运动。未来需引入更丰富的运动模型和多模态信息，提升场景复杂度和真实感。
Plain Language Accessible to non-experts
想象你在厨房里准备一道菜。每次你都要把不同的食材放到锅里，然后用勺子搅拌，让它们变得美味。现在，假设你有一个神奇的机器人，它可以帮你把食材放到锅里，还能自动调整位置，让菜肴看起来既漂亮又好吃。这个机器人会根据你的描述（比如“多放点胡椒”或“让蔬菜更鲜亮”），逐步把食材放到正确的位置，确保每个食材都能保持原有的味道和颜色。它还能让所有食材看起来像是一起做出来的，既自然又协调。这个机器人就像论文里的模型，利用“运动轨迹”让每个物体（食材）逐渐变到理想位置，确保它们都保持自己的特色，最后呈现出一幅完美的菜肴图像。
ELI14 Explained like you're 14
想象你在玩拼图游戏，你有很多不同的拼图块（像玩具或图片），你想把它们拼成一幅漂亮的画。可是，有时候拼图块会掉位、变形或遗漏。现在，有个聪明的机器人帮你拼图，它会先把拼图块随机放在桌子上，然后用一条看不见的“轨迹”把每个拼图块慢慢移动到正确的位置，就像用一条隐形的线把它们拉到合适的地方一样。这个机器人还会记住每个拼图块的样子和颜色，确保它们不会变形或丢失。你只需要告诉它你想要的画面，比如“一个蓝色的天空和一只红色的鸟”，它就会用这个方法，把所有拼图块都拼到一起，变成一幅漂亮的画。这就像论文里的模型，用运动轨迹和智能算法，让多物体在图片中自然地组合在一起，既漂亮又真实。
Glossary
Video Diffusion Model (视频扩散模型)
一种基于扩散过程的生成模型，用于连续生成视频内容，具有强大的时序先验能力。
论文中用来实现多目标合成的核心技术。
CLIP (Contrastive Language-Image Pretraining)
一种多模态预训练模型，用于将文本和图像映射到共同空间，评估内容一致性。
用于目标身份保持和文本引导的匹配。
Synthetic Trajectory (合成轨迹)
人为设计的目标物体运动路径，用于在训练中模拟物体平滑移动。
确保多目标运动的连续性和身份一致性。
High-Resolution Encoding (高分辨率编码)
对目标物体图像进行细节丰富的编码，避免模糊和信息丢失。
增强合成细节还原能力。
Special Text Tokens (特殊文本标记)
在文本中加入特定符号，用以明确指示不同目标或背景区域。
提升多目标文本引导的准确性。
Open Questions Unanswered questions from this research
- 1 如何在非线性复杂运动中保持物体身份和细节的连续性仍未完全解决，未来需引入更复杂的运动模型和场景理解能力。
- 2 现有训练数据多为线性轨迹，难以模拟真实世界中的非线性运动，限制模型的泛化能力。
Abstract
Recent advances in generative AI have dramatically improved photorealistic image synthesis, yet they fall short for studio-level multi-object compositing. This task demands simultaneous (i) near-perfect preservation of each item's identity, (ii) precise background and color fidelity, (iii) layout and design elements control, and (iv) complete, appealing displays showcasing all objects. However, current state-of-the-art models often alter object details, omit or duplicate objects, and produce layouts with incorrect relative sizing or inconsistent item presentations. To bridge this gap, we introduce PLACID, a framework that transforms a collection of object images into an appealing multi-object composite. Our approach makes two main contributions. First, we leverage a pretrained image-to-video (I2V) diffusion model with text control to preserve objects consistency, identities, and background details by exploiting temporal priors from videos. Second, we propose a novel data curation strategy that generates synthetic sequences where randomly placed objects smoothly move to their target positions. This synthetic data aligns with the video model's temporal priors during training. At inference, objects initialized at random positions consistently converge into coherent layouts guided by text, with the final frame serving as the composite image. Extensive quantitative evaluations and user studies demonstrate that PLACID surpasses state-of-the-art methods in multi-object compositing, achieving superior identity, background, and color preservation, with less omitted objects and visually appealing results.