Identity-Preserving Text-to-Video Generation via Agentic Enhancement and Semantic Repair
AESR combines global prompt enhancement and semantic repair to improve identity-preserving video generation.
Key Findings
Methodology
The proposed AESR framework integrates a global agentic prompt enhancement module and a sample-level semantic repair module. The former learns model-specific prompt formats from official documentation and human interaction data, creating a reusable knowledge base that guides prompt construction. The latter uses vision-language models (VLMs) to locate errors in generated videos, designing explicit repair instructions and editing keyframes with a video editing model. A lightweight Mixture-of-Experts (MoE) strategy is employed to select the most reliable outputs. Experiments on ACM MM 2026 challenge datasets show that AESR outperforms baseline methods, achieving a 15% improvement in identity similarity scores and top ranking in the challenge.
Key Results
- The system achieved a facial identity similarity of 0.75 (ArcFace), Motion Smoothness score of 0.88, surpassing baseline by 15%.
- Multi-path generation and refinement significantly reduced identity drift and missing details, especially in multi-subject scenarios.
- The agentic loop enabled automatic learning from failures, continuously refining prompts and repair strategies, leading to robust performance across diverse prompts.
Significance
This work addresses the core challenge of controlling closed-source video generation models for identity preservation without parameter fine-tuning. By focusing on interface-level prompt optimization and targeted semantic repair, it offers a practical solution that enhances controllability and detail fidelity. The approach bridges the gap between black-box models and real-world applications, enabling industries like virtual avatar creation, film post-production, and personalized content generation to adopt AI-driven video synthesis more effectively. Its modular design and knowledge-driven feedback loop set a new standard for controllable generative AI, fostering broader adoption and further innovation.
Technical Contribution
AESR introduces a novel combination of knowledge-driven prompt enhancement and explicit semantic repair, leveraging external datasets and model documentation to build a dynamic, reusable prompt playbook. The integration of VLM-based error localization with targeted editing and a MoE-based output selection strategy enhances reliability. Unlike traditional fine-tuning, AESR operates solely at the interface level, making it adaptable to proprietary models. Its end-to-end design offers a scalable and efficient pathway for improving identity preservation in black-box video generation systems, opening new avenues for controllable synthesis.
Novelty
This study pioneers the systematic integration of global prompt learning with sample-level semantic correction in the context of closed-source video models. Unlike prior work focused on fine-tuning or post-processing, AESR's knowledge accumulation and explicit visual referencing enable high-precision, identity-preserving generation without modifying model weights. Its agentic loop mechanism for continuous knowledge update and the use of visual references for repair are innovative contributions that set it apart from existing approaches.
Limitations
- Dependence on pre-trained VLMs and video editing models may limit performance in highly complex, high-resolution scenarios.
- The agentic loop requires multiple feedback iterations, which may hinder real-time applications.
- Maintaining identity consistency over long sequences with rapid scene changes remains challenging, requiring further temporal modeling enhancements.
Future Work
Future research will explore integrating reinforcement learning to optimize prompt strategies dynamically, developing more efficient semantic repair techniques for real-time processing, and extending the framework to multi-modal video synthesis tasks. Additionally, improving temporal coherence and identity stability in longer videos will be prioritized to broaden practical deployment in entertainment and virtual reality industries.
AI Executive Summary
The rapid advancement of deep generative models has revolutionized visual content creation, yet controlling these models to produce identity-preserving videos remains a significant challenge. Existing commercial solutions like Seedance 2.0 excel in visual fidelity but often suffer from identity drift and incomplete adherence to complex instructions. These models are typically closed-source, making direct parameter tuning infeasible, which limits their adaptability and controllability.
To address these issues, this paper introduces AESR, a novel interface-level framework that combines global prompt enhancement with local semantic repair. The core idea is to leverage external knowledge sources—such as official model documentation and human interaction data—to learn reusable prompt strategies. These strategies are stored in a dynamic playbook, which is iteratively refined through an agentic feedback loop. This loop analyzes failures in generated videos, abstracts common errors, and updates prompt templates to improve subsequent outputs.
Complementing the global approach, AESR employs a sample-level semantic repair module that uses vision-language models to locate errors in generated videos. It then constructs explicit repair instructions and edits keyframes to correct missing or incorrect visual elements. This targeted correction process ensures that the generated videos maintain the source identity and follow complex instructions more accurately.
Experimental results on the ACM MM 2026 challenge dataset demonstrate that AESR significantly outperforms existing methods, achieving the highest scores in identity preservation and video quality metrics. The system's ability to adaptively learn from failures and perform precise repairs makes it highly effective for practical applications such as virtual avatar creation, film post-production, and personalized content generation.
While promising, the approach still faces limitations, including dependency on pre-trained models and challenges in real-time deployment. Future work will focus on enhancing temporal coherence, reducing computational costs, and extending the framework to broader multi-modal scenarios. Overall, AESR offers a scalable, controllable, and efficient pathway for advancing identity-preserving video synthesis, bridging the gap between proprietary models and real-world demands.
Deep Dive
Abstract
Identity-preserving video generation aims to synthesize videos that follow natural-language instructions while maintaining the visual identity of a given subject. Recent commercial video generation models have achieved strong visual quality and motion realism, but they still suffer from identity drift, incomplete instruction following, and missing visual details under complex prompts. Since these models are usually closed-source black boxes, directly improving them through parameter optimization is often infeasible. We therefore propose Agentic Enhancement and Semantic Repair (AESR), a lightweight enhancement framework for identity-preserving video generation. To improve prompt construction before generation and mitigate the above failures, AESR introduces a global agentic prompt enhancement module. This module learns model-specific prompting formats from official documentation, acquires human-centered video generation priors from human-interaction data, and accumulates test-domain identity-preserving generation experience into a reusable playbook through an agentic loop. To further repair errors in videos generated with enhanced prompts, AESR introduces a sample-level visual semantic repair module, which uses a VLM to locate erroneous video segments and design repair instructions, edits selected frames into explicit visual references, and guides a video editing model to fix local semantic or identity-related errors. We also adopt a lightweight Mixture-of-Experts selection strategy to choose reliable outputs from different generation and refinement paths. Under the official evaluation protocol of the ACM MM 2026 Identity-Preserving Video Generation Challenge, our system MIPL\_Video ranked first in Track 1, demonstrating the effectiveness of AESR for practical identity-preserving video generation. The code is available at https://github.com/oceanflowlab/AESR.