VQQA: An Agentic Approach for Video Evaluation and Quality Improvement

The evolution of video synthesis has transitioned from early distribution-matching metrics like IS and FVD to more sophisticated semantic and agentic evaluation frameworks. Early methods focused on statistical similarity, but these metrics poorly correlate with human perception, especially for complex scenes. Recent approaches incorporate multimodal large models (e.g., CLIP, BLIP) to evaluate frame-text alignment, yet they lack temporal sensitivity and actionable feedback. Benchmarks such as VBench and CompBench attempt to disentangle quality axes but rely heavily on large ensembles, incurring high computational costs. Meanwhile, prompt engineering and test-time optimization techniques like VISTA and VideoRepair have made progress but are limited by their local scope or dependence on white-box access. The need for a flexible, interpretable, and efficient evaluation and refinement system remains pressing, motivating the development of VQQA.

Despite rapid advancements, current video generation models struggle to produce content that aligns with complex user intents, especially in terms of temporal coherence, spatial accuracy, and semantic fidelity. Existing evaluation metrics lack the granularity and interpretability needed for targeted improvements, often providing only scalar scores without diagnostic insights. Test-time optimization methods are either computationally prohibitive or require internal model access, limiting their scalability and practical deployment. Consequently, users face tedious trial-and-error prompt tuning, and models often generate artifacts or inconsistencies that degrade content quality. Addressing these issues requires a system capable of detailed, interpretable diagnostics and efficient, model-agnostic optimization.

VQQA introduces a multi-agent question-answering paradigm that converts video evaluation into an interactive, semantic feedback loop. Its key innovations include: 1) Dynamic question generation that targets specific failure modes; 2) QA agents that produce detailed diagnostics, identifying artifacts in spatial, temporal, and compositional aspects; 3) Prompt refinement driven by critiques serving as semantic gradients, enabling gradient-free optimization; 4) A global candidate selection mechanism to prevent semantic drift; 5) Compatibility with diverse modalities and models without fine-tuning. This holistic, task-agnostic approach enables efficient, interpretable, and scalable video quality improvement, surpassing prior passive metrics and localized correction methods.

• �� Input: text prompt p, generation conditions C, pre-trained model M. • Video generation: M(p, C) produces video v. • Evaluation: VQQA’s QA agent analyzes v with questions Q across axes like spatial, temporal, and fidelity, scoring each. • Diagnostics: QA identifies flaws and generates detailed reports. • Prompt update: PR uses low-scoring QA pairs as semantic gradients to refine p. • Global evaluation: VLM assesses candidate videos against initial prompt, selecting the best. • Iteration: Repeat until stopping criteria (score threshold or convergence) are met. • Output: Optimized video and prompt. The process iteratively corrects flaws without internal model access, relying solely on external critiques and natural language interfaces.

The authors evaluate VQQA on T2V-CompBench, VBench2, and VBench-I2V benchmarks, comparing against baselines like VQAScore, VISTA, and random sampling. They employ models such as Gemini-3-Pro and GPT-4o, running 4-5 optimization rounds with 5 candidate samples per round. Metrics include average scores, structural consistency, and artifact reduction. Ablation studies confirm the importance of global candidate selection and dynamic stopping. Results show improvements of +11.57% (T2V) and +8.43% (VBench2) over vanilla generation, with fewer iterations and better artifact correction. The experiments demonstrate robustness across modalities and models, validating the framework’s generalizability and efficiency.

VQQA with Gemini-3-Pro achieves an average score of 53.46% on T2V-CompBench, surpassing baseline methods by over 11%. It effectively reduces artifacts related to compositional errors, spatial relationships, and numerical reasoning. On VBench2, it scores 50.41%, outperforming the best existing method by 8.43%. In I2V tasks, it reaches 97.86% average score with minimal iterations, significantly improving realism and temporal coherence. Ablation results show that the semantic critique-based prompt adjustment and global candidate selection are critical for these gains. The framework’s ability to identify subtle flaws and iteratively correct them highlights its practical utility.

VQQA can be integrated into content creation pipelines for film, animation, and virtual reality, automating quality enhancement without requiring internal model access. Its task-agnostic design makes it suitable for diverse modalities, including text-to-video and image-to-video synthesis. The system enables rapid, interpretable diagnostics, reducing manual prompt engineering and trial-and-error. Long-term, it can facilitate autonomous content generation tailored to user preferences, supporting personalized media, real-time editing, and scalable virtual environment creation, thereby transforming digital content industries.

The reliance on large VLMs introduces hallucination risks, which may lead to misdiagnosis of flaws. Multi-round optimization, while effective, still incurs computational costs, especially for high-resolution videos. The current framework may face challenges with highly dynamic or complex scenes, requiring further robustness improvements. Additionally, the system's performance depends on the quality of critiques; biases or inaccuracies in VLM critiques could limit effectiveness. Future work should focus on reducing computational overhead, enhancing critique accuracy, and extending real-time capabilities.