From Statics to Dynamics: Physics-Aware Image Editing with Latent Transition Priors
PhysicEdit models physical state transitions with 38K video trajectories, improving realism by 5.9%.
Key Findings
Methodology
This paper introduces PhysicEdit, an end-to-end framework combining a frozen Qwen2.5-VL for physics reasoning with learnable transition queries. It leverages PhysicTran38K, a large-scale dataset of 38,000 transition trajectories across five physical domains, obtained via a two-stage filtering and annotation pipeline. The core approach models image editing as predictive physical state transitions, enabling continuous dynamic simulation rather than boundary-condition-based mappings. The framework integrates a diffusion backbone guided by timestep-adaptive visual queries, which are informed by physics-aware reasoning. Training involves supervised learning from the video dataset, enhancing the model’s ability to predict complex causal dynamics, such as refraction and deformation, in a physically plausible manner.
Key Results
- PhysicEdit outperforms Qwen-Image-Edit by 5.9% in physical realism scores, especially in scenarios involving refraction and material deformation, demonstrating superior adherence to physical laws.
- In knowledge-grounded editing tasks, it achieves a 10.1% improvement in accuracy, indicating deep integration of physical understanding.
- Ablation studies confirm that the dual-thinking mechanism and large-scale video supervision significantly improve dynamic modeling, validating the importance of physical state prediction.
Significance
This work advances image editing by embedding physical dynamics into generative models, addressing the longstanding challenge of producing physically consistent results. It bridges the gap between static image manipulation and dynamic physical simulation, opening new avenues for realistic virtual content creation, scientific visualization, and AR/VR applications. The integration of large-scale video data and physics-aware reasoning marks a significant step toward intelligent, physically grounded image synthesis, with broad implications for both academia and industry.
Technical Contribution
The paper’s main technical innovation lies in combining a physics reasoning module based on a pre-trained vision-language model with learnable transition queries within a diffusion framework. This design allows the model to perform continuous physical state prediction, guided by a large-scale dataset of real-world physical trajectories. The approach differs from prior static or boundary-condition-based methods by modeling the entire dynamic process, enabling more accurate and plausible physical effects. The framework also introduces a timestep-adaptive guidance mechanism, improving the fidelity of generated images across different physical scenarios.
Novelty
This is the first work to explicitly model physical state transitions as the core of image editing, leveraging a large-scale video dataset for supervision. Unlike previous methods that rely solely on boundary conditions or static knowledge, PhysicEdit captures the causal and continuous nature of physical phenomena, representing a fundamental shift in how physics is integrated into generative models. Its dual-thinking approach and dataset scale are key innovations that distinguish it from existing techniques.
Limitations
- The model’s generalization to highly complex or unseen physical phenomena remains limited, especially under data-scarce conditions, due to the reliance on supervised video trajectories.
- Training and inference are computationally intensive, limiting real-time applications and scalability.
- Current framework struggles with extreme physical scenarios, such as rapid deformation or chaotic fluid dynamics, indicating room for robustness improvements.
Future Work
Future research will focus on enhancing generalization to diverse physical conditions, reducing computational costs, and enabling real-time interactive editing. Incorporating unsupervised or semi-supervised learning could alleviate data dependency. Extending the framework to multi-physics scenarios and integrating reinforcement learning for autonomous physical reasoning are promising directions. These advancements aim to make physically-aware image editing more practical and widely applicable.
AI Executive Summary
Traditional image editing methods primarily treat the process as a static mapping between input and output images, often neglecting the underlying physical dynamics that govern real-world phenomena. This limitation results in generated images that lack physical plausibility, especially in complex scenarios involving refraction, deformation, or material interactions. To address this, the authors propose PhysicEdit, a novel framework that models image editing as a sequence of predictive physical state transitions. Central to this approach is the construction of PhysicTran38K, a large-scale dataset comprising 38,000 video-based transition trajectories across five physical domains, obtained through a rigorous two-stage filtering and annotation pipeline. This dataset enables the model to learn continuous physical dynamics, moving beyond the traditional boundary-condition paradigm.
PhysicEdit integrates a frozen Qwen2.5-VL model for physics-aware reasoning with learnable transition queries that adaptively guide the diffusion-based image generation process. The dual-thinking mechanism allows the system to reason about physical laws while simultaneously generating visually coherent images. During training, the model leverages the rich supervision from PhysicTran38K, learning to predict the evolution of physical states over time, which enhances its ability to generate realistic, physically consistent edits. Extensive experiments demonstrate that PhysicEdit surpasses existing methods like Qwen-Image-Edit by 5.9% in physical realism scores and 10.1% in knowledge-grounded editing accuracy.
This work marks a significant step forward in the integration of physics into generative models, enabling more accurate and plausible visual content creation. Its potential applications span virtual reality, scientific visualization, and digital content production, where physical fidelity is crucial. Despite its advances, challenges remain in improving generalization to unseen phenomena, reducing computational demands, and achieving real-time performance. Future research will focus on addressing these limitations, expanding multi-physics modeling, and exploring unsupervised learning techniques. Overall, PhysicEdit opens a new frontier in physics-aware image editing, promising more authentic and dynamic virtual worlds.
Deep Dive
Abstract
Instruction-based image editing has achieved remarkable success in semantic alignment, yet state-of-the-art models frequently fail to render physically plausible results when editing involves complex causal dynamics, such as refraction or material deformation. We attribute this limitation to the dominant paradigm that treats editing as a discrete mapping between image pairs, which provides only boundary conditions and leaves transition dynamics underspecified. To address this, we reformulate physics-aware editing as predictive physical state transitions and introduce PhysicTran38K, a large-scale video-based dataset comprising 38K transition trajectories across five physical domains, constructed via a two-stage filtering and constraint-aware annotation pipeline. Building on this supervision, we propose PhysicEdit, an end-to-end framework equipped with a textual-visual dual-thinking mechanism. It combines a frozen Qwen2.5-VL for physically grounded reasoning with learnable transition queries that provide timestep-adaptive visual guidance to a diffusion backbone. Experiments show that PhysicEdit improves over Qwen-Image-Edit by 5.9% in physical realism and 10.1% in knowledge-grounded editing, setting a new state-of-the-art for open-source methods, while remaining competitive with leading proprietary models.