SplashSplat: Reconstructing Splashing Liquids from Real-World Multi-View Videos
SplashSplat reconstructs splashing liquids from multi-view videos, offering lower training costs and more plausible physical motion.
Key Findings
Methodology
SplashSplat utilizes multi-view liquid masks to generate per-frame SDFs, employs level-set transport for coarse velocity fields, and uses Lagrangian carriers for flow correction and resampling, decoding local Gaussians for differentiable rendering.
Key Results
- SplashSplat outperforms state-of-the-art dynamic Gaussian splatting methods on real and synthetic benchmarks, with more plausible physical motion and lower training costs.
- Across 20 scenes, SplashSplat achieves superior PSNR, SSIM, and LPIPS metrics compared to baseline methods.
- The method balances physical plausibility and rendering quality better than existing approaches.
Significance
This research introduces a new method for reconstructing splashing liquids, filling the gap of synchronized multi-view video datasets, and surpassing existing methods in physical plausibility and computational efficiency.
Technical Contribution
SplashSplat imposes physical structure only where observations can constrain it, avoiding boundary condition issues of traditional fluid solvers, offering a novel approach to fluid motion modeling.
Novelty
SplashSplat is the first to reconstruct splashing liquids from real multi-view videos, overcoming limitations of traditional methods in handling weakly textured liquids.
Limitations
- The method may encounter reconstruction errors in scenes with transparent containers due to refraction effects.
- Velocity field estimation may be inaccurate in extremely fast-moving scenarios.
Future Work
Future work could focus on improving reconstruction accuracy in transparent container scenes and extending the method to handle more complex fluid dynamics scenarios.
AI Executive Summary
Reconstructing splashing liquids has been a challenge in computer vision and graphics. Existing methods often focus on smoke or synthetic liquids, lacking real datasets. SplashSplat fills this gap by introducing a benchmark of 20 scenes captured by synchronized multi-view videos.
The core of SplashSplat is imposing physical structure only where observations can constrain it. It fuses multi-view masks to generate liquid SDFs, uses level-set transport for coarse velocity fields, and employs Lagrangian carriers for flow correction and resampling, decoding local Gaussians for differentiable rendering.
Experimental results show that SplashSplat outperforms state-of-the-art dynamic Gaussian splatting methods on both real and synthetic benchmarks, with more plausible physical motion and lower training costs. This method not only improves the accuracy of liquid reconstruction but also opens new possibilities for future fluid simulation and visual effects.
Deep Analysis
Background
Liquid reconstruction is crucial in visual effects and fluid simulation. However, due to view-dependent appearance and weak texture, traditional methods struggle. Existing research often focuses on smoke or synthetic liquids, lacking real datasets.
Core Problem
Reconstructing splashing liquids faces challenges such as lack of multi-view data, weak textures, and rapid geometric changes. Accurately capturing these dynamics is vital for visual effects and fluid simulation.
Innovation
SplashSplat introduces a synchronized multi-view video dataset, combining physical modeling and differentiable rendering to achieve high-precision reconstruction of splashing liquids. Its innovation lies in imposing physical structure only where observations can constrain it.
Methodology
- �� Fuse multi-view liquid masks to generate per-frame SDFs.
- �� Use level-set transport to generate coarse velocity fields.
- �� Employ Lagrangian carriers for flow correction and resampling.
- �� Decode local Gaussians for differentiable rendering.
Experiments
Experiments were conducted on 20 real scenes and the synthetic NeuroFluid benchmark. Baseline methods include Deformable-3DGS, SpacetimeGaussians, and 4D-Scaffold-GS. Evaluation metrics include PSNR, SSIM, LPIPS, etc.
Results
SplashSplat achieves superior PSNR, SSIM, and LPIPS metrics compared to baseline methods, particularly balancing physical plausibility and rendering quality better than existing approaches.
Applications
The method can be used in visual effects for liquid simulation and in fluid dynamics research for data validation and dynamic asset generation.
Limitations & Outlook
The method may encounter reconstruction errors in scenes with transparent containers due to refraction effects. Additionally, velocity field estimation may be inaccurate in extremely fast-moving scenarios.
Plain Language Accessible to non-experts
Imagine a kitchen scene where water is being poured. SplashSplat acts like a super camera that captures the splash from multiple angles simultaneously. It then analyzes these images to reconstruct the water's motion path, much like a clever chef remembering where every drop goes. This method not only lets us see the splash but also helps us understand how the water moves.
ELI14 Explained like you're 14
Imagine you're playing a super cool water gun game with splashes everywhere. SplashSplat is like an ultra-powerful camera that captures these splashes from different angles at the same time. Then, it's like a smart game designer piecing these images together to recreate every detail of the splash. Isn't that awesome? It not only makes the game more realistic but also helps you design cooler water gun effects!
Glossary
SDF (Signed Distance Field)
A method of representing the geometry of objects by calculating the distance from points to the object's surface.
Used to generate the geometry of the liquid for each frame.
Lagrangian carriers
Small units used to track fluid motion, capable of correction and resampling along the flow.
Used in SplashSplat to decode local Gaussians.
Gaussian splatting
A rendering technique that simulates the optical properties of objects using Gaussian functions.
Used for differentiable rendering in SplashSplat.
Level-set transport
A method for tracking interface motion, described by the level-set equation.
Used to generate coarse velocity fields.
Differentiable rendering
A rendering technique that allows the image generation process to be differentiated to optimize model parameters.
Used in the rendering process of SplashSplat.
Open Questions Unanswered questions from this research
- 1 How to reduce refraction errors when dealing with liquids in transparent containers?
- 2 How to improve velocity field estimation accuracy in extremely fast-moving scenarios?
Applications
Immediate Applications
Visual Effects
Reconstruct splashing liquids using SplashSplat to enhance the realism of water effects in films and games.
Long-term Vision
Fluid Dynamics Research
Provide more accurate data validation for fluid simulations, advancing fluid dynamics research.
Abstract
A splash lives for a fraction of a second: sheets tear into ligaments and droplets, appearance is view-dependent and nearly textureless, and little persists long enough to track. Reconstruction research has consequently focused on smoke, synthetic liquids, or gently deforming surfaces. To our knowledge, no synchronized multi-view dataset of splashing liquids exists. We therefore introduce a benchmark of 20 real scenes, from coherent streams to violent splashes, captured by seven synchronized, calibrated 4K cameras at 60 fps, with manually refined per-view liquid and container masks and fixed evaluation splits. We further present SplashSplat, built on a single principle: impose physical structure only where the observations can constrain it. Per-frame liquid SDFs fused from the masks provide the geometry, level-set transport between consecutive SDFs yields a coarse velocity field, and Lagrangian carriers advected along this flow, corrected against each new observation and reseeded where coverage is lost, decode local Gaussians for differentiable rendering. SplashSplat outperforms state-of-the-art dynamic Gaussian splatting methods on our real captures and on a synthetic benchmark, with physically more plausible motion and a lower training cost. The same representation supports temporal interpolation and style transfer without re-optimization.