VirSqueezer: Generating Realistic Deformations and Squeezing Dynamics in VR from Fine-Grained Squeezing Controls
VirSqueezer combines SenseGlove, MPM, and diffusion generation for fine-grained squeezing, but the paper excerpt reports no numeric benchmark values.
Key Findings
Methodology
VirSqueezer synchronizes five-finger flexion, fingertip 3D positions, and 6-DoF palm poses captured by SenseGlove, denoising them with an exponential moving average. Squeezing-conditioned contact area estimation (SC-CAE) finds localized finger contacts. Per-finger force proxies follow ck,t=wk max(ũk,t−β,0)^γ, and an anisotropic displacement field becomes a Dirichlet boundary condition. MPM then performs P2G, grid update, and G2P simulation. GPT-5 infers material type, Young’s modulus, Poisson’s ratio, and yield strength for resistance feedback and thresholds. RGB, depth, particle masks, text, AnimateDiff, Stable Diffusion, and ControlNet generate rupture and overflow.
Key Results
- The paper reports quantitative metrics, human evaluation, and evaluations by multiple large language models, claiming advantages in visual quality, physical commonsense, and squeezing-control consistency. However, the supplied excerpt gives no dataset names, sample counts, scores, or percentage improvements, so specific numerical gains cannot be verified or responsibly reconstructed.
- Qualitative scenarios include localized dents, bursting Coke cans, rebound of soft objects, and juice release. MPM supplies interpretable primary deformation, while generative models add complex appearance dynamics. The excerpt does not provide an itemized numerical comparison against 3DGS, PhysGaussian, OmniPhysGS, or diffusion-only baselines.
- The reported ablation logic identifies SC-CAE, Dirichlet constraints, material priors, and physics-conditioned diffusion as important components. Internal particles represent hidden object structure; their raw simulated effects are less realistic, motivating G-buffer conditioning for visual refinement.
Significance
This work advances VR squeezing from coarse hand gestures to continuous, finger-specific, time-varying control. It addresses a persistent gap: 3D Gaussian Splatting and MPM can model appearance or physically plausible deformation, but struggle with localized contact and rupture or overflow; diffusion video models can look realistic but often disregard physical causality and continuous user input. Academically, the paper demonstrates a complementary physics–generation pipeline. Industrially, it could reduce manual authoring costs for interactive training, games, design, advertising, and haptic experiences.
Technical Contribution
The technical contributions are SC-CAE contact estimation, nonlinear finger-level force proxies, anisotropic soft-weight displacement fields, and Dirichlet-driven local MPM deformation. Stress uses hyperelastic energy Ψ, elastic deformation gradient FE, total gradient F, and correction mapping ψ; FE is updated through return mapping. Resistance is estimated from yield strength times fingertip contact area. Internal particles and G-buffers connect hidden-object dynamics to diffusion generation, while AnimateDiff, Stable Diffusion, and ControlNet preserve spatial and temporal structure more explicitly than prompt-only editing.
Novelty
The authors describe VirSqueezer as the first framework specifically designed for fine-grained virtual squeezing. Its fundamental novelty is system integration rather than a new standalone MPM or diffusion architecture: finger controls are converted into localized contacts, material-aware physical responses, haptic resistance, and physics-conditioned multi-view effects. Compared with prompt-driven 3DGS editing, it preserves continuous user-specific spatiotemporal control and explicitly separates primary deformation from secondary visual dynamics.
Limitations
- The system is not real time: rupture and overflow depend on computationally expensive AnimateDiff, Stable Diffusion, and ControlNet inference, limiting immediate response to continuous hand input.
- GPT-5-derived material parameters may be inaccurate. The excerpt provides no systematic calibration against real force measurements, material databases, or uncertainty bounds.
- The provided paper text omits datasets, training details, hardware specifications, and numerical tables, making reproducibility and cross-material generalization difficult to assess.
Future Work
Future work should distill, cache, or otherwise accelerate diffusion generation to meet VR latency requirements without sacrificing finger-level control or physical consistency. A public benchmark containing materials, contact trajectories, force measurements, and multi-view videos would improve comparability. Real sensor calibration should replace or validate GPT-5 priors. Differentiable physics–generation training could reduce heuristic thresholds, improve fluid and fracture behavior, and support robust generalization across users, viewpoints, and object classes.
AI Executive Summary
Squeezing is a deceptively difficult VR action. It involves five fingers bending at different rates, contacting different locations, and changing pressure over time. Existing 3D Gaussian Splatting systems render scenes efficiently, while MPM methods simulate deformation, but neither reliably combines localized dents with rupture or overflow. Diffusion video models can produce attractive dynamics, yet may drift away from continuous user input and physical causality.
VirSqueezer proposes a hybrid solution. SenseGlove records finger flexion, fingertip positions, and palm 6-DoF motion. SC-CAE projects these signals onto an object surface and estimates localized contacts. A finger-specific nonlinear force proxy drives an anisotropic displacement field, which is imposed as a Dirichlet boundary condition for MPM. The simulator uses P2G, grid update, and G2P steps to compute primary deformation. GPT-5 supplies material priors, including Young’s modulus, Poisson’s ratio, and yield strength, enabling threshold decisions and resistance feedback.
When deformation exceeds a threshold, internal particles represent hidden contents and provide motion cues. RGB images, depth maps, and particle masks are then combined with text prompts to condition AnimateDiff, Stable Diffusion, and ControlNet, producing multi-view rupture and overflow. The paper claims improvements in visual quality, physical commonsense, and squeezing consistency based on quantitative, human, and LLM-based evaluation. Yet the supplied text reports no datasets or numerical scores. VirSqueezer’s main value is therefore its controllable physics-to-generation pipeline; its main obstacle is the non-real-time cost of diffusion inference.
Deep Analysis
Background
3DGS, 4D Gaussians, and VR-GS enabled efficient static and dynamic scene representations. PhysGaussian and OmniPhysGS integrated MPM with visual primitives to improve physical behavior. Stable Diffusion, VideoCrafter, and AnimateDiff offered high-fidelity temporal synthesis. Nevertheless, physics-based systems often target global motion or manually specified parameters, whereas diffusion systems rely heavily on prompts. Neither family adequately handles finger-level localized squeezing followed by physically timed rupture or overflow.
Core Problem
The target mapping is from continuous five-finger flexion, fingertip trajectories, and palm poses to spatially localized, temporally synchronized, material-plausible visual effects. Difficulties include unstable contact estimation, finger overlap, curvature constraints, unknown material properties, and secondary phenomena that ordinary surface simulation cannot convincingly render. The system must also provide resistance feedback consistent with the simulated deformation.
Innovation
- �� SC-CAE estimates finger-specific contact patches using geometric compatibility, normal alignment, curvature, anisotropic scale, and overlap penalties. • ck,t converts flexion into nonlinear, finger-weighted force proxies. • Dirichlet displacement constraints replace unstable direct force application for local MPM control. • GPT-5 infers material properties and squeezing thresholds, while yield strength times contact area estimates haptic resistance. • Internal particles, G-buffers, and physics-conditioned diffusion connect primary deformation to rupture and overflow.
Methodology
- �� Input: SenseGlove supplies normalized five-finger flexion, fingertip 3D positions, and palm 6-DoF poses; an exponential moving average suppresses jitter. • Contact mapping: SC-CAE optimizes contact center x, normal n, and anisotropic scale σ on the object surface. • Control construction: ck,t=wk max(ũk,t−β,0)^γ and Δx(x)=Σkηck,tφk(x)nk produce the displacement field. • Physics: MPM performs P2G, grid updates, and G2P; stress is derived from hyperelastic energy and corrected through return mapping. • Feedback: yield strength and contact area determine resistance; threshold crossing releases internal particles. • Generation: RGB, depth, particle masks, and text condition AnimateDiff, Stable Diffusion, and ControlNet.
Experiments
The paper uses quantitative metrics, human studies, and evaluations by multiple large language models, comparing against 3DGS/MPM and diffusion-oriented approaches. Evaluation targets localized deformation realism, visual quality, physical commonsense, temporal coherence, and control consistency. Demonstrations include Coke cans, soft toys, and fruit. The supplied text does not specify datasets, sample counts, hardware, training schedules, hyperparameters, or metric values; thus the evaluation dimensions are identifiable, but the full protocol is not reproducible from the excerpt.
Results
The authors report superior visual quality, physical commonsense, and squeezing-control consistency, together with localized dents and complex rupture or overflow. Internal particles compensate for the missing interior structure of 3DGS, while G-buffer conditions improve secondary-effect appearance. However, the excerpt contains no percentages, scores, dataset names, or complete baseline tables. Accordingly, “outperforms” should be treated as the paper’s stated conclusion rather than a numerically verifiable claim from the supplied material.
Applications
Potential uses include VR games featuring crushed cans, soft toys, and juicing fruit; immersive advertising; virtual laboratories; product demonstrations; and haptic training. Creators need an object representation, text prompts, calibrated or inferred material priors, a SenseGlove trajectory, and GPU inference. The current system is better suited to offline authoring or experiences that tolerate latency than to unrestricted real-time gameplay.
Limitations & Outlook
The dominant limitation is latency: complex dynamics rely on diffusion inference and therefore cannot yet respond immediately to fine-grained hand input. GPT-5 material inference may diverge from measured physics, and threshold and release-direction assumptions are heuristic. Internal particles are not a full simulation of fracture, fluids, or multiphase materials. Progress requires real force calibration, public benchmarks, accelerated diffusion, differentiable joint optimization, and rigorous evaluation across materials, users, viewpoints, and object categories.
Plain Language Accessible to non-experts
Imagine a smart kitchen that watches a cook squeezing a can or fruit. A special glove records where each finger touches, how much each finger bends, and how the palm moves. The kitchen first decides which spots are being pressed and whether the object is soft, stiff, or filled with something that could spill.
A careful physical assistant then predicts how each spot should dent. It does not squash the whole object uniformly: the thumb may create one dent while the index finger creates another. The glove also pushes back, like a real can resisting your hand. Stronger materials feel harder to squeeze; weaker materials yield sooner.
If the pressure becomes large enough, the object may crack or release its contents. The physical assistant determines where and when that motion should happen. A second assistant then paints the event so it looks realistic from several viewpoints. This division keeps the action connected to the user while improving visual detail.
The trade-off is speed. Painting a convincing rupture takes substantial computation, so the present system is not yet like an ordinary instant-response game. Its promise is a believable link among hand action, object response, appearance, and touch.
ELI14 Explained like you're 14
Picture squeezing a virtual soda can in a game. A basic system might flatten the whole can or play a cool clip that ignores your exact hand motion. VirSqueezer tries to notice the details: which finger moved, where it touched, how far it bent, and when the pressure increased.
SenseGlove works like a super-detailed game controller worn on your hand. Its signals go to a virtual science lab that predicts where the can dents and how hard the material is. It can also send resistance back to the glove, so pressing a tough object feels different from pressing a soft toy. Pretty neat, right?
If you squeeze hard enough, the can might burst or fruit juice might spray out. One part of the system follows the rules of motion and decides the timing and location. Another part, using AnimateDiff, Stable Diffusion, and ControlNet, makes the spray and broken surface look convincing. It is like classmates dividing a project: one handles the science, another handles the artwork.
The catch is that the artwork takes time to generate, so the system is not fully real-time yet. If future researchers make those models faster, VR could let you squeeze, feel resistance, and instantly see a believable result. What would you test first: a sponge, a balloon, or a watermelon?
Glossary
Material Point Method (MPM)
MPM represents a material with particles while using a background grid to compute forces and motion. It is effective for large deformation, soft bodies, and changing material states.
VirSqueezer uses MPM for primary squeezing deformation through P2G, grid update, and G2P.
Dirichlet boundary condition
A boundary condition that directly specifies position or displacement rather than only applying a force. It can impose stable kinematic motion on contact regions.
Estimated finger displacements are converted into Dirichlet constraints on the object surface.
SC-CAE
Squeezing-conditioned contact area estimation maps hand controls to localized surface patches. It jointly considers geometry, normals, curvature, scale, and multi-finger overlap.
SC-CAE is the bridge from SenseGlove signals to local displacement constraints.
G-buffer
A collection of per-pixel auxiliary maps describing scene properties such as color, depth, and masks. These maps provide structural conditions for visual generation.
VirSqueezer uses RGB, depth, and internal-particle masks to guide secondary-effect synthesis.
ControlNet
An architecture that injects spatial conditions into diffusion models. It can preserve structure specified by depth, edges, poses, or masks.
ControlNet constrains the location and geometry of generated rupture and overflow.
Return mapping
A numerical correction step that projects an updated material state back into the valid region of a constitutive model. It is commonly used for constrained elastic or plastic updates.
The MPM G2P stage uses return mapping to correct the updated elastic deformation gradient.
Open Questions Unanswered questions from this research
- 1 The supplied text omits datasets, sample counts, and metric values, so statistical generalization across objects, materials, and users cannot be assessed.
- 2 The error of GPT-5-inferred material parameters relative to physical measurements remains unknown; calibrated force sensors and standardized material data are needed.
- 3 How to preserve multi-view consistency, finger-level control, and physical causality under low-latency diffusion inference remains the central open problem for real-time VR.
Applications
Immediate Applications
VR interactive games
Game developers can record finger-level SenseGlove actions and turn them into dents, bursts, rebounds, or juice release. They need object representations, material priors, prompts, and GPU inference. The current implementation is most suitable for offline authoring or experiences that tolerate generation delay.
Immersive product demonstration
Advertising and education applications can let users squeeze packaging, fruit, or soft objects while viewing deformation and receiving resistance. RGB, depth, and masks support multiple viewpoints, but deployment requires GPU resources, prepared prompts, and carefully chosen material assumptions.
Long-term Vision
Real-time physics-generative VR
With diffusion distillation and differentiable MPM–generation training, users could explore materials and squeezing styles interactively while receiving causal visual and haptic feedback. Major obstacles are latency, parameter calibration, fracture and fluid realism, and consistency across viewpoints.
Abstract
Squeezing is one of the most natural forms of hand manipulation, inherently involving fine-grained, temporally evolving, per-finger flexion. In VR content creation, squeezing plays a unique role in enabling particular visual effects such as localized deformations and dynamic behaviors, e.g., bursting a Coke can or juicing a fruit, thereby expanding the expressive possibilities of VR content. However, existing techniques, such as 3D Gaussian splatting-based methods and diffusion-based video generation models, are limited in their ability to simulate fine-grained virtual squeezing effects. We introduce VirSqueezer, a framework designed to generate both localized deformations (primary effects) and complex squeezing dynamics, such as rupture and overflow (secondary effects). VirSqueezer captures squeezing control signals using a SenseGlove and provides the user with inferred resistance force feedback during the squeezing process. By estimating object contact areas, inferring physical properties, and simulating physical responses, VirSqueezer computes conditions that guide generation models for visual effect generation, ensuring both visual coherence and temporal synchronization with the simulation. Consequently, VirSqueezer enables the generation of physically realistic visual effects directly from continuous, fine-grained squeezing control signals. Our extensive evaluation demonstrates VirSqueezer's ability to reproduce realistic localized deformations, generate convincing visual dynamics, and maintain consistency in fine-grained squeezing controls.