LeHome: A Simulation Environment for Deformable Object Manipulation in Household Scenarios

In the field of robotics research, household environments are of great interest due to their complexity and diversity. Unlike industrial environments, tasks in household scenarios involve interactions with diverse, non-standardized objects and adaptation to unstructured, dynamic environments. These tasks include organizing personal belongings, preparing food, and managing clothing, posing significant challenges. In recent years, simulation platforms for household environments have emerged, such as RoboTwin2.0 and DexGarmentLab, laying a solid foundation for related research. However, most existing methods primarily target rigid and articulated objects, while many household tasks inherently involve diverse deformable objects (e.g., garments and food), for which current support remains limited. These objects lack fixed shapes, deform nonlinearly under applied forces, and exhibit dynamic physical parameters. Consequently, creating large-scale, realistic, and diverse deformable-object datasets for training robust policies remains a major challenge.

Manipulating deformable objects in household scenarios is a complex and challenging problem. Existing simulators lack adequate support for deformable objects, leading to difficulties in both simulation and real-world execution. These objects vary in categories and shapes, have complex dynamics, and diverse material properties, making it difficult for existing simulators to accurately simulate their real interactions. Additionally, collecting real-world household data is prohibitively expensive and labor-intensive, as deformable objects' variable states and the inherent complexity of household environments make it difficult to obtain sufficient high-quality data. Achieving accurate and authentic modeling of deformation is intrinsically hard, as it requires simultaneously capturing complex material properties, nonlinear dynamics, and realistic interactions.

LeHome's core innovations lie in its high-fidelity simulation of deformable objects and support for low-cost robots. First, LeHome integrates multiple physics engines, including Position-Based Dynamics (PBD), Finite Element Method (FEM), and Eulerian Fluid Simulation, to achieve high-fidelity simulation of various deformable objects. Second, LeHome introduces a graphical logical modeling method to ensure causal consistency between actions and outcomes, aligning the simulation with real-world dynamics. Additionally, LeHome supports various low-cost robot platforms, lowering the hardware barrier for household robotics research, enabling end-to-end evaluation of household tasks on resource-constrained hardware.

LeHome's implementation includes the following key steps:

• �� Integration of Physics Engines: Combines Position-Based Dynamics (PBD), Finite Element Method (FEM), and Eulerian Fluid Simulation to achieve high-fidelity simulation of various deformable objects such as liquids, gases, granular objects, linear objects, thin shells, and volumetric objects.

• �� Graphical Logical Modeling: Introduces a graphical logical modeling method to ensure causal consistency between actions and outcomes, aligning the simulation with real-world dynamics.

• �� Robot Platform Support: Supports various low-cost robot platforms, including LeRobot and XLeRobot, lowering the hardware barrier for household robotics research.

• �� Domain Randomization: Enhances existing demonstrations by injecting randomized scene variations, automatically generating more diverse data with richer visual appearances and interaction conditions.

The experimental design includes six representative tasks, covering single-arm and bimanual manipulation, tool use, deformable food interaction, fluids, and rigid object manipulation, rigid-deformable and deformable-deformable interactions. The tasks span multiple rooms (bedroom, kitchen, living room, bathroom) to reflect realistic household diversity. Four representative policies, including Diffusion Policy (DP) and SmolVLA, were used, all trained with the same data budget (50 teleoperated demonstrations per task) and evaluated over 100 test trials.

Experimental results show that the simulation success rate in the LeHome environment significantly improved in various household tasks. For example, in the sausage-cutting task, the DP algorithm achieved a success rate of 93%, significantly higher than other methods. Additionally, training schemes incorporating LeHome simulation data increased the success rate of dual-arm LeRobot from about 15% to about 50%. In garment manipulation tasks, the SmolVLA algorithm performed best, achieving a success rate of 70%. These results indicate that LeHome not only provides a high-fidelity simulation environment but also effectively enhances sim-to-real transfer capabilities.

LeHome's application scenarios include household robotics research and development, testing and evaluation of low-cost robot platforms, and training and validation of deformable object manipulation algorithms. By supporting various low-cost robots, LeHome lowers hardware and deployment barriers, enabling end-to-end evaluation of household tasks on resource-constrained hardware. Additionally, LeHome can be used to generate high-quality training data to support the development of machine learning algorithms.

Despite LeHome's outstanding performance in deformable object manipulation, it still faces challenges in simulating large-deformation, long-horizon garment manipulation tasks, with low success rates. Additionally, its high-fidelity simulation requires significant computational resources, which may limit its application in resource-constrained environments. Future research directions include further optimizing LeHome's physics engines to improve simulation accuracy for large-deformation, long-horizon tasks and developing more efficient algorithms to reduce the computational resource requirements of high-fidelity simulations.