PartialBiGrasp: Inferring Hidden Local Geometry for Bimanual Grasping from Partial Views

Robotic grasping has evolved from simple rule-based control to sophisticated learning-based methods. Early approaches relied on precise geometric models and inverse kinematics, exemplified by classical algorithms like GraspIt! and Dex-Net, which used dense 3D models for grasp synthesis. Recent advances include deep neural networks such as PointNet, DGCNN, and GQ-CNN, which learn features directly from point clouds or images, enabling more flexible grasp prediction. However, these methods typically assume access to complete object geometries, limiting their effectiveness in cluttered or occluded environments. The emergence of implicit geometric representations, such as occupancy networks and signed distance functions, has provided new tools for reasoning about shape completion and local geometry inference. Despite these developments, most existing works focus on single-arm grasping or rely on explicit shape reconstruction, which can be computationally expensive and error-prone, especially under partial observations. The challenge remains to develop methods capable of robustly inferring hidden geometry and generating stable dual-arm grasps in real-world, occlusion-rich scenarios.

The core problem addressed in this work is the difficulty of generating stable dual-arm grasps from partial point clouds. In practical settings, objects are often only partially visible due to occlusions, limited viewpoints, or sensor noise. Traditional methods depend on complete shape information or dense point cloud reconstructions, which are not always feasible. Without accurate knowledge of hidden local geometry—such as surface thickness, edge continuation, or contact regions—robots risk generating grasps that are unstable, collide with the object, or fail to achieve force closure. Moreover, ensuring that two grasping points form a physically stable pair adds complexity, as the joint configuration must satisfy force-closure constraints and avoid collisions. Addressing these issues requires a method capable of inferring the unseen parts of the object and reasoning about grasp stability under incomplete data, which is a significant challenge in robotic manipulation.

This paper introduces several key innovations: 1) The use of convolutional occupancy networks (ConvOccNet) to implicitly learn and infer hidden local geometry from partial point clouds, enabling the reasoning about unobserved surface properties. 2) A hierarchical architecture combining global shape encoding with grasp-centered local geometry encoding, facilitating multi-scale understanding of object structure and contact regions. 3) A force-closure critic network trained to evaluate the stability of grasp pairs, ensuring physically feasible dual-arm configurations. 4) A sampling-based local occupancy optimization that refines grasp poses by predicting and minimizing collisions and contact inconsistencies. These innovations collectively allow the system to generate robust, collision-free, and force-closure compliant dual-arm grasps directly from incomplete observations, a significant step forward from prior methods relying on explicit shape reconstruction or dense supervision.

• �� Input: Single RGB-D image and segmentation mask, projected into point cloud P. • Global geometry encoding: Extract rotation-equivariant features via point-based encoder, project onto triplane, process with shared U-Net to produce continuous feature field. • Local geometry encoding: Sample grasp-centered point crops, encode with shared local encoder to capture fine features like thickness and surface continuation. • Candidate grasp generation: Sample points on the object surface, predict graspability, approach vectors, and pose parameters using multi-head prediction. • Grasp pair evaluation: Concatenate features from two candidate grasps, evaluate with force-closure critic to determine stability. • Grasp refinement: Use local occupancy predictions and sampling-based optimization to iteratively adjust grasp poses, minimizing collisions and maximizing contact stability. • Training: Multi-stage supervision on DG16M dataset, including occupancy, grasp pose, and graspability labels, with loss functions such as BCE and L1. • Testing: Generate grasp pairs, evaluate stability, refine, and execute in simulation and real-world scenarios.

The framework was trained on the DG16M dataset, which contains diverse large objects with annotated grasp contacts. Evaluation involved simulation tests measuring force closure percentage (FC%), grasp success rate (GS%), collision rate (PC%), and coverage of graspable regions. The model was compared against baselines like ContactGraspNet, DAGDiff, RecGen+DAGDiff, and VLM-based methods. In simulation, PartialBiGrasp achieved an FC% of 55.16% and GS% of 67.87%, outperforming baselines significantly. Real-world experiments involved noisy point clouds captured by Realsense D455 sensors, where the method maintained over 51% FC% and 81% GS%, with collision rates below 20%. Ablation studies confirmed the importance of multi-scale feature fusion and local occupancy refinement, with performance drops observed when these components were removed. The system was deployed on physical dual-arm robots (xArm7 and xArm6 Lite), successfully manipulating various large objects under real conditions.

Quantitative results demonstrate that PartialBiGrasp surpasses existing methods in stability and collision avoidance. The force-closure rate of 55.16% and grasp success rate of 67.87% on DG16M indicate high reliability, especially given the partial observation constraint. In real-world tests, the success rate remained above 81%, with collision rates reduced to 17.48%, confirming robustness against sensor noise and occlusion. Ablation experiments showed that removing local occupancy refinement decreased FC% by over 15%, highlighting its critical role. The approach also achieved higher coverage of graspable regions, demonstrating comprehensive surface understanding. These results validate the effectiveness of implicit geometric reasoning for complex, large-object manipulation in practical scenarios.

This method is highly suitable for industrial automation, warehouse logistics, and assistive robotics, where objects are often large, occluded, or partially visible. It enables autonomous systems to generate stable dual-arm grasps without requiring complete shape models, reducing reliance on dense point cloud reconstruction. The framework can be integrated into robotic manipulation pipelines for tasks like loading/unloading, assembly, or disaster response, where quick and reliable grasping is essential. Its ability to handle noisy, incomplete data makes it adaptable to real-world environments, facilitating deployment in unstructured settings. Future integration with motion planning and control modules will further enhance its practical utility.

Currently, the framework does not incorporate explicit kinematic or motion planning constraints, which may lead to infeasible grasps in real robotic systems. The inference of hidden geometry may be unreliable under extreme occlusion or highly complex geometries, affecting grasp stability. Real-time performance in dynamic environments remains a challenge, as the current implementation prioritizes accuracy over speed. Additionally, the method assumes at least one visible contact point per grasp, which may not hold in highly cluttered scenes. Future work should focus on integrating motion planning, sensor fusion, and real-time optimization to address these limitations.