EGAD! an Evolved Grasping Analysis Dataset for diversity and reproducibility in robotic manipulation

Robotic grasping has evolved from simple rule-based approaches to sophisticated deep learning models, driven by datasets like YCB and Dex-Net. These datasets have enabled significant progress but are limited by their small size, low geometric diversity, and lack of standardization. Researchers have recognized the need for more diverse, scalable, and reproducible datasets to improve model robustness and generalization. Recent efforts have explored synthetic data generation, but challenges remain in controlling shape complexity and grasp difficulty. The advent of evolutionary algorithms offers a promising avenue to systematically generate complex, diverse objects. This study builds upon these developments, aiming to create a comprehensive dataset that bridges the gap between simulation and real-world deployment, fostering more resilient robotic manipulation systems.

Current datasets for robotic grasping are insufficient in size and diversity, limiting the training of models capable of handling complex, unseen objects. Manual collection is labor-intensive and often biased toward simple shapes, reducing the model’s ability to generalize. Moreover, the lack of standardized evaluation objects hampers fair comparison across different algorithms. The core challenge is to generate a large, diverse set of objects that span a broad spectrum of shape complexities and grasp difficulties while maintaining reproducibility and ease of use. Addressing this problem requires innovative methods to automate object creation, quantify complexity, and standardize evaluation protocols, ensuring models are robust enough for real-world applications.

This research introduces a novel framework combining MAP-Elites evolutionary algorithm with CPPNs to generate a diverse object set tailored for robotic grasping. Key innovations include: 1) systematic exploration of shape complexity and grasp difficulty spaces; 2) quantitative metrics for shape complexity (entropy) and grasp difficulty (75th percentile Ferrari-Canny score); 3) high-diversity object filtering ensuring broad coverage; 4) precomputing millions of grasp poses for rapid evaluation; 5) providing a standardized, reproducible evaluation set with 49 3D-printable objects. These advancements enable controlled, scalable, and fair benchmarking of grasping algorithms, addressing limitations of prior datasets.

• �� Define a 2D search space based on shape complexity and grasp difficulty, discretized into grid cells.
• �� Encode objects using CPPNs, querying at x, y, z coordinates to generate voxel grids.
• �� Convert voxel grids to meshes via marching cubes, then smooth and prune small features.
• �� Calculate shape complexity using angular deficit entropy; grasp difficulty via Dex-Net’s Ferrari-Canny metric on sampled antipodal grasps.
• �� Measure geometric diversity using topology matching of multiresolution Reeb graphs.
• �� Use MAP-Elites to evolve objects: initialize with random CPPNs, evaluate, assign to grid cells, and iteratively mutate and crossover to fill the space with diverse objects.
• �� Filter objects to keep the most diverse per cell, ensuring coverage and variety.
• �� Precompute grasp poses for each object, storing up to 100 grasps with quality scores.
• �� Final dataset includes 2331 objects, with formats compatible with Dex-Net and mesh files for simulation.

Experiments involved training GG-CNN on the EGAD dataset and testing on the 49-object evaluation set. Each object was grasped 20 times, recording success rates. Hyperparameters included 100 grasp samples per object, 200,000 evolution steps, and a 25×25×25 CPPN resolution. Results showed success rates decreasing from 69% on easy objects to 40% on hard ones, with an overall success of 58%. Comparisons with YCB and Dex-Net datasets demonstrated EGAD’s superior diversity and challenge level. Ablation studies confirmed the importance of shape complexity and grasp difficulty metrics in dataset quality. The experiments validated EGAD’s utility for training robust models and benchmarking grasping performance across object complexities.

EGAD’s 2331 objects cover 93% of the defined search space, with higher diversity than YCB and Dex-Net. Models trained on EGAD achieved a success rate of 58%, outperforming those trained on less diverse datasets, especially on complex objects. The success rate decline from 69% to 40% across difficulty levels highlights the dataset’s challenging nature. Precomputed grasp poses enable rapid evaluation, and the dataset’s diversity improves model robustness. These results demonstrate EGAD’s potential to standardize and elevate robotic grasping research, fostering more generalizable and reliable systems.

EGAD supports training and benchmarking in robotic manipulation, industrial automation, and service robots. Its diverse objects help develop algorithms capable of handling real-world variability. Precomputed grasps facilitate deployment on different robotic platforms, reducing setup time. The standardized evaluation protocol enables fair comparisons, accelerating research progress. Future applications include multi-object manipulation, dynamic scene understanding, and integration with perception systems for autonomous robots in complex environments.

Object generation depends on CPPN parameters, which may not capture all real-world shape nuances, especially deformable or articulated objects. Precomputed grasps are static and may not adapt well to dynamic or multi-object scenarios. Computational costs for large-scale generation and evaluation are high, limiting scalability. The dataset primarily focuses on rigid, static objects, and extending to deformable or articulated shapes remains a challenge. Future work should address these limitations through adaptive learning, real-time grasp planning, and broader object categories.