Bridging the Gap to Real-World Object-Centric Learning

Object-centric learning aims to mimic human perception by decomposing scenes into distinct entities, facilitating reasoning and generalization. Early models like Slot Attention demonstrated success on synthetic datasets such as CLEVR, but faced challenges with complex real-world images due to pixel-level reconstruction limitations. Recent advances incorporated auxiliary signals like motion and depth, yet these are unavailable in most real datasets. Self-supervised learning (e.g., DINO, MoCo) has revolutionized feature representation, offering high-level semantic features without labels. Despite progress, scalable, fully unsupervised object segmentation in natural scenes remains unresolved, especially without auxiliary cues.

Existing unsupervised object segmentation methods rely heavily on pixel-level reconstruction, which emphasizes low-level features like color and texture, failing to capture high-level semantic structures in complex scenes. This results in poor generalization to real-world datasets like COCO and PASCAL VOC. The core challenge is to develop a scalable, semantic-aware, unsupervised framework that can effectively group objects based on high-level features without supervision, overcoming the limitations of pixel-based approaches and auxiliary signals.

The paper introduces a novel approach that leverages self-supervised features (from models like DINO) as the reconstruction target, replacing pixel-level reconstruction. This shift emphasizes semantic information, enabling object grouping based on high-level features. The architecture combines pre-trained ViT or ResNet encoders with Slot Attention for grouping, and simple MLP or Transformer decoders for feature reconstruction. This design simplifies training, enhances scalability, and achieves superior performance on synthetic and real datasets. It uniquely integrates self-supervised learning with object-centric modeling, bridging the gap to real-world applicability.

• �� Feature extraction: Use a frozen pre-trained ViT or ResNet to obtain high-level features from input images. • Feature grouping: Apply Slot Attention to cluster features into object slots via iterative attention. • Feature reconstruction: Use a decoder (MLP or Transformer) to reconstruct features from slots, minimizing L2 loss. • Training: Optimize with Adam, freeze encoder weights, on datasets like MOVi, COCO, PASCAL VOC, with hyperparameters tuned for stability. • Evaluation: Use FG-ARI and mBO metrics, resize masks bilinearly for comparison, perform ablations on feature types and decoder design. • Ablation: Confirm importance of pre-trained features and reconstruction loss for performance gains.

The experiments involve synthetic datasets MOVi-C and MOVi-E, with 1000 realistic 3D objects, and real datasets COCO and PASCAL VOC. The model uses ViT-B/8 and ViT-B/16 encoders, trained for 500k steps on synthetic and 250k on real data. Baselines include Slot Attention, SLATE, and naive block masks. Performance metrics include FG-ARI and mBO. Results show significant improvements over baselines, with FG-ARI scores above 65 on synthetic data and around 34 on COCO. Ablation studies highlight the importance of pre-trained features and simple decoders, with the model demonstrating robustness across scenarios.

On synthetic datasets, FG-ARI scores reach 65.4 (MOVi-C) and 63.8 (MOVi-E), outperforming Slot Attention (~40-45). On real datasets, FG-ARI scores are 34.2 (COCO) and 27.8 (PASCAL VOC), surpassing pixel-based methods. Ablation confirms that pre-trained ViT features and feature reconstruction are critical. The simple MLP decoder offers a good balance of performance and stability, while Transformer decoders, though more powerful, are less stable. The approach generalizes well across synthetic and real data, marking a significant advance in unsupervised object segmentation.

This method can be applied in autonomous driving for scene understanding, robotics for environment perception, and video analysis for object tracking. Its unsupervised nature reduces reliance on labeled data, making it suitable for large-scale deployment. The framework can facilitate real-time scene decomposition, aiding decision-making in complex environments. Future integration with multi-modal data (depth, motion) could further enhance its utility in practical systems.

Dependence on pre-trained self-supervised features means performance drops if features lack semantic richness. Low-resolution feature maps limit mask detail, affecting segmentation precision. Computational costs are high, especially for large-scale datasets, restricting accessibility. The current approach does not incorporate temporal or multi-modal cues, which could improve robustness. Future work should address these issues to broaden applicability and efficiency.