InfiniBench: Infinite Benchmarking for Visual Spatial Reasoning with Customizable Scene Complexity
InfiniBench uses LLM-guided, cluster-based layout and task-aware camera planning to generate infinite customizable 3D scenes, enhancing spatial reasoning benchmarks.
Key Findings
Methodology
This paper introduces a three-stage pipeline: an LLM-based agent refines scene constraints from natural language prompts; a cluster-based optimizer constructs dense, physically plausible layouts; and a task-aware camera trajectory algorithm ensures full object coverage in videos. The framework leverages multi-round iterative constraint refinement, combining semantic understanding with optimization techniques like frontier-based exploration for camera path planning. It integrates specific algorithms such as Infinigen for scene synthesis, CLIP for prompt fidelity evaluation, and Blender Cycles for high-fidelity rendering. This approach enables parameterized control over scene complexity and supports infinite scene variation, outperforming prior procedural and diffusion-based methods in realism and physical coherence.
Key Results
- In high-complexity scenarios, InfiniBench achieved a prompt fidelity of 0.98, CLIP score of 31.8, and nearly zero collision and out-of-bound objects, surpassing baseline methods like Infinigen and LayoutGPT. It maintained fidelity above 0.94 across different density levels, with realism scores reaching 0.93, demonstrating superior scene authenticity and physical plausibility.
- The generated benchmarks effectively support spatial reasoning tasks such as measurement, perspective-taking, and spatiotemporal tracking, showing robustness and scalability. Ablation studies confirmed the importance of cluster optimization and camera path planning in achieving high-quality scene synthesis.
- Compared to existing methods, InfiniBench provides a scalable, flexible, and physically consistent platform, enabling detailed analysis of VLM failures and advancing the development of more capable spatial reasoning models.
Significance
This work addresses a critical bottleneck in evaluating AI spatial reasoning by providing a fully automated, infinitely scalable scene generator guided by natural language. It bridges the gap between semantic understanding and physical scene coherence, facilitating comprehensive benchmarking and model training. The ability to generate diverse, high-fidelity, and physically plausible scenes opens new avenues for research in robotics, virtual reality, and AI understanding of complex environments. InfiniBench’s modular design and parameterized control make it a versatile tool for advancing AI spatial cognition, pushing the boundaries of current evaluation frameworks.
Technical Contribution
The core innovation lies in integrating LLM-driven constraint generation with a cluster-based layout optimizer and a task-aware camera planner, forming a seamless pipeline. This combination allows scalable scene synthesis with high physical realism, overcoming limitations of hierarchical optimization and pure generative models. The framework guarantees scene coherence through iterative feedback and physical plausibility checks, supported by algorithms like Dijkstra for path planning and frontier exploration for camera views. It introduces a novel multi-stage process that balances semantic richness, physical accuracy, and computational efficiency, enabling unlimited scene variation.
Novelty
This is the first framework to unify natural language scene description, iterative constraint refinement, cluster-based dense layout optimization, and task-aware camera planning into an end-to-end system capable of generating infinite, high-fidelity 3D scenes. Unlike prior works limited by rule-based or diffusion models, InfiniBench leverages LLMs for semantic understanding and optimization algorithms for physical coherence, establishing a new paradigm for scalable scene synthesis guided by language. Its ability to produce diverse, complex scenes with physical plausibility marks a significant leap forward in synthetic data generation for spatial reasoning benchmarks.
Limitations
- Despite its strengths, the system can still produce physically implausible layouts in extremely dense or multi-layered scenes, due to optimization complexity and computational costs. Handling dynamic or interactive scenes remains challenging, requiring further algorithmic development.
- Camera path planning, while effective for static scenes, may struggle with scenes involving complex interactions or multiple agents, necessitating multi-objective optimization strategies. Natural language understanding also influences scene quality, especially with ambiguous or complex descriptions.
- High computational demands for scene synthesis and rendering limit real-time applications. Future work should focus on reducing costs, improving semantic parsing, and extending to dynamic or multi-agent environments.
Future Work
Future directions include integrating multi-modal inputs such as audio or tactile cues to enrich scene descriptions, developing real-time scene synthesis capabilities, and enabling dynamic scene generation with interactive elements. Enhancing semantic understanding via fine-tuned LLMs and incorporating reinforcement learning for scene optimization are also promising. Expanding the framework to support multi-agent interactions and temporal dynamics will further broaden its applicability in robotics, AR/VR, and AI research.
AI Executive Summary
The challenge of evaluating AI's spatial reasoning abilities in complex environments has long hindered progress. Existing benchmarks lack the flexibility, diversity, and scalability needed to thoroughly test models under varied conditions. To address this, we introduce InfiniBench, a novel framework that transforms natural language scene descriptions into high-fidelity, physically plausible 3D environments. This system combines an LLM-driven constraint refinement process, a cluster-based layout optimizer, and a task-aware camera trajectory planner, enabling the generation of an infinite variety of scenes tailored to specific spatial reasoning tasks.
By leveraging multi-stage iterative optimization, InfiniBench ensures scene realism, physical coherence, and comprehensive coverage of objects, outperforming traditional procedural and diffusion-based methods. Experimental results demonstrate its ability to produce scenes with prompt fidelity scores of 0.98 and CLIP scores of 31.8, even in highly dense scenarios. These scenes serve as benchmarks for tasks like measurement, perspective-taking, and spatiotemporal tracking, providing researchers with a powerful tool to analyze and improve VLMs.
This work significantly advances the field of synthetic scene generation, offering a scalable, customizable, and physically consistent platform. Its modular design and natural language interface lower barriers for researchers and industry practitioners, fostering broader adoption. Looking ahead, integrating multi-modal inputs, real-time synthesis, and dynamic scene modeling will further expand its impact, paving the way for AI systems with deeper spatial understanding and reasoning capabilities.
Deep Dive
Abstract
Modern vision-language models (VLMs) are expected to have abilities of spatial reasoning with diverse scene complexities, but evaluating such abilities is difficult due to the lack of benchmarks that are not only diverse and scalable but also fully customizable. Existing benchmarks offer limited customizability over the scene complexity and are incapable of isolating and analyzing specific VLM failure modes under distinct spatial conditions. To address this gap, instead of individually presenting benchmarks for different scene complexities, in this paper we present InfiniBench, a fully automated, customizable and user-friendly benchmark generator that can synthesize a theoretically infinite variety of 3D scenes with parameterized control on scene complexity. InfiniBench uniquely translates scene descriptions in natural language into photo-realistic videos with complex and physically plausible 3D layouts. This is achieved through three key innovations: 1) a LLM-based agentic framework that iteratively refines procedural scene constraints from scene descriptions; 2) a flexible cluster-based layout optimizer that generates dense and cluttered scenes previously intractable for procedural methods; and 3) a task-aware camera trajectory optimization method that renders scenes into videos with full object coverage as VLM input. Experiments demonstrate that InfiniBench outperforms state-of-the-art procedural and LLM-based 3D generation methods in prompt fidelity and physical plausibility, especially in high-complexity scenarios. We further showcased the usefulness of InfiniBench, by generating benchmarks for representative spatial reasoning tasks including measurement, perspective-taking and spatiotemporal tracking.