Predictive but Not Plannable: RC-aux for Latent World Models

Deep learning-driven latent world models like DreamerV3, PlaNet, and LeWorldModel have revolutionized pixel-based control by compressing high-dimensional sensory data into manageable latent representations. These models enable environment simulation and planning without pixel reconstruction, making them suitable for offline and goal-conditioned tasks. Early efforts focused on short-term prediction accuracy, using algorithms such as Variational Autoencoders and Recurrent Neural Networks. However, as applications extended to complex environments, limitations emerged: short-term predictive success did not guarantee reliable long-term planning. Researchers introduced value-guided distances, temporal structuring, and quasimetric metrics to improve latent space geometry. Despite these advances, the core issue remained—latent distances often failed to reflect actual environment reachability, leading to suboptimal planning outcomes. This paper builds upon these foundations, proposing a method to explicitly encode finite-horizon reachability into latent representations, addressing a critical gap in the field.

The main challenge in latent world modeling for long-horizon planning is the mismatch between the learned latent geometry and the environment's true reachability constraints. While models can predict short-term dynamics accurately, their latent spaces often contain shortcuts—states that appear close in Euclidean distance but are unreachable within a finite number of actions. This results in planning algorithms favoring infeasible trajectories, reducing success rates. The problem is compounded by the training paradigm: models are typically trained with local, short-horizon supervision, which does not align with the long-horizon evaluation during planning. Consequently, the latent space's geometric structure becomes misaligned with the environment's true dynamics, necessitating a method to encode finite-horizon reachability directly into the training process.

The core innovation of this work lies in integrating a reachability-aware auxiliary objective into latent world model training. This involves two key components: • Multi-horizon open-loop prediction, which extends the training supervision beyond one step, ensuring temporal consistency over multiple steps. • A budget-conditioned reachability predictor, trained with positive pairs, trajectory-induced hard negatives, and batch negatives, to estimate whether a target latent is reachable within a finite horizon. This predictor captures the environment's geometric and temporal constraints, effectively shaping the latent space to reflect true environment reachability. Additionally, during planning, the learned reachability signal is incorporated into the cost function, allowing trajectories to be evaluated not only by proximity to the goal but also by their attainability within the available action budget. This dual approach aligns the latent space geometry with the planning process, overcoming the limitations of traditional short-horizon prediction models.

• �� Data Collection: Collect trajectory data with observations, actions, and goal states.
• �� Multi-horizon Prediction: Train the model to predict future latents over multiple steps, minimizing the open-loop prediction loss to ensure temporal consistency.
• �� Reachability Supervision: Construct positive pairs from trajectory data where the target is within the finite horizon, and negative pairs where it is not, including trajectory-induced hard negatives and batch negatives.
• �� Reachability Head: Train a classifier Rϕ(z, z′, h) to predict whether z′ is reachable from z within horizon h, using BCE loss.
• �� Model Integration: Keep the backbone unchanged; add the reachability loss to the training objective, balancing prediction accuracy and reachability accuracy.
• �� Planning: During inference, incorporate the learned reachability signal into the cost function, modulating the trajectory evaluation with a parameter λplan to favor feasible paths. This results in a planning algorithm that considers both goal proximity and environment constraints.

The evaluation employed five pixel-based goal-conditioned tasks: TwoRoom, Reacher, Push-T, Wall, and Cube. Success rate was the primary metric, measured over multiple fixed evaluation sets. Baselines included the original LeWorldModel and other state-of-the-art models. Hyperparameters such as horizon length, number of candidate action sequences, and λplan were tuned to assess the impact of reachability supervision. Ablation studies isolated the effects of multi-horizon prediction, reachability supervision, and planner coupling. Additional experiments extended to LIBERO-Goal, testing generalization. Results were statistically analyzed, with success rate improvements up to 15% in challenging environments, confirming the effectiveness of the approach.

RC-aux consistently outperformed baseline models, with success rates reaching 98% on TwoRoom and 87.2% on Reacher, surpassing previous methods by significant margins. The largest gains appeared in environments with obstacles, where Euclidean distances failed as proxies for reachability. Ablation results confirmed that combining multi-horizon prediction with reachability supervision was crucial. The integration of learned reachability into planning further improved success, especially in complex scenarios. Visualizations demonstrated more stable progress toward goals, validating the geometric improvements in the latent space. These findings confirm that encoding finite-horizon reachability significantly enhances long-term planning reliability.

This approach is applicable to autonomous navigation, robotic manipulation, and virtual agent control, especially in environments with obstacles or dynamic constraints. It enables agents to plan trajectories that are both goal-directed and feasible within action limits, reducing failure rates. The method relies on high-quality trajectory data and latent space representations, making it suitable for offline training pipelines. Future integration with reinforcement learning can further optimize policy learning, broadening real-world deployment in robotics and autonomous systems.

The method depends on the quality and coverage of trajectory data, which may limit generalization to unseen states. Computational overhead increases due to additional reachability training, impacting scalability. Estimating accurate finite-horizon reachability in high-dimensional latent spaces remains challenging, especially in stochastic or highly dynamic environments. Future work should focus on adaptive budget estimation, real-time reachability updates, and multi-modal data integration to address these issues.