Difference-Aware Retrieval Policies for Imitation Learning

Imitation learning has become a cornerstone of autonomous robot skill acquisition, enabling systems to learn complex behaviors directly from expert demonstrations. Behavior cloning (BC), as a straightforward supervised approach, has shown remarkable success in controlled environments, but its limitations in out-of-distribution generalization have hindered real-world deployment. When robots encounter states outside their training distribution, small errors tend to accumulate rapidly, leading to unpredictable and often failure-prone behaviors. Existing solutions, such as data augmentation, online feedback, or task-specific regularization, often require additional data collection or environment interaction, increasing complexity and cost.

In response to these challenges, the authors propose Difference-Aware Retrieval Policies (DARP), a novel semi-parametric architecture that leverages the training dataset during inference. Unlike traditional BC, which relies solely on a parametric policy network, DARP retrieves a set of k-nearest neighbors from the demonstration data for each query state. It then computes the difference vectors between these neighbors and the query, incorporating this local geometric information into the action prediction process. The neural network fθ takes as input the neighbor states, their actions, and the difference vectors, producing candidate actions conditioned on local context. These multiple predictions are aggregated through a permutation-invariant function gψ, such as averaging or more expressive set functions, resulting in a robust final action.

The core insight is that this neighbor aggregation acts as a form of Laplacian smoothing, effectively filtering high-frequency variance and stabilizing the policy. The authors provide a rigorous spectral analysis, showing that the implicit regularization enforces local Lipschitz continuity and reduces estimator variance, leading to improved stability and generalization. The approach requires no additional data collection, online supervision, or task-specific tuning, making it highly scalable.

Empirical evaluations across diverse benchmarks demonstrate DARP’s effectiveness. In MuJoCo control tasks, success rates improved by up to 20%, with significant reductions in rollout error variance. In robotic manipulation scenarios, success rates increased by 22-25%, especially in tasks with complex visual inputs. The method also outperformed baselines such as R&P, LWR, and transformer-based models, achieving performance gains of 15-46%. These results confirm that local neighborhood information, when properly integrated, can substantially enhance the robustness of imitation policies.

The significance of this work lies in its ability to bridge the gap between parametric and non-parametric methods, providing a scalable, theoretically grounded framework for improving imitation learning. By embedding the neighborhood regularization into the policy architecture itself, DARP simplifies training and hyperparameter tuning while delivering strong empirical gains. Its generality allows extension to multimodal action distributions, high-dimensional visual inputs, and potentially online learning scenarios. Looking ahead, integrating adaptive neighbor selection, learning distance metrics, and online feedback mechanisms could further elevate the capabilities of DARP, bringing autonomous robots closer to reliable, real-world deployment in dynamic environments.