A Hierarchical Spatiotemporal Action Tokenizer for In-Context Imitation Learning in Robotics

With advancements in deep learning, the field of robotic imitation learning has garnered significant attention. Imitation learning (IL) aims to learn generalizable robot policies from expert demonstrations. However, due to the scarcity of high-quality demonstrations, IL often suffers from limited generalization. Inspired by the in-context learning capabilities of large language models (LLMs), in-context imitation learning (ICIL) has emerged as a promising paradigm, showcasing the potential to learn from demonstrations provided at inference time. ICIL allows robotic policies to perform new tasks without retraining, enabling flexible and efficient real-world deployment.

Despite its advantages, ICIL still struggles to learn contextualized action representations from demonstrations. Effective action representations can lead to notable performance gains in ICIL. However, existing methods face challenges in modeling temporal correlations. While positional encoding or vector quantization can be used to preserve temporal order, they often fail to maintain temporal smoothness in action trajectories. Therefore, capturing hierarchical action structures and spatiotemporal dependencies without sacrificing temporal smoothness remains a pressing issue.

This paper introduces a novel hierarchical spatiotemporal action tokenizer (HiST-AT) for in-context imitation learning. The core innovations include:

1. Hierarchical Vector Quantization: By employing two levels of vector quantization, input actions are first assigned to fine-grained subclusters and then mapped to larger clusters. This approach captures hierarchical action structures.

2. Spatiotemporal Reconstruction: By simultaneously reconstructing actions and their timestamps, the method leverages spatial and temporal information, enhancing the smoothness and effectiveness of action representations.

3. Lipschitz Regularization: Ensures the smoothness of action representations and reduces noise.

• �� Hierarchical Vector Quantization: Input actions are first mapped to latent representations through a Lipschitz-regularized network, then assigned to fine-grained subclusters and larger clusters through two levels of vector quantization.
• �� Spatiotemporal Reconstruction: Input actions and corresponding timestamps are reconstructed through spatial and temporal decoders, respectively.
• �� Training Losses: The model is optimized using a combination of hierarchical clustering, spatiotemporal reconstruction, and Lipschitz regularization losses, including encoder, regularizers, subaction and action codebooks, and spatial and temporal decoders.

Experiments were conducted on multiple simulation and real robotic manipulation datasets, including RoboCasa and ManiSkill. Baselines used include BC-Transformer, ACT, and MCR. Success rate was the evaluation metric, with key hyperparameters including codebook sizes and temporal reconstruction weights. Ablation studies were conducted to evaluate the impact of hierarchical clustering and spatiotemporal reconstruction.

On the RoboCasa dataset, HiST-AT achieved an average success rate of 59%, improving by 6% over the previous best method, LipVQ-VAE. On the ManiSkill dataset, HiST-AT achieved 85% success in the Pick Cube task, outperforming LipVQ-VAE by 7%. Ablation studies show that the combination of hierarchical clustering and spatiotemporal reconstruction results in superior performance of HiST-AT over other methods.

HiST-AT has broad application prospects in robotic manipulation tasks. Direct application scenarios include industrial automation, home service robots, and educational robots. Prerequisites include high-quality demonstration data and sufficient computational resources. The industrial impact lies in enhancing the generalization capability of robots across different tasks.

Despite its exceptional performance across multiple benchmarks, HiST-AT may encounter performance bottlenecks when dealing with highly complex action sequences. Additionally, the method requires significant hardware resources, which may not be suitable for resource-constrained environments. Future research directions include optimizing the algorithm to reduce computational overhead and exploring more efficient timestamp prediction methods.