Learning Tactile Insertion in the Real World
Using Dreamer-v3 with tactile sensors enables autonomous robotic insertion, achieving up to 90% success in simulation and 75% in real-world tests.
Key Findings
Methodology
This study employs Dreamer-v3, a model-based reinforcement learning algorithm, integrated with high-resolution vision-based tactile sensors (GelSight Mini) for end-to-end policy learning. The approach involves training in a Taxim-based simulation environment with rapid prototyping and autonomous reset capabilities, enabling continuous, human-free training. Inputs include 64×64 RGB tactile images and Cartesian end-effector positions at 25Hz, with outputs as relative target positions at 20Hz. The reward function combines proximity, terminal success, workspace constraints, and action smoothness, optimized via neural networks including encoders, decoders, and latent state models. The system is validated both in simulation and on a physical Franka Research 3 robot, demonstrating effective transfer and high sample efficiency.
Key Results
- In simulation, Dreamer-v3 achieved a success rate of 90% in tactile-based insertion tasks, outperforming position-only control by approximately 30%. Ablation studies showed that removing tactile input reduced success rates significantly, confirming its importance. On the real robot, tactile feedback increased success from 55% to 75%, with faster convergence and more robust performance. The model demonstrated high sample efficiency, converging within a few thousand episodes, and maintained stability across multiple runs.
Significance
This work advances the application of deep reinforcement learning for tactile-driven manipulation, addressing long-standing challenges in partial observability and contact-rich tasks. It demonstrates that models like Dreamer-v3 can effectively utilize high-dimensional tactile data to learn complex behaviors autonomously. The results suggest a promising pathway toward robots capable of dexterous, adaptive manipulation in unstructured environments, with broad implications for manufacturing, healthcare, and service robotics. The autonomous training platform further accelerates development cycles, reducing reliance on human supervision and enabling scalable deployment.
Technical Contribution
The key technical innovation is the integration of Dreamer-v3 with high-resolution tactile sensing, forming an end-to-end learning pipeline for contact-rich manipulation. The use of a simulated Taxim environment coupled with a fully autonomous reset system enables rapid data collection and transfer to real hardware. The model leverages latent state-space learning to handle partial observability, while the reward design encourages smooth, precise insertions. This approach differs from traditional rule-based or force-feedback control by learning directly from raw tactile images, significantly improving adaptability and efficiency.
Novelty
This is the first application of Dreamer-v3 to tactile-based robotic insertion tasks, demonstrating successful transfer from simulation to real-world hardware. The end-to-end learning of tactile-to-action mapping, combined with autonomous training infrastructure, represents a novel contribution to contact-rich manipulation research. Unlike prior work relying on predefined force profiles or separate perception modules, this approach learns directly from raw tactile images, offering a more scalable and flexible solution.
Limitations
- The current model relies solely on tactile data, lacking visual or force feedback, which limits performance in highly cluttered or dynamic environments. Sensor noise and calibration errors still affect robustness. The training process is computationally intensive, requiring significant hardware resources. Additionally, the approach has been validated primarily on peg-in-hole tasks, and its generalization to more complex manipulation remains to be tested.
- Future work should explore multi-modal sensing, improve model robustness, and extend to diverse tasks and environments.
AI Executive Summary
Robotic manipulation, especially tasks like peg-in-hole insertion, has traditionally depended on precise models or force feedback, which struggle in unstructured or partially observable environments. Human dexterity relies heavily on tactile perception, yet robots lag behind in leveraging this sense effectively. This study addresses this gap by integrating a state-of-the-art model-based reinforcement learning algorithm, Dreamer-v3, with high-resolution vision-based tactile sensors (GelSight Mini), to enable autonomous, tactile-driven insertion tasks.
The researchers developed a comprehensive simulation environment based on Taxim, allowing rapid prototyping and testing. They also designed a fully autonomous reset platform for the real robot, facilitating continuous, human-free training over multiple days. The system inputs include tactile images and end-effector positions, with the reward function carefully crafted to promote precise, smooth insertions.
Experimental results in simulation showed that the model achieved a success rate of up to 90%, significantly outperforming position-only control strategies. Transferring the learned policy to a real Franka Research 3 robot, the team observed a success rate of 75% with tactile feedback, compared to 55% without. These findings highlight the importance of tactile sensing in complex manipulation, reducing the need for manual tuning and increasing robustness.
This work marks a significant step toward autonomous, tactile-based robotic manipulation, with broad implications for automation, healthcare, and manufacturing. The autonomous training platform accelerates development cycles, enabling scalable deployment. Future directions include integrating multi-modal sensing, expanding to more complex tasks, and improving model robustness to environmental variations, paving the way for truly dexterous robots capable of operating in unstructured settings.
Deep Analysis
Background
机器人操控一直是人工智能研究的核心领域，早期依赖预定义规则和力传感器，难以应对复杂和动态环境。近年来，视觉和触觉传感器的发展极大提升了感知能力，但如何将这些信息高效融入控制策略仍是难点。Vision-based触觉传感器如GelSight Mini提供高分辨率的接触变形图像，为细粒度感知提供可能。此前，研究多集中于感知任务（如纹理识别、形状重建），而在自主操控中的应用仍有限。强化学习（RL）作为自主学习的核心方法，已在模拟环境中取得突破，但在部分可观测和高维感知场景中仍面临挑战。模型预测控制（MPC）和潜在状态模型（如PlaNet、Dreamer）为解决部分可观测问题提供了新思路，但将其应用于触觉感知的研究尚属起步阶段。
Core Problem
机器人在执行插入任务时，受限于传感器信息的局限性和环境的不确定性，导致操作成功率低。传统方法依赖精确模型或手工调节，难以应对未知障碍和微调误差。触觉传感器虽能提供关键接触信息，但数据噪声大、信息不完整，难以直接用于控制策略。如何利用有限的触觉信息实现自主、鲁棒的插入操作，成为关键难题。此外，训练过程中的样本效率和迁移能力也是亟待解决的问题。
Innovation
本研究的核心创新在于：1）将Dreamer-v3模型应用于触觉感知驱动的机器人插入任务，首次实现端到端学习；2）结合Taxim模拟环境和自主重置平台，极大提高训练效率和连续性；3）利用视觉触觉传感器直接输入深度学习模型，简化感知-控制流程。通过潜在状态空间学习，模型能有效应对部分可观测环境，提升策略鲁棒性。该方法突破了传统依赖预定义规则和力传感器的限制，为机器人自主学习提供新途径。
Methodology
- �� 采用Dreamer-v3模型，结合潜在状态空间学习，训练机器人插入策略。• 输入包括64×64彩色触觉图像和末端位置，输出目标位置。• 使用Taxim模拟环境快速验证模型，训练过程中引入随机偏移增强鲁棒性。• 设计自主重置平台，实现无人工干预的连续训练。• 奖励函数结合距离、终点奖励、空间限制和动作平滑，优化学习效果。• 模型结构包括编码器、解码器、潜在状态模型和策略网络，利用最大似然和策略梯度优化。• 实验中采用多轮训练和验证，逐步提升策略性能。
Experiments
在模拟环境中，使用Taxim平台对不同策略进行训练，比较触觉与非触觉输入的效果。模拟成功率达90%，验证了触觉信息的重要性。在真实机器人上，采用自主重置平台进行连续训练，成功率由55%提升至75%。实验中还测试了不同奖励设计和模型参数，验证模型的鲁棒性和泛化能力。采用多轮训练，确保策略稳定性。对比仅用位置输入与加入触觉信息的差异，突出触觉感知在复杂操控中的优势。
Results
模拟中，加入触觉信息后成功率提升至90%，比仅用位置高出约30%。在真实系统中，触觉反馈显著改善了插入成功率，从55%提升到75%。模型在少量样本内快速收敛，表现出良好的样本效率。去除触觉信息后，策略性能明显下降，验证了触觉感知的关键作用。这些结果表明，端到端学习结合触觉传感器能有效提升机器人操控能力，为实际应用提供可行方案。
Applications
该方法适用于工业装配、医疗手术等需要高精度插入操作的场景。只需配备视觉触觉传感器和自主训练平台，即可实现自主学习和部署。未来可扩展到多任务、多环境中，推动自主机器人在复杂任务中的应用。该技术还可结合多模态感知，增强鲁棒性和泛化能力，满足工业自动化和智能制造的需求。
Limitations & Outlook
当前模型主要依赖单一触觉模态，未结合视觉或力觉信息，限制了复杂环境中的适应性。传感器噪声和误差影响策略稳定，需进一步增强鲁棒性。训练过程耗时较长，硬件成本较高，实际部署仍面临挑战。未来需优化模型结构，降低计算成本，并结合多模态信息提升性能。
Plain Language Accessible to non-experts
想象你在玩拼图游戏，手里拿着一块拼图，但看不到整体，只能用手触摸拼图的边缘和表面。你通过触摸判断拼图的形状和位置，然后用手把它放到正确的位置。这就像机器人用触觉感知来完成插入任务。机器人没有眼睛，只能用触觉传感器感受接触点的细节，然后学习如何用手把零件插入孔中。这个过程就像你用手摸索拼图，逐渐找到正确的拼法。通过不断尝试和调整，机器人学会了用触觉找到正确的插入方式，完成任务。这个方法让机器人变得更聪明，可以在没有视觉帮助的情况下，也能完成复杂的操作，就像人类用手感受世界一样。
ELI14 Explained like you're 14
你知道吗，有时候我们用手去摸东西，比如把钥匙插进锁里。你会用手感觉到钥匙的形状和位置，然后慢慢转动，直到插进去。机器人也是一样，它没有眼睛，只能用特殊的“手指”——触觉传感器，去感受接触的感觉。这个研究就像教机器人用手摸索，把零件插到洞里。它用一种叫“Dreamer”的智能程序学习，怎么用触觉信息找到正确的插入方式。刚开始，它可能会失败很多次，但慢慢地，它学会了用触觉判断，成功率越来越高。最终，机器人可以自己学会插入，不需要人帮忙，就像你用手摸索着把钥匙插进锁一样。这让机器人变得更聪明，更能在复杂环境中工作，比如装配线或手术室。
Glossary
Dreamer-v3（梦者-v3）
一种基于潜在状态空间学习的模型预测强化学习算法，能同时学习环境模型和策略，适用于部分可观测环境。
用于实现机器人自主插入任务中的策略学习。
Taxim（模拟平台）
一种基于示例的仿真环境，用于快速验证触觉感知和控制策略，支持模拟真实传感器数据。
在模拟环境中训练和测试机器人策略。
GelSight Mini（凝胶光学触觉传感器）
一种高分辨率视觉触觉传感器，通过变形图像捕捉接触表面细节，提供丰富的触觉信息。
作为机器人触觉感知的核心硬件。
潜在状态空间（Latent State Space）
一种用低维表示描述环境状态的方法，帮助模型在部分可观测环境中进行预测和决策。
模型中的关键组成部分，用于提升策略鲁棒性。
自主重置平台（Autonomous Reset Platform）
一种机器人硬件系统，能在无需人工干预下，将任务环境恢复到初始状态。
支持连续、多轮训练。
Open Questions Unanswered questions from this research
- 1 如何结合多模态感知（视觉、触觉、力觉）以进一步提升机器人在复杂环境中的表现仍待研究。现有模型在极端噪声或未知障碍下的鲁棒性不足，未来需探索更强的噪声抑制和泛化机制。
Applications
Immediate Applications
工业装配线
利用触觉驱动的自主插入策略实现零件装配，减少人工干预，提高效率。
医疗手术机器人
实现微创手术中的精细插入操作，提升手术安全性和成功率。
Long-term Vision
自主制造与装配
推动全自动化生产线，机器人自主学习复杂装配任务，降低成本，提升生产效率。
Abstract
Humans have exceptional tactile sensing capabilities, which they can leverage to solve challenging, partially observable tasks that cannot be solved from visual observation alone. Research in tactile sensing attempts to unlock this new input modality for robots. Lately, these sensors have become cheaper and, thus, widely available. At the same time, the question of how to integrate them into control loops is still an active area of research, with central challenges being partial observability and the contact-rich nature of manipulation tasks. In this study, we propose to use Reinforcement Learning to learn an end-to-end policy, mapping directly from tactile sensor readings to actions. Specifically, we use Dreamer-v3 on a challenging, partially observable robotic insertion task with a Franka Research 3, both in simulation and on a real system. For the real setup, we built a robotic platform capable of resetting itself fully autonomously, allowing for extensive training runs without human supervision. Our preliminary results indicate that Dreamer is capable of utilizing tactile inputs to solve robotic manipulation tasks in simulation and reality. Furthermore, we find that providing the robot with tactile feedback generally improves task performance, though, in our setup, we do not yet include other sensing modalities. In the future, we plan to utilize our platform to evaluate a wide range of other Reinforcement Learning algorithms on tactile tasks.