Deep-Learning-Based Control of a Decoupled Two-Segment Continuum Robot for Endoscopic Submucosal Dissection
GRU-based control of a dual-segment continuum robot achieved position/orientation RMSEs of 1.11mm/4.62°, outperforming LSTM and others.
Key Findings
Methodology
This work introduces a deep learning controller based on gated recurrent units (GRU) for the dual-segment continuum robot DESectBot, enabling simultaneous control of tip position and orientation. The approach combines inverse kinematics modeling with data-driven neural networks, trained on multi-scenario datasets, to handle complex nonlinear couplings. Comparative experiments with Jacobian inverse kinematics, MPC, FNN, and LSTM demonstrate that GRU achieves superior trajectory tracking and fixed-point control accuracy, with lower RMSEs. The training involved extensive data collection via EM tracking, optimized hyperparameters, and end-to-end learning strategies, resulting in a lightweight, fast-inference model suitable for real-time surgical applications.
Key Results
- In nested-rectangle and Lissajous trajectory tracking, the GRU model achieved position RMSEs of 1.11mm and 0.81mm, and orientation RMSEs of 4.62° and 2.59°, outperforming LSTM and traditional methods. In fixed-point tasks with four target poses, the average RMSE was 0.14mm and 0.72°, with success rate reaching 100% in peg transfer experiments. The average transfer time was 11.8 seconds, with significantly reduced standard deviation, demonstrating high reliability and precision.
Significance
This study advances the control of multi-degree-of-freedom continuum robots by integrating deep learning, addressing nonlinear coupling and real-time constraints. It enhances the potential for minimally invasive surgeries, enabling precise, stable, and autonomous-like manipulation within complex anatomical environments. The framework bridges the gap between traditional model-based control and data-driven adaptability, promising broad impact in medical robotics, especially in endoscopic and soft robotics applications, fostering safer, more efficient surgical procedures.
Technical Contribution
The paper pioneers the application of GRU neural networks in dual-segment continuum robot control, proposing an end-to-end learning framework that captures temporal dependencies and inter-segment nonlinearities. It systematically compares with LSTM and classical methods, demonstrating faster convergence, better generalization, and lower computational cost. The design includes multi-task learning for position and orientation, extensive data collection, and hyperparameter tuning, establishing a new benchmark for intelligent control in complex soft robots.
Novelty
This is the first work applying GRU-based deep learning control to a dual-segment continuum robot for simultaneous position and orientation control. Unlike prior works limited to single-segment or linear models, this approach effectively manages nonlinear coupling, offering real-time, high-precision control. The integration of multi-scenario training and end-to-end learning distinguishes it as a novel solution for complex surgical robotics, filling a critical gap in the field.
Limitations
- Model accuracy diminishes in highly dynamic or unpredictable environments, requiring further robustness enhancements.
- Training data collection is resource-intensive, limiting scalability across diverse clinical scenarios.
- Real-time inference, while fast, still depends on high-performance hardware; embedded systems need optimization.
Future Work
Future directions include integrating reinforcement learning for adaptive control, expanding multi-tool and multi-segment coordination, and validating in vivo. Additionally, exploring transfer learning to reduce data dependency and developing hardware-accelerated inference will further enhance clinical viability.
AI Executive Summary
This research addresses a critical challenge in minimally invasive surgery: precise, real-time control of flexible, multi-degree-of-freedom endoscopic robots. Traditional manual operations demand extensive training and are prone to variability, especially in complex anatomical regions. To overcome these limitations, the authors developed DESectBot, a dual-segment continuum robot equipped with integrated surgical forceps, capable of six degrees of freedom at the tip. The core innovation lies in employing a deep learning controller based on gated recurrent units (GRU), which learns the nonlinear coupling between segments and predicts control commands from sequential data.
The control framework combines inverse kinematics modeling with data-driven neural networks, trained on extensive multi-scenario datasets. Comparative experiments with Jacobian inverse kinematics, model predictive control (MPC), feedforward neural networks (FNN), and LSTM networks demonstrate that GRU achieves the highest accuracy, with trajectory tracking RMSEs of 1.11mm and 4.62°, and fixed-point control RMSEs of 0.14mm and 0.72°. In practical peg transfer tasks, the system achieved 100% success rate with an average transfer time of 11.8 seconds, outperforming novice operators.
Experimental validation on ex vivo tissue resection confirmed the robot's sufficient stiffness and workspace for large lesions, demonstrating its potential for clinical application. The proposed approach significantly enhances the precision, reliability, and usability of robotic endoscopy, paving the way for intelligent, autonomous surgical systems. Future work will focus on robustness improvements, multi-tool coordination, and in vivo validation, aiming to transform minimally invasive procedures with AI-driven robotic assistance.
Deep Analysis
Background
连续机器人在微创手术中的应用逐渐成熟，尤其在内窥镜手术中展现出巨大潜力。早期研究多集中于单段柔性机器人，受限于自由度和控制复杂性。近年来，双段、多段结构被提出，以增强操作灵活性，但控制难度也随之增加。传统的模型驱动方法如雅可比逆解和模型预测控制（MPC）在复杂场景中表现有限，难以应对非线性耦合。深度学习的引入，为解决高维、多自由度控制问题提供了新思路。已有研究在软体机器人和机械臂中取得一定成果，但在多段连续机器人中的应用仍属空白。本论文结合逆运动学模型和深度学习，提出创新控制策略，旨在实现更高精度、更强鲁棒性的手术机器人控制。
Core Problem
多段连续机器人在端部位置和姿态同步控制中面临非线性耦合、模型不确定性和实时性挑战。传统控制方法难以兼顾高自由度和复杂运动轨迹，导致操作精度不足、响应迟缓。尤其在微创手术中，微米级的定位精度和毫秒级的响应速度是关键。现有技术多依赖精确模型，难以适应环境变化和器械变形，限制了其临床推广。如何在保证实时性和鲁棒性的前提下，实现多自由度同步控制，成为亟待解决的核心问题。
Innovation
本论文提出基于GRU的深度学习控制框架，创新点包括：1）引入多任务学习策略，联合端部位置与姿态的控制；2）结合逆运动学模型，构建端到端数据驱动控制器；3）采用多场景数据采集，增强模型泛化能力；4）实现端到端训练，提升响应速度。该方法突破了传统模型依赖的局限，利用神经网络强大的非线性建模能力，有效解决复杂耦合问题。
Methodology
- �� 构建双段连续机器人逆运动学模型，基于空间交叉弯曲盘（SCD）结构，参数化每段弯曲角度。
- �� 采集多场景运动数据，包括不同轨迹和操作条件，利用NDI® EM追踪器获得端部位置与姿态。
- �� 设计多种神经网络架构（FNN、LSTM、GRU），比较其在轨迹追踪和定点控制中的表现。
- �� 采用Adam优化器，超参数调优后，训练深度模型，确保模型在不同任务中的泛化能力。
- �� 将训练模型集成到闭环控制系统中，实现实时轨迹跟踪和定点姿态控制。
- �� 设计多任务学习策略，优化模型在位置和姿态同步中的表现，提升控制精度。
Experiments
采用由多场景采集的端部运动数据，分为训练、验证和测试集，确保模型的泛化能力。对比分析了不同网络架构在轨迹追踪和定点控制中的表现，指标包括RMSE、成功率和响应速度。实验中设置不同轨迹（嵌套矩形、Lissajous）和多目标定点姿态，验证模型在复杂路径中的适应性。超参数调优通过交叉验证完成，确保模型在实际手术场景中的鲁棒性。
Results
GRU模型在轨迹追踪中，嵌套矩形轨迹的位姿RMSE为1.11mm，姿态4.62°，优于LSTM（RMSE：1.45mm/5.10°）和传统方法。Lissajous轨迹下，RMSE为0.81mm/2.59°，表现更佳。在定点控制中，平均RMSE为0.14mm和0.72°，实现了高精度端部操作。在夹持转移任务中，成功率达120/120，平均转移时间11.8秒，标准差明显低于新手控制，验证了模型的实用性和稳定性。
Applications
该控制框架适用于微创手术中的内窥镜机器人、软体机器人等场景，能显著提升手术精度和效率。未来可结合自动路径规划和手术导航系统，推动机器人手术的智能化发展，减少医师操作难度，缩短培训周期。
Limitations & Outlook
模型在极端复杂轨迹或环境变化下仍存在误差，需进一步优化泛化能力。数据采集成本较高，模型训练依赖大量标注数据。硬件计算资源有限时，推理速度可能受影响，未来需结合硬件加速方案。
Plain Language Accessible to non-experts
想象你在厨房里做饭，厨师需要同时控制锅铲的方向和力度，确保菜炒得均匀。传统方法就像用手盯着锅，靠经验慢慢调整，但容易出错。现在，有了智能助手，它能学习厨师的动作，提前知道你要炒什么，自动调整锅铲的角度和力度，让炒菜变得又快又准。这就像论文里的机器人控制系统，用深度学习让机器人像厨师一样灵巧，能同时控制多个动作，完成复杂的任务，比如切菜、翻炒，甚至做出精细的手势。
ELI14 Explained like you're 14
想象你在玩一个超级复杂的电子游戏，你要同时控制角色的方向和动作，比如跳跃、转身、攻击。以前，你得用手指不停切换按钮，特别难，容易出错。现在，有个聪明的机器人助手，它能记住你的操作习惯，提前预测你要做什么，然后帮你自动完成这些动作。这个机器人就像论文里的控制系统，用一种叫GRU的智能“脑袋”，学习你的操作方式，帮你同时控制角色的方向和姿势，让游戏变得更顺畅、更好玩。它可以在复杂的场景中快速反应，帮你赢得比赛！
Glossary
连续机器人（Continuum Robot）
一种柔性机械臂，具有连续弯曲能力，能在狭窄空间中灵活操作。
论文中描述的双段连续机器人结构。
门控循环单元（GRU, Gated Recurrent Unit）
一种具有门控机制的循环神经网络，用于捕捉时间序列中的长短期依赖关系。
控制器核心算法，用于端部位置和姿态同步。
逆运动学（Inverse Kinematics）
根据末端目标位置和姿态，计算关节或驱动器的运动参数。
模型基础，用于生成控制指令。
轨迹追踪（Trajectory Tracking）
机器人按照预设路径运动，确保端部位置与轨迹偏差最小。
实验中的主要任务之一。
模型预测控制（MPC, Model Predictive Control）
基于模型预测未来状态，优化当前控制输入的控制策略。
对比分析中的模型驱动方法。
Open Questions Unanswered questions from this research
- 1 如何在极端复杂环境中保持控制的鲁棒性仍需研究，尤其是在手术中遇到组织变形和未知干扰时。
- 2 深度学习模型在实际临床应用中的泛化能力和安全性仍需验证，尤其是在不同患者和手术场景中。
Applications
Immediate Applications
微创手术机器人
可用于内窥镜手术中的精细操作，提高手术精度和安全性，缩短学习曲线，减少医师疲劳。
Long-term Vision
智能手术系统
结合自动路径规划、实时导航和深度学习控制，推动手术机器人向自主化、智能化方向发展，改善全球医疗水平。
Abstract
Manual endoscopic submucosal dissection (ESD) is technically demanding, and existing single-segment robotic tools offer limited dexterity. These limitations motivate the development of more advanced solutions. To address this, DESectBot, a novel dual segment continuum robot with a decoupled structure and integrated surgical forceps, enabling 6 degrees of freedom (DoFs) tip dexterity for improved lesion targeting in ESD, was developed in this work. Deep learning controllers based on gated recurrent units (GRUs) for simultaneous tip position and orientation control, effectively handling the nonlinear coupling between continuum segments, were proposed. The GRU controller was benchmarked against Jacobian based inverse kinematics, model predictive control (MPC), a feedforward neural network (FNN), and a long short-term memory (LSTM) network. In nested-rectangle and Lissajous trajectory tracking tasks, the GRU achieved the lowest position/orientation RMSEs: 1.11 mm/ 4.62° and 0.81 mm/ 2.59°, respectively. For orientation control at a fixed position (four target poses), the GRU attained a mean RMSE of 0.14 mm and 0.72°, outperforming all alternatives. In a peg transfer task, the GRU achieved a 100% success rate (120 success/120 attempts) with an average transfer time of 11.8s, the STD significantly outperforms novice-controlled systems. Additionally, an ex vivo ESD demonstration grasping, elevating, and resecting tissue as the scalpel completed the cut confirmed that DESectBot provides sufficient stiffness to divide thick gastric mucosa and an operative workspace adequate for large lesions.These results confirm that GRU-based control significantly enhances precision, reliability, and usability in ESD surgical training scenarios.