DexDrummer: In-Hand, Contact-Rich, and Long-Horizon Dexterous Robot Drumming

Dexterous manipulation is a crucial research area in robotics, involving complex finger-object interactions. Existing studies primarily focus on short-horizon tasks or isolated aspects of dexterity, such as in-hand object reorientation, grasping, and tool use. These studies provide valuable insights into dexterity but often emphasize short-horizon tasks or study dexterity aspects in isolation. In contrast, many real-world tasks, such as assembly or cooking, require dexterous skills that combine in-hand control, robustness to external perturbations, and long-horizon robustness. For example, assembling parts often involves reorienting a fastener in the hand while applying force to connect components, and cooking requires both holding utensils stably and stirring against resistance. Motivated by the need for a compelling testbed, we propose drumming, a long-horizon, contact-rich dexterous manipulation task. Drumming inherently requires balancing in-hand control – maintaining and adjusting the grasp of the stick with fine finger control – and external contact – forcefully and repeatedly striking drums. To play long songs, this control becomes even more crucial: drumming requires a policy robust to these contacts for extended periods of time.

In robotics, dexterous manipulation remains an unsolved challenge, especially in tasks involving long-horizon coordination and contact-rich interactions. Existing research often addresses these challenges separately, without combining these skills into a single complex task. To further test the capabilities of dexterity, we propose drumming as a testbed for dexterous manipulation. Drumming naturally integrates all three challenges: it involves in-hand control for stabilizing and adjusting the drumstick, contact-rich interaction through repeated striking of the drum surface, and long-horizon coordination when switching between drums and sustaining rhythmic play.

DexDrummer's core innovation lies in its hierarchical bimanual drumming policy, achieving sim-to-real transfer through simulation training. Its framework combines trajectory planning with residual reinforcement learning (RL) corrections for fast transitions between drums. A dexterous manipulation policy handles contact-rich dynamics, guided by rewards that explicitly model both finger-stick and stick-drum interactions. DexDrummer is the first to apply dexterous manipulation to the complex task of drumming, combining trajectory planning and residual RL. Compared to existing work, it not only performs well in simulation but also successfully transfers to real-world environments, showcasing its innovation in complex tasks.

DexDrummer's implementation involves the following key steps:

• �� High-Level Policy: Introduces parameterized motion primitives to generate task-space drumstick trajectories from musical inputs. These trajectories are converted into arm motions via motion planning, producing nominal control commands for the robot arms. A residual RL policy then learns corrective adjustments on top of this planner to compensate for tracking errors during fast transitions between drums.

• �� Low-Level Dexterous Policy: Trains a dexterous manipulation policy to handle the contact-rich dynamics of drumming. Learning is structured using contact-targeted rewards, which explicitly address two types of interactions: in-hand contacts and external contacts. In-hand contacts correspond to finger-stick interactions that manipulate the drumstick through fingertip contact and a fulcrum grasp and stabilize through arm energy penalty. External contacts correspond to interactions between the stick and the drum surface. To learn robust striking behavior, trajectory-guided rewards and a contact curriculum are introduced to stabilize learning of impacts.

The experimental design includes a simulated drum environment created in the ManiSkill framework, consisting of a bimanual robot setup and a full drum set (snare, tom, ride, hi-hat, and crash). This requires controlling and coordinating two arms and hands under a single policy that can simultaneously play different drums. Three types of tasks are designed for evaluation: bimanual full-drum set songs in simulation, single-drum tasks that emphasize dexterity in both simulation and the real world, and bimanual two-drum songs in the real world.

In experiments, DexDrummer demonstrates outstanding performance in both simple and complex songs in simulation. It outperforms a fixed grasp policy by 1.87x in F1 scores across easy songs and 1.22x across hard songs. In real-world tasks, DexDrummer achieves an F1 score of 1.0 when playing both the training song and its extended version. These results indicate that DexDrummer not only performs well in simulation but also successfully transfers to real-world environments. Ablation studies show that removing the residual RL policy reduces the F1 score to 0.8, and further removing motion planning drops it to 0.5.

DexDrummer's application scenarios include robotic music performance, dexterous manipulation in complex tasks, and other fields requiring long-horizon coordination and contact-rich interactions. Its application in music performance demonstrates the potential of robots in the arts, while its dexterous manipulation capabilities provide new possibilities for industrial automation and service robots. By combining trajectory planning and residual RL, DexDrummer offers new insights into solving coordination and transition challenges in complex tasks.

DexDrummer's speed in playing multi-drum songs is still not on par with humans, mainly limited by current hardware capabilities and algorithm optimization. Additionally, DexDrummer may not perform as well in completely unseen drum transitions as it does in transitions seen during training. Due to reliance on simulated environments, DexDrummer may not perform well under certain dynamic changes in the real world. Future research directions include improving DexDrummer's speed and flexibility in multi-drum songs, exploring more complex musical styles, and validating its effectiveness in more real-world scenarios. Additionally, further optimizing the algorithm to reduce dependency on simulated environments is an important research direction.