MV-BMR: A real-time Motion and Vision Sensing Integration based Agile Badminton Robot

This paper presents the Motion and Vision Sensing Integration-based Agile Badminton Robot (MV-BMR), a real-time system that plays badminton with human players. Current badminton robots excel at handling low-speed strikes, such as high clears and net shots, but struggle with high-speed cases, particularly short shots. This challenge arises from two key factors: the shuttlecock’s short flight time, which ranges from 500 to 1000 ms for flat and low flat shots, and the extensive range of the robot’s hit zone. This lingering problem highlights the necessity of designing a dynamic and precise badminton robot. We propose an innovative two-stage approach that incorporates trajectory prediction and control modules to address this challenge. In the first stage, we design the Shuttlecock Early Prediction Network (SEPNet) to estimate the robot’s hit zone with an Inertial Measurement Unit (IMU) mounted on the racket so that the robot can move immediately after a player hits the shuttlecock. In the second stage, we employ a data-driven method, which exploits detected trajectories of shuttlecocks to determine hit points and control the corresponding robot to accurately hit the shuttlecock with Nonlinear Model Predictive Control (NMPC). We have implemented such a real-time system and conducted extensive experiments. The average successful hit rate for short shots of 92.2% and the most extended rallies of 68 demonstrates that our design effectively overcomes the challenges. The video demonstration is available at: https://youtu.be/lQo1Ls5Rj3o.