Finger flow: Reactive reach-while-grasp generation for robotic arms with multi-fingered hands

Abstract

Humans effortlessly grasp both stationary and moving objects in one-shot motions, fluidly adapting to disturbances and automatically recovering from failed attempts. In contrast, robots with multi-fingered hands often rely on pre-planned, sequential “reach-then-grasp” strategies, which result in slow, unnatural motions and restrict the robot’s ability to react dynamically to changes in the object’s location. Moreover, open-loop execution oftentimes leads to grasp failures. To address these challenges, we introduce Finger Flow (FF), a reactive motion generator that uses the visual feedback from an onboard camera and position feedback from fingers and arms to robustly reach and grasp stationary and moving objects with unpredictable behavior. During the reaching, FF continuously guides the hand to avoid finger-object collisions and adjusts the hand’s reactive opening and closure based on its relative position to the object. This state-dependent behavior results in automatic recovery from failed grasp attempts. We also provide formal guarantees of convergence and collision avoidance for stationary spherical objects. We evaluate FF on the DLR humanoid robot neoDavid, equipped with a multi-fingered hand, and quantitatively assess its performance in a series of grasping experiments involving fast and reactive grasping of a stationary or unpredictable spatially moving object. Running in a closed loop at 3 kHz, FF achieves an 87 % grasp success rate on the stationary object placed at random positions over 130 attempts. Interactive and adversarial human-to-robot handover experiments further demonstrate the robustness and effectiveness of FF.