Position-based visual servoing for pallet picking by an articulated-frame-steering hydraulic mobile machine

Abstract

This paper addresses a visual servoing problem for a mobile manipulator. Specifically, it investigates pallet picking by using visual feedback using afork lift truck. A manipulator with limited degrees of freedom and differential constraint mobility together with large dimensions of the machine require reliable visual feedback (pallet pose) from relatively large distances. To address this challenge, we propose a control architecture composed of three main sub-systems: (1) pose estimation: body and fork pose estimation in the pallet frame; (2) path planning: from the current pose to the origin (pallet frame); and (3) feedback motion control. In this architecture, the pallet becomes the local earth fixed frame in which poses are resolved and plans are formulated. Choosing the pallet as the origin provides a natural framework for fusing the wheel odometry/inertial sensor data with vision, and planning is required only once the pallet is detected for the first time (because the target is always the origin). Visual pallet detection is non-real-time and unreliable, especially owing to large distances, unfavorable vibrations, and fast steering. To address these issues, we introduce a simple and efficient method that integrates the vision output with odometry and realizes smooth and non-stop transition from global navigation to visual servoing. Real-world implementation on a small-sized forklift truck demonstrates the efficacy of the proposed visual servoing architecture.