Real-time path planning for multi-scenario headland turns in agricultural vehicles using panoramic vision and sequential frame correlation method

Headland turning poses a significant challenge to the vision-based autonomous navigation of agricultural vehicles. Compared to in-field crop row tracking, headland turning requires a greater variety of maneuvers with larger turning angles. Existing research on agricultural vehicle navigation is limited to using a single front-facing camera as the sensor. However, during turning maneuvers, the field of view of a single front-facing vision sensor is restricted, inevitably losing perception of the headland area. Unfortunately, no practical purely vision-based headland turning method has been proposed yet. To address this critical gap in agricultural automation, which demands robust industrial information integration engineering (IIIE) solutions, a panoramic surround view (PSV) system incorporating four fisheye cameras was developed in this study. The PSV image generated enabled a continuous perception of the relative position between the agricultural vehicle and the headland. Moreover, due to the vast disparity in perceived viewpoints and navigational references, previous methods for detecting navigational lines in the field are no longer applicable. Therefore, leveraging the integrated panoramic data stream, a multi-scenario headland turning path planning algorithm tailored for panoramic vision has been developed by extracting valuable correlations between sequential panoramic frames and integrating deep learning techniques. Robust detection of the headland area in PSV images was accomplished through semantic segmentation. Functional area delineation of the headland was achieved by correlating the headland locations within adjacent frames, thus enabling accurate turn path planning. Leveraging the lightweight model CGNet, the mean intersection over union (mIoU) for the semantic segmentation of the headland reached 92.84%. The average deviation of the detected headland boundary was 5.63 pixels in an image with a resolution of 640×340. The proposed algorithm demonstrated an inference speed of 18.45 frames per second. Field navigation experiments have yielded promising results, demonstrating that the proposed method effectively addresses the practical challenge of vision-only headland turning for agricultural vehicles, thereby laying the groundwork for a fully vision-based IIIE navigation system in agricultural applications, contributing to the broader goals of industrial integration and informatization in agriculture.