FPGA-based stereo matching for crop height measurement using monocular camera

Abstract

We have proposed a hardware-accelerated drone to analyze the condition of farmland right then and there; as a first step, we report that the proposed system can take crop height measurements with high accuracy using a monocular camera. The proposed three-dimensional farmland is generated using stereo matching, where a drone with a monocular camera can extend the parallax distance as the length between two positions when taking a ground image. This means that our approach can improve the accuracy of a reconstructed 3D farmland. In addition, toward real-time computation and low power consumption, the proposed hardware design accelerates image processing efficiently. Thus, to achieve this, we propose a strategy that combines the semi-global matching (SGM) with single path direction and a sum of absolute difference (SAD) with reduced disparity searching length. For example, a semi-global matching (SGM) was employed to smooth the disparity map result before checking the consistency, where the scan line was performed in one direction, from left to right, to speed up the computation time. The experimental result shows that the computation time performed by Xilinx Zynq ZCU102 FPGA achieves 0.77 s for the stereo data set images with 1536 × 1024 pixels resolution. To meet the real-time application and reduce the FPGA resources toward lower power consumption, the experiment discusses reducing the disparity searching length for the SAD computation. In our experiment, the execution time is less than 40 milliseconds, and the circuit volume is around 9,500 LUTs, equivalent to a small-size FPGA. Finally, we also estimated the object's height; a value of 0.43 m was estimated for the object with a physical height of 0.45 m. Meanwhile, for the object with a physical height of 0.65 m, a value of 0.63 m was estimated.