Object-based spectral library for knowledge-transfer-based crop detection in drone-based hyperspectral imagery

Abstract

Crop mapping or crop recognition specifies the types of agricultural crops that grow in a selected region. Hyperspectral imaging (HSI) acquires spectral reflectance profiles of materials in hundreds of narrow and continuous spectral bands in the optical electromagnetic spectrum. The emerging compact HSI sensors mountable on ground-based platforms and drones are promising data sources for crop classification at sub-field level. Forming part of the knowledge engineering domain in developing spectral imaging-based systems for autonomous mapping of crops, Spectral Knowledge Transfer (SKT) is a data-driven image classification paradigm for precision crop mapping. Reflectance spectral libraries provide valuable reference reflectance databases. However, spectral diversity and heterogeneity in natural farms limit the relevance and accuracy of spectra-alone based spectral libraries for crop mapping. In addition, many crops are differentiated by a combination of geometrical and spectral features. Acquiring high-resolution HSI datasets using a VNIR hyperspectral imaging system mounted on ground and drone-based platforms, this research has explored the development and demonstration of an object-based spectral library for semi-autonomous classification of drone-based hyperspectral imagery for crop mapping at plant-level. Laying a factorial designed experimental setup on the research farms of the University of Agricultural Sciences, Bengaluru, India, three vegetable crops: tomato (Solanumlycopersicum L.), eggplant (Solanummelongena L.) and cabbage (Brassica oleracea L.), each treated with different nitrogen levels were grown. Altering the view angle and flying altitudes, ground and drone-based HSI datasets were acquired at different growth stages. Adapting to the shape of the crop, thousands of crop patches were extracted from the HSI datasets, considering nitrogen levels, illumination, and altitude regions. Structured in a RDBMS-compatible database architecture, a spectral library, named as Object-Based Spectral Library (OBSL), incorporating spatial, and spectral characteristics of plants at different altitudes is developed. Further, the OBSL has been experimentally implemented for the knowledge-transfer based classification of drone-based HSI for the plant-level mapping of cabbage and eggplant. Computing accuracy metrics such as overall accuracy (OA), F1-score, and defining a new metric, Inverse Turndown Ratio (ϕ), for an objective comparison of the accuracy estimates across flying heights, the classification performance was analyzed for changes across the flying heights and crop-composition of the imagery. The best estimates of accuracy are about 69% and 86% respectively for the pixel-based and object-based crop classification. Quantified by the Inverse Turndown Ratio, the knowledge-transfer effected through the OBSL is good and consistent across the flying heights with 86% and 90% reproducibility for the pixel-based and object-based approach. While the results from object-based approach call for optimizing flying height, overall, the results highlight the prospects of plant-level crop mapping and knowledge-transfer based hyperspectral image analysis for agriculture.