Advanced Classification of Failure-Related Patterns on Solar Photovoltaic Farms Through Multiview Photogrammetry Thermal Infrared Sensing by Drones and Deep Learning

Abstract

Here, we propose an approach that relies on digital photogrammetry and deep learning to classify thermal infrared patterns sheltering potential failures within solar panels from aerial imagery collected by drones. We collect images from a solar plant using a rotary-wing drone equipped with an onboard thermal camera. The captured images are processed using a photogrammetric pipeline that stitches the images together producing a georeferenced thermal orthomosaic. The solar panels are digitized, extracted from the orthomosaic, labeled into 4 classes, augmented using transformations acting on their geometry and radiometry then utilized to constitute a dataset to train from scratch and validate a developed deep learning classifier. The latter consists of a convolutional neural network architecture comprising two core blocks: (1) a convolutional block that produces multi-level feature maps from the images, followed by (2) a multi-layer perceptron block that classifies the constructed feature maps according to the considered categories. The final developed model scores an F1-score of 98.2% on our validation sub-dataset, which confirms both its high performance and generalizability on additional data. The proposed approach elaborates an efficient, comprehensive and cost-effective framework to monitor solar farms through the use of drone-based thermal sensing, photogrammetry and deep learning, alongside addressing the drawbacks related to the use of classic techniques.