Metric collection on semantically segmented highspeed lightning footage with machine learning

Abstract

An investigation of the accuracy of a system using semantic segmentation in determining the number of individual strokes, direction of leaders and strike points for high-speed lightning footage. The paper uses a pre-trained DeepLabv3+ network, chosen to allow for the lowest computer requirements, where the last layers of the model are retrained on lightning footage. The network produces semantic segmented images where each pixel has a numerical label which are evaluated to count strokes. Regions of interest are created per strike and used to reduce noise. The system has a stroke detection efficiency of 70.1%, direction accuracy of 80% and a strike point accuracy of 89.5% when evaluated on 15 videos.