Generation of metrics by semantic segmentation of high speed lightning footage using machine learning

Abstract

An investigation into the application of semantic segmentation in generating metrics from high speed footage is presented. Five pre-trained networks were assessed, DeepLabV3performed the best with 87.5 mean accuracy given limited GPU ram for training. The temporal part of the system uses the segmentation labels to determine information about the lightning events. A direction finding procedure based on DBscan is implemented. The performance of the system as a whole across 7 points for per-frame analysis is between 48% and 81.5% based on 27 events. The system parameters require adjusting to better extract the metrics from the events. The system is not able to replicate manual classification but is an aid in classification.