ECO(VI)2SE: Expert-computer vision integration for visual coral status exploration

Cold-water coral reefs and associated habitats are of high ecological relevance and are subject to a diverse spectrum of anthropogenic stressors. Consequently, being able to evaluate the biodiversity and health status of cold-water coral reefs is of high importance. A web application for large-scale assessment of multiple cold-water coral reefs would improve our understanding of the effects of these stressors, and provide an important knowledge base for future planning of human activities in relation to these reefs. In this work, we present a new computational approach to the bottleneck problem of analyzing 77 h of ROV video from cold-water coral reef health status assessments. By combining domain expert knowledge, state-of-the-art deep learning image segmentation and information visualization, we have developed an efficient and sustainable workflow for analyzing visual cold-water coral monitoring data on a continuous basis. The deep learning segmentation network detected and segmented Desmophyllum pertusum, Paragorgia arborea, other gorgonians and sponges from the background in the test set with an intersection over union values of (81.77%, 85.64%, 63.64%, 40.5%, 96.13%) despite fluctuations in water quality and marine snow. Comparisons with manual ROV video evaluations from field personnel showed that the results from the computational approach correlated with the expert-based assessment.