Probabilistic Graphical Models Meet Deep Learning for Semantic Segmentation: Mathematical connections and recent developments

Abstract

Semantic segmentation, also known as spatially dense image classification, plays a crucial role in image analysis, bridging the fields of image processing and machine learning. It has wide applications, ranging from land cover mapping in Earth observation to medical diagnostics using biomedical images, fault detection in industrial imagery, and so on. This article focuses on the mathematical connections between two pivotal families of methodological approaches—probabilistic graphical models (PGMs) and deep learning (DL)—and explores the potential of their integration for semantic segmentation tasks. After providing a comprehensive overview of state-of-the-art techniques from both families, the article highlights recent developments that combine these approaches, either through theoretical equivalence or direct integration. Examples of results are provided for renowned benchmark datasets in computer vision and remote sensing, and the article concludes with a discussion of promising future research directions.