Quantitative verification of learning-enabled systems using ProbStar reachability

Abstract

Deep neural networks (DNN) verification primarily focuses on qualitative verification, determining whether a DNN violates safety or robustness properties. This paper introduces a novel approach for quantitative verification of Feedforward Neural Networks (FFNN), transforming qualitative assessments into probabilistic evaluations. The resulting quantitative verification method not only can answer YES or NO questions but also can compute the probability of a property being violated. To do that, we introduce the concept of a probabilistic star (or shortly ProbStar), a new variant of the well-known star set, in which the predicate variables belong to a Gaussian distribution. We further propose an approach to compute the probability of a probabilistic star in high-dimensional space. Unlike existing works dealing with constrained input sets, our work considers the input set as a truncated multivariate normal (Gaussian) distribution, i.e., besides the constraints on the input variables, the input set has a probability of the constraints being satisfied. The input distribution is represented as a probabilistic star set and propagates through a network to construct the output reachable set containing multiple ProbStars, which are used to verify the safety or robustness properties of the network. In case a property is violated, the violation probability can be computed precisely by an exact verification algorithm or approximately by an over-approximate verification algorithm. Building on this foundation, we extend our quantitative verification framework to Learning-Enabled Cyber-Physical Systems (Le-CPS), where a piecewise linear FFNN controls a linear physical plant model. Our approach enables the construction of probabilistic reachable sets for Le-CPS, allowing for both qualitative Safe/Unsafe assessments and quantitative probability computations of property violations. We have implemented our verification framework in a tool named StarV and evaluated its effectiveness on benchmarks including HorizontalCAS and ACASXu networks, a rocket landing system, as well as advanced emergency braking and adaptive cruise control systems within the Le-CPS domain.