Towards machine learning as AGM-style belief change

Abstract

Artificial Neural Networks (ANNs) are powerful computational models that are able to reproduce complex non-linear processes, and are being widely used in a plethora of contemporary disciplines. In this article, we study the statics and dynamics of a certain class of ANNs, called binary ANNs, from the perspective of belief-change theory. A binary ANN is a feed-forward ANN whose inputs and outputs take binary values, and as such, it is suitable for a wide range of practical applications. For this type of ANNs, we point out that their knowledge (expressed via their input-output relationship) can symbolically be represented in terms of a propositional logic language. Furthermore, in the realm of belief change, we identify the process of changing (revising/contracting) an initial belief set to a modified belief set, as a process of a gradual transition of intermediate belief sets — such a gradualist approach to belief change is more congruent with the behaviors of real-world agents. Along these lines, we provide natural metrics for measuring the distance between these intermediate belief sets, effectively quantifying the disparity in their encoded knowledge. Thereafter, we demonstrate that, similar to belief change, the training process of binary ANNs, through backpropagation, can be emulated via a sequence of successive transitions of belief sets, the distance between which is intuitively related through one of the aforementioned metrics. We also prove that the alluded successive transitions of belief sets can be modeled by means of rational revision and contraction operators, defined within the fundamental belief-change framework of Alchourrón, Gärdenfors and Makinson (AGM). Thus, the process of machine learning (specifically, training binary ANNs) is framed as an operation of AGM-style belief change, offering a modular and logically structured perspective on neural learning.