Towards an effective practice of learning from data and knowledge

Abstract

We discuss some recent advances on combining data and knowledge in the context of supervised learning using Bayesian networks. A first set of advances concern the computational efficiency of learning and inference, and they include a software-level boost based on compiling Bayesian network structures into tractable circuits in the form of tensor graphs, and algorithmic improvements based on exploiting a type of knowledge called unknown functional dependencies. The used tensor graphs capitalize on a highly optimized tensor operation (matrix multiplication) which brings orders of magnitude speedups in circuit training and evaluation. The exploitation of unknown functional dependencies yields exponential reductions in the size of tractable circuits and gives rise to the notion of causal treewidth for offering a corresponding complexity bound. Beyond computational efficiency, we discuss empirical evidence showing the promise of learning from a combination of data and knowledge, in terms of data hungriness and robustness against noise perturbations. Sometimes, however, an accurate Bayesian network structure may not be available due to the incompleteness of human knowledge, leading to modeling errors in the form of missing dependencies or missing variable values. On this front, we discuss another set of advances for recovering from certain types of modeling errors. This is achieved using Testing Bayesian networks which dynamically select parameters based on the input evidence, and come with theoretical guarantees on full recovery under certain conditions.