Being Bayesian about learning Bayesian networks from hybrid data

We develop a new Bayesian model to infer the structure of Bayesian networks from hybrid data, that is, data containing a mix of continuous (Gaussian) and discrete (categorical) variables. In line with state-of-the-art hybrid Bayesian network models, we do not allow discrete variables to have continuous parents. However, our new model differs from existing approaches by incorporating discrete variables through multivariate linear regression rather than mixture modeling. In our model, the continuous variables follow a multivariate Gaussian distribution with a shared covariance matrix, while the mean vector varies across different configurations.

As with all Bayesian network models, we use directed acyclic graphs (DAGs) to represent conditional dependency relations among the continuous variables. For our Gaussian distribution, this requires the covariance matrix to be consistent with the structure of the DAG. Our key idea is to apply multivariate linear regression, using the discrete variables as potential covariates to adjust the mean vector of the multivariate Gaussian distribution. Each continuous variable is associated with its own regression model and discrete parent set. Since the values of the discrete variables vary across observations, the mean vector becomes observation-specific.

This enables mean-adjustment of the continuous variables for their discrete parents while simultaneously inferring a Gaussian Bayesian network among them. In simulation studies, we compare our new model against two state-of-the-art hybrid Bayesian network models and demonstrate that both existing models have conceptual shortcomings, positioning our new hybrid Bayesian network model as a strong alternative.