A hybrid generative-discriminative learning algorithm for Bayesian network structure

The discriminative learning of Bayesian networks benefits the classification accuracy as compared to generative learning. Previous approaches mostly learn either the structure or the parameters in a discriminative manner based on the scoring+ search paradigm. Many works have focused on structure learning by optimizing a discriminative scoring function but the resulted structure is still generative in the sense that the class variable is not conditioned on attribute variables. On the other hand, searching Markov Blanket in a constrained space can generate a hybrid generative-discriminative structure. In this paper, we propose a new Hybrid Generative-Discriminative (HGD) algorithm for learning Bayesian network structure. The algorithm searches the neighboring structures by optimizing a cross-validated classification rate (CR) criterion to give a really discriminative structure. We select the initial structure and design neighborhood operators appropriately such that the learning procedure is computationally feasible. Our empirical study on a large suite of bench-mark dataseis shows that the proposed HGD+ CR algorithm yields better classification results than BN classifiers with only discriminative scores.