Learning Bayesian Network Structure from Large-Scale Datasets

• Published in: 2016 International Conference on Advanced Cloud and Big Data (CBD) pp. 258 - 264
• Main Authors: Yu Hong, Xiaoling Xia, Jiajin Le, Xiangdong Zhou
• Format: Conference Proceeding
• Language: English
• Published: IEEE 01.08.2016
• Subjects: AIC; Bayesian network structure learning; Big data; big data graph algorithm; information theory; large-scale datasets; Tabu Search
• DOI: 10.1109/CBD.2016.052

Bayesian network is one of the most classical and effective models in big data graph algorithms. Aiming at the problem of learning Bayesian network structure from large-scale datasets, a novel algorithm with the combination of Information theory, Tabu search and Akaike Information Criterion (AIC) called ITA is proposed. Firstly, a dimensionreduction algorithm based on information theory is used to filter non-target variables. The variables closely related to the target are picked as the vertexes in Bayesian network. Then choosing AIC as the scoring method and Tabu Search as the heuristic algorithm, a new learning algorithm is adopted to build the global optimal structure. Experimental results demonstrate that ITA algorithm can obtain core causal relationships from largescale datasets in certain area accurately and construct clean and straightforward Bayesian network structure at a lower time cost. Therefore, ITA is an effective and efficient big data graph algorithm for learning Bayesian network structure from largescale datasets.