Knowledge Base Completion by Variational Bayesian Neural Tensor Decomposition

• Published in: Cognitive computation Vol. 10; no. 6; pp. 1075 - 1084
• Main Authors: He, Lirong, Liu, Bin, Li, Guangxi, Sheng, Yongpan, Wang, Yafang, Xu, Zenglin
• Format: Journal Article
• Language: English
• Published: New York Springer US 01.12.2018; Springer Nature B.V
• Subjects: Algorithms; Artificial Intelligence; Bayesian analysis; Biomedical and Life Sciences; Biomedicine; Computation by Abstract Devices; Computational Biology/Bioinformatics; Decomposition; Graphs; Inference; Information retrieval; Iterative methods; Knowledge bases (artificial intelligence); Knowledge representation; Machine learning; Mathematical analysis; Methods; Multilayers; Neural networks; Neurosciences; Query expansion; Tensors; Knowledge base completion; Variational Bayesian; Neural networks
• ISSN: 1866-9956; 1866-9964
• DOI: 10.1007/s12559-018-9565-x

Knowledge base completion is an important research problem in knowledge bases, which play important roles in question answering, information retrieval, and other applications. A number of relational learning algorithms have been proposed to solve this problem. However, despite their success in modeling the entity relations, they are not well founded in a Bayesian manner and thus are hard to model the prior information of the entity and relation factors. Furthermore, they under-represent the interaction between entity and relation factors. In order to avoid these disadvantages, we provide a neural-inspired approach, namely Bayesian Neural Tensor Decomposition approach for knowledge base completion based on the Stochastic Gradient Variational Bayesian framework. We employ a multivariate Bernoulli likelihood function to represent the existence of facts in knowledge graphs. We further employ a Multi-layered Perceptrons to represent more complex interactions between the latent subject , predicate , and object factors. The SGVB framework can enable us to make efficient approximate variational inference for the proposed nonlinear probabilistic tensor decomposition by a novel local reparameterization trick. This way avoids the need of expensive iterative inference schemes such as MCMC and does not make any over-simplified assumptions about the posterior distributions, in contrary to the common variational inference. In order to evaluate the proposed model, we have conducted experiments on real-world knowledge bases, i.e., FreeBase and WordNet. Experimental results have indicated the promising performance of the proposed method.