Discovering hierarchical common brain networks via multimodal deep belief network

• Published in: Medical image analysis Vol. 54; pp. 238 - 252
• Main Authors: Zhang, Shu, Dong, Qinglin, Zhang, Wei, Huang, Heng, Zhu, Dajiang, Liu, Tianming
• Format: Journal Article
• Language: English
• Published: Netherlands Elsevier B.V 01.05.2019; Elsevier BV
• Subjects: Belief networks; Brain; Brain architecture; Brain mapping; Common brain networks; DBN; Deep learning; DTI, FMRI; Feature extraction; Functional anatomy; Functional magnetic resonance imaging; Hierarchical structure; Machine learning; Magnetic resonance imaging; Mapping; Medical imaging; Neuroimaging; Neurology; Organizations; Structural hierarchy; Structure-function relationships; DBN; DTI, FMRI; Common brain networks; Hierarchical structure
• ISSN: 1361-8415; 1361-8423; 1361-8431; 1361-8423
• DOI: 10.1016/j.media.2019.03.011

•Explore the relationship between brain structure and function via deep believe network (DBN).•Efficient connectivity descriptors are adopted to describe connectivity at voxel level.•Multimodal DBN (fMRI/DTI) is used to represent hierarchical brain networks.•Promising results are obtained to represent common brain networks. Studying a common architecture reflecting both brain's structural and functional organizations across individuals and populations in a hierarchical way has been of significant interest in the brain mapping field. Recently, deep learning models exhibited ability in extracting meaningful hierarchical structures from brain imaging data, e.g., fMRI and DTI. However, deep learning models have been rarely used to explore the relation between brain structure and function yet. In this paper, we proposed a novel multimodal deep believe network (DBN) model to discover and quantitatively represent the hierarchical organizations of common and consistent brain networks from both fMRI and DTI data. A prominent characteristic of DBN is that it is capable of extracting meaningful features from complex neuroimaging data with a hierarchical manner. With our proposed DBN model, three hierarchical layers with hundreds of common and consistent brain networks across individual brains are successfully constructed through learning a large dimension of representative features from fMRI/DTI data. [Display omitted] The proposed computational framework. (a) Steps to obtain the structural connectivity trace-map and functional connectivity trace-map for each vertex. (b) Feature integration. (c) The 3-layer DBN model using the functional and structural profiles acquired from (a) as the inputs to derive the hierarchical representation across the subjects. (d) Explore and study the hierarchical common brain networks and their characteristics.