MSAFF: Multi-Way Soft Attention Fusion Framework With the Large Foundation Models for the Diagnosis of Alzheimer's Disease

• Published in: IEEE transaction on neural networks and learning systems Vol. 36; no. 10; pp. 17541 - 17555
• Main Authors: Bi, Xia-An, Shen, Wenzhuo, Shan, Yinglu, Chen, Dayou, Xu, Luyun, Chen, Ke, Liu, Zhonghua
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.10.2025
• Subjects: Algorithms; Alzheimer Disease - diagnosis; Alzheimer Disease - diagnostic imaging; Alzheimer Disease - genetics; Alzheimer’s disease (AD); Attention; Attention mechanisms; Data mining; disease diagnosis; Feature extraction; Foundation models; Functional magnetic resonance imaging; Humans; Imaging; imaging genetics; large foundation model (LFM); Mathematical models; multi-way soft attention fusion framework; Neural Networks, Computer; Noise; Pathogenesis; pathogeny extraction; Training
• ISSN: 2162-237X; 2162-2388; 2162-2388
• DOI: 10.1109/TNNLS.2025.3545101

Complementary information in multi-omics data are crucial for understanding the pathogenesis of Alzheimer's Disease (AD). However, existing studies face challenges in addressing the high-level noise and heterogeneity in multi-omics data. This article presents a novel approach that combines large foundation models (LFMs) with soft attention mechanisms to enhance, select, and fuse multi-omics features, thereby improving the performance of disease classification. Specifically, we first propose a mathematical model based on soft attention mechanisms. This model employs multi-head attention (MHA) and self-attention (SA) for feature selection, and uses cross-attention (CA) for feature fusion. Then, a multi-way soft attention fusion framework (MSAFF) with LFMs is proposed. In this approach, biomedical LFMs are used to construct low-noise biomedical features. The multi-way soft attention algorithm implements effective feature selection and fusion described in the mathematical model. Experimental results on the public imaging genetics datasets demonstrate the advanced performances of MSAFF in both disease classification and AD-related pathogeny discrimination. This article provides intelligent support for the diagnosis and pathogenesis research of AD. Our code can be accessed at github.com/fmri123456/MSAFF.