Explainable artificial intelligence model to predict brain states from fNIRS signals

• Published in: Frontiers in human neuroscience Vol. 16; p. 1029784
• Main Authors: Shibu, Caleb Jones, Sreedharan, Sujesh, Arun, KM, Kesavadas, Chandrasekharan, Sitaram, Ranganatha
• Format: Journal Article
• Language: English
• Published: Switzerland Frontiers Research Foundation 19.01.2023; Frontiers Media S.A
• Subjects: Accuracy; Artificial intelligence; brain state classification; brain-computer interface; Classification; Communication; convolutional neural networks; Deep learning; Electroencephalography; Feature selection; functional near-infrared spectroscopy; Hemodynamics; Hemoglobin; Human Neuroscience; Infrared spectroscopy; Long short-term memory; Machine learning; Memory; Mental task performance; Neural networks; Support vector machines; Wavelet transforms; deep learning; brain-computer interface; functional near-infrared spectroscopy; explainable AI; convolutional neural networks; brain state classification; long short-term memory
• ISSN: 1662-5161; 1662-5161
• DOI: 10.3389/fnhum.2022.1029784

Objective: Most Deep Learning (DL) methods for the classification of functional Near-Infrared Spectroscopy (fNIRS) signals do so without explaining which features contribute to the classification of a task or imagery. An explainable artificial intelligence (xAI) system that can decompose the Deep Learning mode’s output onto the input variables for fNIRS signals is described here. Approach: We propose an xAI-fNIRS system that consists of a classification module and an explanation module. The classification module consists of two separately trained sliding window-based classifiers, namely, (i) 1-D Convolutional Neural Network (CNN); and (ii) Long Short-Term Memory (LSTM). The explanation module uses SHAP (SHapley Additive exPlanations) to explain the CNN model’s output in terms of the model’s input. Main results: We observed that the classification module was able to classify two types of datasets: (a) Motor task (MT), acquired from three subjects; and (b) Motor imagery (MI), acquired from 29 subjects, with an accuracy of over 96% for both CNN and LSTM models. The explanation module was able to identify the channels contributing the most to the classification of MI or MT and therefore identify the channel locations and whether they correspond to oxy- or deoxy-hemoglobin levels in those locations. Significance: The xAI-fNIRS system can distinguish between the brain states related to overt and covert motor imagery from fNIRS signals with high classification accuracy and is able to explain the signal features that discriminate between the brain states of interest.