End-to-end deep learning approach to mouse behavior classification from cortex-wide calcium imaging

• Published in: PLoS computational biology Vol. 20; no. 3; p. e1011074
• Main Authors: Ajioka, Takehiro, Nakai, Nobuhiro, Yamashita, Okito, Takumi, Toru
• Format: Journal Article
• Language: English
• Published: United States Public Library of Science 13.03.2024; Public Library of Science (PLoS)
• Subjects: Analysis; Artificial intelligence; Artificial neural networks; Behavior; Biology and Life Sciences; Brain; Calcium; Calcium imaging; Classification; Computer and Information Sciences; Decoders; Decoding; Deep learning; Engineering and Technology; Limbs; Machine learning; Mann-Whitney U test; Methods; Mice; Neural coding; Neural networks; Neuroimaging; Neurosciences; Physiological aspects; Recurrent neural networks; Research and Analysis Methods; Social Sciences; Somatosensory cortex; Temporal lobe; Japan
• ISSN: 1553-7358; 1553-734X; 1553-7358
• DOI: 10.1371/journal.pcbi.1011074

Deep learning is a powerful tool for neural decoding, broadly applied to systems neuroscience and clinical studies. Interpretable and transparent models that can explain neural decoding for intended behaviors are crucial to identifying essential features of deep learning decoders in brain activity. In this study, we examine the performance of deep learning to classify mouse behavioral states from mesoscopic cortex-wide calcium imaging data. Our convolutional neural network (CNN)-based end-to-end decoder combined with recurrent neural network (RNN) classifies the behavioral states with high accuracy and robustness to individual differences on temporal scales of sub-seconds. Using the CNN-RNN decoder, we identify that the forelimb and hindlimb areas in the somatosensory cortex significantly contribute to behavioral classification. Our findings imply that the end-to-end approach has the potential to be an interpretable deep learning method with unbiased visualization of critical brain regions.