Foundation model of neural activity predicts response to new stimulus types

• Published in: Nature (London) Vol. 640; no. 8058; pp. 470 - 477
• Main Authors: Wang, Eric Y., Fahey, Paul G., Ding, Zhuokun, Papadopoulos, Stelios, Ponder, Kayla, Weis, Marissa A., Chang, Andersen, Muhammad, Taliah, Patel, Saumil, Ding, Zhiwei, Tran, Dat, Fu, Jiakun, Schneider-Mizell, Casey M., Reid, R. Clay, Collman, Forrest, da Costa, Nuno Maçarico, Franke, Katrin, Ecker, Alexander S., Reimer, Jacob, Pitkow, Xaq, Sinz, Fabian H., Tolias, Andreas S.
• Format: Journal Article
• Language: English
• Published: London Nature Publishing Group UK 10.04.2025; Nature Publishing Group
• Subjects: 14; 14/28; 14/69; 631/114/116/1925; 631/114/1305; 631/114/2397; 631/378/2613/2614; 631/378/3917; Accuracy; Algorithms; Animals; Artificial intelligence; Brain; Brain research; Connectome; Datasets; Deep Learning; Dendrites; Eye movements; Female; Humanities and Social Sciences; Machine learning; Male; Mice; Models, Neurological; multidisciplinary; Neural coding; Neural networks; Neurons; Neurons - cytology; Neurons - physiology; Neurosciences; Noise prediction; Photic Stimulation; Recording sessions; Science; Science (multidisciplinary); Statistical models; Training; Visual cortex; Visual Cortex - cytology; Visual Cortex - physiology; Visual pathways
• ISSN: 0028-0836; 1476-4687; 1476-4687
• DOI: 10.1038/s41586-025-08829-y

The complexity of neural circuits makes it challenging to decipher the brain’s algorithms of intelligence. Recent breakthroughs in deep learning have produced models that accurately simulate brain activity, enhancing our understanding of the brain’s computational objectives and neural coding. However, it is difficult for such models to generalize beyond their training distribution, limiting their utility. The emergence of foundation models 1 trained on vast datasets has introduced a new artificial intelligence paradigm with remarkable generalization capabilities. Here we collected large amounts of neural activity from visual cortices of multiple mice and trained a foundation model to accurately predict neuronal responses to arbitrary natural videos. This model generalized to new mice with minimal training and successfully predicted responses across various new stimulus domains, such as coherent motion and noise patterns. Beyond neural response prediction, the model also accurately predicted anatomical cell types, dendritic features and neuronal connectivity within the MICrONS functional connectomics dataset 2 . Our work is a crucial step towards building foundation models of the brain. As neuroscience accumulates larger, multimodal datasets, foundation models will reveal statistical regularities, enable rapid adaptation to new tasks and accelerate research. A foundation model trained on neural activity of visual cortex from multiple mice accurately predicts responses to video stimuli and cell types, dendritic features and connectivity within the MICrONS functional connectomics dataset.