Personalized visual encoding model construction with small data

• Published in: Communications biology Vol. 5; no. 1; pp. 1382 - 12
• Main Authors: Gu, Zijin, Jamison, Keith, Sabuncu, Mert, Kuceyeski, Amy
• Format: Journal Article
• Language: English
• Published: London Nature Publishing Group UK 17.12.2022; Nature Publishing Group; Nature Portfolio
• Subjects: 59; 59/36; 631/114/116/2395; 631/114/116/2396; 631/378/2613/2615; Accuracy; Animals; Biology; Biomedical and Life Sciences; Brain; Brain - diagnostic imaging; Brain - physiology; Brain mapping; Brain Mapping - methods; Datasets; Functional magnetic resonance imaging; Humans; Life Sciences; Magnetic Resonance Imaging - methods; Visual stimuli
• ISSN: 2399-3642; 2399-3642
• DOI: 10.1038/s42003-022-04347-z

Quantifying population heterogeneity in brain stimuli-response mapping may allow insight into variability in bottom-up neural systems that can in turn be related to individual’s behavior or pathological state. Encoding models that predict brain responses to stimuli are one way to capture this relationship. However, they generally need a large amount of fMRI data to achieve optimal accuracy. Here, we propose an ensemble approach to create encoding models for novel individuals with relatively little data by modeling each subject’s predicted response vector as a linear combination of the other subjects’ predicted response vectors. We show that these ensemble encoding models trained with hundreds of image-response pairs, achieve accuracy not different from models trained on 20,000 image-response pairs. Importantly, the ensemble encoding models preserve patterns of inter-individual differences in the image-response relationship. We also show the proposed approach is robust against domain shift by validating on data with a different scanner and experimental setup. Additionally, we show that the ensemble encoding models are able to discover the inter-individual differences in various face areas’ responses to images of animal vs human faces using a recently developed NeuroGen framework. Our approach shows the potential to use existing densely-sampled data, i.e. large amounts of data collected from a single individual, to efficiently create accurate, personalized encoding models and, subsequently, personalized optimal synthetic images for new individuals scanned under different experimental conditions. An ensemble encoding approach uses existing individual encoding models with densely-sampled data to predict brain responses to visual stimuli in novel individuals with small amounts of data.