Light attention predicts protein location from the language of life

• Published in: Bioinformatics advances Vol. 1; no. 1; p. vbab035
• Main Authors: Stärk, Hannes, Dallago, Christian, Heinzinger, Michael, Rost, Burkhard
• Format: Journal Article
• Language: English
• Published: England Oxford University Press 2021
• Subjects: Bioinformatics; Information processing; Localization; Neural networks; Original; Predictions; Proteins; Proteomes
• ISSN: 2635-0041; 2635-0041
• DOI: 10.1093/bioadv/vbab035

Summary Although knowing where a protein functions in a cell is important to characterize biological processes, this information remains unavailable for most known proteins. Machine learning narrows the gap through predictions from expert-designed input features leveraging information from multiple sequence alignments (MSAs) that is resource expensive to generate. Here, we showcased using embeddings from protein language models for competitive localization prediction without MSAs. Our lightweight deep neural network architecture used a softmax weighted aggregation mechanism with linear complexity in sequence length referred to as light attention. The method significantly outperformed the state-of-the-art (SOTA) for 10 localization classes by about 8 percentage points (Q10). So far, this might be the highest improvement of just embeddings over MSAs. Our new test set highlighted the limits of standard static datasets: while inviting new models, they might not suffice to claim improvements over the SOTA. Availability and implementation The novel models are available as a web-service at http://embed.protein.properties. Code needed to reproduce results is provided at https://github.com/HannesStark/protein-localization. Predictions for the human proteome are available at https://zenodo.org/record/5047020. Supplementary information Supplementary data are available at Bioinformatics Advances online.