Solving the RNA design problem with reinforcement learning

• Published in: PLoS computational biology Vol. 14; no. 6; p. e1006176
• Main Authors: Eastman, Peter, Shi, Jade, Ramsundar, Bharath, Pande, Vijay S.
• Format: Journal Article
• Language: English
• Published: United States Public Library of Science 01.06.2018; Public Library of Science (PLoS)
• Subjects: Algorithms; Bioengineering; Biology and life sciences; Computer and Information Sciences; Computer applications; Computer Simulation; Computer-Aided Design - instrumentation; Decomposition; Design; Game theory; Laboratories; Learning; Machine Learning; Machine translation; Mutation; Neural networks; Nucleic Acid Conformation; Nucleotide sequence; Physical Sciences; Problem Solving; Protein structure; Reinforcement; Research and Analysis Methods; Ribonucleic acid; RNA; RNA - chemistry; RNA Folding - physiology; RNA processing; Secondary structure; Software; Technology application; Training; United States > US
• ISSN: 1553-7358; 1553-734X; 1553-7358
• DOI: 10.1371/journal.pcbi.1006176

We use reinforcement learning to train an agent for computational RNA design: given a target secondary structure, design a sequence that folds to that structure in silico. Our agent uses a novel graph convolutional architecture allowing a single model to be applied to arbitrary target structures of any length. After training it on randomly generated targets, we test it on the Eterna100 benchmark and find it outperforms all previous algorithms. Analysis of its solutions shows it has successfully learned some advanced strategies identified by players of the game Eterna, allowing it to solve some very difficult structures. On the other hand, it has failed to learn other strategies, possibly because they were not required for the targets in the training set. This suggests the possibility that future improvements to the training protocol may yield further gains in performance.