Automated discovery of algorithms from data

• Published in: Nature Computational Science Vol. 4; no. 2; pp. 110 - 118
• Main Authors: Blazek, Paul J, Venkatesh, Kesavan, Lin, Milo M
• Format: Journal Article
• Language: English
• Published: United States Nature Publishing Group 01.02.2024
• Subjects: Algorithms; Artificial Intelligence; Automation; Computers; Decision trees; Deep learning; Distillation; Humans; Machine Learning; Neural networks; Neural Networks, Computer; Neurons; Support vector machines
• ISSN: 2662-8457
• DOI: 10.1038/s43588-024-00593-9

To automate the discovery of new scientific and engineering principles, artificial intelligence must distill explicit rules from experimental data. This has proven difficult because existing methods typically search through the enormous space of possible functions. Here we introduce deep distilling, a machine learning method that does not perform searches but instead learns from data using symbolic essence neural networks and then losslessly condenses the network parameters into a concise algorithm written in computer code. This distilled code, which can contain loops and nested logic, is equivalent to the neural network but is human-comprehensible and orders-of-magnitude more compact. On arithmetic, vision and optimization tasks, the distilled code is capable of out-of-distribution systematic generalization to solve cases orders-of-magnitude larger and more complex than the training data. The distilled algorithms can sometimes outperform human-designed algorithms, demonstrating that deep distilling is able to discover generalizable principles complementary to human expertise.