Zen-NAS: A Zero-Shot NAS for High-Performance Image Recognition

• Published in: Proceedings / IEEE International Conference on Computer Vision pp. 337 - 346
• Main Authors: Lin, Ming, Wang, Pichao, Sun, Zhenhong, Chen, Hesen, Sun, Xiuyu, Qian, Qi, Li, Hao, Jin, Rong
• Format: Conference Proceeding
• Language: English
• Published: IEEE 01.10.2021
• Subjects: Architecture; Computational modeling; Computer architecture; Efficient training and inference methods; Graphics processing units; Image recognition; Inference algorithms; Machine learning architectures and formulations; Recognition and classification; Training
• ISSN: 2380-7504
• DOI: 10.1109/ICCV48922.2021.00040

Accuracy predictor is a key component in Neural Architecture Search (NAS) for ranking architectures. Building a high-quality accuracy predictor usually costs enormous computation. To address this issue, instead of using an accuracy predictor, we propose a novel zero-shot index dubbed Zen-Score to rank the architectures. The Zen-Score represents the network expressivity and positively correlates with the model accuracy. The calculation of Zen-Score only takes a few forward inferences through a randomly initialized network, without training network parameters. Built upon the Zen-Score, we further propose a new NAS algorithm, termed as Zen-NAS, by maximizing the Zen-Score of the target network under given inference budgets. Within less than half GPU day, Zen-NAS is able to directly search high performance architectures in a data-free style. Comparing with previous NAS methods, the proposed Zen-NAS is magnitude times faster on multiple server-side and mobile-side GPU platforms with state-of-the-art accuracy on ImageNet. Searching and training code as well as pre-trained models are available from https://github.com/idstcv/ZenNAS.