Simplified Space Based Neural Architecture Search

Zixiang Ding, Yaran Chen, Nannan Li, Dongbin Zhao

Research output: Chapter in Book or Report/Conference proceedingConference Proceedingpeer-review

4 Citations (Scopus)

Abstract

In this paper, we propose Simplified Space based Neural Architecture Search (SSNAS), an efficient approach for automatic architecture search. Inspired by the popular convolutional neural networks which have strong capability for feature extraction with small convolutional kernels, we design a simplified search space of convolutional operations with small kernel and construct a large model based on it. Furthermore, we use an Long Short-Term Memory (LSTM) to sample child model from the large model in a way of selective activation. The probability distribution of selective activation is obtained by training the LSTM with reinforcement learning for maximizing the expected reward of selected child model on the validation set. Moreover, the trained weights are saved in a large model which concludes all child models. Each weight of new sampled child model will be restored when being selected by another one again instead of training from scratch so that SSNAS can greatly reduce the expensive computation than traditional approaches for automatic model design. Extensive experiments on CIFAR-10, ImageNet and VOC show that the proposed approach excels in discovering transferable architectures with high performance.

Original languageEnglish
Title of host publication2019 IEEE Symposium Series on Computational Intelligence, SSCI 2019
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages43-49
Number of pages7
ISBN (Electronic)9781728124858
DOIs
Publication statusPublished - Dec 2019
Event2019 IEEE Symposium Series on Computational Intelligence, SSCI 2019 - Xiamen, China
Duration: 6 Dec 20199 Dec 2019

Publication series

Name2019 IEEE Symposium Series on Computational Intelligence, SSCI 2019

Conference

Conference2019 IEEE Symposium Series on Computational Intelligence, SSCI 2019
Country/TerritoryChina
CityXiamen
Period6/12/199/12/19

Keywords

  • image classification
  • neural architecture search
  • object detection
  • simplified space

Fingerprint

Dive into the research topics of 'Simplified Space Based Neural Architecture Search'. Together they form a unique fingerprint.

Cite this