Multi-Objective Neural Architecture Search for Light-Weight Model

Neural architecture search (NAS) has achieved superior performance in visual tasks by automatically designing an effective neural network architecture. In recent years, deep neural networks are increasingly applied to resource-constrained devices. As a result, in addition to the model performance, model size is another very important factor that requires to consider when designing powerful neural network architectures. Therefore, we propose the multi-objective neural architecture search for light-weight model and name it Light-weight NAS. On one hand, the Light-weight NAS introduces Multiply-ACcumulate (MAC) into the optimize objective to get the architecture with fewer parameters. On the other hand, we simplify the search space and adopt weight sharing to make the search process more efficient. Experimental results indicate that the searched architecture can perform competitive classification accuracy with few parameters on the image classification task, while using less computation cost than the most existing multi-objective NAS approaches.