Compressing deep networks by neuron agglomerative clustering

Li Na Wang, Wenxue Liu, Xiang Liu, Guoqiang Zhong, Partha Pratim Roy, Junyu Dong*, Kaizhu Huang

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

4 Citations (Scopus)


In recent years, deep learning models have achieved remarkable successes in various applications, such as pattern recognition, computer vision, and signal processing. However, high-performance deep architectures are often accompanied by a large storage space and long computational time, which make it difficult to fully exploit many deep neural networks (DNNs), especially in scenarios in which computing resources are limited. In this paper, to tackle this problem, we introduce a method for compressing the structure and parameters of DNNs based on neuron agglomerative clustering (NAC). Specifically, we utilize the agglomerative clustering algorithm to find similar neurons, while these similar neurons and the connections linked to them are then agglomerated together. Using NAC, the number of parameters and the storage space of DNNs are greatly reduced, without the support of an extra library or hardware. Extensive experiments demonstrate that NAC is very effective for the neuron agglomeration of both the fully connected and convolutional layers, which are common building blocks of DNNs, delivering similar or even higher network accuracy. Specifically, on the benchmark CIFAR-10 and CIFAR-100 datasets, using NAC to compress the parameters of the original VGGNet by 92.96% and 81.10%, respectively, the compact network obtained still outperforms the original networks.

Original languageEnglish
Article number6033
Pages (from-to)1-16
Number of pages16
JournalSensors (Switzerland)
Issue number21
Publication statusPublished - 1 Nov 2020


  • Agglomerative clustering
  • Deep learning
  • Feature maps
  • Network compression
  • Neurons


Dive into the research topics of 'Compressing deep networks by neuron agglomerative clustering'. Together they form a unique fingerprint.

Cite this