Learning from Few Samples with Memory Network

Shufei Zhang, Kaizhu Huang*, Rui Zhang, Amir Hussain

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

15 Citations (Scopus)

Abstract

Neural networks (NN) have achieved great successes in pattern recognition and machine learning. However, the success of a NN usually relies on the provision of a sufficiently large number of data samples as training data. When fed with a limited data set, a NN’s performance may be degraded significantly. In this paper, a novel NN structure is proposed called a memory network. It is inspired by the cognitive mechanism of human beings, which can learn effectively, even from limited data. Taking advantage of the memory from previous samples, the new model achieves a remarkable improvement in performance when trained using limited data. The memory network is demonstrated here using the multi-layer perceptron (MLP) as a base model. However, it would be straightforward to extend the idea to other neural networks, e.g., convolutional neural networks (CNN). In this paper, the memory network structure is detailed, the training algorithm is presented, and a series of experiments are conducted to validate the proposed framework. Experimental results show that the proposed model outperforms traditional MLP-based models as well as other competitive algorithms in response to two real benchmark data sets.

Original languageEnglish
Pages (from-to)15-22
Number of pages8
JournalCognitive Computation
Volume10
Issue number1
DOIs
Publication statusPublished - 1 Feb 2018

Keywords

  • Memory
  • Multi-layer perceptron
  • Neural network
  • Prior knowledge
  • Recognition

Cite this