Long short-term attention

Guoqiang Zhong*, Xin Lin, Kang Chen, Qingyang Li, Kaizhu Huang

*Corresponding author for this work

Research output: Chapter in Book or Report/Conference proceedingConference Proceedingpeer-review

2 Citations (Scopus)

Abstract

Attention is an important cognition process of humans, which helps humans concentrate on critical information during their perception and learning. However, although many machine learning models can remember information of data, they have no the attention mechanism. For example, the long short-term memory (LSTM) network is able to remember sequential information, but it cannot pay special attention to part of the sequences. In this paper, we present a novel model called long short-term attention (LSTA), which seamlessly integrates the attention mechanism into the inner cell of LSTM. More than processing long short term dependencies, LSTA can focus on important information of the sequences with the attention mechanism. Extensive experiments demonstrate that LSTA outperforms LSTM and related models on the sequence learning tasks.

Original languageEnglish
Title of host publicationAdvances in Brain Inspired Cognitive Systems - 10th International Conference, BICS 2019, Proceedings
EditorsJinchang Ren, Amir Hussain, Huimin Zhao, Jun Cai, Rongjun Chen, Yinyin Xiao, Kaizhu Huang, Jiangbin Zheng
PublisherSpringer
Pages45-54
Number of pages10
ISBN (Print)9783030394301
DOIs
Publication statusPublished - 2020
Event10th International Conference on Brain Inspired Cognitive Systems, BICS 2019 - Guangzhou, China
Duration: 13 Jul 201914 Jul 2019

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume11691 LNAI
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference10th International Conference on Brain Inspired Cognitive Systems, BICS 2019
Country/TerritoryChina
CityGuangzhou
Period13/07/1914/07/19

Keywords

  • Attention mechanism
  • Long short-term attention
  • Long short-term memory
  • Machine learning
  • Sequence learning

Fingerprint

Dive into the research topics of 'Long short-term attention'. Together they form a unique fingerprint.

Cite this