Bidirectional long short-term memory network for vehicle behavior recognition

Jiasong Zhu, Ke Sun, Sen Jia, Weidong Lin, Xianxu Hou, Bozhi Liu, Guoping Qiu*

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

16 Citations (Scopus)


Vehicle behavior recognition is an attractive research field which is useful for many computer vision and intelligent traffic analysis tasks. This paper presents an all-in-one behavior recognition framework for moving vehicles based on the latest deep learning techniques. Unlike traditional traffic analysis methods which rely on low-resolution videos captured by road cameras, we capture 4K (3840 × 2178) traffic videos at a busy road intersection of a modern megacity by flying a unmanned aerial vehicle (UAV) during the rush hours. We then manually annotate locations and types of road vehicles. The proposed method consists of the following three steps: (1) vehicle detection and type recognition based on deep neural networks; (2) vehicle tracking by data association and vehicle trajectory modeling; (3) vehicle behavior recognition by nearest neighbor search and by bidirectional long short-term memory network, respectively. This paper also presents experimental results of the proposed framework in comparison with state-of-the-art approaches on the 4K testing traffic video, which demonstrated the effectiveness and superiority of the proposed method.

Original languageEnglish
Article number887
JournalRemote Sensing
Issue number6
Publication statusPublished - 1 Jun 2018
Externally publishedYes


  • Behavior recognition
  • Deep neural networks
  • Long short-term memory
  • Unmanned aerial vehicles (UAVs)
  • Vehicle detection
  • Vehicle tracking


Dive into the research topics of 'Bidirectional long short-term memory network for vehicle behavior recognition'. Together they form a unique fingerprint.

Cite this