Abnormal event detection for video surveillance using an enhanced two-stream fusion method

Abnormal event detection is a critical component of intelligent surveillance systems, focusing on identifying abnormal objects or unusual human behaviours in video sequences. However, conventional methods struggle due to the scarcity of labelled data. Existing solutions typically train on normal data, establish boundaries for regular events, and identify outliers during testing. These approaches are often inadequate as they do not efficiently leverage the geometry and image texture information, and they lack a specific focus on different types of abnormal events. This paper introduces a novel two-stream fusion algorithm for abnormal event detection to address these diverse abnormal events better. We first extract the object, pose, and optical flow features. Then, the object and pose information is combined early on to eliminate occluded pose graphs. The trusted pose graphs are fed into a Spatio-Temporal Graph Convolutional Network (ST-GCN) to detect abnormal behaviours. Simultaneously, we propose a video prediction framework that identifies abnormal frames by measuring the difference between predicted and ground truth frames. Lastly, we execute a decision-level fusion between the classification and prediction streams to achieve the final results. Our results on the UCSD PED1 dataset indicate the enhanced performance of the fusion model for various abnormal events. Furthermore, experimental results on the UCSD PED2 dataset and the ShanghaiTech campus dataset underscore our approach's effectiveness compared to other related works.