Learning to Predict Pedestrian Intention via Variational Tracking Networks

We propose a new deep learning based system for short term prediction of pedestrian behavior in front of a vehicle. To achieve this, we first develop a framework for class-specific object tracking and short term path prediction based on a variant of a Variational Recurrent Neural Network (VRNN), which incorporates latent variables corresponding to a dynamic state space model. The low level visual features learned from this system were found to be highly informative for the discrete intention prediction task (i.e., predicting whether a pedestrian is stopping or crossing), and achieved high performance on the Daimler benchmark. This is despite a much smaller training dataset than is normally used for training deep learning models. To the best of our knowledge, we are the first to apply deep learning to this problem without using externally trained pedestrian pose estimation systems. Our system performs comparable to the state-of-the-art approach that relies on pose estimation, and runs in real time.