Multi-modal Neural Network for Traffic Event Detection

Cities are composed of complex systems with Cyber, Physical, and Social (CPS) components. The advances in the Internet of Things (IoTs) and social networking services help people understand the dynamics of cities. Traffic event detection is an important while complex task in transportation modeling and management of smart cities. In this paper, we address the task of detecting traffic events using two types of data, i.e. physical sensor observations and social media text. Unlike most existing studies focused on either analysing sensor observations or social media data, we identify traffic events with both types of data that may complement each other. We propose a Multi-modal Neural Network (MMN) to process sensor observations and social media texts simultaneously and detect traffic events. We evaluate our model with a real-world CPS dataset consisting of sensor observations, event reports, and tweets collected from Twitter about San Francisco over a period of 4 months. The evaluation shows promising results and provides insights into the analysis of multi-modal data for detecting traffic events.