VideoModerator: A Risk-Aware Framework for Multimodal Video Moderation in E-Commerce

Tan Tang, Yanhong Wu, Yingcai Wu, Lingyun Yu, Yuhong Li

Research output: Contribution to journalArticlepeer-review

16 Citations (Scopus)


Video moderation, which refers to remove deviant or explicit content from e-commerce livestreams, has become prevalent owing to social and engaging features. However, this task is tedious and time consuming due to the difficulties associated with watching and reviewing multimodal video content, including video frames and audio clips. To ensure effective video moderation, we propose VideoModerator, a risk-Aware framework that seamlessly integrates human knowledge with machine insights. This framework incorporates a set of advanced machine learning models to extract the risk-Aware features from multimodal video content and discover potentially deviant videos. Moreover, this framework introduces an interactive visualization interface with three views, namely, a video view, a frame view, and an audio view. In the video view, we adopt a segmented timeline and highlight high-risk periods that may contain deviant information. In the frame view, we present a novel visual summarization method that combines risk-Aware features and video context to enable quick video navigation. In the audio view, we employ a storyline-based design to provide a multi-faceted overview which can be used to explore audio content. Furthermore, we report the usage of VideoModerator through a case scenario and conduct experiments and a controlled user study to validate its effectiveness.

Original languageEnglish
Pages (from-to)846-856
Number of pages11
JournalIEEE Transactions on Visualization and Computer Graphics
Issue number1
Publication statusPublished - 1 Jan 2022


  • e-commerce livestreaming
  • video moderation
  • video visualization

Cite this