Skip to main navigation Skip to search Skip to main content

QDRF: Quantized and distilled robust fusion for incomplete data in multimodal sentiment analysis

  • Jiachen Hou
  • , Zheyan Cheng
  • , Chen Xing
  • , Yushi Li
  • , Nan Xiang
  • , Shaoyu Cai
  • , Jie Liu
  • , Yushan Pan*
  • *Corresponding author for this work
  • Xi'an Jiaotong-Liverpool University
  • University of Liverpool
  • Air Force Medical University
  • National University of Singapore
  • Northwestern Polytechnical University Xian
  • Shanghai Jiao Tong University

Research output: Contribution to journal › Article › peer-review

Abstract

Despite the success of Multimodal Sentiment Analysis in ideal settings, real-world data corruption often leads to “variance explosion” in continuous latent spaces and “semantic drift” during teacher–student alignment. To overcome these challenges, we introduce Quantized and Distilled Robust Fusion (QDRF), a framework that replaces fragile continuous imputation with a Finite Scalar Quantization bottleneck to improve robustness against latent-space perturbations through a discrete semantic pivot. We further stabilize the system using an asymmetric Exponential Moving Average distillation framework that enforces dual alignment at both the feature and weight levels. Evaluated on CMU-MOSI, CMU-MOSEI, and CH-SIMS, QDRF achieves state-of-the-art or highly competitive performance on CMU-MOSEI and CH-SIMS, demonstrating that linguistic-prior-guided discrete representation learning is effective for robust sentiment prediction in incomplete data environments.

Original languageEnglish
Article number123966
JournalInformation Sciences
Volume758
DOIs
Publication statusPublished - 5 Jan 2027

Keywords

  • Finite scalar quantization
  • Missing modalities
  • Multimodal sentiment analysis
  • Teacher-student distillation

Cite this