Micro-Expression Recognition Using Convolutional Variational Attention Transformer (ConVAT) with Multihead Attention Mechanism

Hafiz Khizer Bin Talib, Kaiwei Xu, Yanlong Cao*, Yuan Ping Xu, Zhijie Xu, Muhammad Zaman, Adnan Akhunzada*

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

Abstract

Micro-Expression Recognition is crucial in various fields such as behavioral analysis, security, and psychological studies, offering valuable insights into subtle and often concealed emotional states. Despite significant advancements in deep learning models, challenges persist in accurately handling the nuanced and fleeting nature of micro-expressions, particularly when applied across diverse datasets with varied expressions. Existing models often struggle with precision and adaptability, leading to inconsistent recognition performance. To address these limitations, we propose the Convolutional Variational Attention Transformer (ConVAT), a novel model that leverages a multi-head attention mechanism integrated with convolutional networks, optimized specifically for detailed micro-expression analysis. Our methodology employs the Leave-One-Subject-Out (LOSO) cross-validation technique across three widely used datasets: SAMM, CASME II, and SMIC. The results demonstrate the effectiveness of ConVAT, achieving impressive performance with 98.73% accuracy on the SAMM dataset, 97.95% on the SMIC dataset, and 97.65% on CASME II. These outcomes not only surpass current state-of-the-art benchmarks but also highlight ConVAT's robustness and reliability in capturing micro-expressions, marking a significant advancement toward developing sophisticated automated systems for real-world applications in micro-expression recognition.

Original languageEnglish
Pages (from-to)20054-20070
Number of pages17
JournalIEEE Access
Volume13
DOIs
Publication statusPublished - 2025
Externally publishedYes

Keywords

  • ConVAT
  • convolutional neural networks
  • LOSO cross-validation
  • micro-expression recognition
  • multi-head attention

Fingerprint

Dive into the research topics of 'Micro-Expression Recognition Using Convolutional Variational Attention Transformer (ConVAT) with Multihead Attention Mechanism'. Together they form a unique fingerprint.

Cite this