Knowledge Distillation from Monolingual to Multilingual Models for Intelligent and Interpretable Multilingual Emotion Detection

Yuqi Wang, Zimu Wang, Nijia Han, Wei Wang*, Qi Chen, Haiyang Zhang, Yushan Pan, Anh Nguyen

*Corresponding author for this work

Research output: Chapter in Book or Report/Conference proceedingConference Proceedingpeer-review

1 Citation (Scopus)

Abstract

Emotion detection from text is a crucial task in understanding natural language with wide-ranging applications. Existing approaches for multilingual emotion detection from text face challenges with data scarcity across many languages and a lack of interpretability. We propose a novel method that leverages both monolingual and multilingual pre-trained language models to improve performance and interpretability. Our approach involves 1) training a high-performing English monolingual model in parallel with a multilingual model and 2) using knowledge distillation to transfer the emotion detection capabilities from the monolingual teacher to the multilingual student model. Experiments on a multilingual dataset demonstrate significant performance gains for refined multilingual models like XLM-RoBERTa and E5 after distillation. Furthermore, our approach enhances interpretability by enabling better identification of emotion-trigger words. Our work presents a promising direction for building accurate, robust and explainable multilingual emotion detection systems.

Original languageEnglish
Title of host publicationWASSA 2024 - 14th Workshop on Computational Approaches to Subjectivity, Sentiment, and Social Media Analysis, Proceedings of the Workshop
EditorsOrphee De Clercq, Valentin Barriere, Jeremy Barnes, Roman Klinger, Joao Sedoc, Shabnam Tafreshi
PublisherAssociation for Computational Linguistics (ACL)
Pages470-475
Number of pages6
ISBN (Electronic)9798891761568
Publication statusPublished - 2024
Event14th Workshop on Computational Approaches to Subjectivity, Sentiment, and Social Media Analysis, WASSA 2024 - Bangkok, Thailand
Duration: 15 Aug 2024 → …

Publication series

NameWASSA 2024 - 14th Workshop on Computational Approaches to Subjectivity, Sentiment, and Social Media Analysis, Proceedings of the Workshop

Conference

Conference14th Workshop on Computational Approaches to Subjectivity, Sentiment, and Social Media Analysis, WASSA 2024
Country/TerritoryThailand
CityBangkok
Period15/08/24 → …

Fingerprint

Dive into the research topics of 'Knowledge Distillation from Monolingual to Multilingual Models for Intelligent and Interpretable Multilingual Emotion Detection'. Together they form a unique fingerprint.

Cite this