Projects per year
Abstract
Social-media sarcasm may be conveyed by discourse context, speaker expectations, cultural norms, or purely linguistic irony; in image–text posts, however, a frequent and experimen tally accessible cue is tension between the literal text and visible evidence. We present CAMC, a conflict-aware multimodal clue model for visually grounded multimodal sarcasm recognition. CAMC does not treat image–text contradiction as a complete theory of sarcasm; rather, it exposes contradiction as an explicit intermediate signal when sarcasm is grounded in visual context. The Cross-Modal Conflict Localizer (CMCL) extracts text-token and image-token states from the same projected Qwen2.5-VL decoder space and computes spatial conflict maps using coverage weighted residuals between text tokens and visual patches. The Gaussian Clue Generator (GCG-K) converts salient conflict peaks into scale-adaptive soft-focus views and region-anchored textual clues. A reinforcement-learning objective further optimizes prediction accuracy, output compliance, and conflict-grounded reasoning structure. On MMSD2.0, CAMC improves Qwen2.5 VL baselines by 4.65 and 4.24 points in accuracy and macro-F1 at the 3B scale, and by 4.40 and 4.98 points at the 7B scale. The ablation experiments show that explicit contradiction localization improves multimodal sarcasm recognition while retaining interpretable visual and textual evidence.
| Original language | English |
|---|---|
| Journal | IEEE Transactions on Affective Computing |
| DOIs | |
| Publication status | Accepted/In press - 2026 |
Keywords
- Affective computing
- Reinforcement learning
- Sarcasm recognition
- Vision language models
Projects
- 6 Active
-
Suzhou Industrial Park Affective Computing & Interactive Health Interdisciplinary Innovation (Research) Platform
Xu, Z. (PI), lim, E. (Team member), Leach, M. (Team member), Yue, Y. (Team member), Selig, T. (Team member), Wang, W. (Team member), Zhang, X. (Team member), Zhu, X. (Team member), Pan, Y. (Team member), Xiang, N. (Team member), Zhang, H. (Team member), Wang, Y. (Team member), Chen, Y. (Team member), Zhang, C. (Team member), Liu, P. (Team member), Wang, J. (Team member), Li, Y. (Team member), 黎秋宇 (Team member), 刘禹含 (Team member), 高一凡 (Team member), 朱铭徽 (Team member), 白万里 (Team member), 张吉儿 (Team member), 徐豪谡 (Team member), 胡晓松 (Team member) & Li, W. (Team member)
1/10/25 → 30/03/28
Project: Governmental Research Project
-
Suzhou Intelligent Manufacturing Global AI Data Collaboration Industrial Innovation Consortium/苏州智造全球 AI 数据协同产业创新联合体
Pan, Y. (PI), Wang, S. (PI), Yang, Z. (PI), Xu, Z. (CoPI), Wang, W. (CoPI), Xiang, N. (CoPI), Zhang, H. (Team member), Chen, Y. (Team member), Wang, J. (Team member), Wang, Y. (Team member), Wang, T. (Team member), Wang, H. (Team member), Wang, L. (Team member), Wang, C. (Team member), Yang, H. (Team member), Fei, J. (CoPI), Fang, F. (CoPI), Li, Q. (Team member), Gao, Y. (Team member), Liang, S. (Team member) & Li, Y. (Team member)
1/05/25 → 30/04/28
Project: Governmental Research Project
Cite this
- APA
- Author
- BIBTEX
- Harvard
- Standard
- RIS
- Vancouver