MFISN: Modality Fuzzy Information Separation Network for Disease Classification

Fengtao Nan, Bin Pu*, Jiayi Pan, Yingchun Fan, Jiewen Yang, Xingbo Dong, Zhaozhao Xu, Shuihua Wang

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

1 Citation (Scopus)

Abstract

Most of the previous machine learning-based models for multimodal medical diagnosis, primarily designed for unimodal images, usually do not fully leverage the potential of multimodal medical images, leading to limited classification accuracy. These conventional methods typically focus only on the intermodality common information, neglecting the intramodality specific information and assuming that the common information is more effective in disease diagnosis. Moreover, they do not adequately address the impact of fuzzy information between different medical imaging modalities on diagnostic results. To this end, we propose a modality fuzzy information separation network for disease classification, which extracts both common and specific information from fuzzy information to construct a comprehensive representation of multimodal medical images. Specifically, we extract modality invariant features as common information by explicitly modeling and maximizing loss constraints on mutual information. For specific information extraction, a constraint on feature space independence between specific and common information is imposed on each modality. Above-mentioned two steps, we concatenate common information and specific information to construct a comprehensive multimodal representation for separating fuzzy information. Finally, we purposely design a decoder network to reconstruct medical images from unimodal specific information and common information to demonstrate the effectiveness of the modality fuzzy information separation network. We conducted a validation of the proposed method's performance in classifying cardiomegaly, pneumothorax, edema, and skin disease. The experimental results substantiate the effectiveness of our proposed approach.

Original languageEnglish
Pages (from-to)5545-5556
Number of pages12
JournalIEEE Transactions on Fuzzy Systems
Volume32
Issue number10
DOIs
Publication statusPublished - 2024

Keywords

  • Comprehensive information
  • disease classification
  • fuzzy information
  • multimodal medical image
  • multimodal neural network

Fingerprint

Dive into the research topics of 'MFISN: Modality Fuzzy Information Separation Network for Disease Classification'. Together they form a unique fingerprint.

Cite this