MFISN: Modality Fuzzy Information Separation Network for Disease Classification

Fengtao Nan, Bin Pu, Jiayi Pan, Yingchun Fan, Jiewen Yang, Xingbo Dong, Zhaozhao Xu, Shuihua Wang

Research output: Contribution to journalArticlepeer-review


Most of the previous machine learning-based models for multi-modal medical diagnosis, primarily designed for unimodal images, usually do not fully leverage the potential of multimodal medical images, leading to limited classification accuracy. These conventional methods typically focus only on the intermodality common information, neglecting the intra-modality specific information and assuming that the common information is more effective in disease diagnosis. Moreover, they do not adequately address the impact of fuzzy information between different medical imaging modalities on diagnostic results. To this end, we propose a Modality Fuzzy Information Separation Network for disease classification, which extracts both common and specific information from fuzzy information to construct a comprehensive representation of multi-modal medical images. Specifically, we extract modality invariant features as common information by explicitly modeling and maximizing loss constraints on mutual information. For specific information extraction, a constraint on feature space independence between specific and common information is imposed on each modality. Above two steps, we concatenate common information and specific information to construct a comprehensive multi-modal representation for separating fuzzy information. Finally, we purposely design a decoder network to reconstruct medical images from uni-modal specific information and common information to demonstrate the effectiveness of the modality fuzzy information separation network. We conducted a validation of the proposed method's performance in classifying cardiomegaly, pneumothorax, edema, and skin disease. The experimental results substantiate the effectiveness of our proposed approach.

Original languageEnglish
Pages (from-to)1-12
Number of pages12
JournalIEEE Transactions on Fuzzy Systems
Publication statusAccepted/In press - 2024


  • Correlation
  • Data mining
  • Diseases
  • Feature extraction
  • Medical diagnosis
  • Medical diagnostic imaging
  • Multi-modal medical image
  • Mutual information
  • comprehensive information
  • disease classification
  • fuzzy information
  • multi-modal neural network


Dive into the research topics of 'MFISN: Modality Fuzzy Information Separation Network for Disease Classification'. Together they form a unique fingerprint.

Cite this