TY - JOUR
T1 - MFISN
T2 - Modality Fuzzy Information Separation Network for Disease Classification
AU - Nan, Fengtao
AU - Pu, Bin
AU - Pan, Jiayi
AU - Fan, Yingchun
AU - Yang, Jiewen
AU - Dong, Xingbo
AU - Xu, Zhaozhao
AU - Wang, Shuihua
N1 - Publisher Copyright:
IEEE
PY - 2024
Y1 - 2024
N2 - Most of the previous machine learning-based models for multi-modal medical diagnosis, primarily designed for unimodal images, usually do not fully leverage the potential of multimodal medical images, leading to limited classification accuracy. These conventional methods typically focus only on the intermodality common information, neglecting the intra-modality specific information and assuming that the common information is more effective in disease diagnosis. Moreover, they do not adequately address the impact of fuzzy information between different medical imaging modalities on diagnostic results. To this end, we propose a Modality Fuzzy Information Separation Network for disease classification, which extracts both common and specific information from fuzzy information to construct a comprehensive representation of multi-modal medical images. Specifically, we extract modality invariant features as common information by explicitly modeling and maximizing loss constraints on mutual information. For specific information extraction, a constraint on feature space independence between specific and common information is imposed on each modality. Above two steps, we concatenate common information and specific information to construct a comprehensive multi-modal representation for separating fuzzy information. Finally, we purposely design a decoder network to reconstruct medical images from uni-modal specific information and common information to demonstrate the effectiveness of the modality fuzzy information separation network. We conducted a validation of the proposed method's performance in classifying cardiomegaly, pneumothorax, edema, and skin disease. The experimental results substantiate the effectiveness of our proposed approach.
AB - Most of the previous machine learning-based models for multi-modal medical diagnosis, primarily designed for unimodal images, usually do not fully leverage the potential of multimodal medical images, leading to limited classification accuracy. These conventional methods typically focus only on the intermodality common information, neglecting the intra-modality specific information and assuming that the common information is more effective in disease diagnosis. Moreover, they do not adequately address the impact of fuzzy information between different medical imaging modalities on diagnostic results. To this end, we propose a Modality Fuzzy Information Separation Network for disease classification, which extracts both common and specific information from fuzzy information to construct a comprehensive representation of multi-modal medical images. Specifically, we extract modality invariant features as common information by explicitly modeling and maximizing loss constraints on mutual information. For specific information extraction, a constraint on feature space independence between specific and common information is imposed on each modality. Above two steps, we concatenate common information and specific information to construct a comprehensive multi-modal representation for separating fuzzy information. Finally, we purposely design a decoder network to reconstruct medical images from uni-modal specific information and common information to demonstrate the effectiveness of the modality fuzzy information separation network. We conducted a validation of the proposed method's performance in classifying cardiomegaly, pneumothorax, edema, and skin disease. The experimental results substantiate the effectiveness of our proposed approach.
KW - Correlation
KW - Data mining
KW - Diseases
KW - Feature extraction
KW - Medical diagnosis
KW - Medical diagnostic imaging
KW - Multi-modal medical image
KW - Mutual information
KW - comprehensive information
KW - disease classification
KW - fuzzy information
KW - multi-modal neural network
UR - http://www.scopus.com/inward/record.url?scp=85187018107&partnerID=8YFLogxK
U2 - 10.1109/TFUZZ.2024.3371678
DO - 10.1109/TFUZZ.2024.3371678
M3 - Article
AN - SCOPUS:85187018107
SN - 1063-6706
SP - 1
EP - 12
JO - IEEE Transactions on Fuzzy Systems
JF - IEEE Transactions on Fuzzy Systems
ER -