MIDCAN: A multiple input deep convolutional attention network for Covid-19 diagnosis based on chest CT and chest X-ray

Yu Dong Zhang, Zheng Zhang, Xin Zhang*, Shui Hua Wang

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

84 Citations (Scopus)

Abstract

Background: COVID-19 has caused 3.34m deaths till 13/May/2021. It is now still causing confirmed cases and ongoing deaths every day. Method: This study investigated whether fusing chest CT with chest X-ray can help improve the AI's diagnosis performance. Data harmonization is employed to make a homogeneous dataset. We create an end-to-end multiple-input deep convolutional attention network (MIDCAN) by using the convolutional block attention module (CBAM). One input of our model receives 3D chest CT image, and other input receives 2D X-ray image. Besides, multiple-way data augmentation is used to generate fake data on training set. Grad-CAM is used to give explainable heatmap. Results: The proposed MIDCAN achieves a sensitivity of 98.10±1.88%, a specificity of 97.95±2.26%, and an accuracy of 98.02±1.35%. Conclusion: Our MIDCAN method provides better results than 8 state-of-the-art approaches. We demonstrate the using multiple modalities can achieve better results than individual modality. Also, we demonstrate that CBAM can help improve the diagnosis performance.

Original languageEnglish
Pages (from-to)8-16
Number of pages9
JournalPattern Recognition Letters
Volume150
DOIs
Publication statusPublished - Oct 2021
Externally publishedYes

Keywords

  • Automatic differentiation
  • COVID-19
  • Chest CT
  • Chest X-ray
  • Convolutional neural network
  • Data harmonization
  • Deep learning
  • Multimodality
  • Multiple input

Fingerprint

Dive into the research topics of 'MIDCAN: A multiple input deep convolutional attention network for Covid-19 diagnosis based on chest CT and chest X-ray'. Together they form a unique fingerprint.

Cite this