Synthesizing Multi-Contrast MR Images Via Novel 3D Conditional Variational Auto-Encoding GAN

Huan Yang, Xianling Lu*, Shui Hua Wang, Zhihai Lu, Jian Yao, Yizhang Jiang, Pengjiang Qian*

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

19 Citations (Scopus)


As two different modalities of medical images, Magnetic Resonance (MR) and Computer Tomography (CT), provide mutually-complementary information to doctors in clinical applications. However, to obtain both images sometimes is cost-consuming and unavailable, particularly for special populations. For example, patients with metal implants are not suitable for MR scanning. Also, it is probably infeasible to acquire multi-contrast MR images during once clinical scanning. In this context, to synthesize needed MR images for patients whose CT images are available becomes valuable. To this end, we present a novel generative network, called CAE-ACGAN, which incorporates the advantages of Variational Auto-Encoder (VAE) and Generative Adversarial Network (GAN) with an auxiliary discriminative classifier network. We apply this network to synthesize multi-contrast MR images from single CT and conduct experiments on brain datasets. Our main contributions can be summarized as follows: 1)We alleviate the problems of images blurriness and mode collapse by integrating the advantages of VAE and GAN; 2) We solve the complicated cross-domain, multi-contrast MR synthesis task using the proposed network; 3) The technique of random-extraction-patches is used to lower the limit of insufficient training data, enabling to obtain promising results even with limited available data; 4) By comparing with other typical networks, we are able to yield nearer-real, higher-quality synthetic MR images, demonstrating the effectiveness and stability of our proposed network.

Original languageEnglish
Pages (from-to)415-424
Number of pages10
JournalMobile Networks and Applications
Issue number1
Publication statusPublished - Feb 2021
Externally publishedYes


  • 3D
  • Auto-encoding
  • Generative adversarial network
  • MR synthesis
  • Multi-contrast


Dive into the research topics of 'Synthesizing Multi-Contrast MR Images Via Novel 3D Conditional Variational Auto-Encoding GAN'. Together they form a unique fingerprint.

Cite this