Three-dimensional shape generation via variational autoencoder generative adversarial network with signed distance function

Ebenezer Akinyemi Ajayi, Kian Ming Lim*, Siew Chin Chong, Chin Poo Lee

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

Abstract

Mesh-based 3-dimensional (3D) shape generation from a 2-dimensional (2D) image using a convolution neural network (CNN) framework is an open problem in the computer graphics and vision domains. Most existing CNN-based frameworks lack robust algorithms that can scale well without combining different shape parts. Also, most CNN-based algorithms lack suitable 3D data representations that can fit into CNN without modification(s) to produce high-quality 3D shapes. This paper presents an approach that integrates a variational autoencoder (VAE) and a generative adversarial network (GAN) called 3 dimensional variational autoencoder signed distance function generative adversarial network (3D-VAE-SDFGAN) to create a 3D shape from a 2D image that considerably improves scalability and visual quality. The proposed method only feeds a single 2D image into the network to produce a mesh-based 3D shape. The network encodes a 2D image of the 3D object into the latent representations, and implicit surface representations of 3D objects corresponding to those 2D images are subsequently generated. Hence, a signed distance function (SDF) is proposed to maintain object inside-outside information in the implicit surface representation. Polygon mesh surfaces are then produced using the marching cubes algorithm. The ShapeNet dataset was used in the experiments to evaluate the proposed 3D-VAE-SDFGAN. The experimental results show that 3D-VAE-SDFGAN outperforms other state-of-the-art models.

Original languageEnglish
Pages (from-to)4009-4019
Number of pages11
JournalInternational Journal of Electrical and Computer Engineering
Volume13
Issue number4
DOIs
Publication statusPublished - Aug 2023
Externally publishedYes

Keywords

  • 3D shape generation
  • Convolution neural network
  • Generative adversarial network
  • Signed distance function
  • Variational autoencoder

Fingerprint

Dive into the research topics of 'Three-dimensional shape generation via variational autoencoder generative adversarial network with signed distance function'. Together they form a unique fingerprint.

Cite this