HSGAN: Hierarchical Graph Learning for Point Cloud Generation

Yushi Li, George Baciu*

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

11 Citations (Scopus)

Abstract

Point clouds are the most general data representations of real and abstract objects, and have a wide variety of applications in many science and engineering fields. Point clouds also provide the most scalable multi-resolution composition for geometric structures. Although point cloud learning has shown remarkable results in shape estimation and semantic segmentation, the unsupervised generation of 3D object parts still pose significant challenges in the 3D shape understanding problem. We address this problem by proposing a novel Generative Adversarial Network (GAN), named HSGAN, or Hierarchical Self-Attention GAN, with remarkable properties for 3D shape generation. Our generative model takes a random code and hierarchically transforms it into a representation graph by incorporating both Graph Convolution Network (GCN) and self-attention. With embedding the global graph topology in shape generation, the proposed model takes advantage of the latent topological information to fully construct the geometry of 3D object shapes. Different from the existing generative pipelines, our deep learning architecture articulates three significant properties HSGAN effectively deploys the compact latent topology information as a graph representation in the generative learning process and generates realistic point clouds, HSGAN avoids multiple discriminator updates per generator update, and HSGAN preserves the most dominant geometric structures of 3D shapes in the same hierarchical sampling process. We demonstrate the performance of our new approach with both quantitative and qualitative evaluations. We further present a new adversarial loss to maintain the training stability and overcome the potential mode collapse of traditional GANs. Finally, we explore the use of HSGAN as a plug-and-play decoder in the auto-encoding architecture.

Original languageEnglish
Article number9409723
Pages (from-to)4540-4554
Number of pages15
JournalIEEE Transactions on Image Processing
Volume30
DOIs
Publication statusPublished - 2021
Externally publishedYes

Keywords

  • GAN
  • Generative Adversarial Network
  • gradient penalty
  • graph learning
  • machine learning
  • point cloud generation
  • self-attention

Cite this