Enhanced Text-to-Image Synthesis With Self-Supervision

Yong Xuan Tan, Chin Poo Lee*, Mai Neo, Kian Ming Lim, Jit Yan Lim

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

6 Citations (Scopus)
Plum Print visual indicator of research metrics
  • Citations
    • Citation Indexes: 6
  • Captures
    • Readers: 13
  • Mentions
    • News Mentions: 1
see details

Abstract

The task of Text-to-Image synthesis is a difficult challenge, especially when dealing with low-data regimes, where the number of training samples is limited. In order to address this challenge, the Self-Supervision Text-to-Image Generative Adversarial Networks (SS-TiGAN) has been proposed. The method employs a bi-level architecture, which allows for the use of self-supervision to increase the number of training samples by generating rotation variants. This, in turn, maximizes the diversity of the model representation and enables the exploration of high-level object information for more detailed image construction. In addition to the use of self-supervision, SS-TiGAN also investigates various techniques to address the stability issues that arise in Generative Adversarial Networks. By implementing these techniques, the proposed SS-TiGAN has achieved a new state-of-the-art performance on two benchmark datasets, Oxford-102 and CUB. These results demonstrate the effectiveness of the SS-TiGAN method in synthesizing high-quality, realistic images from text descriptions under low-data regimes.

Original languageEnglish
Pages (from-to)39508-39519
Number of pages12
JournalIEEE Access
Volume11
DOIs
Publication statusPublished - 2023
Externally publishedYes

Keywords

  • GAN
  • generative adversarial networks
  • generative model
  • self-supervised learning
  • Text-to-image synthesis

Fingerprint

Dive into the research topics of 'Enhanced Text-to-Image Synthesis With Self-Supervision'. Together they form a unique fingerprint.

Cite this

Tan, Y. X., Lee, C. P., Neo, M., Lim, K. M., & Lim, J. Y. (2023). Enhanced Text-to-Image Synthesis With Self-Supervision. IEEE Access, 11, 39508-39519. https://doi.org/10.1109/ACCESS.2023.3268869