Continuous Image Outpainting with Neural ODE

Penglei Gao, Xi Yang, Rui Zhang*, Kaizhu Huang

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

Abstract

Generalised image outpainting is an important and active research topic in computer vision, which aims to extend appealing content all-side around a given image. Existing state-of-the-art outpainting methods often rely on discrete extrapolation to extend the feature map in the bottleneck. They thus suffer from content unsmoothness, especially in circumstances where the outlines of objects in the extrapolated regions are incoherent with the input sub-images. To mitigate this issue, we design a novel bottleneck with Neural ODEs to make continuous extrapolation in latent space, which could be a plug-in for many deep learning frameworks. Our ODE-based network continuously transforms the state and makes accurate predictions by learning the incremental relationship among latent points, leading to both smooth and structured feature representation. Experimental results on three real-world datasets both applied on transformer-based and CNN-based frameworks show that our methods could generate more realistic and coherent images against the state-of-the-art image outpainting approaches. Our code is available at https://github.com/PengleiGao/Continuous-Image-Outpainting-with-Neural-ODE.

Original languageEnglish
Article number203
JournalACM Transactions on Multimedia Computing, Communications and Applications
Volume20
Issue number7
DOIs
Publication statusPublished - 25 Apr 2024

Keywords

  • Image outpainting
  • neural ODE
  • transformer
  • u-shaped structure

Fingerprint

Dive into the research topics of 'Continuous Image Outpainting with Neural ODE'. Together they form a unique fingerprint.

Cite this