TY - JOUR
T1 - Outpainting Natural Scenery Images by Fusing Forecasting Information
AU - Geng, Yujie
AU - Gao, Penglei
AU - Yang, Xi
AU - Yan, Yuyao
AU - Huang, Kaizhu
N1 - Publisher Copyright:
© Published under licence by IOP Publishing Ltd.
PY - 2022/6/1
Y1 - 2022/6/1
N2 - Image outpainting receives less attention due to the challenges of predicting the spatial content consistency and maintaining the high quality in generated images. All-sides image prediction is a generalized task in outpainting, aiming to extrapolate the outside content for all sides of an image. It is a challenging task to maintain spatial and semantic consistency between the original input and the generated multi-step regions. In this paper, we embed a novel Multi-view Recurrent Content Transfer module into an Encoder-Decoder architecture for long-range all-side image outpainting. A multi-head attention mechanism is leveraged to fuse information from different representation sub-spaces at different positions to enhance the consistency of generated images and the original input. Our model could obtain sufficient temporal information in predicting the extended feature maps, which improves the quality of long-range images extrapolation. We experimentally demonstrate that our proposed method could produce visually appealing results for outside image outpainting against the state-of-the-art image inpainting and outpainting approaches. Modelling the temporal relationship could help generate the outside regions and reconstruct the input regions smoothly and realistically. In addition, an attempt to possibly allow for arbitrary output resolutions is supported as well.
AB - Image outpainting receives less attention due to the challenges of predicting the spatial content consistency and maintaining the high quality in generated images. All-sides image prediction is a generalized task in outpainting, aiming to extrapolate the outside content for all sides of an image. It is a challenging task to maintain spatial and semantic consistency between the original input and the generated multi-step regions. In this paper, we embed a novel Multi-view Recurrent Content Transfer module into an Encoder-Decoder architecture for long-range all-side image outpainting. A multi-head attention mechanism is leveraged to fuse information from different representation sub-spaces at different positions to enhance the consistency of generated images and the original input. Our model could obtain sufficient temporal information in predicting the extended feature maps, which improves the quality of long-range images extrapolation. We experimentally demonstrate that our proposed method could produce visually appealing results for outside image outpainting against the state-of-the-art image inpainting and outpainting approaches. Modelling the temporal relationship could help generate the outside regions and reconstruct the input regions smoothly and realistically. In addition, an attempt to possibly allow for arbitrary output resolutions is supported as well.
UR - http://www.scopus.com/inward/record.url?scp=85132012677&partnerID=8YFLogxK
U2 - 10.1088/1742-6596/2278/1/012031
DO - 10.1088/1742-6596/2278/1/012031
M3 - Conference article
AN - SCOPUS:85132012677
SN - 1742-6588
VL - 2278
JO - Journal of Physics: Conference Series
JF - Journal of Physics: Conference Series
IS - 1
M1 - 012031
T2 - 2022 6th International Conference on Machine Vision and Information Technology, CMVIT 2022
Y2 - 25 February 2022
ER -