TY - GEN
T1 - View synthesis from silhouette using deep convolutional neural network
AU - Jammal, Samer
AU - Tillo, Tammam
AU - Xiao, Jimin
N1 - Publisher Copyright:
© Proceedings of 2019 the 9th International Workshop on Computer Science and Engineering, WCSE 2019 SPRING. All rights reserved.
PY - 2019
Y1 - 2019
N2 - Multiview video could be the basis to support various applications, such as Three-Dimensional video (3DV), Virtual Reality (VR), and Free Viewpoint Video (FVV). Multiview data is intrinsically redundant, in fact, the semantic contents of different views are almost similar, and this is especially true for small baseline views. Obviously, wide baseline views might significantly differ in their contents, where some objects might be completely absent in some views. However, the current approaches of representing multiview data, even if they exploit inter-view correlation, require large bandwidth for transmission. This bandwidth is almost linear with the number of transmitted views. Thus, in this paper we propose to address this problem by representing lateral views solely using their edges, while dropping their texture content. The texture content is synthesized, at the receiver side, by a convolutional neural network (CNN) exploiting the edges and the information in the central view. The edges of the lateral views represent the location of the “objects” in their corresponding views, whereas, we assume that their texture in other views does not change significantly, consequently there is no need to represent them in the lateral views. In this work, in addition to the proposed paradigm of representing multiview data, we also propose a training framework for the CNN network. Experimental results demonstrate the effectiveness of the proposed framework and demonstrate that the network is able to synthesis accurate and reliable lateral views starting from their edges.
AB - Multiview video could be the basis to support various applications, such as Three-Dimensional video (3DV), Virtual Reality (VR), and Free Viewpoint Video (FVV). Multiview data is intrinsically redundant, in fact, the semantic contents of different views are almost similar, and this is especially true for small baseline views. Obviously, wide baseline views might significantly differ in their contents, where some objects might be completely absent in some views. However, the current approaches of representing multiview data, even if they exploit inter-view correlation, require large bandwidth for transmission. This bandwidth is almost linear with the number of transmitted views. Thus, in this paper we propose to address this problem by representing lateral views solely using their edges, while dropping their texture content. The texture content is synthesized, at the receiver side, by a convolutional neural network (CNN) exploiting the edges and the information in the central view. The edges of the lateral views represent the location of the “objects” in their corresponding views, whereas, we assume that their texture in other views does not change significantly, consequently there is no need to represent them in the lateral views. In this work, in addition to the proposed paradigm of representing multiview data, we also propose a training framework for the CNN network. Experimental results demonstrate the effectiveness of the proposed framework and demonstrate that the network is able to synthesis accurate and reliable lateral views starting from their edges.
KW - Convolutionalneural network
KW - Edge map
KW - Silhouette
KW - Stereo video
KW - View synthesis
UR - http://www.scopus.com/inward/record.url?scp=85081106833&partnerID=8YFLogxK
M3 - Conference Proceeding
AN - SCOPUS:85081106833
T3 - Proceedings of 2019 the 9th International Workshop on Computer Science and Engineering, WCSE 2019 SPRING
SP - 17
EP - 21
BT - Proceedings of 2019 the 9th International Workshop on Computer Science and Engineering, WCSE 2019 SPRING
PB - International Workshop on Computer Science and Engineering (WCSE)
T2 - 2019 9th International Workshop on Computer Science and Engineering, WCSE 2019 SPRING
Y2 - 27 February 2019 through 1 March 2019
ER -