TY - JOUR
T1 - Convolutional neural network for intermediate view enhancement in multiview streaming
AU - Yu, Li
AU - Tillo, Tammam
AU - Xiao, Jimin
AU - Grangetto, Marco
N1 - Publisher Copyright:
© 2017 IEEE.
PY - 2018/1
Y1 - 2018/1
N2 - Multiview video streaming continues to gain popularity due to the great viewing experience it offers, as well as its availability that has been enabled by increased network throughput and other recent technical developments. User demand for interactive multiview video streaming that provides seamless view switching upon request is also increasing. However, it is a highly challenging task to stream stable and high quality videos that allow real-time scene navigation within the bandwidth constraint. In this paper, a convolutional neural network (ConvNet)-assisted seamless multiview video streaming system is proposed to tackle the challenge. The proposed method solves the problem from two perspectives. First, a ConvNet-assisted multiview representation method is proposed, which provides flexible interactivity without compromising on multiview video compression efficiency. Second, a bit allocation mechanism guided by a navigationmodel is developed to provide seamless navigation and adapt to network bandwidth fluctuations at the same time. These two blocks work closely to provide an optimized viewing experience to users. They can be integrated into any existing multiview video streaming framework to enhance overall performance. Experimental results demonstrate the effectiveness of the proposed method for seamless multiview streaming.
AB - Multiview video streaming continues to gain popularity due to the great viewing experience it offers, as well as its availability that has been enabled by increased network throughput and other recent technical developments. User demand for interactive multiview video streaming that provides seamless view switching upon request is also increasing. However, it is a highly challenging task to stream stable and high quality videos that allow real-time scene navigation within the bandwidth constraint. In this paper, a convolutional neural network (ConvNet)-assisted seamless multiview video streaming system is proposed to tackle the challenge. The proposed method solves the problem from two perspectives. First, a ConvNet-assisted multiview representation method is proposed, which provides flexible interactivity without compromising on multiview video compression efficiency. Second, a bit allocation mechanism guided by a navigationmodel is developed to provide seamless navigation and adapt to network bandwidth fluctuations at the same time. These two blocks work closely to provide an optimized viewing experience to users. They can be integrated into any existing multiview video streaming framework to enhance overall performance. Experimental results demonstrate the effectiveness of the proposed method for seamless multiview streaming.
KW - Convolutional neural network (convnet)
KW - Multiview navigation
KW - Multiview video representation
KW - Multiview video streaming
UR - http://www.scopus.com/inward/record.url?scp=85028938072&partnerID=8YFLogxK
U2 - 10.1109/TMM.2017.2726900
DO - 10.1109/TMM.2017.2726900
M3 - Article
AN - SCOPUS:85028938072
SN - 1520-9210
VL - 20
SP - 15
EP - 28
JO - IEEE Transactions on Multimedia
JF - IEEE Transactions on Multimedia
IS - 1
ER -