VHS to HDTV video translation using multi-task adversarial learning

Hongming Luo; Guangsen Liao; Xianxu Hou; Bozhi Liu; Fei Zhou; Guoping Qiu

doi:10.1007/978-3-030-37731-1_7

VHS to HDTV video translation using multi-task adversarial learning

Hongming Luo, Guangsen Liao, Xianxu Hou, Bozhi Liu, Fei Zhou^*, Guoping Qiu

^*Corresponding author for this work

Research output: Chapter in Book or Report/Conference proceeding › Conference Proceeding › peer-review

Abstract

There are large amount of valuable video archives in Video Home System (VHS) format. However, due to the analog nature, their quality is often poor. Compared to High-definition television (HDTV), VHS video not only has a dull color appearance but also has a lower resolution and often appears blurry. In this paper, we focus on the problem of translating VHS video to HDTV video and have developed a solution based on a novel unsupervised multi-task adversarial learning model. Inspired by the success of generative adversarial network (GAN) and CycleGAN, we employ cycle consistency loss, adversarial loss and perceptual loss together to learn a translation model. An important innovation of our work is the incorporation of super-resolution model and color transfer model that can solve unsupervised multi-task problem. To our knowledge, this is the first work that dedicated to the study of the relation between VHS and HDTV and the first computational solution to translate VHS to HDTV. We present experimental results to demonstrate the effectiveness of our solution qualitatively and quantitatively.

Original language	English
Title of host publication	MultiMedia Modeling - 26th International Conference, MMM 2020, Proceedings
Editors	Wen-Huang Cheng, Junmo Kim, Jung-Woo Choi, Wei-Ta Chu, Peng Cui, Min-Chun Hu, Wesley De Neve
Publisher	Springer
Pages	77-86
Number of pages	10
ISBN (Print)	9783030377304
DOIs	https://doi.org/10.1007/978-3-030-37731-1_7
Publication status	Published - 2020
Externally published	Yes
Event	26th International Conference on MultiMedia Modeling, MMM 2020 - Daejeon, Korea, Republic of Duration: 5 Jan 2020 → 8 Jan 2020

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	11961 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	26th International Conference on MultiMedia Modeling, MMM 2020
Country/Territory	Korea, Republic of
City	Daejeon
Period	5/01/20 → 8/01/20

Keywords

GAN
HDTV
Multi-task learning
Unsupervised
VHS
Video translation

Access to Document

10.1007/978-3-030-37731-1_7

Cite this

Luo, H., Liao, G., Hou, X., Liu, B., Zhou, F., & Qiu, G. (2020). VHS to HDTV video translation using multi-task adversarial learning. In W.-H. Cheng, J. Kim, J.-W. Choi, W.-T. Chu, P. Cui, M.-C. Hu, & W. De Neve (Eds.), MultiMedia Modeling - 26th International Conference, MMM 2020, Proceedings (pp. 77-86). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11961 LNCS). Springer. https://doi.org/10.1007/978-3-030-37731-1_7

Luo, Hongming ; Liao, Guangsen ; Hou, Xianxu et al. / VHS to HDTV video translation using multi-task adversarial learning. MultiMedia Modeling - 26th International Conference, MMM 2020, Proceedings. editor / Wen-Huang Cheng ; Junmo Kim ; Jung-Woo Choi ; Wei-Ta Chu ; Peng Cui ; Min-Chun Hu ; Wesley De Neve. Springer, 2020. pp. 77-86 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{a2cc2c58919c409787851119e324130a,

title = "VHS to HDTV video translation using multi-task adversarial learning",

abstract = "There are large amount of valuable video archives in Video Home System (VHS) format. However, due to the analog nature, their quality is often poor. Compared to High-definition television (HDTV), VHS video not only has a dull color appearance but also has a lower resolution and often appears blurry. In this paper, we focus on the problem of translating VHS video to HDTV video and have developed a solution based on a novel unsupervised multi-task adversarial learning model. Inspired by the success of generative adversarial network (GAN) and CycleGAN, we employ cycle consistency loss, adversarial loss and perceptual loss together to learn a translation model. An important innovation of our work is the incorporation of super-resolution model and color transfer model that can solve unsupervised multi-task problem. To our knowledge, this is the first work that dedicated to the study of the relation between VHS and HDTV and the first computational solution to translate VHS to HDTV. We present experimental results to demonstrate the effectiveness of our solution qualitatively and quantitatively.",

keywords = "GAN, HDTV, Multi-task learning, Unsupervised, VHS, Video translation",

author = "Hongming Luo and Guangsen Liao and Xianxu Hou and Bozhi Liu and Fei Zhou and Guoping Qiu",

note = "Publisher Copyright: {\textcopyright} 2020, Springer Nature Switzerland AG.; 26th International Conference on MultiMedia Modeling, MMM 2020 ; Conference date: 05-01-2020 Through 08-01-2020",

year = "2020",

doi = "10.1007/978-3-030-37731-1_7",

language = "English",

isbn = "9783030377304",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer",

pages = "77--86",

editor = "Wen-Huang Cheng and Junmo Kim and Jung-Woo Choi and Wei-Ta Chu and Peng Cui and Min-Chun Hu and {De Neve}, Wesley",

booktitle = "MultiMedia Modeling - 26th International Conference, MMM 2020, Proceedings",

}

Luo, H, Liao, G, Hou, X, Liu, B, Zhou, F & Qiu, G 2020, VHS to HDTV video translation using multi-task adversarial learning. in W-H Cheng, J Kim, J-W Choi, W-T Chu, P Cui, M-C Hu & W De Neve (eds), MultiMedia Modeling - 26th International Conference, MMM 2020, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 11961 LNCS, Springer, pp. 77-86, 26th International Conference on MultiMedia Modeling, MMM 2020, Daejeon, Korea, Republic of, 5/01/20. https://doi.org/10.1007/978-3-030-37731-1_7

VHS to HDTV video translation using multi-task adversarial learning. / Luo, Hongming; Liao, Guangsen; Hou, Xianxu et al.
MultiMedia Modeling - 26th International Conference, MMM 2020, Proceedings. ed. / Wen-Huang Cheng; Junmo Kim; Jung-Woo Choi; Wei-Ta Chu; Peng Cui; Min-Chun Hu; Wesley De Neve. Springer, 2020. p. 77-86 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11961 LNCS).

Research output: Chapter in Book or Report/Conference proceeding › Conference Proceeding › peer-review

TY - GEN

T1 - VHS to HDTV video translation using multi-task adversarial learning

AU - Luo, Hongming

AU - Liao, Guangsen

AU - Hou, Xianxu

AU - Liu, Bozhi

AU - Zhou, Fei

AU - Qiu, Guoping

PY - 2020

Y1 - 2020

N2 - There are large amount of valuable video archives in Video Home System (VHS) format. However, due to the analog nature, their quality is often poor. Compared to High-definition television (HDTV), VHS video not only has a dull color appearance but also has a lower resolution and often appears blurry. In this paper, we focus on the problem of translating VHS video to HDTV video and have developed a solution based on a novel unsupervised multi-task adversarial learning model. Inspired by the success of generative adversarial network (GAN) and CycleGAN, we employ cycle consistency loss, adversarial loss and perceptual loss together to learn a translation model. An important innovation of our work is the incorporation of super-resolution model and color transfer model that can solve unsupervised multi-task problem. To our knowledge, this is the first work that dedicated to the study of the relation between VHS and HDTV and the first computational solution to translate VHS to HDTV. We present experimental results to demonstrate the effectiveness of our solution qualitatively and quantitatively.

AB - There are large amount of valuable video archives in Video Home System (VHS) format. However, due to the analog nature, their quality is often poor. Compared to High-definition television (HDTV), VHS video not only has a dull color appearance but also has a lower resolution and often appears blurry. In this paper, we focus on the problem of translating VHS video to HDTV video and have developed a solution based on a novel unsupervised multi-task adversarial learning model. Inspired by the success of generative adversarial network (GAN) and CycleGAN, we employ cycle consistency loss, adversarial loss and perceptual loss together to learn a translation model. An important innovation of our work is the incorporation of super-resolution model and color transfer model that can solve unsupervised multi-task problem. To our knowledge, this is the first work that dedicated to the study of the relation between VHS and HDTV and the first computational solution to translate VHS to HDTV. We present experimental results to demonstrate the effectiveness of our solution qualitatively and quantitatively.

KW - GAN

KW - HDTV

KW - Multi-task learning

KW - Unsupervised

KW - VHS

KW - Video translation

UR - http://www.scopus.com/inward/record.url?scp=85078518391&partnerID=8YFLogxK

U2 - 10.1007/978-3-030-37731-1_7

DO - 10.1007/978-3-030-37731-1_7

M3 - Conference Proceeding

AN - SCOPUS:85078518391

SN - 9783030377304

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 77

EP - 86

BT - MultiMedia Modeling - 26th International Conference, MMM 2020, Proceedings

A2 - Cheng, Wen-Huang

A2 - Kim, Junmo

A2 - Choi, Jung-Woo

A2 - Chu, Wei-Ta

A2 - Cui, Peng

A2 - Hu, Min-Chun

A2 - De Neve, Wesley

PB - Springer

T2 - 26th International Conference on MultiMedia Modeling, MMM 2020

Y2 - 5 January 2020 through 8 January 2020

ER -

Luo H, Liao G, Hou X, Liu B, Zhou F, Qiu G. VHS to HDTV video translation using multi-task adversarial learning. In Cheng WH, Kim J, Choi JW, Chu WT, Cui P, Hu MC, De Neve W, editors, MultiMedia Modeling - 26th International Conference, MMM 2020, Proceedings. Springer. 2020. p. 77-86. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-030-37731-1_7

VHS to HDTV video translation using multi-task adversarial learning

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this