PAG: Protecting Artworks from Personalizing Image Generative Models

Zhaorui Tan; Siyuan Wang; Xi Yang; Kaizhu Huang

doi:10.1007/978-981-99-8070-3_33

PAG: Protecting Artworks from Personalizing Image Generative Models

Zhaorui Tan, Siyuan Wang, Xi Yang^*, Kaizhu Huang

^*Corresponding author for this work

Department of Intelligent Science

Research output: Chapter in Book or Report/Conference proceeding › Conference Proceeding › peer-review

Abstract

Recent advances in conditional image generation have led to powerful personalized generation models that generate high-resolution artistic images based on simple text descriptions through tuning. However, the abuse of personalized generation models may also increase the risk of plagiarism and the misuse of artists’ painting styles. In this paper, we propose a novel method called Protecting Artworks from Personalizing Image Generative Models framework (PAG) to safeguard artistic images from the malicious use of generative models. By injecting learned target perturbations into the original artistic images, we aim to disrupt the tuning process and introduce the distortions that protect the authenticity and integrity of the artist’s style. Furthermore, human evaluations suggest that our PAG model offers a feasible and effective way to protect artworks, preventing the personalized generation models from generating similar images to the given artworks.

Original language	English
Title of host publication	Neural Information Processing - 30th International Conference, ICONIP 2023, Proceedings
Editors	Biao Luo, Long Cheng, Zheng-Guang Wu, Hongyi Li, Chaojie Li
Publisher	Springer Science and Business Media Deutschland GmbH
Pages	433-444
Number of pages	12
ISBN (Print)	9789819980697
DOIs	https://doi.org/10.1007/978-981-99-8070-3_33
Publication status	Published - 2024
Event	30th International Conference on Neural Information Processing, ICONIP 2023 - Changsha, China Duration: 20 Nov 2023 → 23 Nov 2023

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	14450 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	30th International Conference on Neural Information Processing, ICONIP 2023
Country/Territory	China
City	Changsha
Period	20/11/23 → 23/11/23

Keywords

Conditional image generation
Image protection
Personalizing generation models

Access to Document

10.1007/978-981-99-8070-3_33

Cite this

Tan, Z., Wang, S., Yang, X., & Huang, K. (2024). PAG: Protecting Artworks from Personalizing Image Generative Models. In B. Luo, L. Cheng, Z.-G. Wu, H. Li, & C. Li (Eds.), Neural Information Processing - 30th International Conference, ICONIP 2023, Proceedings (pp. 433-444). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 14450 LNCS). Springer Science and Business Media Deutschland GmbH. https://doi.org/10.1007/978-981-99-8070-3_33

Tan, Zhaorui ; Wang, Siyuan ; Yang, Xi et al. / PAG : Protecting Artworks from Personalizing Image Generative Models. Neural Information Processing - 30th International Conference, ICONIP 2023, Proceedings. editor / Biao Luo ; Long Cheng ; Zheng-Guang Wu ; Hongyi Li ; Chaojie Li. Springer Science and Business Media Deutschland GmbH, 2024. pp. 433-444 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{efaeffb4a40f439cae1d92de066945fa,

title = "PAG: Protecting Artworks from Personalizing Image Generative Models",

abstract = "Recent advances in conditional image generation have led to powerful personalized generation models that generate high-resolution artistic images based on simple text descriptions through tuning. However, the abuse of personalized generation models may also increase the risk of plagiarism and the misuse of artists{\textquoteright} painting styles. In this paper, we propose a novel method called Protecting Artworks from Personalizing Image Generative Models framework (PAG) to safeguard artistic images from the malicious use of generative models. By injecting learned target perturbations into the original artistic images, we aim to disrupt the tuning process and introduce the distortions that protect the authenticity and integrity of the artist{\textquoteright}s style. Furthermore, human evaluations suggest that our PAG model offers a feasible and effective way to protect artworks, preventing the personalized generation models from generating similar images to the given artworks.",

keywords = "Conditional image generation, Image protection, Personalizing generation models",

author = "Zhaorui Tan and Siyuan Wang and Xi Yang and Kaizhu Huang",

note = "Publisher Copyright: {\textcopyright} 2024, The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.; 30th International Conference on Neural Information Processing, ICONIP 2023 ; Conference date: 20-11-2023 Through 23-11-2023",

year = "2024",

doi = "10.1007/978-981-99-8070-3_33",

language = "English",

isbn = "9789819980697",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Science and Business Media Deutschland GmbH",

pages = "433--444",

editor = "Biao Luo and Long Cheng and Zheng-Guang Wu and Hongyi Li and Chaojie Li",

booktitle = "Neural Information Processing - 30th International Conference, ICONIP 2023, Proceedings",

}

Tan, Z, Wang, S, Yang, X & Huang, K 2024, PAG: Protecting Artworks from Personalizing Image Generative Models. in B Luo, L Cheng, Z-G Wu, H Li & C Li (eds), Neural Information Processing - 30th International Conference, ICONIP 2023, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 14450 LNCS, Springer Science and Business Media Deutschland GmbH, pp. 433-444, 30th International Conference on Neural Information Processing, ICONIP 2023, Changsha, China, 20/11/23. https://doi.org/10.1007/978-981-99-8070-3_33

PAG: Protecting Artworks from Personalizing Image Generative Models. / Tan, Zhaorui; Wang, Siyuan; Yang, Xi et al.
Neural Information Processing - 30th International Conference, ICONIP 2023, Proceedings. ed. / Biao Luo; Long Cheng; Zheng-Guang Wu; Hongyi Li; Chaojie Li. Springer Science and Business Media Deutschland GmbH, 2024. p. 433-444 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 14450 LNCS).

Research output: Chapter in Book or Report/Conference proceeding › Conference Proceeding › peer-review

TY - GEN

T1 - PAG

T2 - 30th International Conference on Neural Information Processing, ICONIP 2023

AU - Tan, Zhaorui

AU - Wang, Siyuan

AU - Yang, Xi

AU - Huang, Kaizhu

PY - 2024

Y1 - 2024

N2 - Recent advances in conditional image generation have led to powerful personalized generation models that generate high-resolution artistic images based on simple text descriptions through tuning. However, the abuse of personalized generation models may also increase the risk of plagiarism and the misuse of artists’ painting styles. In this paper, we propose a novel method called Protecting Artworks from Personalizing Image Generative Models framework (PAG) to safeguard artistic images from the malicious use of generative models. By injecting learned target perturbations into the original artistic images, we aim to disrupt the tuning process and introduce the distortions that protect the authenticity and integrity of the artist’s style. Furthermore, human evaluations suggest that our PAG model offers a feasible and effective way to protect artworks, preventing the personalized generation models from generating similar images to the given artworks.

AB - Recent advances in conditional image generation have led to powerful personalized generation models that generate high-resolution artistic images based on simple text descriptions through tuning. However, the abuse of personalized generation models may also increase the risk of plagiarism and the misuse of artists’ painting styles. In this paper, we propose a novel method called Protecting Artworks from Personalizing Image Generative Models framework (PAG) to safeguard artistic images from the malicious use of generative models. By injecting learned target perturbations into the original artistic images, we aim to disrupt the tuning process and introduce the distortions that protect the authenticity and integrity of the artist’s style. Furthermore, human evaluations suggest that our PAG model offers a feasible and effective way to protect artworks, preventing the personalized generation models from generating similar images to the given artworks.

KW - Conditional image generation

KW - Image protection

KW - Personalizing generation models

UR - http://www.scopus.com/inward/record.url?scp=85178602262&partnerID=8YFLogxK

U2 - 10.1007/978-981-99-8070-3_33

DO - 10.1007/978-981-99-8070-3_33

M3 - Conference Proceeding

AN - SCOPUS:85178602262

SN - 9789819980697

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 433

EP - 444

BT - Neural Information Processing - 30th International Conference, ICONIP 2023, Proceedings

A2 - Luo, Biao

A2 - Cheng, Long

A2 - Wu, Zheng-Guang

A2 - Li, Hongyi

A2 - Li, Chaojie

PB - Springer Science and Business Media Deutschland GmbH

Y2 - 20 November 2023 through 23 November 2023

ER -

Tan Z, Wang S, Yang X, Huang K. PAG: Protecting Artworks from Personalizing Image Generative Models. In Luo B, Cheng L, Wu ZG, Li H, Li C, editors, Neural Information Processing - 30th International Conference, ICONIP 2023, Proceedings. Springer Science and Business Media Deutschland GmbH. 2024. p. 433-444. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-981-99-8070-3_33