Learning Adapters for Text-Guided Portrait Stylization with Pretrained Diffusion Models

Mintu Yang, Xianxu Hou, Hao Li, Linlin Shen*, Lixin Fan

*Corresponding author for this work

Research output: Chapter in Book or Report/Conference proceedingConference Proceedingpeer-review

Abstract

This paper presents a framework for text-guided face portrait stylization using a pre-trained large-scale diffusion model. To balance style transformation and content preservation, we introduce an adapter that modifies specific components of the diffusion model. By training the adapter to only modify these components, we reduce the tuning parameter space, resulting in an efficient solution for face portrait stylization. Our approach captures the target style and at the same time, preserves the source portrait content, making it an effective method for personalized image editing. Experimental results show its superiority over state-of-the-art techniques in various stylization tasks.

Original languageEnglish
Title of host publicationPattern Recognition and Computer Vision - 6th Chinese Conference, PRCV 2023, Proceedings
EditorsQingshan Liu, Hanzi Wang, Rongrong Ji, Zhanyu Ma, Weishi Zheng, Hongbin Zha, Xilin Chen, Liang Wang
PublisherSpringer Science and Business Media Deutschland GmbH
Pages247-258
Number of pages12
ISBN (Print)9789819984282
DOIs
Publication statusPublished - 2024
Event6th Chinese Conference on Pattern Recognition and Computer Vision, PRCV 2023 - Xiamen, China
Duration: 13 Oct 202315 Oct 2023

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume14425 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference6th Chinese Conference on Pattern Recognition and Computer Vision, PRCV 2023
Country/TerritoryChina
CityXiamen
Period13/10/2315/10/23

Keywords

  • Diffusion model
  • Portrait stylization

Fingerprint

Dive into the research topics of 'Learning Adapters for Text-Guided Portrait Stylization with Pretrained Diffusion Models'. Together they form a unique fingerprint.

Cite this