Text-to-image diffusion models for GAN domain adaptation

Home

Discover

Bookmarks

Keep Up With Latest Trending Papers. Computer Science, AI and more.Subscribe

CC BY

Source arXiv

Computer Vision

Text-to-image diffusion models for GAN domain adaptation

Diffusion Guided Domain Adaptation of Image Generators

We show that the classifier-free guidance can be leveraged as a critic and enable generators to distill knowledge from large-scale text-to-image diffusion models to efficiently shift into new domains indicated by text prompts without access to groundtruth samples from target domains.

The proposed method is the first attempt at incorporating large-scale pre-trained diffusion models and distillation sampling for text-driven image generator domain adaptation and gives a quality previously beyond possible.

We demonstrate the effectiveness and controllability of our method through extensive experiments and extend our work to 3d-aware style-based generators and dreambooth guidance.

Authors

Kunpeng Song, Ligong Han, Bingchen Liu, Dimitris Metaxas, Ahmed Elgammal

Related Topics

Introduction

Text-to-image diffusion models have witnessed a remarkable rise in image-generation tasks with their ability to cover a wide range of visual semantics from different image domains.

These models usually require lots of iterative generative steps, which is computationally demanding and makes it undesirable in many application scenarios.

On the other hand, gans preserve their advantage over diffusion models in terms of generation speed and is more computational-friendly to train.

In this work, we explore using diffusion to improve the performance of a text-driven image generator for domain adaptation.

Our new domain adaptation method takes advantage of a pre-trained image diffusion model, providing well-aligned guidance directly from the image domain to help train the gan model.

Read the Paper

Read Paper

Bookmarks

Bookmarks

Keep Up With Latest Trending Papers. Computer Science, AI and more.Subscribe

CC BY

Source arXiv

Computer Vision

Text-to-image diffusion models for GAN domain adaptation

Diffusion Guided Domain Adaptation of Image Generators

We demonstrate the effectiveness and controllability of our method through extensive experiments and extend our work to 3d-aware style-based generators and dreambooth guidance.

Authors

Kunpeng Song, Ligong Han, Bingchen Liu, Dimitris Metaxas, Ahmed Elgammal

Related Topics

Introduction

Text-to-image diffusion models have witnessed a remarkable rise in image-generation tasks with their ability to cover a wide range of visual semantics from different image domains.

These models usually require lots of iterative generative steps, which is computationally demanding and makes it undesirable in many application scenarios.

On the other hand, gans preserve their advantage over diffusion models in terms of generation speed and is more computational-friendly to train.

In this work, we explore using diffusion to improve the performance of a text-driven image generator for domain adaptation.

Our new domain adaptation method takes advantage of a pre-trained image diffusion model, providing well-aligned guidance directly from the image domain to help train the gan model.

Read the Paper

Read Paper

Home

Discover

Bookmarks

◐ Latest Activity

◐ Recommended Follows