Imagen Editor: A cascaded diffusion model for text-guided image inpainting

Home

Discover

Bookmarks

Keep Up With Latest Trending Papers. Computer Science, AI and Machine Learning and more.Subscribe

Liked byJean De Dieu

CC BY

Source arXiv

Computer Vision

Artificial Intelligence

Imagen Editor: A cascaded diffusion model for text-guided image inpainting

Imagen Editor and EditBench: Advancing and Evaluating Text-Guided Image Inpainting

Text-guided image editing can have a transformative impact in supporting creative applications.

A key challenge is to generate edits that are faithfulto input text prompts, while consistent with input images.

We present imageneditor, a cascaded diffusion model built, by fine-tuning, by using object detectors to propose inpainting masks during training, on text-guided inpainting.

To improve qualitative and quantitative evaluation, we introduce editbench, a systematic benchmark for text-guided image inpainting.

Editbench evaluates inpainting edits on natural and generated images exploring objects, attributes, and scenes.

Authors

Su Wang, Chitwan Saharia, Ceslee Montgomery, Jordi Pont-Tuset, Shai Noy, Stefano Pellegrini, Yasumasa Onoe, Sarah Laszlo, David J. Fleet, Radu Soricut, Jason Baldridge, Mohammad Norouzi, Peter Anderson, William Chan

Related Topics

Introduction

Text-guided image can enhance the image generation experience by supporting interactive refinement.

This paper contributes to the modeling and evaluation of text-guided image inpainting, where a user provides an image, a masked area, and a text prompt and the model fills the masked area, consistent with both the prompt and the image context.

We propose, a curated evaluation dataset that captures a wide variety of language, types of images, and levels of difficulty.

Each editbench example consists of (i) a masked input image, (ii) an input text prompt, and (iii) a high-quality output image that can be used as reference for automatic metrics.

Our modeling contribution is, a cascaded diffusion model that combines large scale language representations with fine-grained control to produce high fidelity outputs.

Result

We present two new text-guided image inpainting tools, imagen editor and editbench.

Key to both tools is the addition of new convolution layers to enable high-resolution editing, and the use of a object masking policy for training.

We find that both tools outperform other tools on editbench in both human evaluation and automatic metrics.

Read the Paper

Read Paper

Bookmarks

Bookmarks

Keep Up With Latest Trending Papers. Computer Science, AI and Machine Learning and more.Subscribe

Liked byJean De Dieu

CC BY

Source arXiv

Computer Vision

Artificial Intelligence

Imagen Editor: A cascaded diffusion model for text-guided image inpainting

Imagen Editor and EditBench: Advancing and Evaluating Text-Guided Image Inpainting

Text-guided image editing can have a transformative impact in supporting creative applications.

A key challenge is to generate edits that are faithfulto input text prompts, while consistent with input images.

We present imageneditor, a cascaded diffusion model built, by fine-tuning, by using object detectors to propose inpainting masks during training, on text-guided inpainting.

To improve qualitative and quantitative evaluation, we introduce editbench, a systematic benchmark for text-guided image inpainting.

Editbench evaluates inpainting edits on natural and generated images exploring objects, attributes, and scenes.

Authors

Related Topics

Introduction

Text-guided image can enhance the image generation experience by supporting interactive refinement.

We propose, a curated evaluation dataset that captures a wide variety of language, types of images, and levels of difficulty.

Each editbench example consists of (i) a masked input image, (ii) an input text prompt, and (iii) a high-quality output image that can be used as reference for automatic metrics.

Our modeling contribution is, a cascaded diffusion model that combines large scale language representations with fine-grained control to produce high fidelity outputs.

Result

We present two new text-guided image inpainting tools, imagen editor and editbench.

Key to both tools is the addition of new convolution layers to enable high-resolution editing, and the use of a object masking policy for training.

We find that both tools outperform other tools on editbench in both human evaluation and automatic metrics.

Read the Paper

Read Paper

Home

Discover

Bookmarks

◐ Recommended Follows

◐ Latest Activity