Paella: A Speed-Painting Text-to-Image Model

Home

Discover

Bookmarks

Keep Up With Latest Trending Papers. Computer Science, AI and Machine Learning and more.Subscribe

Bookmarked byMark

Paella: A Speed-Painting Text-to-Image Model

Fast Text-Conditional Discrete Denoising on Vector-Quantized Latent Spaces

We introduce a novel text-to-image model requiring less than 10 steps to sample high-fidelity images, using a speed-optimized architectureallowing to sample a single image in less than 500 ms, while having 573mparameters.

The model operates on a compressed & quantized latent space, it is conditioned on clip embeddings and uses an improved sampling function over previous works.

Aside from text-conditional image generation, our model is ableto do latent space interpolation and image manipulations such as inpainting, outpainting, and structural editing.

Authors

Dominic Rampas, Pablo Pernias, Elea Zhong, Marc Aubreville

Related Topics

Introduction

Text-to-image generation has yielded stunning progress in terms of diversity, quality, and variation of generated images, however, the impressive output quality of these models has come with the tradeoff that they require many sampling steps, leading to slow inference speeds unideal for end-user applications.

In this work, we propose a novel technique for text-conditional image generation which is neither transformer-nor diffusion-based, but utilizes a fully convolutional neural network architecture.

With our model, we can sample images with as few as 8 steps while still achieving high-fidelity results, making the model attractive for use-cases that are limited by requirements on latency, memory or computational complexity.

Result

We present a text-to-image generation system using a novel training objective, an improved sampling strategy and a fully-convolutional architecture.

We show that our model can generate high-fidelity images despite being substantially smaller and requiring less steps for sampling than existing models while still reaching competitive numerical results.

In addition to abilities to model relationships from the textual into the image domain, its specific architecture and conditioning strategy allows for a variety of image manipulations tasks such as inpainting, outpainting, variations, etc.

Read the Paper

Read Paper

Bookmarks

Bookmarks

Keep Up With Latest Trending Papers. Computer Science, AI and Machine Learning and more.Subscribe

Bookmarked byMark

Paella: A Speed-Painting Text-to-Image Model

Fast Text-Conditional Discrete Denoising on Vector-Quantized Latent Spaces

The model operates on a compressed & quantized latent space, it is conditioned on clip embeddings and uses an improved sampling function over previous works.

Aside from text-conditional image generation, our model is ableto do latent space interpolation and image manipulations such as inpainting, outpainting, and structural editing.

Authors

Dominic Rampas, Pablo Pernias, Elea Zhong, Marc Aubreville

◐ Recommended Follows

◐ Latest Activity