Diffusion Models for 3D Image Generation

Home

Discover

Bookmarks

Keep Up With Latest Trending Papers. Computer Science, AI and Machine Learning and more.Subscribe

Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generation

We propose a novel estimation scheme for image generative models that backpropagate the score of a diffusion model through the jacobian of a differentiable renderer, which weinstantiate to be a voxel radiance field.

This setup aggregates 2d scores at multiple camera viewpoints into a 3d score, and repurposes a pretrained 2dmodel for 3d data generation.

We identify a technical challenge of distributionmismatch that arises in this application, and propose an estimation mechanism to resolve it.

We run our algorithm on several off-the-shelf diffusion models, including the recently released stablediffusion trained on the large-scale laion dataset.

Authors

Haochen Wang, Xiaodan Du, Jiahao Li, Raymond A. Yeh, Greg Shakhnarovich

◐ Recommended Follows

◐ Latest Activity