Sparse voxel radiance fields for novel view synthesis based on few views on ShapeNet

Home

Discover

Bookmarks

Keep Up With Latest Trending Papers. Computer Science, AI and Machine Learning and more.Subscribe

SPARF: Large-Scale Learning of 3D Sparse Radiance Fields from Few Input Images

We present a large-scale synthetic dataset for novel view synthesis based on few views on shapenet.

The dataset consists of 17 million images rendered from nearly 40,000 shapes at high resolution (400 x 400 pixels) and includes morethan one million 3d-optimized radiance fields with multiple voxel resolutions.

Furthermore, we propose a novel pipeline (surfnet) that learns to generate high-quality sparse voxel radiance fields that can be rendered from only few views.

This is done by using the densely collected dataset and 3d sparse convolutions.

Our approach achieves state-of-the-art results in the task of unconstrained novel view synthesis as comparedto recent baselines.

Authors

Abdullah Hamdi, Bernard Ghanem, Matthias Nießner

Related Topics

Introduction

In this work, we treat the distribution of sparse-voxel radiance fields (srfs) as a 3d data structure and try to learn a generative model (dubbed) on the distribution of radiance fields conditioned on a few images to generalize to unseen 3d shapes.

We construct a large and high-resolution dataset (sparf) of posed multi-view images from shapenet that correspond to the same 13 classes originally used in the neural radiance field (nmr) dataset, but with an order of magnitude more images and pixels (17m 1m images and 400 @xmath1 400 64 @xmath2 64 pixels).

We also provide more than optimized sparse radiance fields of spherical harmonics and densities that allow for the novel view synthesis of the 40k models using plenoxels.

The idea of learning a prior (2d cnn/vit) on radiance fields in order to enhance the few-view setup of novel view synthesis is previously investigated by several works @cite14.

Result

We propose a large-scale dataset of sparse radiance fields (srfs) that include around one million srfs and 17 million posed images of 3d shapes.

The dataset aims to move the community in the direction of treating radiance fields as a 3d data structure, instead of optimization results and mlp fitting.

Leveraging the utility of the dataset, we propose a surfnet pipeline to train a conditional generative model to generate sparse voxel grids from few input images (1 or 3) distilled as partial srfs.

Surfnet allows generating radiance fields from single images of unseen shapes, which allows for rendering high-quality images from novel views, reaching state-of-the-art performance in unconstrained novel view synthesis compared to other methods.

Read the Paper

Read Paper

Bookmarks

Bookmarks

Keep Up With Latest Trending Papers. Computer Science, AI and Machine Learning and more.Subscribe

Sparse voxel radiance fields for novel view synthesis based on few views on ShapeNet

SPARF: Large-Scale Learning of 3D Sparse Radiance Fields from Few Input Images

We present a large-scale synthetic dataset for novel view synthesis based on few views on shapenet.

Furthermore, we propose a novel pipeline (surfnet) that learns to generate high-quality sparse voxel radiance fields that can be rendered from only few views.

This is done by using the densely collected dataset and 3d sparse convolutions.

Our approach achieves state-of-the-art results in the task of unconstrained novel view synthesis as comparedto recent baselines.

Authors

Abdullah Hamdi, Bernard Ghanem, Matthias Nießner

◐ Recommended Follows

◐ Latest Activity