Real-time 3D Representation of Neural Fields using PlenOctrees

Home

Discover

Bookmarks

Keep Up With Latest Trending Papers. Computer Science, AI and Machine Learning and more.Subscribe

Liked byShimon

Real-time 3D Representation of Neural Fields using PlenOctrees

PlenOctrees for Real-time Rendering of Neural Radiance Fields

We introduce a method to render neural radiance fields (nerfs) in real time using an octree-based 3d representation which supports view-dependent effects such as specularities.

Our method can render 800x800 images at more than 150fps, which is over 3000 times faster than conventional neural rendering methods.

We do so without sacrificing quality while preserving the ability of nerfs to perform free-viewpoint rendering of scenes with arbitrary geometry and view-dependenteffects.

Specifically, we show that it is possible to train the neural network to predict a sphericalharmonic representation of radiance, removing the viewing direction as an inputto the neural network.

Furthermore, we show that plenoctrees can be directly optimized to further minimize the reconstruction loss, which leads to equal or better quality compared to competing methods.

Our real-time neural renderingapproach may potentially enable new applications such as 6-dof industrial and product visualizations, as well as next generation ar/vr systems.

Authors

Alex Yu, Ruilong Li, Matthew Tancik, Hao Li, Ren Ng, Angjoo Kanazawa

◐ Recommended Follows

◐ Latest Activity