Full-body Neural Avatars of a Person Captured by an egocentric Fisheye Camera

Home

Discover

Bookmarks

Keep Up With Latest Trending Papers. Computer Science, AI and more.Subscribe

CC BY

Source arXiv

Computer Vision

Full-body Neural Avatars of a Person Captured by an egocentric Fisheye Camera

EgoRenderer: Rendering Human Avatars from Egocentric Camera Images

Full-body neural avatars of a person captured by a wearable, egocentric fisheye camera that is mounted on a cap or a vr headset come with unique challenges due to the top-down view and large distortions.

We present egorenderer, a system for rendering full-body neural avatars of a person captured by a wearable, egocentric fisheye camera that is mounted on a cap or a vr headset.

Our system renders photorealistic novel views of the actorand her motion from arbitrary virtual camera locations.

We decompose the rendering process into several steps, including texture synthesis, pose construction, and neural image translation.

For texture synthesis, we propose a neural network that infers dense correspondences between the input fisheye images and an underlying parametric body model, and to extract textures from egocentric inputs.

For pose generation, we first estimate body pose from the egocentric view using a parametric model, and synthesize an external free-viewpoint pose image by projecting the parametric model to the user-specified target viewpoint.

We next combine the target pose image and the textures into a combined feature image, which is transformed into the output color image using a neural image translation network.

In addition, to encode dynamic appearances, our approach also learns an implicit texture stack that captures detailed appearance variation across poses and viewpoints.

Experimental evaluations show that egorenderer is capable of generating realistic free-viewpoint avatars of a person wearing an egocentriccamera.

Authors

Tao Hu, Kripasindhu Sarkar, Lingjie Liu, Matthias Zwicker, Christian Theobalt

Related Topics

Neural network

Parametric model

Parametric body model

Egorenderer

Neural image translation network

Neural image translation

Ego-dpnet

Read the Paper

Read Paper