3D scene reconstruction from monocular spherical video with motion parallax
Kenji Tanaka
In this paper, we describe a method to capture nearly entirely spherical (360
degree) depth information using two adjacent frames from a single spherical
video with motion parallax. After illustrating a spherical depth information
retrieval using two spherical cameras, we demonstrate monocular spherical
stereo by using stabilized first-person video footage. Experiments demonstrated
that the depth information was retrieved on up to 97% of the entire sphere in
solid angle. At a speed of 30 km/h, we were able to estimate the depth of an
object located over 30 m from the camera. We also reconstructed the 3D
structures (point cloud) using the obtained depth data and confirmed the
structures can be clearly observed. We can apply this method to 3D structure
retrieval of surrounding environments such as 1) previsualization, location
hunting/planning of a film, 2) real scene/computer graphics synthesis and 3)
motion capture. Thanks to its simplicity, this method can be applied to various
videos. As there is no pre-condition other than to be a 360 video with motion
parallax, we can use any 360 videos including those on the Internet to
reconstruct the surrounding environments. The cameras can be lightweight enough
to be mounted on a drone. We also demonstrated such applications.