Software that builds 3D space simply by loading movies shot by 360 degree camera appears and can be applied to VR

There is no need for dedicated equipment such as a depth sensor and software capable of constructing a 3D stereoscopic space from an actual landscape is developed if there is only a camera capable of shooting 360 degrees around. With this software, it is possible to convert the real city scenery very easily to 3D.

Maxime Lhuillier's home page.
http://maxime.lhuillier.free.fr/

Jaw-dropping software converts 360 video into 3D model for VR - 360 Rumors
http://360rumors.com/2017/11/software-institut-pascal-converts-360-video-3d-model-vr.html

I developed software,French National Center for Scientific ResearchAnd Institut Pascal (Pascal Research Institute) etc. Mr. Maxim Luirier who specializes in computer vision field. 360 degree shooting action camera made by GARMINVIRB 360Based on the images taken with the notebook PC, you can feel the wonderful thing when you see the following demonstration video that you can analyze using the software installed on your laptop computer and build 3D space and move freely.

Forest scan using helmet-held Virb 360 camera - YouTube


Mr. Luirier photographed the landscape of the forest with VIRB 360 attached to the helmet, and let the computer process it. Video data is 2496 × 2496 pixels in size and the frame rate is 30 fps.


We wander around in the forest for 744 seconds and try to build a 3D space based on the material we have taken.


The picture actually taken is like this. The scenery by the fisheye lens is not much coming to the pin even by humans, but when we analyze this with the software developed by Mr. Luirier ......


It's like a Google Street view. But the most different thing from Street View is that this landscape is completely 3D and can walk around freely.


Within the software, processing to analyze the actual three-dimensional structure is performed based on the image. The depth sensor such as infrared rays and laser is not used, and it is an epochal point to analyze the appearance of space completely based only on the image of the 360 ​​degree camera.


There are countless silver marks in the woods in the movie. These are points where software is set as a key frame for recognizing solid.


It is a little confusing, but it is also possible to move like this from the original eye to a high point. Please see the actual movie by playing the movie with the video.


The whole forest is constructed as one such space.


Mr. Louilier, 3D data created in the same way, free online 3D viewerSketchfabIt is open to the public. The reason why the scenery is somewhat rough is that the upper limit of the data capacity that can be uploaded seems to be the reason, but actually thinking that creating a three dimensional space from photographs and moving pictures is quite amazing thing realized .

These 3D models can be displayed in VR headsets such as "HTC Vive" and "Oculus Rift" and enter that space. Also, "Google Cardboard" and WebVR compatibleFirefox NightlyIt is said that a similar experience can be done by combining browsers such as.