"Pokemon GO" announces technology "IDV" that may be able to move in conjunction with Pokemon and background

Massachusetts Institute of Technology (MIT)'s Computer Science and Artificial Intelligence Research Institute (CSAIL) developed the technology "Interactive Dynamic Video (IDV)" that enables objects moved in the camera to be moved in real time by manipulating them as images It announced.

For example, if you turn on AR mode on the Pokémon capture screen of "Pokémon GO (Pokemon GO)", the real landscape reflected on the camera will appear behind Pokemon, but if you use IDV It is said that it might be possible to move this landscape together, and demo images using Pokemon GO are actually made.

Interactive Dynamic Video
http://www.interactivedynamicvideo.com/


Reach in and touch objects in videos with "Interactive Dynamic Video" | MIT News
http://news.mit.edu/2016/touching-objects-in-videos-with-interactive-dynamic-video-0802

The movie whose outline is understood is Kore.

Interactive Dynamic Video - YouTube


Until now, subjects shot with cameras were "fixed ones", they could not be moved. However, introduction of computers and algorithms overturns common sense.


First, shoot a picture with a subject slightly vibrated.


Analyze and process the image as to how the subject moves. Then, the image which can move the subject without touching the thing like this was completed. This technology is "Interactive Dynamic Video (IDV)".


The magnitude of the shake when moving it can be adjusted.


No special equipment is required for photographing the subject itself, OK with a normal camera and a tripod.


If you shoot a picture of a plant shaking in the wind for a few seconds and then process it, you will be able to pull it in the direction you want and move it.


Applications include structural health monitoring that checks the status of pillars, beams, etc. of buildings and buildings, for example ......


It is thought of directing in a movie with budget limited.


Virtual characters can not interfere with what actually exists.


Taking the movie "Jurassic World" as an example, this is a picture taken by placing in the forest, but since the dinosaur was made with CG, just because the dinosaur was moved as it is, the plant steps It will not be crushed.


Therefore, plants that dinosaurs will interfere are also made with CG.



However, if it is IDV, you can move the real landscape according to the movement of the virtual character.


The video does not necessarily have to be photographed by oneself. For example, suppose you found a bridge wind tunnel experiment video on YouTube.


With IDV ......


When "The attacked alien robot" descends to the bridge, the bridge will deflect.



When a man wearing a glove uses "magic" toward a plant ... ...


The plant moved with the bulk. It is not moving as a real landscape, but it can be moved by IDV.


Although this technology is not planned to be introduced directly to "Pokemon GO", demo images using Pokemon GO are also made.

Pokemon GO and Interactive Dynamic Video - YouTube


In Pokemon GO, the production using AR (augmented reality) is used.


This is to make the Pokemon as if it were in reality by making the background picture of the camera appear as the background when fighting the gym and capturing Pokemon.


However, in AR, there is no mutual influence between what actually exists and the virtual one in AR space.


Even if you put your hand in this way, it is "the background reflected in the camera" to the last, Latta reflected on the screen does not react to the hand.


However, if it is IDV ...


Pokemon will appear on the video being shot on the smartphone and the background will move according to the movement of that Pokemon.


When Pikachu rebounds, the play equipment will move accordingly, it may become possible to produce such as "more Pokemon exists".


By the way, it is not touched especially in these demos, but it may be useful for applications to "psychic tremor" in games and images ....