Google's real-time space recognition technology "Tango" has the "Area Learning" function that can grasp the features of hundreds of structures at a time and store the "space" itself

Real time 3D recognition technology under development by GoogleTango"By memorizing features in space, you can memorize the space itself"Area LearningFunction. Demonstration that Area Learnig's technology indispensable for indoor 3D mapping is advanced so far,Google I / O 2016It is announced in.

Introducing Project Tango Area Learning - Google I / O 2016 - YouTube


Mr. Wim Muessun, a developer of Project Tango who is surprised that the audience seats are full. Today I will explain the technology "Area Learning".


"People who know about Project Tango," almost everyone raised hands.


"Then, who's heard about Area Leaning?"


I can only raise my hands.


Mr. Muessun talking about wanting to acquire knowledge about Area Learning by all means today, first from the characteristics of Tango.


Tango is a terminal equipped with a special camera, and Project Tango is a project to realize the same performance on a mobile terminal as humans recognize space.


What is important as a characteristic of Tango is "Motion tracking" "Depth recognition" and "Area Learning"


Motion tracking is a technique to estimate motion from changes in external space.


Start from a certain place, for example ...


Go ahead and add the action of returning at the return point to the Tango terminal.


When returning to the original place, Tango recognizes that it is the starting point and can measure exactly how far it moved.


On the other hand, depth recognition is a function to acquire depth information of an object.


You can measure the depth using a special infrared camera for depth measurement in Tango.


And the third is "Area Learning" which is the main theme of today. It is a function to memorize space.


For example, if you have experience of entering a particular building, when you visit the same place again after a time you should feel "I've been to this space."


This is because I remember the characteristic structures in the building. It is Area Learning that let Tango have a function similar to this.


So, how does Area Learning work?


Tango's camera is wide-angle and catches several characteristic structures visible in the field of vision.


Even in the same structure arrangement, as Tango's position and orientation change, its appearance will change. However, it is possible to determine that the mutual relative position has not changed by calculating the arrangement of multiple structures.


Even if a certain structure is directly in front of you, we recognize that Tango is the same place if you know from the position of the rest of the structure that there is no change in placement itself. This is a function Area Learning which can memorize space arrangement and memorize place.


So, demonstration of Area Learning.


When the Area Learning function is off, only the motion tracking function is working.


When Area Learning function is turned on ... ...


Recognizing a characteristic structure in the space ...


Yellow point has been fixed. This, Tango is say that stores the arrangement of the structure in the space. Note that the Tango can store know the hundreds of structural features at a time.


Here, try blindfolding with Tango's camera ......


The yellow point memorized in the space disappeared.


Taking a blindfold and looking around at Tango ......


When the place I memorized was taken, a yellow dot appeared, I realized that it is the same as the place I remember.


What is the condition that Area Learning can not be used?


The room on the left is the room where the clothes are messy and the room on the right is tidy. The rooms themselves are the same, but if objects to be marked are scattered, Tango is confusing and Area Learning is difficult to function.


Besides, in the day and night room where light condition is different ... ...


Retrograde or ......


Where there are differences in presence or absence of spectators ......


It is a problem that overcoming the space which is not good, such as pure white and uncharacteristic space, is an issue.


In addition, another demonstration.


Let's put a virtual cube box on the table.


Of course, even if you are away from the sight, even if the box disappears from the sight, once you reprint the table, the box will show up.


However, if the Area Learning function is OFF, moving the Tango terminal violently ...


You may lose sight of the box.


On the other hand, when the Area Learning function is ON.


Tango violently moving the terminal ......


Although I lose sight of the box again ...


Resurrected instantly. Because I remember the space, I can reproduce the virtual object.


Next is the sharing function of Area Learning. It is an experiment that shares the memory information of space among those who wear Tango tablet like a VR headset.


If flow can grasp the surrounding structure, memorize the space, share its memory information, if all members can recognize the same space OK.


Whether three people can accurately share the space is tested by passing the tablet terminal.


This is the picture one person wearing Tango headset is watching. Three people wearing the Tango headset are watching such VR space. A white object moving to the center of the image is a virtual tablet created in the virtual space.


The right person hands the virtual tablet to the left person ... ....


Naturally, the left person succeeds in receiving. Because three people are watching images of different VR space, it is a trick that can not be done unless precise location information is shared.


Mr. Muesson said, "The development of Area Learning technology has only just started from the starting point."


Before Area Learning technology evolved, indoor type 3D mapping that will give AR information inside the building will be realized.


The Tango project partnered with Lenovo to manufacture Tango tablets for developers.


According to Mr. Muessen, it is planned to develop Tango smartphones for general purpose not for developers in collaboration with Lenovo and release in late 2016.