Disney develops technology to make paintings 3D in real time and move them as alive

Disney developed Disney technology which makes characters on the smartphone and tablet 3D in real time when painting colors on the characters on the page. Textures such as colored pencils are well reproduced, and characters are shaking and dancing to fit the movement of turning pages.

Disney Research »Live Texturing of Augmented Reality Characters from Colored Drawings
http://www.disneyresearch.com/publication/live-texturing-of-augmented-reality-characters/

You can check how the painting starts to move as if it is alive from the following movie.

Live Texturing of Augmented Reality Characters from Colored Drawings - YouTube


Coloring painting is one of the creative expressive acts that children do, but the paintings so far have been "If it gets painted, it is over", the child was quickly losing interest.


Therefore, Disney developed the technique of 3D painting and display it with a smartphone or tablet. The state of the very moment painting the color is displayed in 3D and handwriting and texture are expressed as it is, so you can work as if you are painting a color on a three-dimensional character.


Moreover, the 3D character painted color is not just three-dimensionalized, but it is shaking and walking and moving.


In order to track paintings in real time, Disney developed "Lookup map". By doing this, you can make irregularities on characters that are supposed to be flat, and you can detect how colors are applied to each pixel.


If this can not be detected well, the color will not be placed in the correct place and it will be messed up as follows.


In order to correctly express the texture of painting, we conducted a test with cooperation of 314 people. In the figure below, "Input" is displayed on the left side, "Result A" in the middle, "Result B" on the right side, and you can see that the completed figure changes greatly depending on the method of 3Dization. Subjects seem to pick the most favorable completion drawing from the choices.




As a result of the test, it turned out that the subject did not like "finish for children". Therefore, it seems that the completed figure avoids becoming an extremely childish finish.


Also, since the painting is a mechanism that the character moves in response to the swing and movement of the page, not only "paint" but also "fun" moving character is born.


Where 40 completed subjects used the completed application for development finish ... ...


Many opinions that "painting became fun", "the feeling connected to the character got bigger" and "I would like to use the application in the future" were said to be highly satisfying applications.


A colorful elephant dancing with Pyon Pyeong on paper. From adults to children, I can enjoy painting more than ever.