Mar 19, 2019 12:43:00

NVIDIA announces astonishing drawing technology 'GauGAN' that can convert graffiti into realistic landscape photos in real time

NVIDIA Research has developed an innovative technology that converts rough illustrations, such as those drawn using Microsoft Paint and a mouse, into realistic landscape photos in an instant. The interactive software called ' GauGAN ' uses deep learning with a large amount of learning data. With GauGAN, anyone can shape their ideas simply by drawing objects or drawing lines.

GauGAN Turns Doodles into Stunning, Realistic Landscapes | NVIDIA Blog
https://blogs.nvidia.com/blog/20019/03/18/gaugan-photorealistic-landscapes-nvidia-research/

You can see how great GauGAN is by watching the following movie.

Changing Sketches into Photorealistic Masterpieces-YouTube


The movie featured developer technology engineer Chris Hebert and lead science researcher Ming-Yu Liu.

What two people are watching is the following screen. The left side of the screen shows a solid illustration like painted in Microsoft Paint, and the right side shows a realistic image like a landscape picture.

At the bottom of the screen, there are a total of 20 buttons, such as 'sky', 'tree', 'cloud', and 'mountain'. Then click the 'rock' button.

And if you click on the green area painted with 'mountain', the illustration on the left of the screen will be solid in brown. However, the landscape on the right has changed from a mountain-expanding picture to a rocky picture.

Furthermore, if I draw a line with the light blue 'water' ...

For a photo of a waterfall flowing out of a rock.

'The NVIDIA technology has made it possible to create an amazingly high-performance smart paint brush,' said

Bryan Catanzaro, who heads NVIDIA's deep learning research application department. With this technology, neural networks can draw details to fill in the shape by 'drawing only the shape of the object', and anyone can make their ideas a realistic picture.

Draw a line on the water and draw a rock ...

A huge rock is completed in no time. The surrounding elements such as water movement and reflection are also adjusted automatically.

Landscape painting of the meadow.

When the sky is repainted from 'sky' to 'cloud', the blue sky changes to a cloudy sky. The color and appearance of the ground also changed accordingly.

Draw a line with 'mountain' ...

At first only outline. There is no particular change in the water surface at this point.

However, at the moment when the mountain was filled, the mountain appeared in the lake. This is because the neural network is learning that 'water reflects'.

The picture of the meadow is ...

The appearance of the sky changes the moment the material of the ground is changed to 'snow'. This is also because it has been learned that 'the appearance of the sky also changes when it snows.'

The technology is very useful for architects, designers, and people who develop robots using virtual worlds such as self-driving cars, says Catanzaro.

The artist only needs to specify the position and shape, and the colors, textures, shadows, etc. are all handled by the neural network learned from the huge real-world image data. The variety and fidelity of images that can be composited is a remarkable level that has never been seen before.

NVIDIA is also releasing a service called 'AI Playground'. AI Playground was announced in 2018 '

Image InPainting ' ' Artistic Style Transfer ', ' Photorealistic Image Synthesis contains three technology called'.

Slide into the Latest Deep Learning Research in the NVIDIA AI Playground | The Official NVIDIA Blog
https://blogs.nvidia.com/blog/20019/03/18/nvidia-ai-playground/

Image InPainting is a technology that edits unnecessary content as follows and replaces it with realistic computer generated content.

Also, Artistic Style Transfer can convert photos into Gogh-like styles ...

Photorealistic Image Synthesis is a new deep learning-based system that generates photorealistic images and virtual environments using conditional generation hostile networks.

You can experience AI Playground from the following.

Live Interactive Demos | NVIDIA Research
https://www.nvidia.com/en-us/research/ai-playground/