NVIDIA publishes "Image Inpainting" technology that automatically corrects images to a level that makes AI feel uncomfortable just by painting obstructive objects

Technique for automatically correcting missing parts of photo information "Image Inpainting"Was released by NVIDIA. Image Inpainting realized by using deep learning makes it clean and clean everything you do not need in the picture. It is also possible to complement the natural content by understanding the meaning of the missing part of the image.

[1804.07723] Image Inpainting for Irregular Holes Using Partial Convolutions
https://arxiv.org/abs/1804.07723

New AI Imaging Technique Reconstructs Photos with Realistic Results - NVIDIA Developer News Center NVIDIA Developer News Center
https://news.developer.nvidia.com/new-ai-imaging-technique-reconstructs-photos-with-realistic-results/

Demonstration of Image Inpainting is available in the following movie. It shows natural image correction technology without incongruity.

Research at NVIDIA: AI Reconstructs Photos with Realistic Results - YouTube


By applying the Image Inpainting technique to the left image, the right image is corrected automatically by AI and the image is corrected.


Cut out obstructing objects at the back of the screen and ...


The object disappeared and the image of the cutout portion was corrected by Image Inpainting.


Next, if you cut out an obstructive line running in the center of the screen ... ...


The line has disappeared. There is no unnaturalness in the corrected image.


Then fill the flag ...


The flag disappears ......


Fill the stone ......


Stones are gone.


The white bar on the right of the screen is also erased. By using Image Inpainting, it is possible to erase from the image as much as it is beautiful just by painting obstructive objects in the photo.


Fill the road with the bridge at the bottom left of the photo ...


The road is gone, only the cliffs are in a state.


Fill the rocks on the cliff ... ...


The rock is gone.


A picture of a room where massive books were collected.


When painting the pillar of the back shelf ... ...


The pillar disappeared and it was filled with a book.


When painting the boundary between the ceiling and the book ... ...


The decoration of the crown molding was gone.


Even if you paint it like drawing a curve, it is corrected with natural feeling.


Image Inpainting is a technique to compensate for the optimum information on the missing part of AI, so if applied to parts of human face it can be reborn as a different person. Filling the women's eyes ......


A completely different eye appeared.


I also paint the remaining eyes ......


It became a totally different person. There is no discomfort in this image alone.


Looking over my nose ......


Fixed the nose bridge. Like cosmetic surgery, you can change the shape of face parts.


Cutting out a part of the man 's hair ......


Hair was corrected without feeling any discomfort.


Fill the majority ... ...


The forehead is exposed. It is a natural correction including including the hairline.


I paint a fine eyebrow of an old man ... ...


The personality of the eyebrows has been lost. Then fill the chin ... ...


No irregularities to the beautiful skin.


When tracing a goblin line ......


I rejuvenated at once.


NVIDIA's research team, Guylyn Liu et al.Tesla V 100"Learning Framework"cuDNN"I trained a neural network and realized Image Inpainting. In the training, more than 50,000 random masks (fills) are used, and another about 25,000 masks are used for accuracy verification. In Image Inpainting, the neural network learned how to reconstruct the pixel by examining both the missing region and the original picture. Mr. Ryu says, "The developed model can respond to repair irrespective of the hole shape or hole location where information is missing," better than the conventional model predicting the missing part from the surrounding pixel information We are doing it.