Waseda University researchers developed a technology to supplement natural images instantaneously in areas where the image was painted

Researchers at Waseda University are using image supplementation technology using a deep network (convolution neural network)Globally and Locally Consistent Image Completion"Has been released. It shows the accuracy with different level from existing image supplementing technology.

Irizuka Risa - image complement by deep network (SIGGRAPH 2017)
http://hi.cs.waseda.ac.jp/~iizuka/projects/completion/ja/

Thesis was officially adopted at SIGGRAPH 2017. It is a method to complement complicated images by deep learning. You can erase unnecessary objects in the image and change the person's face.https://t.co/iHccQXmqEhPic.twitter.com/zhmVjIajcy

- Satoshi Iizuka (@ stsiizk)April 30, 2017

If you look at the movie published on the research presentation page, you can understand the magnanimity of the image complementation technology "Globally and Locally Consistent Image Completion (GLCIC)" with a single shot.

Globally and Locally Consistent Image Completion


In GLCIC, it is a major feature that combines two types of identification networks with a technology that supplements natural images with blanks.


In order to try out the performance of GLCIC, fill in the image with white and try to make "blank". This blank space is complemented automatically by GLCIC using neural network automatically.


When you click on the mouse, it looks like the original image is displayed. Actually this is an image automatically complemented by GLCIC.


When comparing the original image (left) and the complemented image (right) side by side, the part painted with white is subtly different. However, if you do not compare it in detail, you can see that it has a level of "reproducibility" that can not be interpreted as a complemented image.


Using the GLCIC image complement function, it is also possible to erase objects in the image,


Fill the man behind the fence with white ...


One click. The fence reappeared, but the man disappeared. Complementing images is supplementing ability of accuracy which is hard to believe.


If you use GLCIC's image complement function, you can change face.


Fill the eyes around ... ...


Complementing another eye looks like this. In GLCIC, natural images can be complemented by using two kinds of identification networks that evaluate consistency and local alignment in the entire image.


Fill the mouth ...


Complement with another mouth. It is possible to complement with different images so that size and placement appear most natural.


Filling the whole face and a larger area ...


You can swap the face altogether.


You can complement not only face but also hair style.


Fill the top of the head ...


How, the part changed. It is natural finish that I can not imagine replacing it with a different image in a moment.


Leave it blank to cover your nose and mouth ......


Make a different nose and mouth. There is no unnatural place in the expression.


Fill from the eye to the head ...


To another face. With GLCIC, you can freely change the image.


The GLCIC consists of three networks, "complementary network" which complements the image, "global identification network" which evaluates the image, "local identification network". When the complementary network, in which all layers are composed of a convolution network, outputs an image complementing the loss region, the global identification network evaluates the entire image and the complementary region respectively "comprehensively" the complementary region. Evaluation is fed back to the complementary network, and after complementing accuracy is improved, the global identification network / local identification network evaluates the output. By repeating the alternate learning, the complementary network learns the natural image complement.


The superbness of GLCIC is even more prominent when compared with the image supplementing technology announced so far. The input image (original image) with the blank entry at the top left is this2009,year 2012,2014,2016Complementing by using the method devised in GLCIC in the lower right and this method it is like this. You can see that the image of a boy who is sleeping with headphones can be restored (accurately guessed) by GLCIC including the form of headphones.


When using a perforated image of a cabin, only the GLCIC was able to accurately reproduce the entrance pillar.


Even in the image of a building built near the lake, only GLCIC can naturally reproduce the boundaries between buildings, trees, and the sky.


GLCIC with complementary performance which is not exaggeration to say that the level is different from the conventional image supplementing technology,Risa IizukaTechnology developed by Assistant Professor Team, Iizuka Assoc.Simosela · EdgarAssistant professor,Hiroshi IshikawaThe paper by the professorSIGGRAPH 2017Adopted. The paper and GLCIC source code are coming soon.

Incidentally, co-authors Edgar Associate Professor et al. Are also known for developing pencil software to automatically insert pencil rough images.

Waseda University's Laboratory Develops Awesome Neural Network Technology to Automatically Pen in Rough Pictures to Draw Lines - GIGAZINE