"PixelNN" that can generate high resolution images even from low resolution images

One of the pixel interpolation methods for low resolution images / pictures that do not know what the original was likeNearest neighbor methodAnd the use of a neural network, "PixelNN" technology "reproduces" with high resolution, Carnegie Mellon UniversityAayush BansalMr. et al. Developed it.

PixelNN
http://www.cs.cmu.edu/~aayushb/pixelNN/


PixelNN: Example-based Image Synthesis
(PDF file)https://arxiv.org/pdf/1708.05349.pdf

In the site and PDF, it is shown what kind of image that can actually be generated with this technology, you can understand its terribleness.

For example, this is a high resolution photo (right) generated based on a low resolution photo (left).


This isNormal mappingI am generating a figure of a dog based on.


Cats generated from contour extraction.


Multiple output examples. In the example of the top woman, it seems that there are as many as 4 different women.


The left end is normal mapping, the second from the left is the intermediate product, the third from the left is an example. The right end is a picture that became an original, so you can see how much you can "reproduce" by comparing the right two.


Aayush Bansal and his colleagues have realized reproduction to a more original form by grasping the contents of frequencies in the image by frequency analysis using Fast Fourier Transform. The following shows the frequency spectra of the four images "input", "intermediate product", "output" and "original" respectively.


"Edges 2 cats" in which figures drawn appropriately are automatically converted to "cats"Ya"The fotogenerator" where hand-painted line drawings are strangely transformed into real and terrible illustrationsThere is a prior art of this, but if this around gathers, if a certain amount of "original picture" can be drawn, the time will come when technology will somehow come. Considering from the early stage of progress, I feel like a future that is not far.