DeNA announces "PSGAN" to generate animation characters' high-resolution images and attach them to animation using AI

DeNADeveloped a technique capable of generating a clear animated character image by using artificial intelligence (AI), and a moving image generating technology capable of making movement on the created character.

Develop technology to make infinite whole-body animated characters with AI | DeNA · Inc. 【DeNA】
http://dena.com/jp/article/2018/05/11/003622/index.html

Full-body high-resolution Anime Generation with Progressive Structure-conditional Generative Adversalial Networks | DeNA Co., Ltd.
https://dena.com/intl/anime-generation/

Generative Adversarial Network(GAN) has made it possible to generate high-resolution images of some parts such as faces and hands using AI, but in existing methods it is possible to create "character whole body" etc. important for industrial applications etc. It was not possible to generate. Of course, GANs capable of generating images based on poses and faces have also been proposed, but they seem to have had the problem that the image quality is insufficient for commercial use. A new framework "PSGAN (Progressive Structure - Conditional Generative Adversarial Networks)" which can solve such constraints, generate high - resolution images of the whole body of animated characters based on pose information using AI, ) "Is proposed by DeNA.

The high resolution image (512 × 512 pixels) by PSGAN and the process of moving image generation are as follows.

First of all, many animated characters (character 1) are generated from latent variables using PSGAN. By interpolating the latent value corresponding to the generated animation character with PSGAN, a new animated character (character 2) is generated. This makes it possible to generate whole body images of various looking characters.


You can see how PSGAN actually generates various animated characters in the following movies.

Full-body anime generation at 512x512 with Progressive Structure-conditional GANs - YouTube


At this point, only the animation character's whole body high resolution image is generated, there is only a single pose. By modifying the latent variable and giving the continuous pause sequence to the PSGAN, it seems that animation of the generated animated character can be generated. Specifically, we will map the representation of the designated animated character to the latent variable of latent space which is the input vector of PSGAN.


If you watch the following movie, you can see what happens when you animate PSGAN to the whole body image of an animated character that actually has only a single pose.

Adding action to full-body anime characters with Progressive Structure-conditional GANs - YouTube