Aug 28, 2020 19:00:00

It is possible to extract only the costume part from the animation image and automatically generate the cosplay costume image

Doshisha University and

ZOZO Technologies, which is responsible for the development of ZOZOTOWN, a fashion online shopping site, are jointly conducting research to generate costume images for cosplay from anime character images.

[2008.11479] Anime-to-Real Clothing: Cosplay Costume Generation via Image-to-Image Translation
https://arxiv.org/abs/2008.11479

Anime-to-Real Clothing: Cosplay Costume Generation viaImage-to-Image Translation
(PDF) https://arxiv.org/pdf/2008.11479.pdf

Ryosuke Goto , one of the authors of the research paper, explains on his Twitter account that he is 'proposing a task to generate cosplay costume images from images of anime characters' about the paper.

This is a joint research between Doshisha University and ZOZO Technologies. We propose a task to generate a cosplay costume image from an image of an anime character. https://t.co/1XZTlVota5

— AlphaGoto (@Ryosuke0624) August 27, 2020

``Cosplay was born from a gathering of fans and has grown into a billion dollar global dress market. In this paper, we will show the actual cosplay costume from animated images. 'We are proposing an 'Automatic costume image generation method for converting animation images into costume images' to expand the imagination for creating it.'

Since costumes for anime and cartoon characters reproduced in cosplay have a great variety, it seems difficult to convert only the costume part of an anime image into an image with a conventional image generation algorithm such as pix2pix . In order to solve this problem, in the same research, images were collected on the web and a data set prepared with multiple 'pairs of anime images and cosplay costume images' was created.

Next, we created a new architecture for Adversarial

Generation Networks (GANs) to facilitate the generation of high quality cosplay costume images. This GAN makes some improvements based on pix2pix to bridge the gap between the two domains (anime and reality) and improve the overall and local consistency of the generated images. It is said that. Specifically, the domain discriminator is improved by adding new features such as true/false discriminator, feature matching loss, and input consistency loss to pix2pix.

The graph below shows the results of comparing the cosplay costume image automatic generation algorithm created by the research team with other image generation algorithms such as pix2pix. 'FID' is an index for evaluating the quality of generated images, and the lower the number, the better the result. 'LPIPS' is an index for evaluating the variety of generated images. The higher the number, the greater the diversity. Is large. Numerically, the research team's automatic cosplay costume image generation algorithm is outstanding and the quality of the generated images is high.

The following is a summary of what costume images each image generation algorithm generated from 'Anime images (Input)'. 'Ours' is the image output by the cosplay costume image automatic generation algorithm created by the research team, and it can be visually seen that compared to other image generation algorithms, it is clearly possible to output highly accurate images.