Introducing the Ukiyo-e generation AI 'Evo-Ukiyoe' and the Ukiyo-e colorization AI 'Evo-Nishikie'
Sakana AI, a Japanese AI company, has announced two AI models: 'Evo-Ukiyoe,' which generates ukiyo-e, one of Japan's most representative art forms, and 'Evo-Nishikie,' which colors ukiyo-e.
AI that learned from Japanese beauty: Ukiyo-e style image generation model Evo-Ukiyoe and Ukiyo-e colorization model Evo-Nishikie released
Ukiyo-e is well known around the world and is known to have influenced many great artists such as Monet and Van Gogh. However, existing image generation models have not been trained with a focus on ukiyo-e, and many people have tried to get image generation AI to output ukiyo-e, but have been unable to generate images that look like ukiyo-e.
Sakana AI's new AI models, Evo-Ukiyoe and Evo-Nishikie, are based on the company's Japanese-language-compatible image generation model 'Evo-SDXL-JP,' and are trained on 24,038 digital images of ukiyo-e prints housed at the Ritsumeikan University Art Research Center.
To develop Evo-Ukiyoe, Sakana AI first generated captions for ukiyo-e images using a technique called few-shot prompting , which teaches the model by giving concrete examples. It then trained the base model, Evo-SDXL-JP, using pairs of image data and captions as a learning dataset, and fine-tuned it using LoRA.
Evo-Ukiyoe, which was created in this way, can generate images that are similar to ukiyo-e by using prompts such as cherry blossoms, Mt. Fuji, kimonos, and birds that are often featured in ukiyo-e. On the other hand, if prompts are used for things that did not exist in the Edo period, such as computers or hamburgers, the images generated may not look like ukiyo-e.
For example, if you use the prompt 'There are plants and flowers. Butterflies are flying. Top quality spoke ukiyo-e.', the result will be as follows. Note that the phrase 'spoke ukiyo-e' is a special token used during learning, that is, a token that has been trained differently from normal words in order to give it a specific function.
If you use the prompt 'A crane is standing in the garden. It's snowing. The highest quality Japanese woodblock print.' you get something like this.
And below is the output from the prompt 'A cat wearing a kimono drinking tea in a garden. A top quality ukiyo-e print.'
Another generative AI, Evo-Nishikie, is based on Evo-Ukiyoe and is a model that uses condition images created by removing noise such as insect holes and blemishes from ukiyo-e images as a training dataset. It can turn monochrome ukiyo-e such as sumizuri-e into multi-colored ukiyo-e, also known as nishikie.
Below is a monochrome ukiyo-e print (left) from the Ritsumeikan University Art Research Center collection, which has been colorized using Evo-Nishikie (center and right).
Ryo Akama, director of Ritsumeikan University's Art Research Center, said, 'There are contemporary artists all over the world who create works based on ukiyo-e. The ukiyo-e created by Evo-Ukiyoe will not only inspire these artists' creative desire, but will also provide an opportunity for general enthusiasts to explore more deeply the meaning of authentic ukiyo-e as classics.'
In addition, one of the developers of Evo-Ukiyoe and Evo-Nishikie, Sakana AI's Karanuwat Tarin, is a person who has developed the model 'RURI' used in the AI cursive character recognition application 'Mio'.
I tried using the AI cursive character recognition app 'miwo' - GIGAZINE
Related Posts:
