Sep 15, 2022 20:00:00

Summary of how to use model data “Waifu-Diffusion” specialized for drawing illustrations with image generation AI “Stable Diffusion”

“ Waifu-Diffusion ” is an image generation AI that has been tuned with a dataset of more than 4.9 million two-dimensional illustrations of the image generation AI “

Stable Diffusion, ” which was released to the public in August 2022. In order to run this Waifu-Diffusion in a local Windows environment, I actually downloaded the model data and installed it.

GitHub - harubaru/waifu-diffusion: stable diffusion finetuned on danbooru
https://github.com/harubaru/waifu-diffusion

◆Example of generation using Waifu Diffusion
In fact, I set the prompt, Sampling Steps, Sampling method, CFG Scale, and seed value exactly the same and generated images with Stable Diffusion (left) and Waifu-Diffusion (right). You can compare the images below by moving the slide bar in the center left and right.

The prompt is 'Hatsune Miku in anime style cooking in the beach, high quality, pixiv', Sampling Steps is 120, Sampling method is Euler a, CFG Scale is 7, Seed value is 793512668. For some reason, the shape of the hands of the one generated with Stable Diffusion is strange and the structure of the clothes is a bit disorganized, but the balance of the face and body of Waifu-Diffusion is quite normal. As per the prompt 'cooking in the beach', something that looks like cooking is drawn in front of you.

The prompt is 'girl with dark skin, summer blue sky, sharpness , makoto shinkai, mamoru hosoda, okama, ookami uo, 4k,8k, manga, moe, ponytail, pixiv ranking', Sampling Steps is 20, Sampling method is Euler a, CFG Scale is 7, seed value is 3081179311. In the image generated by Stable Diffusion, the lower half of the girl's face no longer resembles that of a human, but in the image generated by Waifu-Diffusion, the side profile is the same, but the composition is much more drawn out. You can see that her face has become much better.

As you can see from the example above, Waifu-Diffusion is an image generation AI that specializes in two-dimensional illustrations. This way, the illustrations will be finished at a level that can be viewed properly.

◆How to install Waifu-Diffusion
This time, we introduced Waifu-Diffusion model data to the ``AUTOMATIC1111 version Stable Diffusion web UI,'' which allows you to easily operate Stable Diffusion from the UI and is multifunctional. You can understand how to install AUTOMATIC1111 version Diffusion web UI by reading the article below.

Summary of how to install the definitive version 'Stable Diffusion web UI (AUTOMATIC 1111 version)' that allows you to easily run various functions such as image generation AI 'Stable Diffusion' on 4GB GPU & learning your own patterns on Google Colabo and Windows - GIGAZINE

In addition, the basic usage of the AUTOMATIC1111 version Stable Diffusion web UI is explained in the following article.

Basic usage of ``Stable Diffusion web UI (AUTOMATIC1111 version)'' that allows you to easily use ``GFPGAN'' that can clean the face that tends to collapse with image generation AI ``Stable Diffusion'' - GIGAZINE

◆Download model data
To download Waifu-Diffusion model data, access the following page.

waifu-diffusion/README.md at main · harubaru/waifu-diffusion · GitHub
https://github.com/harubaru/waifu-diffusion/blob/main/docs/en/weights/danbooru-7-09-2022/README.md

There are Google Drive, Torrent, HTTPS mirror, and HTTP mirror, but this time I downloaded from the fastest (Fastest) HTTPS mirror. Please note that the file size is approximately 7.2GB, which is quite large.

Save the downloaded model data 'wd-v1-2-full-ema.ckpt' in the 'models\Stable-diffusion' folder in the AUTOMATIC1111 version Stable Diffusion web UI folder.

◆Model data changes
Next, start the AUTOMATIC1111 version Stable Diffusion web UI and click the 'Settings' tab at the top.

You can change the usage model by changing the 'Stable Diffusion checkpoint' in the 'Stable Diffusion' item. The default is models.ckpt (Stable Diffusion v1.4), but change this to 'wd-v1-2-full-ema.ckpt'.

Click Apply Settings at the top of the settings to apply the settings. However, model changes will be reflected after restarting, so shut down the AUTOMATIC1111 version Stable Diffusion web UI once.

Now you are ready to introduce Waifu-Diffusion. Run 'webui-user.bat' to restart the AUTOMATIC1111 version Stable Diffusion web UI.

Access the URL displayed in the command prompt to display the UI. Then, just like Stable Diffusion, you can generate an image with Waifu-Diffusion by entering the prompt, setting various elements, and clicking 'Generate'.

According to harubaru, the creator of Waifu-Diffusion, Waifu-Diffusion is a collection of 4.9 million images collected on Danbooru , an overseas illustration community site, from May 24, 2005 to December 31, 2021. Training is being performed on the dataset “ Danbooru2021 ”, which consists of the above and its metadata and has a total of 4.9TB.

waifu-diffusion/dataset.md at main · harubaru/waifu-diffusion · GitHub
https://github.com/harubaru/waifu-diffusion/blob/main/docs/en/training/dataset.md#organizing-the-dataset

The tags used in Danbooru are registered in the metadata included in Danbooru2021, and are associated with tags based on work name, character name, and sexuality. Also, content that suggests sexual content or pornographic content is not prohibited, and the evaluation value of ``explicitness'' is included in the metadata.

·postscript
Waifu-Diffusion v1.3, which was in beta version, has been officially released.

Official Release Notes for Waifu Diffusion 1.3 · GitHub
https://gist.github.com/harubaru/f727cedacae336d1f7877c4bbe2196e1