Sep 22, 2022 06:00:00

Introduction and usage summary of `` txt2mask for Stable Diffusion '' that automatically creates a mask that partially corrects the image from the character string with the image generation AI `` Stable Diffusion ''

An add-on for '

AUTOMATIC 1111 version Stable Diffusion web UI ' that allows you to easily operate the image generation AI ' Stable Diffusion ' from a web browser. ' txt2mask for Stable Diffusion ' creates a mask for correction. By introducing this add-on, you no longer need to select an image with the brush tool and create a mask.

GitHub - ThereforeGames/txt2mask: Automatically create masks for Stable Diffusion inpainting using natural language.
https://github.com/ThereforeGames/txt2mask

The following article summarizes how to introduce the AUTOMATIC1111 version of Stable Diffusion web UI to the local environment.

Image generation AI ``Stable Diffusion'' works even with 4 GB GPU & various functions such as learning your own pattern can be easily operated on Google Colabo or Windows Definitive edition ``Stable Diffusion web UI (AUTOMATIC 1111 version)'' installation method summary - GIGAZINE

Also, you can understand the basic usage of the AUTOMATIC 1111 version Stable Diffusion web UI by reading the following article.

Basic usage of ``Stable Diffusion web UI (AUTOMATIC 1111 version)'' that can easily use ``GFPGAN'' that can clean the face that tends to collapse with image generation AI ``Stable Diffusion''-GIGAZINE

And the basic usage of 'img2img' to generate images from images is as follows.

Simple usage summary of `` img2img '' that can automatically generate images with composition and color similar to the original image with image generation AI `` Stable Diffusion web UI (AUTOMATIC 1111 version) '' and change only the specified part - GIGAZINE

To install txt2mask for Stable Diffusion, simply clone or download the repository hosted on GitHub and place it in the base directory of the Automatic1111 version of the Stable Diffusion web UI. Click 'Code' on the above GitHub page and select 'Download ZIP'. Download the txt2mask for Stable Diffusion package in ZIP format. The file size is about 1.13MB.

Unzip the downloaded ZIP file with free software such as Explzh or the standard function of Windows, and overwrite the contents in the AUTOMATIC 1111 version Stable Diffusion web UI folder.

The image I want to rewrite this time is Steve Jobs generated by Stable Diffusion. The prompt is 'A beautiful oil painting portrait of steve jobs wearing medieval clothes and holding an iPhone by rembrandt and dino valls and geaorge stubbs and greg rutkowski' with 30 steps, a sampler of Euler a, a CFG scale of 7 and a seed value of It is 3167673378.

Run the Automatic1111 version Stable Diffusion web UI, select the 'img2img' tab, and load the above image.

Select 'txt2mask v 0.0.5' from the 'Script' pull-down.

This time I will change the black clothes that Jobs is wearing to Spider-Man's clothes. First, specify 'black suits' in 'Mask prompt' of txt2mask v 0.0.5. 'Mask precision' indicates the selection precision of the mask, and 'Mask padding' indicates how much of the masked part is filled with the specified part. Also, if you check 'Show mask in output?', the image of the alpha value of the automatically applied mask will be output together.

Then, specify 'spiderman suits' in the prompt field above 'Image for img2img' and click 'Generate'.

Then, the specified part of the image was automatically rewritten like this.

This is the image after rewriting. The part other than the neck and the iPhone is certainly a Spider-Man suit. However, since the hand part has come off the mask, both hands in the previous image remain as they are.

The image of the alpha value of the automatically masked part looks like this. Certainly, by using txt2mask for Stable Diffusion, the part you want to mask with only the character string is automatically selected, and you can save a lot of trouble of tracing with the brush tool.