Summary of how to use 'Aesthetic Gradients' which can learn and apply illustration style and pattern with image generation AI 'Stable Diffusion' AUTOMATIC1111 version



Image generation AI '

Stable Diffusion ' can be easily used on a browser The most vigorously developed web UI has been developed, and it can now be said to be the definitive Stable Diffusion GUI. The `` Aesthetic Gradients'' function has been added to the ``Aesthetic Gradients '' feature, which allows you to set specific directions, so I actually tried using it.

[2209.12330] Personalizing Text-to-Image Generation via Aesthetic Gradients
https://arxiv.org/abs/2209.12330

Extensions · AUTOMATIC1111/stable-diffusion-webui Wiki · GitHub
https://github.com/AUTOMATIC1111/stable-diffusion-webui/wiki/Extensions#aesthetic-gradients

GitHub - AUTOMATIC1111/stable-diffusion-webui-aesthetic-gradients: Aesthetic gradients extension for web ui
https://github.com/AUTOMATIC1111/stable-diffusion-webui-aesthetic-gradients

The following article summarizes how to install AUTOMATIC1111 version Stable Diffusion webUI.

Summary of how to install the definitive version 'Stable Diffusion web UI (AUTOMATIC 1111 version)' that allows you to easily run various functions such as image generation AI 'Stable Diffusion' on 4GB GPU & learning your own patterns on Google Colabo and Windows - GIGAZINE



In addition, you can understand the basic usage of the AUTOMATIC1111 version Stable Diffusion webUI by reading the article below.

Basic usage of ``Stable Diffusion web UI (AUTOMATIC1111 version)'' that allows you to easily use ``GFPGAN'' that can clean the face that tends to collapse with image generation AI ``Stable Diffusion'' - GIGAZINE



Aesthetic Gradients is a function that adds orientation using pre-prepared image data to the ``embedding'' process, which converts text input in Stable Diffusion into vector expression and provides orientation for image generation. By performing Aesthetic Gradients with data composed of pictures you have drawn, it is possible to make the pattern of the generated image closer to your own pattern.

First, access the AUTOMATIC1111 version Stable Diffusion webUI directory with Git and execute the code below.
[code]git clone https://github.com/AUTOMATIC1111/stable-diffusion-webui-aesthetic-gradients extensions/aesthetic-gradients[/code]



When you run the code, the ``aesthetic-gradients'' repository will be cloned into the ``stable diffusion webui/extensions/'' folder. Aesthetic Gradients are not implemented in the AUTOMATIC1111 version Stable Diffusion webUI, but are added as extensions.



To direct image generation, embedding data used for Aesthetic Gradients is required. This embedding data is published by Víctor Gallego, author of the Aesthetic Gradients paper, on the GitHub page below.

stable-diffusion-aesthetic-gradients/aesthetic_embeddings at main · vicgalle/stable-diffusion-aesthetic-gradients · GitHub
https://github.com/vicgalle/stable-diffusion-aesthetic-gradients/tree/main/aesthetic_embeddings

Access the above site and click on the embedding data you want to download.



Click 'Download'.



Save the embedding data in the 'stable diffusion webui/extensions/aesthetic-gradient/aesthetic_embeddings' folder. This time I saved aivazovsky.pt, flower_plant.pt, laion_7plus.pt, and sac_8plus.pt.



Start the AUTOMATIC1111 version Stable Diffusion webUI. There is a section called 'Open for Clip Aesthetic!' at the bottom of the settings panel, so click it.



The Aesthetic Gradients settings panel will then expand.



Aesthetic Gradients mainly has the following four setting items.

・Aesthetic Weight : Weight per step of Aesthetic Gradients. The higher you set it, the stronger the embedding effect will be.
・Aesthetic Steps : Number of steps for Aesthetic Gradients. The more you increase the number, the stronger the influence of the embedding will be.
・Aesthetic learning rate : Learning rate of Aesthetic Gradients. According to Gallego, the default value of 0.0001 works well and there is no need to change it in most cases.
・Aesthetic img embedding : Select data for embedding.



I checked how much the actual use of Aesthetic Gradients would affect the generated images. The following is 'Portrait of a full body of beautiful young female detective, d & d, sleeveless turtleneck, pleated skirt, fantasy, flat lighting, intricate, highly detailed, digital painting, artstation, concept art, smooth, sharp focus, illustration, misa amane, art by simon bisley and greg rutkowski and alphonse mucha, natural tpose' prompt, set the resolution to 512 x 512 pixels, the number of generation steps to 73, the generation sampler to Euler a, the CFG scale to 7, and the seed value to 8923. This is an image generated by



The following is the result of running Aesthetic Gradients with the settings at the time of generation as they are, Aesthetic Weight 0.9, Aesthetic Steps 3, Embedding data aivazovsky.pt. aivazovsky.pt is composed of works by

Ivan Aivazovsky , a painter who painted many landscape paintings with sea motifs, and if you look closely, you can see something like a ship model in the woman's left hand.



flower_plant.pt is data consisting of over 2000 images of plants and flowers. A meadow appears in the background of the woman, with wild flowers and trees.



laion_7plus.pt uses the embedding data used in

LAION-Aesthetics_Predictor V1 . For some reason, the woman's hairstyle is swollen like a mushroom, but the clothes and background are more detailed.



sac_8plus.pt is based on the embedding data used in

Simulacra Aesthetic Captions . The design has changed completely, and the design now looks a lot like Japanese manga or anime.



Below is the prompt 'empty sidewalk of a cyberpunk megacity, dramatic lighting, detailed background, gorgeous view, realistic, high detail, depth of field, lightrays, atmospheric, digital art', with the number of generation steps set to 80 and the generation sampler set to Euler a , generated with CFG scale set to 7, seed value set to 20221031, and batch size set to 4. I don't use Aesthetic Gradients. It depicts a background that looks like something out of a science fiction movie and people walking through it.



When I run Aesthetic Gradients with the same settings, Aesthetic Weight 0.9, Aesthetic Steps 15, Embedding data aivazovsky.pt, it looks like this. Perhaps because I set the Aesthetic Steps to a fairly high setting, I was greatly influenced by Aivazovsky's work, and it turned into a painting depicting the sea. One of the four pieces is now in a frame, making it look like a painting that could be hung in a museum.



In flower_plant.pt, plants are depicted in the picture, but the buildings are also drawn in the foreground compared to the original image, giving it a more civilized look.



In laion_7plus.pt, not only the buildings and plants were added, but the original style of drawing changed significantly. It is very different from the original image that was generated with the cyberpunk style, and it has a somewhat fantasy-like atmosphere.



On the other hand, sac_8plus.pt is a generated image that is quite close to the original generated image, but instead of a devastated land, the background depicts a rather civilized and futuristic cityscape.



In addition, you can create the embedding data using images you have prepared yourself. Click the 'Create aesthetic embedding' tab, enter the name of the data in 'Name', the directory containing the image in Source Directory, and click 'Create Images embedding'.



However, according to the explanation video below by Australian engineer koiboi , it is necessary to prepare several thousand images for embedding data in order to obtain a sufficient effect.

How to use Aesthetic Gradients: Stable Diffusion Tutorial - YouTube

Related Posts:

in Review,   Software,   Web Service,   Web Application,   Video,   Art,   , Posted by log1i_yk