Image generation AI 'Stable Diffusion' AUTOMATIC 1111 version extension 'Deforum' that can automatically generate animations from prompts and spells



The image generation AI '

Stable Diffusion ' can automatically generate images from input sentences called prompts or spells. ' Deforum ', which can generate animations using images generated by Stable Diffusion as frames, can now be added as an extension of the AUTOMATIC 1111 version Stable Diffusion web UI, which can be said to be the definitive version of web UI, so you can actually generate animations. I tried to.

deforum/deforum_stable_diffusion – Run with an API on Replicate
https://replicate.com/deforum/deforum_stable_diffusion

GitHub - deforum-art/deforum-for-automatic1111-webui: Deforum extension script for AUTOMATIC1111's Stable Diffusion webui
https://github.com/deforum-art/deforum-for-automatic1111-webui

In order to use Deforum's extensions, you need to update the AUTOMATIC1111 version of Stable Diffusion webUI to a version after October 22, 2022. The following article summarizes how to install and update AUTOMATIC1111 version Stable Diffusion webUI.

Summary of how to install the definitive version 'Stable Diffusion web UI (AUTOMATIC 1111 version)' that allows you to easily run various functions such as image generation AI 'Stable Diffusion' on 4GB GPU & learning your own patterns on Google Colabo and Windows - GIGAZINE



In addition, you can understand the basic usage of the AUTOMATIC1111 version Stable Diffusion webUI by reading the article below.

Basic usage of ``Stable Diffusion web UI (AUTOMATIC1111 version)'' that allows you to easily use ``GFPGAN'' that can clean the face that tends to collapse with image generation AI ``Stable Diffusion'' - GIGAZINE



First, right-click the Stable Diffusion web UI folder 'stable-diffusion-webui' and select 'Git Bash Here'.



A Git command prompt will open, so enter the code below and run it.
[code]git clone https://github.com/deforum-art/deforum-for-automatic1111-webui/ extensions/deforum[/code]



When you run the code, the Deforum repository will be cloned to 'stable-diffusion-webui/ectensions' and a 'deforum' folder will be generated.



Create a new “Deforum” folder in stable-diffusion-webui/models.



To create a 3D movie with Deforum, you need to save the following two files in 'stable-diffusion-webui/models/Deforum'.

・(Direct link: 1.28GB)

https://github.com/intel-isl/DPT/releases/download/1_0/dpt_large-midas-2f21e586.pt
・(Direct link: 896MB) https://cloudflare-ipfs.com/ipfs/Qmd2mMnDLWePKmgfS8m6ntAg4nhV5VkUyAydYBp8cWWeB7/AdaBins_nyu.pt

When saved to stable-diffusion-webui/models/Deforum it looks like this.



When you start the AUTOMATIC1111 version Stable Diffusion webUI, a tab called 'Deforum' appears, so click it. The left part is the settings panel, and the right part is the panel that displays the generated results.



In the settings panel, select the 'Keyframes' tab to change the settings for the generated animation. You can set 'animation_mode' to 2D or 3D, 'max_frames' to the maximum

frame rate , and 'border' to set how the frames are connected.



In the 'Prompt' field, you can enter the prompt for the animation to be generated. Unlike the prompts you typically enter in Stable Diffusion, you need to use numbers to indicate when to display the generated image.



The following Google document explains how to input the prompt.

Quick Guide to Deforumv05 - Google Docs

https://docs.google.com/document/d/1pEobUknMFMkn8F5TMsv8qRzamXX_75BShMMXV8IFslI/

An example has been entered in the prompt field from the beginning, so this time just click 'Generate' to generate a 12fps 2D animation.



It took about 20 minutes to generate on a PC equipped with an NVIDIA graphics board with 6GB of VRAM.



The resulting animation looks like this. The story is about a car driving on a snowy road that transforms into a fruit-like silhouette, and eventually a durian appears.

Image generation AI 'Stable Diffusion' Animation automatically generated with the Deforum extension of the AUTOMATIC version web UI - YouTube


The generated animation and frame images are output to the 'Deforum' folder generated in 'stable-diffusion-webui/outputs/img2img-images'.



In the case of 2D animation, it was output in GIF format.



This time we generated a 2D animation, but in the case of 3D animation, a movie that expresses depth and is drawn three-dimensionally is generated. However, according to the ReadMe on GitHub, if you want to generate 3D animation, you need at least 6.4GB of VRAM, or at least 3.8GB. On Twitter, various works generated with Deforum are tweeted, and if you look at the works below, you can understand what the 3D animations generated with Deforum are like.

The Halloween theme includes pumpkins, spooky buildings, and trees with scary faces.



A movie where Saitama from the anime ' One Punch Man ' appears with rabbit ears.



Taking advantage of the characteristics of 3D, there was also a movie released that made it look like you were walking through the forest.

Related Posts:

in Review,   Web Service,   Web Application,   Hardware,   Video,   Anime,   Art,   , Posted by log1i_yk