Mar 01, 2023 17:00:00

``Scribble Diffusion'' that allows you to draw a picture in the image generation AI with the same composition as the illustration drawn on the browser appears

Stable Diffusion of image generation AI can generate images by entering text and issuing instructions, but it is quite difficult to generate the picture you want with only text. ' Scribble Diffusion ' can generate a picture close to your wishes with Stable Diffusion from graffiti drawn on the browser and text input, and you can try the operation demo from the browser.

Scribble Diffusion

https://scribblediffusion.com/

This Scribble Diffusion applies the neural network 'ControlNet' to additionally provide image composition and depth information as prompts. You can understand what kind of images can be generated by using ControlNet by reading the following article.

I tried a combination technique of ``ControlNet'' & ``Stable Diffusion'' that allows you to specify poses and compositions and quickly generate your favorite illustration images Review-GIGAZINE

When you access the Scribble Diffusion demo page , it looks like this. Draw an illustration on the square canvas, enter a text description in the input field below it, and click the 'Go' button.

The example below was entered by default, and it was an illustration and text showing the sun peeking through the mountains. depicted in Also, there was no depiction of an existence that looked like ``the sun that can be seen from between the mountains''.

Below is what I drew thinking what would happen if I drew the illustration in more detail. The text is simply 'A cat playing a guitar', and the illustration is drawn accordingly.

The generated image is below. Although the expression has become harsh, the picture is drawn according to the composition, and the picture output is almost the expected result.

This time 'Earon mask sprinkles sugar on the cake like

Salt Bay '

The result is like this. Elon Musk is very reproducible, but the cake is transformed into a mysterious light.

So, after writing a little more on the right hand and cake part of the illustration as shown below, I tried to output with the same text. Then, I was able to output a picture of Elon Mask that sprinkles sugar on something that looks like a cake.

The Scribble Diffusion code is published on GitHub and can be built on your own server.

GitHub - replicate/scribble-diffusion: Turn your rough sketch into a refined image using AI
https://github.com/replicate/scribble-diffusion