'Paints-Undo' allows you to create a production process movie as if you were taking a time-lapse picture of a hand-drawn illustration by inputting an illustration
An AI called ' Paints-Undo ' has been developed that can output a time-lapse movie of the production flow of an illustration just by inputting the illustration. The developer is
PaintsUndo: A Base Model of Drawing Behaviors in Digital Paintings
https://lllyasviel.github.io/pages/paints_undo/
Paints-Undo is an AI model that can receive an image as input and output a drawing sequence of that image, and can output a 'movie that looks like an illustration is being drawn by trial and error by drawing and undoing lines. The name Paints-Undo comes from the 'undo' function that is often used when creating illustrations.
llyasviel introduces examples of multiple inputs and outputs on the Paints-Undo demo page . The images used for input on the demo page include both original people and copyrighted characters, but all are AI-generated images.
Below is an AI-generated illustration used in the Paints-Undo demo.
Below is the movie that results when this illustration is entered into Paints-Undo.
There are several other demos, some of which start with a fairly rough rough sketch. The left image in the following image is the input illustration, and the right is a part of the movie output by Paints-Undo.
In actual illustration production, many people create a rough line drawing and then do not change the appearance significantly with subsequent coloring, but in the case of Paints-Undo, the illustration is far from the finished product even when coloring begins. Since the process of 'changing the outline while coloring' is different from the general work process by humans, it is not 'totally indistinguishable from a human'.
In the end, the lines and colors were close to the original illustration. In the demo below, the mouth was different from the original illustration until the end, but the mouth was corrected at the end of the output movie to match the original illustration.
With Paints-Undo, you can also output the production process by specifying how rough your initial sketch is.
The images that can be input are not limited to digital illustration-style images; it also appears possible to use images with a comic book feel, game characters, or monochrome watercolor painting style images.
On the other hand, there are some images that are difficult to output, such as those that create realistic illustrations from photographs, contain many people or complex backgrounds, or have special situations such as splashes of water. In addition, a time-lapse-style movie of the production process is output, but it may contain scenes such as 'It is impossible to actually redo this here at this timing.'
Paints-Undo combines two models, 'paints_undo_single_frame' and 'paints_undo_multi_frame'. The single-frame model is an AI model that assumes that 'a work is always completed in 1000 operations' and divides the process from a white canvas to a completed work into 1000 steps. The Paints-Undo time-lapse movie is a simulation of how many times 'Ctrl + Z (undo)' is included in this. In addition, the multi-frame model takes two images as input and outputs 16 intermediate frames between them, and outputs them as a movie in a way that complements the 'keyframes' obtained by the single-frame model.
Depending on the environment and settings used, it takes about 5 to 10 minutes to convert one image into a 25-second movie. Llyasviel also said that the demo was tested on NVIDIA 's GeForce RTX 4090 and GeForce RTX 3090 Ti with 24GB VRAM, and that it may work with 16GB VRAM, but it will not work with 8GB, so be careful.
More information about Paints-UNDO is available on GitHub.
GitHub - lllyasviel/Paints-UNDO: Understand Human Behavior to Align True Needs
https://github.com/lllyasviel/Paints-UNDO
Related Posts:
in Software, Posted by log1e_dh