Stability AI releases “Stable Video Diffusion”, an AI that generates videos from text and images



Stability AI, which develops the image generation AI 'Stable Diffusion,' has released ' Stable Video Diffusion, ' a latent video diffusion model that can generate high-resolution videos from text and images.

Introducing Stable Video Diffusion — Stability AI Japan

https://ja.stability.ai/blog/stable-video-diffusion



Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets — Stability AI
https://stability.ai/research/stable-video-diffusion-scaling-latent-video-diffusion-models-to-large-datasets



Stable Video Diffusion is open as a research preview and the source code is publicly available in a GitHub repository.

GitHub - Stability-AI/generative-models: Generative Models by Stability AI
https://github.com/Stability-AI/generative-models

You can also check the weights required to run the model locally on HuggingFace.

stabilityai/stable-video-diffusion-img2vid-xt · Hugging Face
https://huggingface.co/stabilityai/stable-video-diffusion-img2vid-xt

Stable Video Diffusion is released in two Image to Video models that can generate 14 frames and 25 frames, allowing you to generate videos at customizable frame rates from 3fps to 30fps.

If you enter 'Ice dragon in the mountains', the exact animation will be generated.



'Astronaut walking on the moon'



'Two blue jays on the top of building'



Stability AI has published the bar graph below as a result of comparing user evaluations of video quality (vertical axis) with Runway Research's

GEN-2 and pika.art's PikaLabs . In the case of Stable Video Diffusion (purple) that can generate 14 frames, it looks like this.



The case of Stable Video Diffusion XT (purple), which can generate 25 frames, is shown below.



Stability AI said, 'We are pleased to add Stable Video Diffusion to our diverse range of models. Stability AI's portfolio, which spans image, language, audio, 3D, code, and other modalities, harnesses the power of human imagination to the fullest. It's a testament to Stability AI's mission to transform

◆Forum now open
A forum related to this article has been set up on the GIGAZINE official Discord server . Anyone can write freely, so please feel free to comment! If you do not have a Discord account, please create one by referring to the article explaining how to create an account!

• Discord | 'Do you think video generation AI will become as popular as image generation AI?' | GIGAZINE
https://discord.com/channels/1037961069903216680/1176823847232741376

in Software, Posted by log1i_yk