Movie generation AI 'Text2Video-Zero' that can be used for free has appeared, so I actually used it to make a quick movie

On March 25, 2023, a demo app for movie generation AI ' Text2Video-Zero ' was released. If you use Text2Video-Zero, you can quickly generate a movie for several seconds just by entering sentences, so I tried to see what kind of movie can actually be generated.

Text2Video-Zero - a Hugging Face Space by PAIR

To generate a movie, enter 'movie description (prompt)' in the entry field labeled 'Prompt' and click 'Run'. This time, I entered 'A cat is playing baseball.' and clicked 'Run'.

Click 'Run' to start generating the movie, so wait for a while.

After waiting for a while, the movie will be displayed on the right side of the screen. The movie can be played by clicking the play button in the lower left. It took about 13 minutes to generate the movie this time.

Below is a 4-second movie actually generated by entering the prompt 'A cat is playing baseball.' A place like a baseball field and an object like a ball are reflected with cats.

``Movie where cats play baseball'' created with movie generation AI ``Text2Video-Zero''-YouTube


To save the generated movie, select 'Save movie as' from the right-click menu OK.

Text2Video-Zero's demo app includes prompts like 'an astronaut waving the arm on the moon' and 'a sloth surfing on a wakeboard' for example usage. There are multiple sets of generated results and you can click on each prompt to browse the generated results.

In addition to the function 'input text and generate movie', Text2Video-Zero also has the function 'input text and reference movie and generate movie'.

In addition, there is also a function called 'Generate a movie by entering sentences and reference poses'.

It is also possible to operate 'generate movie by entering sentences and outline data'.

In addition, it is also possible to generate a movie with your favorite pattern using the additionally learned DreamBooth model and contour data.

The files required to run Text2Video-Zero are published at the link below. However, a GPU with 12GB or more of video memory is required to run Text2Video-Zero locally.

GitHub - Picsart-AI-Research/Text2Video-Zero: Text-to-Image Diffusion Models are Zero-Shot Video Generators
https://github.com/Picsart-AI-Research/Text2Video-Zero