Adobe releases beta version of video generation AI 'Firefly Video Model', Premiere Pro video generation function to be released in stages
At
Adobe Launches Firefly Video Model and Enhances Image, Vector and Design Models
https://news.adobe.com/news/2024/10/101424-adobe-launches-firefly-video-model
Regarding the Firefly Video Model, it was announced in September 2024 that 'a beta version will be released in the second half of 2024.' You can find out what kind of model the Firefly Video Model is by reading the following article.
Adobe Firefly introduces 'Firefly Video' model with video generation function, beta version available in late 2024 - GIGAZINE
Adobe has not yet announced a general release date for the Firefly Video Model, but at the time of writing, the web app version of Adobe Firefly is currently offering beta versions of 'Text-to-Video' and 'Image-to-Video,' with access only available to users on the waitlist.
With Text-to-Video, users can generate videos simply by entering a text description of what they want to generate. The app can also emulate a variety of camera styles, including cinematic, 3D animation, and stop motion, and the generated clips can be further adjusted by selecting camera controls that simulate camera angles, movements, shooting distances, etc.
'Image-to-Video' goes a step further by allowing users to add reference images alongside a text prompt, giving them more control over the outcome. Adobe has hinted that this could be used to create B-roll from images and photos, or even upload stills from existing footage to help visualize reshoots.
Below is an example of a video input to 'Image-to-Video.' It shows a scene where a machine with a yellow cable plugged in is turned on.
And the image generated based on this input image is shown below, and instead of turning off the switch, the yellow cable is pulled out. However, if you look closely, there are some unnatural points such as the yellow cable behind the pulled cable shaking and the blurred background gradually deforming, so it seems that the accuracy is not enough to completely replace it.
At the time of writing, the maximum length of the video that can be generated is 5 seconds, with a maximum image quality of 720p and 24fps. The generation itself takes about 90 seconds, and Adobe says it is working on developing a 'turbo mode' to shorten this generation time.
Adobe also announced that it will gradually install the AI in its video editing software Premiere Pro. Adobe announced that Premiere Pro will have new functions using generative AI in April 2024, which will enable functions such as 'adding and deleting objects in a video,' 'adding frames,' and 'generating video materials.'
Adobe Premiere Pro will be equipped with AI-generated features to 'add and remove objects in video,' 'add frames,' and 'generate video materials' in 2024 - GIGAZINE
Premiere Pro will first release Generative Extend, a feature that allows you to extend 720p or 1080p 24fps footage for about two seconds, making it possible to generate the beginning and end of the clip using AI.
Adobe says that all of its generative AI models are commercially safe because they are trained on content that is licensed for learning use. Adobe also says that the generated video content can be embedded with 'Content Credentials' that indicate the creator's information, social media links, and whether or not it was generated by AI, and that the use of AI and ownership can be disclosed by using the web app Adobe Content Authenticity.
Adobe launches 'Adobe Content Authenticity' web app that lets users apply attribution to digital content and opt out of AI training - GIGAZINE
'Firefly has become a widespread presence in Adobe's creative applications, and it's inspiring to see how the creative community is using it to push the boundaries of what's possible,' said Ellie Greenfield, chief technology officer, digital media, Adobe. 'We're excited to give creative professionals even more tools for ideation and creation that are commercially safe by design.'
Related Posts:
