Adobe reveals a part of video editing by generative AI 'Firefly', it is possible to change color, generate sound effect, and automatically generate storyboard



Adobe's proprietary generative AI `` Firefly '' learns only with content whose copyright has expired and license-free content, and can automatically generate images and text effects. Adobe has introduced Firefly to Creative Cloud and revealed that it is under development so that it can be used for video editing and audio editing.

Reimagining Our Video and Audio Tools with Adobe Firefly | Adobe Blog

https://blog.adobe.com/en/publish/2023/04/17/reimagining-video-audio-adobe-firefly

Adobe has already introduced AI technology to the Creative Cloud. For example, 'Adobe Sensei' using machine learning realizes ' fill according to content ' in Photoshop and ' auto reframe ' in Premiere Pro.

Developers explain the secret of Adobe's AI 'Adobe Sensei' that realizes that 'image search has come this far' - GIGAZINE



Generative AI announced in March 2023 by Adobe, which has been developing AI and machine learning frameworks for more than 10 years, is `` Adobe Firefly ''. Firefly is trained on images from Adobe Stock, Adobe's photo stock service, as well as license-free and out-of-copyright public domain content, thus violating the intellectual property of other creators and brands A major feature is that it does not generate content that does.

Adobe announces its own generative AI 'Firefly', integration into Creative Cloud such as Photoshop and Illustrator - GIGAZINE



As for what you can actually do with Firefly, examples of usage in Photoshop and Illustrator were demonstrated at a briefing session by Mr. Shoichi Nishiyama, chief digital officer of Adobe, and the contents can be confirmed in the following article. increase.

People inside have shown me what they can do with AI 'Firefly', which will be able to generate text-based images and effects even in Adobe's Photoshop and Illustrator - GIGAZINE



And Adobe has released what will be possible if Firefly is introduced to Premiere Pro, a video editing software.

For example, when you want to add BGM to the video you are editing, you can enter text such as 'bright, adventurous, energetic music' and generate BGM.



You can also add sound effects generated by Firefly.



In addition, you can change the color of the screen by entering text.



It has been possible to change the color scheme of the screen so far, but instead of changing the actual numerical values by yourself, you can automatically change the atmosphere and color by simply entering text.



Furthermore, if you enter 'brighten your face', AI will automatically recognize your face.



I was able to change the brightness of only the face part.



Automatically generate captions for short movies



Automatically place captions in the movie as it is.



Enter 'iron that melts in 3D'.



The word MOLTEN was drawn with the effect of melting metal.



The B-roll function is to input a script in advance, create a sub-video (B-roll) that complements the main video, and add it to the editing.




In addition to B-roll, by reading the script, the storyboards are automatically created according to the content.




It will also be possible to create

a previs based on this storyboard.




At the time of writing the article, it seems that the application of Adobe Firefly for video editing is still under development, but Adobe wants to release it in the second half of 2023.

in Software, Posted by log1i_yk