People inside have shown me what they can do with AI 'Firefly', which will be able to generate text-based images and effects even in Adobe's Photoshop and Illustrator
On March 21, 2023, Adobe announced its own generative AI '
Adobe announces its own generative AI 'Firefly', integration into Creative Cloud such as Photoshop and Illustrator - GIGAZINE
Firefly has released a private beta version along with the announcement, but the demonstration using this beta version will be shown by Shoichi Nishiyama, Chief Digital Officer (CDO) of Adobe, media I participated in an online information session for
After all, Adobe's commitment to AI innovation has been going on for over a decade. In recent years, it has released AI tools like Adobe Sensei , and Adobe already offers hundreds of intelligent functions. Adobe's stance on AI tool development is to develop in accordance with AI ethical principles based on accountability, social responsibility, and transparency, and naturally Firefly is also developed along these AI ethical principles. . In addition, Adobe's AI tools 'assist creators to become more confident in their own creativity as a 'co-pilot' that removes barriers from a blank slate to the realization of an image.' Nishiyama emphasizes that it is not a tool designed to be a replacement.
Firefly is the latest generative AI developed by Adobe. Firefly seems to be a particularly important point that it is designed to be able to perform 'image generation specialized for commercial use'. Firefly's initial models are trained on images from Adobe Stock, open license images, and public domain images with expired copyrights. Firefly will be installed in tools such as Photoshop, Illustrator, and Experience Manager.
Firefly aims to be a custom trainable AI, such as using company owned assets to train Firefly. Furthermore, it seems that Firefly is also looking for a structure that ``returns profits to creators who provide assets'', which has been done with Adobe Stock and
Firefly uses
The private beta version has been released as a web tool, and there are only two functions that can be used: 'Text to image' and 'Text effects'. Adobe explains that the private beta is ``not only to encourage creators of all skill levels to create works, but also to encourage sharing, experimentation, and feedback.''
'Image generation from text'
The UI is as follows, and the images generated by the user are lined up. By entering a prompt in the text box at the bottom of the screen, it is possible to generate an image according to the instructions.
In the beta version, four candidate images are generated based on the prompt, and the image finish can be adjusted with the parameters on the right side of the screen.
Text effects allow you to add effects to the characters you type using Firefly. Enter the character string you want to add the effect to in the 'Input text' part, and enter the effect you want to apply in the text box at the bottom of the screen. By using the parameters on the right side of the screen, you can change not only the effect type but also the font.
The differences between Firefly and other generative AIs are summarized in the following five points.
・Designed with image generation specialized for commercial use in mind
・Training with the highest quality assets
・Integrated workflows in
・Creator-focused approach: custom training and revenue return
・Advocate the transparency of digital content through the content credential function (content authentication initiative: more than 900 participating organizations and companies)
In addition, Nishiyama CDO has also released some usage concepts for future development of Firefly.
For example, in the case of Photoshop, we are planning to implement a function that allows you to add an object to an image simply by entering 'underwater city'.
For Illustrator, Firefly offers features that help you create illustrations, artwork, and graphic designs. Specifically, it is proposed to vectorize handwritten fonts using Firefly.
In the case of Premiere Pro, Firefly may play an active role as one of the video editing functions. For example, by inputting 'change scene to winter day', it was proposed to change one scene of the video to winter at once.
Substance 3D plans to apply Firefly to the production of 3D models. Specifically, it is expected that Firefly will create a 3D model based on the entered text, or change the texture of the created 3D model.
Design tool Express expects Firefly to create templates that can be used for marketing images and social media posts.
In addition, during
Adobe Sensei GenAI services are native to Adobe Experience Cloud, making generative AI services accessible anywhere in your marketing workflow. The Adobe Sensei GenAI service aims to be a co-pilot for marketers, significantly improving productivity and efficiency while maintaining brand tone and manner with complete creative control. Adobe
Firefly will also be integrated with Experience Manager (AEM) Assets and Adobe Express. Firefly on the Adobe Experience Cloud generated content that was acceptable for commercial use, and we were able to get consumer insights on the performance of Firefly-generated images, so we could see how the business of Firefly-generated content could grow. It is also possible to build a feedback loop to evaluate effectiveness.
In addition, since the Adobe Sensei GenAI service utilizes large-scale language models (LLMs) such as Microsoft Azure OpenAI and open source FLAN T-5, it can be customized to meet business needs without relying on a single LLM. is possible. Specifically, it will be able to create marketing copy, provide chat experiences, be responsible for creating audiences and journeys, and be responsible for generating captions.
In addition, a Firefly demonstration was also presented by Nishiyama CDO. In addition, since the beta version provided for Adobe employees was used in the demonstration, there is a possibility that the UI may be slightly different from the private beta version.
The private beta version only offers 'Text to image' and 'Text effects', and text input is only available in English.
In 'Text to image', if you enter 'European tourists enjoying Mario Kart on the roads of Tokyo', the following image is generated. It seems that the feature is that elements that infringe on intellectual property (IP) are not generated, so I tried inputting the names of various world-famous characters, but they said, 'Images that infringe on IP are generated. There was no such thing,' says Nishiyama CDO.
In 'Text effects', when you enter the text 'Adobe Summit' to add the effect 'Las Vegas night', the following image is generated.
When I changed the effect to 'Las Vegas slot machine', it became like this.
That's why the demonstration was over, so the question and answer (Q & A) started.
Q:
It seems that information indicating that it was generated by Firefly is included in the generated image, but what happens to this information if the image generated by Firefly is edited or synthesized after the fact with Photoshop etc.?
A:
Adobe's content credential function is a function that records the processing history of images. Not only the information 'generated with Firefly' but also the information 'processed with Photoshop' is recorded. However, as the content credentials feature is still in development at the time of writing, we are unable to reveal any further details.
Q:
About returning profits to creators. Are there any other services other than Adobe Stock that are currently monetized by Adobe?
A:
Specifically, it will be Adobe Stock only, but it is possible to sell plug-ins etc. on the marketplace (in the Creative Cloud desktop app). Behance is not a service to buy and sell content, but a service that connects ``creators and companies that want to order work to creators'' like LinkedIn for creators, so it is different from Adobe Stock and the marketplace.
Q:
There are some images in Adobe Stock that have been pointed out that the rights are questionable, but if you are using these for training, can you really use Firefly for commercial use? Also, if a third party files an infringement claim for using an image from Adobe Stock, I think you can receive compensation from Adobe according to the terms, but the image generated by Firefly can also be claimed to infringe the right. If so, is this also eligible for compensation?
A:
Photo stock services, not just Adobe Stock, have schemes for judging content. To be honest, it is not necessarily the case that this screening scheme is 100% complete. Of course, it is a major premise that the rights relationship is dubious based on the scheme, but as I said earlier, it is not 100%. Therefore, compensation is provided in case of such an eventuality. The same is true for Firefly, so it will be subject to compensation, but at the moment Firefly is a private beta version, so it is difficult to answer what will happen when the service is provided. However, I would like to emphasize that it is designed to ensure transparency, including these points.
Q:
Where did the Firefly name come from?
A:
Not published.
Q:
I believe Adobe Stock, which Firefly is trained on, already includes images generated by external generative AI. Will these be included in the training dataset? If so, are the rights relationships clear?
A:
For one thing, I'm not in a position to give a clear answer on whether Firefly's training includes images generated by external generative AI. The current understanding is that Adobe Stock is a learning source, and hundreds of millions of images in this are used as training sources. This does not mean that all Adobe Stock images are used for training. However, as mentioned above, the image is used for training on the premise that the rights relationship is clear, and development should be carried out with the stance of compensating in case of emergency.
Q:
I felt that the demo for Premiere Pro, etc., would be very difficult to implement, but is it at the stage where it would be nice if it could be done, or is it at the level where the technical prospects are already in place?
A:
This is what I'm aiming for, and I can't answer how far development has progressed.
Q:
About Japanese correspondence.
A:
Support for languages other than English, including Japanese, is also planned, but the schedule is undecided.
Q:
Is the copyright of the content created with Firefly on the Adobe side or on the creator side?
A:
Under consideration.
Q:
Adobe Stock has the tag 'generative AI', will Firefly-generated content have the same tag?
A:
It should be yes, but it's still in beta so I can't say 100%.
Q:
Will it support Japanese at the time of service in?
A:
Hope is so, but I can't say it.
Q:
Does the text effect support hiragana and kanji?
A:
Although it is not supported at the moment, it is possible to apply effects to Japanese because it recognizes Japanese as images instead of characters even at this stage. However, you cannot use all functions such as changing the typeface.
Q:
How are the materials in Stock licensed for training? Did such items exist in the first place?
A:
That's right. There is a description to the effect that it will be used for the development of Adobe products.
Q:
Please let me know if there is a roadmap for Firefly.
A:
At this time, we can only say what has been announced so far.
Q:
Discussions are progressing in the direction that images generated by generative AI are not subject to copyright, but in the case of commercial use, isn't this a point of concern for companies to use?
A:
Since it is unknown what kind of license it will be when Firefly is commercialized, it is difficult to answer clearly, but in the case of Adobe Stock, the license is royalty-free. Since it is royalty-free, there is no copyright infringement, but it cannot cover the risk of using the same image as competitors. In the same way, Firefly is designed within the scope of Adobe's thinking that there is no risk in commercial use. I think it's quite possible. However, at this point in time, it is difficult to give an in-depth answer.
Related Posts:
in Software, Posted by logu_ii