FLUX's Black Forest Labs Announces 'Self-Flow,' a Multimodal AI Learning Method for Generating Images, Video, and Audio with High Efficiency and Accuracy



Black Forest Labs, known for its FLUX series of image-generating AI, has announced a new approach to training generative AI called ' Self-Supervised Flow Matching (Self-Flow) .' Self-Flow is a mechanism for efficiently training generative models using

self-supervised learning , achieving significant improvements in learning efficiency and text rendering during image generation.

Black Forest Labs - Frontier AI Lab
https://bfl.ai/research/self-flow




Black Forest Labs' new Self-Flow technique makes training multimodal AI models 2.8x more efficient | VentureBeat
https://venturebeat.com/technology/black-forest-labs-new-self-flow-technique-makes-training-multimodal-ai

Self-Flow is a self-supervised flow matching framework that integrates representation and generation in AI models for generating images, videos, and audio. It consistently outperforms existing generative AI learning methods without the need for external models or training data. The graph below compares Self-Flow (orange) with REPA (blue), a conventional method for matching features within an image. The horizontal axis represents the number of training steps, and the vertical axis represents the evaluation of how realistic the results of the generative AI are. Since the evaluation of how realistic a result is 'the closer it is to 0 (lower),' the graph shows that with the same number of steps, Self-Flow can generate more realistic images, videos, and audio, while significantly reducing the number of steps required to generate the same level of quality.



The following images show the results of inputting the instruction 'Elegant typography 'From the Black Forest with love' in gold and rose gold letters against a dark forest background.' The left image shows the results of the AI trained using conventional learning methods, while the right image shows the results of the AI trained using Self-Flow. Self-Flow is said to bring significant improvements in the accuracy of text rendering, as well as structural consistency such as human faces and hands.



Furthermore, video samples show that the technology can generate more natural-looking human movements and facial expressions, animal movements, and liquid flow patterns while significantly reducing the number of training steps compared to conventional training methods.



Self-supervised learning, one of Self-Flow's features, is a method that allows AI to learn the structures and relationships within data on its own, without being given external training data labels. Generative AI such as Stable Diffusion and FLUX use '

diffusion models ' that primarily learn to remove noise, and have been criticized for having difficulty cultivating sufficient semantic understanding as internal representations. Using self-supervised learning promises to improve learning efficiency and reduce costs.

Self-Flow also employs a 'self-distillation' architecture, where a 'teacher' (EMA model) and a 'student' are prepared using the same data with different noise levels, and the student predicts a cleaner internal representation. This approach forces the model to deepen its internal semantic understanding, effectively learning how to perceive the world while learning how to generate it.

The implementation of Self-Flow resulted in faster learning across modalities, reaching a plateau up to 2.8 times more efficiently, and also improved temporal consistency in video and sharper text and typography.

Black Forest Labs mentions world models as a future challenge. Self-Flow, a training method for generative AI, may provide a path to AI that can understand physical laws and object relationships with a 'perceptual basis,' rather than simply imitating appearance. In fact, by fine-tuning a 675 million parameter version of Self-Flow, they have reported achieving significantly higher success rates in complex multi-step tasks.



The technical details of Self-Flow are available on GitHub. At the time of writing, Self-Flow is in a research preview stage, but it is likely to be incorporated into commercial APIs and open source products in the future.

GitHub - black-forest-labs/Self-Flow: Code and website for Self-Flow: Self-Supervised Flow Matching for Scalable Multi-Modal Synthesis · GitHub
https://github.com/black-forest-labs/Self-Flow/

Related Posts:

in AI, Posted by log1e_dh