Dec 11, 2024 12:00:00

NitroFusion, an open source AI model that can instantly generate images on affordable household hardware, is released

The Surrey Human Centred Artificial Intelligence Laboratory at the University of Surrey in the UK has announced that it has created an AI model called ' NitroFusion ' that generates images instantly using only modest, affordable hardware.

Surrey announces the world's first AI model for near-instant image creation on consumer-grade hardware | University of Surrey

https://www.surrey.ac.uk/news/surrey-announces-worlds-first-ai-model-near-instant-image-creation-consumer-grade-hardware

NitroDiffusion
https://chendaryen.github.io/NitroFusion.github.io/

Typically, high-speed image generation requires a large investment in hardware, but NitroFusion can generate images instantly using only a single consumer-grade graphics card, though there is no information on which graphics card can be used.

Professor Yi Jae Song, co-director of the Surrey Human-Centred Artificial Intelligence Institute, called NitroFusion 'a paradigm shift that eliminates the need for large-scale computing resources and makes AI accessible to everyone.'

Here's how NitroFusion works: It condenses a multi-step teacher model into a one-step student generator through training.

The project page shows examples of actual generation and comparisons with other models. On the far left is an image generated over 25 steps using Stable Diffusion XL (SDXL) for comparison, followed by SDXL-Turbo and SDXL-Lightning for comparison with one step each. DMD2 is the teacher model, and one-step and four-step generated images are shown, with NitroSD-Realism on the far right being the NitroFusion model.

An example of image generation using the 'NitroSD-Vibrant' model, which uses 'Hyper-SDXL' as a teacher model, looks like this. On the left half, an example of image generation using the same comparative model as before is shown.

The NitroFusion model is characterized by its ability to produce a reasonable level of quality in image generation in a single step, but by performing multiple steps, the quality of the details is said to improve.

The model is available on HuggingFace , and the code is available on GitHub . The licenses for 'NitroSD-Realism' and 'NitroSD-Vibrant' are CC BY-NC-SA 4.0 and cannot be used commercially, respectively, and 'NitroSD-Realism' and 'NitroSD-Vibrant' are Open RAIL++-M and can be used commercially.