PabloNet is a generative AI that turns anything it sees into art and turns it into a wall hanging image
' PabloNet ', created by machine learning engineer Matthew Le Cochois, is a system that creates AI art from what is captured by a camera in real time using the explosive image generation pipeline '
PabloNet
https://www.matthieulc.com/posts/pablonet/
Although traditional diffusion models are good at generating images from text or image prompts, their performance may not be sufficient for real-time interactions. StreamDiffusion is a new approach devised to deal with scenarios involving 'continuous input' such as the metaverse and live video streaming. A typical generation AI outputs an image corresponding to a series of prompts after a loading time, but in the StreamDiffusion demo video, you can see that the generated image changes in real time with each character of the prompt.
Introducing 'StreamDiffusion', an explosive image generation pipeline capable of generating over 100 images per second - GIGAZINE
'It's great that anyone can express their inner world through art by generating art with AI,' said Koshowa, who said he liked the idea of using AI to be creative. However, when he was trying out StreamDiffusion, he felt that while real-time image generation was fun, he was often dealing with anime-like images, and that it was just a temporary thing of entering a prompt and enjoying the image being generated.
So Kochowa created PabloNet. PabloNet is a picture frame with a monitor inside and a hole just above the monitor. This hole is a webcam, and the images it captures in real time are converted into art in real time using StreamDiffusion.
Below is a video of PabloNet generating the image in real time while Kochowa is standing in front of it with his smartphone. Click on the image to see the real-time switching in a movie.
Below is a photo of Koshowa's friend, who is holding both hands in front of PabloNet, which is reflected in real time by PabloNet.
PabloNet can react to any photographed object, not just people. In the example below, a tennis racket is placed in front of PabloNet to create art.
According to Kochowa, the system of turning video into art in real time means that 'the art changes just by passing in front of you, and it functions as interactive art that is not temporary. You can also change the object you place in front of you to decorate it according to your mood, and it feels permanence as a constantly changing canvas.' According to Kochowa, there are still problems such as 'the frame rate of the output art video is low,' so there is still room for improvement.
The back of PabloNet looks like this: a 10.1-inch screen, an infrared light, and a Pi camera for
The PabloNet code is also available on GitHub.
GitHub - mlecauchois/pablonet
https://github.com/mlecauchois/pablonet
Related Posts:
