Nov 01, 2019 07:00:00

Created to visualize the music a 'living painting' by the cooperation of AI and a human

Machine learning researcher

Xander Steenbrugge created a new audiovisual experience through collaboration between humans and AI. The various paintings AI makes to the music are so impressive that you can see them with beautiful and nightmare art.

Emtremely excited to finally share a side-project I've been working on for the past few months: Neural Synesthesia, visualizing music with #GANs !

So many ideas left to explore, this journey is just getting started ..

More HD renders on my new YT channel: https://t.co/kKoXEuXOi3 pic.twitter.com/jkRGflSfjg

— Xander Steenbrugge (@xsteenbrugge) October 28, 2019

In creating AI, Mr. Steenbrugge first gathered a painting data set that determines the theme and visual style of paintings drawn by AI, and learned that algorithm. Along with this visual style, AI created a new image that imitated painting over several days. In addition, I added music so that I could get information from that music.

In this way, AI was created to create unique paintings from music one after another. Steenbrugge arranged this and synthesized it to create the final work. “Movies that visualize music” by AI and humans can be confirmed from the following.

When the painting comes to life ...- YouTube


The first thing that appeared was a painting of a blue mountain.

This will change gradually along with the music. At first it was a soft painting, but it gradually changed to a style that understands the handwriting ...

Something gradually emerges.

Although it is a human being, it has a mysterious impression that multiple faces overlap.

It becomes a clear portrait ...

Impressed as an Italian merchant of “

Arnolfini statue ” drawn by Jan van Eyck.

The painting changes slowly and sometimes violently to match the rhythm of the music.

Something strange that can be taken as a portrait or a sculpture.

Something like a building.

Change to landscape painting ……

A rural scene that seems to be depicted in a

millet painting.

Contours and lines change their colors while maintaining the handwriting.

The development that seems to have no context is unique to AI.

It feels like an abstract painting.

Give a disturbing atmosphere ...

A landscape painting reminiscent of Impressionism.

If you think, depending on the music, paintings like cubism were drawn.

There are also mysterious paintings that mix various elements of art.