OpenAI develops AI 'Jukebox' that automatically generates music with lyrics from specified genre and artist

OpenAI, a non-profit organization that researches artificial intelligence, has released AI ' Jukebox, ' which uses neural networks to generate music with lyrics from specified genres and artists.


OpenAI introduces Jukebox, a new AI model that generates genre-specific music with lyrics-The Verge

For example, when Jukebox creates a song in the style of Frank Sinatra , a pop singer representing the 20th century, it looks like this. Although there are some English words that can be heard about the lyrics, it seems that it is not established as a sentence.
OpenAI · Classic Pop, in the style of Frank Sinatra-OpenAI Jukebox

Below is a song by Ella Fitzgerald , the Queen of Jazz.
OpenAIJazz , in the style of Ella Fitzgerald-OpenAI Jukebox

The Rage- like song that represents German metal, the tempo and beat peculiar to heavy metal are well reproduced.
OpenAI Heavy Metal, in the style of Rage-OpenAI Jukebox

All 7131 sample songs can be checked below.

Jukebox Sample Explorer

According to OpenAi, there have been attempts to generate music by machine learning, but the problem was that the amount of data contained in the music was too large to analyze. To address this issue, Jukebox discards perceptually irrelevant information from the song and compresses the raw data with a convolutional neural network .

Generate music from this compressed data ...

It is said that it has been upsampled and restored to the original sound quality.

It uses 1.2 million songs (600,000 of which are English songs) as a dataset to train Jukebox. In addition, metadata such as lyrics, artist, album, year of release, genre, keywords, etc. is associated with songs in the dataset by crawling .

From various information such as metadata, Jukebox classifies artists by genre. Artists placed close to each other in the diagram below are classified by Jukebox as 'closely related'.

The classification table on the official Jukebox page shows the name when you hover over an artist's face photo. In one example, an artist similar to Elton John was John Lennon .

`` While Jukebox takes a step forward in the ability to condition music quality, consistency, audio sample length, artist, genre, and lyrics, the OpenAI research team says there is a big gap with human-made songs. `` In one example, the song Jukebox generated contains chord patterns, solo parts, etc. in line with tradition, but there is no very common musical structure such as chorus repeat, '' Jukebox said Acknowledge that there is.

The source code of Jukebox is available on Github.

GitHub-openai / jukebox: Code for 'Jukebox: A Generative Model for Music'

in Science, Posted by darkhorse_log