An attempt to compose music using a neural network

by Spencer Imbrock

A type of recursive neural network · A music generator using LSTM (Long short-term memory) actually composes various piano music and chamber music.

Generating Classical Music with Neural Networks - Floyd Hub Blog
https://blog.floydhub.com/generating-classical-music-with-neural-networks/

Clara: A Neural Net Music Generator - Christine Payne
http://christinemcleavey.com/clara-a-neural-net-music-generator/

This music generator " Clara (Clara) " was created by Christine McLay Bay Pain, a research fellow of OpenAI, a nonprofit organization for artificial intelligence research and a pianist from the Juilliard School of Music . The name is a wife of Robert Schumann composer and honored to Clara Schumann, a prominent woman pianist of the 19th century.

Mr. Christin's idea is that "you can generate music in the same way you generate a language." In training the language model, the researcher sends a paragraph to the neural network and predicts the next word. The difficult part is in terms of language, in terms of music, how to convert notes into shapes that can be sent to the model.

Mr. Christine uses MIT's music21 library from MIDI files centered on piano and chamber music in classical archives to acquire notes in music score and its timing offset (where to ring / how long to ring) . I converted that information into a form conforming to my own code rule / note rule. However, music does not continue to sound all the time, there is a "holiday" until the next sound is produced. According to Mr. Christin, about 40% of the songs correspond to this "holiday", and I also let this neural network learn this point.

In addition, the training was carried out in the range of 62 keys out of the 88 key piano key, which is almost unused key. In the data set, only three notes of this removed part came out. And while there are many cases in general music generators to limit the number of notes to be placed at the same time to be limited to 4, Christine did not limit Clara.

Although songs actually composed by Clara are released, it seems to be quite difficult to distinguish when people are mixed with what they composed. This is a song called "Notewise Generation".


This is "Jazz Piano Notewise Generation". As the name suggests, it is jazzy.


"Chamber Music Generation" is a chamber music style with piano and violin.


"Chordwise (reproduced an actual Mozart piece)" is a reproduction of Mozart's sonata .


The code of "Clara" is published on GitHub.

GitHub - mcleavey / musical - neural - net: Train an LSTM to generate piano or violin / piano music.
https://github.com/mcleavey/musical-neural-net