AI 'Piano Genie' capable of playing 88 key pianos to a high level with just 8 buttons

" Magenta " started from the Google Brain Team 's research on Google's AI development is developing an intelligent controller " Piano Genie " that maps eight button inputs to 88 key pianos in real time.

Piano Genie: An Intelligent Musical Interface
https://magenta.tensorflow.org/pianogenie

Actually playing the piano with "Piano Genie" can be seen in the following movie. There are only 8 input buttons and it seems only to be pushing randomly, but it is showing off an unexpectedly seductive performance.

Piano Genie Improvisation # 1 - YouTube


"Magenta" is a research project to explore the role of machine learning in the process of creating art and music. It has also been involved in the development of a new concept synthesizer " Google NSynth Super " that makes it possible to create completely new sounds.

"Piano Genie" by Magenta consists of two recurrent neural networks , "bidirectional LSTM encoder" which assigns the piano MIDI sequence to 8 buttons and "unidirectional LSTM decoder" which reflects button input on the actual piano It has been. According to Magenta, the training data set of the encoder uses the one that converted the performance data of about 1400 songs in the international piano-e competition into a MIDI file.

Since the mapping of buttons and notes varies with time, even when the same button is pressed, the sound emitted by the timing will change. The part of the encoder works when training from the data set in advance, and only the decoder is used when playing. Also, since the pitch of the piano to be assigned corresponds to the position of the button, it seems that a higher sound is generated by pressing the right button and a lower sound is produced by pressing the left button. The following image shows the actual piano roll (top) and the sequence output (bottom) of the 8 button, so you can see that the movement of the two sequences is synchronized.

Demo "Piano Genie" can be played on web browser is released. By inputting "1" to "8" on the keyboard, the corresponding keyboard will be played.

Piano Genie
https://tensorflow.github.io/magenta-demos/piano-genie/

Actually playing the demo version feels like the following movie. Every time you enter a key, you can see how the pitch assigned to the button changes on the lower key.

I tried the "Piano Genie" demo that you can freely play the 88 key piano with just 8 buttons - YouTube