An app called 'RollTab' has been released that can automatically generate the continuation of a piano performance using AI.
An iPhone app called '
Training a 125M-parameter Model to Autocomplete Piano - SimEdw's Blog
https://simedw.com/2026/08/20/midi-autocomplete/
RollTab is available for free on iPhone/iPad. You can install it from the following page.
RollTab App - App Store
https://apps.apple.com/gb/app/rolltab/id6799679058
To use RollTab, first connect your MIDI keyboard to your smartphone.
Once you start playing, what you've played will be recorded on your smartphone screen.
Then, when you stop playing, the AI automatically generates the rest of the melody from the short melody you recorded. In the demo Edwardson released, it took less than two seconds from the time he stopped playing until the next part was generated.
On Edwardson's blog , you can listen to samples of music generated by AI, such as the background music from 'Pokémon Red and Green' ('Pallet Town'), 'Tina's Theme' from 'Final Fantasy VI,' and 'Für Elise,' which you can input.
Edwardson trained a Transformer-based AI model with 125 million parameters to create RollTab. What makes this system unique is that, unlike typical speech-generating AIs, it works with MIDI files.
Unlike other audio formats such as MP3, MIDI files have the characteristic of 'saving music as a series of events rather than saving recorded sounds.' For example, information such as 'a key was pressed at a specific pitch and speed,' 'a key was released,' 'the state of the sustain pedal changed,' or 'the instrument was changed' is saved as events, not sounds. In Edwardson's project, since it focuses on piano performance, he mainly kept piano-like material and deleted or reduced other material.
According to Edwardson, in order to train the Transformer model based on the characteristics of these MIDI files, it was first necessary to convert the MIDI events into discrete sequences that the model could read and predict. To convert the MIDI events into 'tokens' that the AI could handle, they tried methods such as tokenizing each event like 'NOTE_ON,' 'NOTE_OFF,' and 'TIME_SHIFT,' as well as assigning 'pitch, volume, and length' to the notes, but Edwardson says that these methods resulted in problems such as the AI being unable to manage the sounds being played, a decrease in processing speed, and limitations on the length of the performance that could be input.
The method ultimately adopted was 'NOTE(pitch, delta_onset, duration, velocity)'. In this method, for example, 'C4' is played, and then 'D4' is played 24 steps later. Instead of the previous method which involved four conversion processes to generate the attributes of the notes, this conversion method advances the music one note at a time, allowing even large iPhone models to achieve a processing speed of approximately 108 notes per second, far exceeding the speed required for live performances by humans.
Furthermore, by holding down the sustain pedal on a piano, you can extend the duration of the sound's resonance for as long as you keep it pressed. Adding this sustain pedal operation as an event would complicate implementation, so by incorporating it into the 'note length,' the model is simplified to only predict 'pitch,' 'start,' 'duration,' and 'speed,' enabling a quick response even with a relatively small model.
The dataset Edwardson used for training was mainly collected from old classical music in the public domain, ultimately resulting in hundreds of thousands of MIDI files and approximately 300 million note events. However, according to Edwardson, simply increasing the amount of data did not improve performance; cleaning and selecting the data was more important than increasing the amount of data.
When evaluating the created model, Edwardson initially compared the generated performances manually by listening to them. However, this was time-consuming, and the short input of only about four notes provided little musical context, making judgment difficult. Therefore, Edwardson used Gemini 3.5 Flash to compare two generated results and determine 'which one better continues the input performance,' which allowed him to build a fairly large preference dataset.
Using the evaluation data obtained in this way, additional training called ' Direct Preference Optimization (DPO) ' was also performed. DPO is a method developed to precisely control unsupervised training. It generates multiple answers from the same input, selects the result that is evaluated as 'better,' and adjusts the AI so that it generates that answer with a higher probability. It seems that DPO had the greatest effect, improving the automatic music generation model to a level where it could generate more reliable songs.
Finally, by exporting the completed model to Apple's 'Core ML' and quantizing the weights to INT8, we created 'RollTab,' an app that works perfectly on the iPhone.
Edwardson said, 'This was a very enjoyable project. There are many interesting papers on music generation, but at first I deliberately avoided reading them in depth. I wanted to experience the joy of tackling the problem myself rather than simply implementing someone else's research. It was only later that I compared my approach to existing literature. My project is still far from perfect; sometimes the generated melodies loop, it's difficult to generate a consistent song from short inputs, and the input instructions themselves are difficult to understand. You could say it's like a 'piano version of GPT-2.' Nevertheless, I've finally reached a point where I genuinely enjoy sitting down at the piano, playing a few notes, and seeing what comes out of it.'
Related Posts:
in AI, Posted by log1e_dh