Succeeded in reconstructing 'music I was listening' from brain waves, there is also a movie that listens to reconstructed music

A research team at the University of California, Berkeley has developed a technology to analyze brain waves and reconstruct ``music you were listening''. The developed technology can reconstruct music so clearly that changes in scale and parts of phrases can be distinguished.

Music can be reconstructed from human auditory cortex activity using nonlinear decoding models | PLOS Biology
https://doi.org/10.1371/journal.pbio.3002176

Brain recordings capture musicality of speech — with help from Pink Floyd | Berkeley
https://news.berkeley.edu/2023/08/15/releases-20230811

When the research team performed brain surgery to treat epilepsy , they played Pink Floyd's song ' Another Brick in the Wall, Part 1 ' in the operating room and recorded the patient's brain waves.

As a result of comprehensively analyzing the brain waves of 29 patients, we succeeded in reconstructing the music with such clarity that the phrase 'All in all it was just a brick in the wall' could be heard. Furthermore, they succeeded in extracting a voice that could barely discern a change in scale from the electroencephalogram of one patient. By playing the following movie, you can check the music actually reconstructed from the brain waves.

Analyze brain waves and reconstruct ``music you were listening''-YouTube


Robert Knight, a member of the research team, said, '(Technology that reconstructs music from brain waves) adds musicality to brain implants for patients such as amyotrophic lateral sclerosis (ALS). Not only can you use language to express what you are feeling, but you will also be able to express prosody and emotion.”

However, to use the technology created in this study, it is necessary to perform surgical operations on the brain and place electrodes. According to Ludovic Berrier, a member of the research team, it is not possible to accurately read brain waves with a non-invasive method that does not require surgery. ``At the moment, non-invasive methods are not exactly accurate. I'm here.

In addition, a research team of Google and Osaka University is also working on the idea of ``reconstructing music from brain activity'', and in July 2023, an AI that reconstructs music from brain activity data read by MRI ' Brain2Music ” has been announced. A demo of the music reconstructed by Brain2Music can be found at the link below.

Brain2Music
https://google-research.github.io/seanet/brain2music/