Mar 31, 2020 16:00:00

AI to convert brain waves into sentences is under development

The power of reading the hearts of others has appeared in various novels and films. A research team

at the University of California, San Francisco, is developing an AI that converts brain waves detected by electrodes inserted into the brain into sentences that make such a supernatural ability of 'reading the mind' a reality.

Machine translation of cortical activity to text with an encoder–decoder framework | Nature Neuroscience
https://www.nature.com/articles/s41593-020-0608-8

Scientists develop AI that can turn brain activity into text | Science | The Guardian
https://www.theguardian.com/science/2020/mar/30/scientists-develop-ai-that-can-turn-brain-activity-into-text

In the treatment of epilepsy , there is a case where a treatment method called 'embedding electrodes on or inside the brain' is used. The research team received this treatment and asked four epilepsy patients with electrodes implanted in the brain to become subjects, and conducted an experiment called 'Generate sentences from brain waves detected by electrodes'.

In this experiment, subjects who monitored brain waves read out 50 short sentences aloud multiple times, such as ``

Tina Turner is a pop singer '', `` Thief stole 30 jewels '', and detected it It converts the brainwaves into character strings using AI using machine learning algorithms. At the start of the experiment, AI generated meaningless sentences, but by comparing the sentence read and the list of generated words, the accuracy was improved.

Although the accuracy of the text generated by each subject varied, one participant generated text that needed to be modified by only 3% of the total. According to the Guardian, who reported the news, even a sentence transcribed by a human stenographer contains about 5% of errors in the whole, so this result can be said to be quite accurate. However, the sentences analyzed by AI are limited to 50 short sentences, and if you try to analyze other than these sentences, the accuracy seems to be worse.

Also, the generated text was sometimes very wrong. The sentence `` Those musicians harmonise marvellously (the musicians play wonderful harmony) '' is described as `` The spinach was a famous singer '', or `` A roll of wire lay near the wall '' (A wire was wrapped near the wall) 'was changed to' Will robin wear a yellow lily '(Is Robin wearing a yellow lily)?

`` The AI we created recognizes common English grammar, identifies words from brain activity, and combines learning about specific sentences, '' said Joseph McKin, the author of the study. It is thought that it is generating sentences '' It is a long way to go, but I hope that it should be the basis of a device that automatically writes sentences '.

According to the results of this research, Dr. Christian Helf of the University of Maastricht, who is conducting an experiment to apply machine learning technology to the cranial nerve interface, said, `` Since we use subjects who can read sentences aloud, we read sentences aloud. It cannot be applied to patients with inaccessible disability. ' However, 'This type of study typically requires millions of hours of experimentation. However, in this study, the experiment time per subject was less than 40 minutes and the number of times each sentence was read was limited. That's a stimulating result for us, 'he said.