Apple's AR headset 'Vision Pro' developer talks about 'technology to read mind from user's brain and body'?



Apple's MR headset ' Vision Pro ' was announced at Apple's developer conference ' WWDC23 ' held on June 6, 2023 (Japan time). Mr. Sterling Crispin, a developer who was involved in the development of this Vision Pro, tweeted a comment about Vision Pro on Twitter and has become a hot topic.



What kind of device Vision Pro is, the contents of the announcement are summarized in the following article.

Apple announces AR headset 'Vision Pro', freed from the limitation of display, allowing you to operate your favorite applications with your eyes, hands and voice wherever you like-GIGAZINE



Also, in the following article, you can see pictures and movies of the actual Vision Pro machine exhibited at Apple Park at Apple's headquarters.

Actual machine photos & videos & overseas reviews of Apple's AR headset 'Vision Pro' appeared - GIGAZINE



Mr. Crispin, who worked as a Neurotechnology Prototyping Researcher in the technology development group at Apple, said, ``I spent 10% of my life developing VisionPro. I've been working on AR and VR for 10 years, and Vision Pro is in many ways the culmination of the entire industry into one product.'

Mr. Crispin has been with Apple for three and a half years until he leaves Apple at the end of 2021, and two of those years have been spent on Vision Pro, mindfulness experiences, ■■■ products, and more ambitious projects using neurotechnology. It is said that he spent it on basic development of magnificent research. Of course, Mr. Crispin has a non-disclosure agreement (NDA) with Apple, so he can't specifically talk about much of the work he actually did. However, some of the information has already been published in the form of patents, so Mr. Crispin mentions some.

For example, Mr. Crispin said he was researching and developing a technology that 'detects the user's mental state based on data from the user's body and brain in the immersive experience.' By measuring the movement of the user's gaze, electrical activity in the brain, heart rate and rhythm, muscle activity, blood density in the brain, blood pressure, etc., it is possible to determine whether the user is curious or scared. , It seems that they aimed to predict various cognitive states, such as whether they are alert or remembering the past.



Some of the patent applications that Apple has already submitted include Mr. Crispin's name, and among them, the technology that 'predicts the click before the user actually clicks' is the most amazing result. Mr Crispin said. The pupil moves before the click because humans expect something to happen after the click. Therefore, by monitoring the movement of the pupil, redesigning the UI in real time, and drawing out the reaction of the pupil, it seems that he succeeded in drawing out biological feedback from the brain. ``It's a crude BCI (brain computer interface) through the eyes, but it's very cool compared to invasive brain surgery,'' Crispin commented.

He also developed a technology to estimate the user's cognitive state by quickly flashing the appearance and sound at a level that the user cannot perceive and measuring the reaction to it. He also developed a method to predict concentration, relaxation, and learning effects using machine learning, using signals from the body and brain. In addition, it seems that there was also an attempt to update the virtual environment created by the device by receiving feedback from the user's body and brain state.



“Many people have been waiting for the Vision Pro for a long time,” Crispin said. It will take years,” he said.

``It's been more than five years since I started this project,'' Crispin said. I hope that the whole of Vision Pro will be bigger than the sum of individual projects, and that it will move everyone's hearts.'

In addition, Mr. Crispin also commented on a post featuring his tweets on the social news site Hacker News.

Hey I appreciate the general lack of cruelty so far in the comments, sometimes H... | Hacker News
https://news.ycombinator.com/item?id=36218557



Crispin said, 'The reason I'm phrasing weird in my tweets is because I can't talk about the work I've done because of NDAs, other than what I do and patents. But I'm in a pretty unusual role. I also tried to make it clear that my work was not intended to be interpreted as existing, unannounced features or capabilities of Vision Pro. research done or contributed to, and published through patents.'

Mr. Crispin also said, ``I said that I spent 10% of my life on this, but this is a solid expression. This is a self-reflection tweet, I'm not trying to brag about being involved with the Vision Pro, it's an unusual product and I was in an unusual role, the Vision Pro is just cool I hope everyone will enjoy it with a great device.'

Related Posts:

in Hardware, Posted by log1i_yk