Microsoft's MR headset "HoloLens" contains a coprocessor for deep learning

Mixed Reality (MR) headset developed by Microsoft "HoloLens"It was revealed that a chip dedicated to the deep learning process was installed.

Second version of HoloLens HPU will incorporate AI coprocessor for implementing DNNs - Microsoft Research
https://www.microsoft.com/en-us/research/blog/second-version-hololens-hpu-will-incorporate-ai-coprocessor-implementing-dnns/

In order to process the information from various sensors, such as head tracking cameras and inertial measurement unit in HoloLens, Microsoft has developed a custom multi-processor "HPU". For that HPU, the "HPU 2.0" to be installed in next-generation HoloLens may be dedicated AI coprocessor for implementing deep neural network (DNN) is mounted,CVPR 2017Harry · Shum's keynote speech was held at the same time.

HPU 2.0 - YouTube


The purpose of implementing DNN with HPU 2.0 is to utilize deep learning in MR content. However, deep learning requires a large amount of labeled data for training and computational resources for multilayered processing of neural networks, and this calculation process is not efficient with general CPUs and memories, The necessity of dedicated chip was increased in terms of computation speed and battery consumption. Therefore, next HoloLens will be able to deep learning processing with HoloLens which is a mobile terminal by incorporating a dedicated AI coprocessor for implementing DNN.


With HPU 2.0's AI coprocessor built in, HoloLens will be able to handle DNN natively, but according to Harry, this new feature is just a part of the features we are developing for the next-generation HoloLens Thing. Microsoft is assuming that MR devices and AI are the future of computing itself, and long-term and huge investment is planned.