A strong man who will date Hatsune Miku and the real world in Microsoft's Holo Lens (Holo Lens) appeared

Although it may be delusive with animation of two dimensional world and character of cartoon and "I want to do such things and such", it is possible to realize it actually by taking advantage of the latest technology and programming technology etc. with that hands There should be few people who have sought out methods. However,Let's have oneMr. Hatsune is going to date Hatsune Miku in the real world, he has made his own programming skill and HoloLens to realize his wish date.

Date with Hatsune Miku in AR through HoloLens - I tried date with Hatsune Miku - YouTube


A movie starts from the self-introduction of the developer saying "There are various things done with AR (Augmented reality)"


I have a date with Hatsune Miku using AR glasses that I have already owned in 2012, but ....


Microsoft's HoloLens is now available in Japan in 2017.


So, in order to enjoy Hatsune Miku and more sophisticated date, it seems that I tried again AR date with HoloLens.


When I say "Switch on" in nothing place ......


You can summon Hatsune Miku.


Hatsune Miku that came down to the real world is like this. The model of Hatsune Miku being displayed is created by Foil ToriiMMDIt is said that.


When walking while HoloLens is worn, Hatsune Miku follows along side by side.



In this AR date we are using HoloLens, but since HoloLens measures the shape of the surrounding environment ......


Three-dimensional objects such as stairs can be recognized.


Hence, Miku Hatsune will go up and down stairs by firmly recognizing the steps.



Furthermore, it is also possible to stroke your head with gestures with fingers.


Extend one finger with this feeling ......


You can sweep your head when you shake it to the left or right. However, "It is still unnatural, so I'd like to detect my hand somehow."


Next, I was tired of walking around, so I decided to take a break on the bench.


A Japanese crane with a HoloLens was seated on the bench.


You can see that the viewpoint is lower than Hatsune Miku and sitting on the bench.


Then, the position where HoloLens wearer sitting is detected ......


How, Hatsune Miku sat next to me.


You can see that the bench does not overlap with the 3DCG model, and it seems that you are sitting on the bench securely.


Other, when bringing my face closer ......



It shows a special action.


Hatsune Miku sitting on the bench.


In addition, at the end of the movie, there will be a table seat sitting opposite to the bench, but a sister will sit down.


Then Miku Hatsune moved to the chair which is in front of the eaves where HoloLens is wearing, and I sat down. Thanks to HoloLens' s advanced object detection function, Hatsune Miku can sit also in chairs without any discomfort.



Mr. Ogune, who has a developer, said, "With the release of HoloLens, it has become possible to make Miku more sophisticated, about 100 times easier than our own AR glasses.You can access a database and have a conversation Miku I am going to be able to do it.


If you say "Switch off", Hatsune Miku will disappear.


At the end of the movie the viewpoint of the HoloLens wearer gradually decreases ... ...



Hatsune Miku changes the pose at exquisite timing, it shows the action.


The movie below is the prototype of this movie. Instead of HoloLens a head mounted display with motion sensor deviceKinectI used my own AR goggles attached. The action using hands has become larger than the latest version.

Date with Hatsune Miku - Date with Hatsune Miku in Augmented Reality environment - YouTube


The following movie is the first time Hatsune Miku was able to sit on chair. According to the movie, we use our proprietary technology to recognize the surrounding environment, and we can see that the object is not recognized by the AR marker and so on.

I tried living with Hatsune Miku - Stay with Hatsune Miku in Augmented Reality environment - YouTube


Furthermore, it seems that the action of kissing the face close to it was created by applying the system in the following movie.

Kissing Hatsune Miku in Augmented Reality environment - I tried kissing with Hatsune Miku - YouTube


In the past realized various interactive actions such as touching and high touching a tie, so if you can accurately detect the movement of the hand of HoloLens wearing person, you can make a more advanced date It looks likely.