Sep 27, 2022 16:03:00

Technology will be developed to reproduce the movement of the lower body only with the headset and controller

Sony will release the next-generation VR device ``

PlayStation VR2 (PSVR2) '' in 2023, and Meta hasannounced that it will introduce a new VR headset in October 2022. VR headsets are used as a device to casually enter a virtual space at home, but these only track the human body with a device on the head and controllers held in both hands, so they can only move the head and hands. cannot be reproduced exactly. Newly announced that researchers at Meta have devised a method to reproduce the movement of the lower body with only a headset and controller.

[2209.09391] QuestSim: Human Motion Tracking from Sparse Sensors with Simulated Avatars
https://arxiv.org/abs/2209.09391

Meta Demonstrates Quest 2 Body Tracking Without Trackers
https://uploadvr.com/meta-quest-2-body-tracking-without-trackers/

Existing common VR systems only use headsets and controllers, so they can only track head and hand movements. Elbow, torso, and leg positions can be inferred using a type of algorithm called inverse kinematics, but elbow movements lack precision and leg movements are nearly impossible to reproduce.

However, Meta researchers have announced that they have succeeded in estimating the pose of the whole body with only the headset and controller using the in-house VR headset ' Meta Quest 2 ' and machine learning.

Below is the state of the demonstration experiment. You can see that the whole body movement of a human wearing only a headset and a controller is reproduced.

QuestSim: Human Motion Tracking from Sparse Sensors with Simulated Avatars-YouTube


The left half of the video is a live-action shot of the movement of the user holding the VR headset and controller, and the right half is data that estimates the user's movement based on the data collected from the headset and controller. I'm here. When the wearer runs, it reproduces how the legs bend and run correctly.

Even complex dance-like movements involving the upper and lower body are reproduced with some degree of accuracy.

If you put obstacles in the VR space, you can make the avatar kick the obstacles or make the avatar fall.

However, these reproductions are only 'guess' and do not match the correct position of the lower body. Also, the delay for this reproduction is 160 ms, which is good for seeing other people's movements, but it seems that it is not suitable for looking down and looking at your own body.

Meta researchers said, ``It is very unpleasant for people to have legs that are different from the actual legs in their avatar.We are working on reproducing natural looking legs next. is our current strategy.'

UploadVR, a news site that covered this technology, wrote, ``Because Meta's AR/VR event will be held soon , body tracking announcements may be made during that period.''