Anyone with a free browser and webcam can easily use ``OpenLive3D'' that can track and move facial expressions and finger movements with VRM format 3D avatars

Expensive tracking equipment and cameras are required to reflect your movements in 3D avatars. Therefore, volunteers are developing a method to capture body and facial movements with a webcam. ' OpenLive3D ' is an open source capture system that allows you to easily track the movement of your face and hands with a webcam.

GitHub - OpenLive3D/OpenLive3D.github.io: Connect Camera to VRM with Web

https://github.com/OpenLive3D/OpenLive3D.github.io

You can experience OpenLive3D at the following site.

openlive3d.com/
https://openlive3d.com/

When you access the above site, the browser will ask you to use the camera, so select the camera and click 'Allow'.

If you access and wait for a while to load, you will see a female avatar with long hair.

When you show your face on your webcam, your facial expressions are reflected in your avatar.

The top of the left icon is a button to switch hand capture on / off. When hand capture was enabled, not only hand movements but also finger movements were captured.

Below the hand capture icon are icons that change facial expressions. For example, this is the face when angry.

Sad face

smile

surprised face

Smile

straight face

'AUTO' is a mode that reads facial expressions with camera capture.

The bottom '0' will make the eyes '][' and the mouth movements will not be captured either.

Click the top left logo.

Then, the menu of OpenLive3D came out. By selecting 'Upload VRM Model' from 'Model Manager', you can upload your

This time, I loaded the VRM file of

In 'Video Control', you can display the point being tracked by the webcam.

In 'Setting', it is possible to set each parameter in detail.

In 'BACKGROUND' of 'Setting', it is possible to change the background color and upload a background image.

In “Parameters”, you can check parameter changes.

In addition, when I actually used it, the operation was quite heavy, and when I moved my hand vigorously, the VRM model sometimes disappeared for a moment. On GitHub , it was recommended to enable the hardware acceleration of the browser, but it was quite slow even if it was actually enabled, so it seems that a certain PC spec is required to run comfortably.