May 27, 2023 23:06:00

I tried using the application 'Kalidoface Lab' that anyone can become a VTuber that can track facial expressions and actions with just a webcam

To become a VTuber, you need a tracking device and software to read facial movements and expressions in addition to a 3D model. However, the difficulty is that it costs money to prepare '3D model' and 'tracking device & software'. With the free web application ' Kalidoface Lab ' developed by Mr.

Rich , an AI researcher at Google, you can easily become a VTuber by tracking facial expressions and body movements with just a webcam, so I actually tried using it. rice field.

Kalidoface Lab
https://lab.kalidoface.com/

When you access Kalidoface Lab from the link above, a 3D avatar will be displayed on the screen. To perform tracking using a webcam, click the camera button on the right side of the screen.

When asked for permission to use the camera, click 'Allow'.

Now you can reflect your movements on your 3D avatar simply by changing your expression or moving your face towards the webcam.

Below is how the 3D avatar is actually moved. In addition to moving the face left and right and turning the neck, 'opening and closing the mouth', 'opening and closing the eyes', and 'gaze' are also reproduced.

I tried using 'Kalidoface Lab' that can become a VTuber with just a webcam - YouTube


It is also possible to change the pose of the 3D avatar in Kalidoface Lab. To switch poses, click 'Preference' at the top left of the screen, select 'Avatar', and click 'Resting Pose'.

Then the pose selection field will be displayed, so click it.

Various poses such as 'sitting' and 'operating a smartphone' are available as selection candidates. Try 'Sitting' and click 'Done'.

The pose of the 3D avatar changed as instructed.

Kalidoface Lab can also load VRM files and control your favorite 3D avatars. To load the VRM file, click 'Upload' at the top left of the screen.

When the file selection screen is displayed, select the VRM file you want to move. This time, I selected the VRM file of GIGAZINE's anthropomorphic character '

GIGAZINE-chan (provisional) '.

After waiting for a while, GIGAZINE-chan (temporary) was displayed on the screen. For some reason the front and back were reversed, so drag the screen to adjust the camera direction.

He turned to me.

Below is how I tried to move GIGAZINE-chan (provisional). If you use Kalidoface Lab, you can move the 3D avatar without downloading software, so it is convenient when you say ``I have prepared a 3D avatar but I don't know how to move it''.

I tried moving GIGAZINE-chan (provisional) using the free web application 'Kalidoface Lab' - YouTube