A movie that reads facial expressions in real time and moves 3D animation grin gully

The real-time tracking technology continues to progress day by day, and it is possible to operate the PC by detecting the movement of the hands and fingers of the userLeap Motion, And tracking the human facial expression in real timeA movie that can be a 2D characterAnd so on are appearing one after another. International conference on CG technology "SIGGRAPH (SIGGRAPH) 2014"Technology that allows you to move the face of 3D animation by reading the outline, eye direction, orientation, facial expression, etc. of a human figure in real time in real timeZhejiang UniversityIt has been published by the research team. It is unnecessary to input facial data for each user, and movies that move 3D animation by anyone are released as soon as they appear in the camera.

Displaced Dynamic Expression Regression for Real-time Facial Tracking and Animation
http://gaps-zju.org/DDE/


You can see how the user's movements and facial expressions are actually reflected in 3D avatar of 3D animation and moving.

Siggraph 2014: Displaced Dynamic Expression Regression for Real-time Facial Tracking and Animation - YouTube


A man sat in front of a PC equipped with a web camera. Open your mouth ......


When drawing a mouth corner and making a smile, the two faces on the right side of the screen make the same expression as men at the same time.


It is also possible to change the face orientation while making facial expressions.


Even when a man stands up and replaces it with a woman, the two faces are tracking the trend of the face of the woman exactly.


Wink is OK.


Even when facing upwards, the two faces make the same expression as a woman.


Further change the subject ... ...


Every time I try it, it's a technology that can accurately track real time and reflect it in animation.


The gray animation that made expressions according to the movements of the subjects is based on what men have read various expressions in advance.


It is possible to trace the expression of the person in the camera as animation by the base made from men.


Also, compared with existing facial expression recognition systems, the Zhejiang University system on the right side accurately recognizes contours, lip lines and so on.



"Turning diagonally and winking" ... ...


For example, looking at different expressions such as "smiling a face", you can see the accuracy of real-time tracking.


This real-time tracking system is a dark place and ...


Even in outdoor environments, you can recognize the target person and reflect it on 3D animation and digital avatar.


Although we slightly misalign it if you cover your face with your hands ... ...


Even if you wear sunglasses, there is no problem at all, it is an excellent item.