An amazing technology to read facial expressions and transfer them to others in real time will be developed

A research team such as Stanford University publishes a technique to read the "display" of a person, transfer the expression to another person, and draw in real time. The image drawn in real time has become an astounding level that it is no longer difficult to distinguish it from the real one.

Real-time Expression Transfer for Facial Reenactment
http://graphics.stanford.edu/~niessner/thies2015realtime.html

You can check the state of "transplanting" just like a person's "facial expression" directly to other people in the following movie.

Real-time Expression Transfer for Facial Reenactment - YouTube


This is an experiment to transfer the facial expression of the right person to the left person. Although the expression of the two people is displayed on the center monitor, the expression of the left person is converted to the image of the right person in real time.


When the left person moves the eyebrow upwards, the image of the left person also changes to a state in which the eyebrow has risen.


Too realistic images have little delay and are quite natural.


Facial expressions can be transferred between men and women.


The facial expression of the right woman has been transplanted to the man on the left in real time, but it has become a picture without discomfort.


The expression of the man in the upper left is portrayed as a picture on the right, which is ported to the man in the lower left.


Even if you move your face up, down, left and right, you can convert facial expressions without problems.


There is no problem even if the face is made oblique.


In this technique, expression is divided into three elements of "shape" "reflectance" "texture".


By analyzing three parts in real time, it is possible to read facial expressions accurately.


Furthermore, motion capture of eyebrows, eyes, nose, mouth changes which are parts of the face.


By combining them, we can create facial expressions freely.


Too natural synthetic,Eerie valleyIt seems that it is completely over.


The upper picture shows the error rate of the shape, the lower picture shows the error rate of the luminance, and the red color shows the error rate is high. We said that combining not only color information (RGB) but also depth information (D) reduces the error rate of facial expression recognition.


If you disturb your real-time facial expression transplant by waving your hand ...


I will fail to read facial expressions for just a moment ... ...


It instantly recovers.


There seems to be little interference.


It can respond flexibly to face movement by high reading performance. I am facing up ... ....


Even if it is covered with both hands ......


You can recognize facial expressions immediately.


There is no problem at all even if the face is facing sideways.


This real-time facial expression transfer technology is said to be done using commercially available RGB-D sensor, and it is possible to immediately analyze each parameter such as "shape" "reflectance" "texture" "light amount" By rendering it, I have succeeded in creating a natural "expression" that can hardly be distinguished from reality. If this technology is installed in a video chat such as Skype or video call and furthermore "voice" can be converted in the same way in some way, a new type of oleole fraud is completed, a movie that has become an entertainer or an actor has been created Or, various possibilities are likely to spread.