Disney develop Disney "FaceDirector" technology which synthesizes multiple images and creates expressions such as anger, sorrow, and pleasure linearly

A skilled actor skillfully manipulates facial expressions and intonation makes it possible to create completely different scenes in the same dialogue, but image technology that freely adjusts emotional expression by synthesizing multiple images "FaceDirector"Disney ResearchDeveloped.

Disney Research »FaceDirector: Continuous Control of Facial Performance in Video
http://www.disneyresearch.com/publication/facedirector/

The following movie explains what kind of image technology FaceDirector is.

FaceDirector: Continuous Control of Facial Performance in Video - YouTube


FaceDirector is a technology that can synthesize two different facial expressions linearly without failure. A man who speaks a line of speech. Emotions appear in the expression, but the timing at which emotions appear is usually different for each shooting.


In Take 2, while expressing the same dialogue, the facial expressions themselves are largely changed.


Even if you try to simply synchronize without changing the timing of sounding ... ...


As the timing of words is different, naturally the expression also falls apart.


FaceDirector analyzes the timing of actors' facial expressions and cues (signs to start dialog) of two pictures and graphs it, and seamlessly synthesizes the two videos accordingly.


By doing this, not only will the two movies shot timings to be spoken ....


By changing the "proportion" of take 1 and take 2, "expression" itself can be adjusted linearly. The graph between Take 1 and Take 2 displayed on the left is a graph showing that the time is taken on the horizontal axis, the takeoff increases as the vertical axis goes up, and the proportion of take 2 increases as going down, as shown in the graph The image synthesized on the right side is displayed. Since the shape of the graph can be freely changed, expressions can be freely created.


You can adjust your emotions with FaceDirector. First of all, I shot with emotional expression of "anger".


The following take 2 expresses the feelings of "sorrow".


As shown in the graph, first let me display it with feelings of sadness of take 1 ....


When it changes to the feeling of anger of take 2 at a stretch at a certain timing it is like this. As the graph plummets, emotional expression changes from sadness to anger at a stretch, but you can confirm that the synthesized image changes smoothly without collapse.


Emotions of "fear" and ...


Combining "natural" emotions.


Natural facial expressions ......


To the expression of terror with the change of the graph as the boundary.


If you wave the graph ......


From nature to fear → to nature, you can change your emotional expression.


It is also possible to change the expression extremely. From the depths of sorrow ......


To the peak of happiness at a stretch.


FaceDirector can handle scenes with motion. A woman who speaks serifs while walking on the bridge.


Taking a second shot while walking on the same bridge.


While synthesizing emotion expression by FaceDirector, it also corrects the inclination of the image.


You can adjust the video seamlessly without failing even in scenes with motion.


According to Disney Research who developed FaceDirector, it is not unusual to take several scenes of the same scene in search of the best images in film shooting, which is a big cause of the shooting cost . Since FaceDirector can process images shot with a normal camera without software using a special camera such as a 3D camera, by using FaceDirector to adjust and modify facial expressions, Film director is able to get the best footage, so shooting cost dramatically lowers, and even creators who do not have abundant production cost can shoot high quality movies.