"3D Face Reconstruction" which can also create 3D model from facial features and replace it with others' face

"3D Face Reconstruction"Can generate a 3D face model from the face of a person in the image or movie. The generated face model can synthesize only a part of eyes etc. to the face of another person or replace the whole face.

GitHub - Yadira F / PRNet: The source code of 'Joint 3D Face Reconstruction and Dense Alignment with Position Map Regression Network'.
https://github.com/YadiraF/PRNet

3D Face reconstruction is a paper published by Mr. Yao Feng of Shanghai Jiao Tong University(PDF file)Joint 3D Face Reconstruction and Dense Alignment with Position Map Regression NetworkHe actually implemented it based on himself.

The main functions of 3D Face reconstruction are the following five points.

◆ Face position detection
Identify the face part from the person's image and detect the position of the vertex on the face. Although it is possible to synthesize images that are not facing the front, it is impossible to make 3D models of parts that can not be confirmed from images.


◆ 3D face synthesis
Acquire the color corresponding to each vertex from the image and build up the face model.


Determination of face angle
By accurately determining which direction the person in the image is facing, it is possible to eliminate the sense of incompatibility when synthesizing different faces.


◆ Perception of depth
As in the following image, even if 3D modeling of the face of a person shown at an oblique angle does not create a 3D model that has an ear at a position away from the eyes, reproduces a positional relationship without discomfort I can do it.


◆ Texture editing
By composing the face of a person as a 3D model, it is possible to synthesize the eyes of an animated character on the face of a person as follows ... ...


It is also possible to replace the face of President Trump as a face of various people.


Moreover, it is said that you can generate 3D models at high speed with GPU, and if you use GeForce GTX 1080 you can generate face models in real time from 100 fps movies or more. The appearance of actually generating the face model in real time can be confirmed in the following movie.

Joint 3D Face Reconstruction and Dense Alignment with Position Map Regression Network - YouTube


3D Face ReconstructionPythonFor 2.7 and developed libraries necessary for installation and execution,3D Face Reconstruction documentIt is explained in.