Official TP-GAN Tensorflow implementation for the ICCV17 paper "Beyond Face Rotation: Global and Local Perception GAN for Photorealistic and Identity Preserving Frontal View Synthesis" by Huang, Rui and Zhang, Shu and Li, Tianyu and He, Ran.
The goal is to recover a frontal face image of the same person from a single face image under any poses.
Here are some examples from the paper.
Synthesized testing images of all poses, corresponding illumination in Setting 2 (and its cropped input) in MultiPIE can be obtained here Google Drive.
Synthesized images for other illumination condition and/or training set can be obtained upon request. If you would like to access the original MultiPIE dataset, please contact MultiPIE.
Here are random examples of 10 testing image pairs for each degree.
15 and 30 degrees:
45 and 60 degrees:
75 and 90 degrees:
It was initially written in Tensorflow 0.12.
This is an initial release of code, which may not be fully tested. Refinement, input data example, pre-trained models, and precomputed testing image features will come later.
The input is cropped with the Matlab script face_db_align_single_custom.m, which accepts 5 keypoints and outputs a cropped image and transformed keypoints.
Some example cropping outputs is shown in folder data-example.
The keypoints can be extracted from off-the-shelf landmark detectors, e.g. 'Zhang et al. Combining Data-driven and Model-driven Methods for Robust Facial Landmark Detection, 2016'. The synthesis performance is similar to using manually labelled keypoints.
We thank Xiang Wu for providing the face feature network. We load it as DeepFace in the code, the weights are from a custom Light-CNN cafeemodel file.
If you like our work or find our code useful, welcome to cite our paper!
Any suggestion and/or comment would be valuable. Please send an email to Rui at huangrui@cmu.edu or other authors.
@InProceedings{Huang_2017_ICCV,
author = {Huang, Rui and Zhang, Shu and Li, Tianyu and He, Ran},
title = {Beyond Face Rotation: Global and Local Perception GAN for Photorealistic and Identity Preserving Frontal View Synthesis},
booktitle = {The IEEE International Conference on Computer Vision (ICCV)},
month = {Oct},
year = {2017}
}
This code is freely available for free non-commercial use, and may be redistributed under the conditions set by the license. Please, see the license for further details. For commercial queries, please contact Rui Huang and Ran He.