End-to-End 3D Face Reconstruction with Expressions and
Specular Albedos from Single In-the-wild Images
Qixin Deng , Binh Huy Le, Aobo Jin, and Zhigang Deng
ACM International Conference on Multimedia 2022
Abstract: Recovering 3D face models from in-the-wild face images has numerous potential applications. However, properly modeling complex lighting effects in reality, including specular lighting, shadows, and occlusions, from a single in-the-wild face image is still considered as a widely open research challenge. In this paper, we propose a convolutional neural network based framework to regress the face model from a single image in the wild. The outputted face model includes dense 3D shape, head pose, expression, diffuse albedo, specular albedo, and the corresponding lighting conditions. Our approach uses novel hybrid loss functions to disentangle face shape identities, expressions, poses, albedos, and lighting. Besides a carefullydesigned ablation study, we also conduct direct comparison experiments to show that our method can outperform state-of-art methods both quantitatively and qualitatively.
Demo Video on Youtube