DeepFaceDrawing: Deep Generation of Face Images from Sketches

 

Shu-Yu Chen1 *          Wanchao Su2 *          Lin Gao1 †          Shihong Xia1          Hongbo Fu2      

 

 

1Institute of Computing Technology, Chinese Academy of Sciences

 

2City University of Hong Kong      

 

*Authors contributed equally  

 

Corresponding author  

 

 

 

Accepted by Siggraph 2020

 

 

 

 

Figure: Our DeepFaceDrawing system allows users with little training in drawing to produce high-quality face images (Bottom) from rough or even incomplete freehand sketches (Top). Note that our method faithfully respects user intentions in input strokes, which serve more like soft constraints to guide image synthesis.

 

 

 

Abstract

 

Recent deep image-to-image translation techniques allow fast generation of face images from freehand sketches. However, existing solutions tend to overfit to sketches, thus requiring professional sketches or even edge maps as input. To address this issue, our key idea is to implicitly model the shape space of plausible face images and synthesize a face image in this space to approximate an input sketch. We take a local-to-global approach. We first learn feature embeddings of key face components, and push corresponding parts of input sketches towards underlying component manifolds defined by the feature vectors of face component samples. We also propose another deep neural network to learn the mapping from the embedded component features to realistic images with multi-channel feature maps as intermediate results to improve the information flow. Our method essentially uses input sketches as soft constraints and is thus able to produce high-quality face images even from rough and/or incomplete sketches. Our tool is easy to use even for non-artists, while still supporting fine-grained control of shape details. Both qualitative and quantitative evaluations show the superior generation ability of our system to existing and alternative solutions. The usability and expressiveness of our system are confirmed by a user study.

 

 

 

paper thumbnail

Paper

 

DeepFaceDrawing: Deep Generation of Face Images from Sketches

Supplemental Materials

 

Code

 

Jittor     Pytorch[Comming soon]

 

Video

 

Demo    YouTube    SIGGRAPH Technical Papers Preview Trailer

 

System

 

Online System   

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Popular Press

 

 

 

 

paper thumbnail paper thumbnail paper thumbnail paper thumbnail paper thumbnail paper thumbnail
paper thumbnail paper thumbnail paper thumbnail paper thumbnail paper thumbnail paper thumbnail
paper thumbnail paper thumbnail paper thumbnail paper thumbnail paper thumbnail paper thumbnail
paper thumbnail paper thumbnail paper thumbnail paper thumbnail paper thumbnail paper thumbnail

 

 

 

 

 

 

BibTex

 

@article {chenDeepFaceDrawing2020,
    author = {Chen, Shu-Yu and Su, Wanchao and Gao, Lin and Xia, Shihong and Fu, Hongbo},
    title = {{DeepFaceDrawing}: Deep Generation of Face Images from Sketches},
    journal = {ACM Transactions on Graphics (Proceedings of ACM SIGGRAPH 2020)},
    year = {2020},
    volume = 39,
    pages = {72:1--72:16},
    number = 4
}

 

 

 

 

 

 

 


 

Last updated on July, 2020.

free hit counter
free hit counter