💡 Using The Face of Arts and DST to generate geometry-aware face caricatures.
📖 This project in details with analysis on Medium: Geometry-Aware Style Transfer: Implementation and Analysis
- Python 3
- pytorch, torchvision, cudatoolkit, numpy, PIL, matplotlib, sklearn, cv2, imutils, scikit-image
Set up the virtual environment:
python3 -m venv venv
source venv/bin/activate
pip3 install -r requirements.txt
after setup, you can just run source venv/bin/activate to activate the venv later on.
preprocessing.sh is used to convert all Face of Art style images' correspondence points into .txt, this only need to be run once if you don't already have data/style/pts containing all Face of Art style images.
run.sh is a script automating the two steps for geometry style transfer.
Currently, run.sh serve as a demo with Barack Obama as the content image and
Vincent van Gogh's self-portrait with the style image.
This step will obtain the facial landmarks for the content image with dlib.
python face_landmark_detection.py ${content_path} ${style_path} ${content_pts_path} ${style_pts_path_dlib}
Now, run deformable style transfer, this step will take ~1.5 minutes on a single GPU.
Please check parameter adjustment in DST repo [code].
python main.py ${content_path} ${style_path} ${content_pts_path} ${style_pts_path} \
${output_dir} ${output_prefix} ${im_size} ${max_iter} \
${checkpoint_iter} ${content_weight} ${warp_weight} ${reg_weight} ${optim} \
${lr} ${verbose} ${save_intermediate} ${save_extra} ${device}
Result from run.sh with Barack Obama as the content image and
Vincent van Gogh's self-portrait with the style image. Images from left to right are: input content image, input style image, a standard style transfer output, demo output.
- Dilb facial landmark
- Deformable Style Transfer (DST) [code] [paper]
- The Face of Art: Landmark Detection and Geometric Style in Portraits [website] [paper]
- Style Transfer by Relaxed Optimal Transport and Self-Similarity. Nicholas Kolkin, Jason Salavon and Gregory Shakhnarovich. CVPR 2019. [paper] [code] [David Futschik's implementation]
- WarpGAN: Automatic Caricature Generation. Yichun Shi, Debayan Deb and Anil K. Jain. CVPR 2019. [paper] [code]
- Neural Best-Buddies: Sparse Cross-Domain Correspondence. Kfir Aberman, Jing Liao, Mingyi Shi, Dani Lischinski, Baoquan Chen and Daniel Cohen-Or. SIGGRAPH 2018. [paper] [code]