SurfaceNet: Adversarial SVBRDF Estimation from a Single Image

Giuseppe Vecchio, Simone Palazzo and Concetto Spampinato

Overview

This is the official PyTorch implementation for paper "SurfaceNet: Adversarial SVBRDF Estimation from a Single Image".

Abstract

In this paper we present SurfaceNet, an approach for estimating spatially-varying bidirectional reflectance distribution function (SVBRDF) material properties from a single image. We pose the problem as an image translation task and propose a novel patch-based generative adversarial network (GAN) that is able to produce high-quality, high-resolution surface reflectance maps. The employment of the GAN paradigm has a twofold objective: 1) allowing the model to recover finer details than standard translation models; 2) reducing the domain shift between synthetic and real data distributions in an unsupervised way.

An extensive evaluation, carried out on a public benchmark of synthetic and real images under different illumination conditions, shows that SurfaceNet largely outperforms existing SVBRDF reconstruction methods, both quantitatively and qualitatively. Furthermore, SurfaceNet exhibits a remarkable ability in generating high-quality maps from real samples without any supervision at training time.

Method

Instructions

Install dependencies

# clone project   
git clone https://github.com/perceivelab/surfacenet

# install requirements 
cd surfacenet/src
pip install -r requirements.txt

Run training

# src folder
cd surfacenet/src

# run training   
accelerate launch train.py --tag ... --dataset ... --logdir ...

Run inference

# src folder
cd surfacenet/src

# run inference   
python eval.py --ckpt ... --input ... --size ...

Citation

@inproceedings{vecchio2021surfacenet,
  title={SurfaceNet: Adversarial SVBRDF Estimation from a Single Image},
  author={Vecchio, Giuseppe and Palazzo, Simone and Spampinato, Concetto},
  booktitle={Proceedings of the IEEE/CVF International Conference on Computer Vision},
  pages={12840--12848},
  year={2021}
}