High-frequency Stereo Matching Network

High-Frequency Stereo Matching Network
CVPR 2023, Highlight
Haoliang Zhao, Huizhou Zhou, Yongjun Zhang, Jie Chen, Yitong Yang and Yong Zhao

@inproceedings{zhao2023high,
  title={High-Frequency Stereo Matching Network},
  author={Zhao, Haoliang and Zhou, Huizhou and Zhang, Yongjun and Chen, Jie and Yang, Yitong and Zhao, Yong},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  pages={1327--1336},
  year={2023}
}

Software Requirements (Recommended)

PyTorch 1.12.0
CUDA 11.7

pip install scipy
pip install tqdm
pip install tensorboard
pip install opt_einsum
pip install imageio
pip install opencv-python
pip install scikit-image
pip install einops

The program runs in a variety of environments, but the results may vary slightly.

Required Data

To evaluate/train High-Frequency Stereo Matching Network, you will need to download the required datasets.

• Sceneflow
• Middlebury
• KITTI-2015

By default stereo_datasets.py will search for the datasets in these locations. You can create symbolic links to wherever the datasets were downloaded in the datasets folder

├── datasets
    ├── FlyingThings3D
        ├── frames_cleanpass
        ├── frames_finalpass
        ├── disparity
    ├── Monkaa
        ├── frames_cleanpass
        ├── frames_finalpass
        ├── disparity
    ├── Driving
        ├── frames_cleanpass
        ├── frames_finalpass
        ├── disparity
    ├── KITTI
        ├── testing
        ├── training
        ├── devkit
    ├── Middlebury
        ├── MiddEval3

Build Sampler (Optional)

cd sampler
rm -r build corr_sampler.egg-info dist
python setup.py install && cd ..

Train

bash ./train.sh

Evaluate

Set the arguments in evaluate_stereo.py and execute

python evaluate_stereo.py

Pretrained Weights

• Sceneflow
• Middlebury
• KITTI-2015

Acknowledgement

Special thanks to RAFT-Stereo for providing the code base for this work.

RAFT-Stereo [BibTeX]

@inproceedings{lipson2021raft,
  title={RAFT-Stereo: Multilevel Recurrent Field Transforms for Stereo Matching},
  author={Lipson, Lahav and Teed, Zachary and Deng, Jia},
  booktitle={International Conference on 3D Vision (3DV)},
  year={2021}
}