NeXt-Stereo: Focusing on Cost Aggregation and Disparity Refinement for Lightweight Stereo-Matching

"### Manuscript in Submission

I am currently in the process of submitting my manuscript to IEEE Transactions on Instrumentation and Measurement. Upon acceptance of the paper, I will update this section with the necessary details. Thank you for your interest and understanding."

How to use

Environment

• Python 3.8
• Pytorch 1.10

Install

Create a virtual environment and activate it.

conda create -n nextstereo python=3.8
conda activate nextstereo

Dependencies

conda install pytorch torchvision torchaudio cudatoolkit=11.3 -c pytorch -c nvidia
pip install opencv-python
pip install scikit-image
pip install tensorboard
pip install matplotlib 
pip install tqdm
pip install timm
pip install basicsr

Install casual-conv1d

cd causal-conv1d

python setup.py install

Install mamba

cd mamba

python setup.py install

Data Preparation

Download Scene Flow Datasets, KITTI 2012, KITTI 2015

Train

Firstly, train network for 24 epochs,

python main_sceneflow.py --attention_weights_only True --logdir ./checkpoints/sceneflow/complete

Use the following command to train model on KITTI (using pretrained model on Scene Flow),

python main_kitti.py --loadckpt ./checkpoints/sceneflow/complete/checkpoint_000023.ckpt --logdir ./checkpoints/kitti

Evaluation on Scene Flow and KITTI

Pretrained Model

NeXt-Stereo

• sceneflow.ckpt

Generate disparity images of KITTI test set,

python save_disp.py

Submitted to KITTI benchmarks

python save_disp.py