Performance gap in Test sets of Scannet vs. Scannet 200?

[TerenceWallel]

Wonderful job. My collaborators and I have found that using PTV3'S code with the weights file, the model performs consistently in the Val set of Scannet and Scannet200, but there is a serious performance degradation in the Test set of Scannet and Scannet200 (over than 3% in Scannet and Scannet200).

We would like to ask if this performance degradation is normal? What steps do we need to follow to get points in the Test set that are consistent with the paper results?

[TerenceWallel]

According to the previous issue #25, I'm guessing that the test set metrics in the paper use more trick or TTA compare with Val set of Scannet. In order to further replicate PTv3, I'm wondering if it would be possible to provide a list of tricks for the test set using in the paper, as well as the associated code implementation?

[Gofinge]

According to the previous issue #25, I'm guessing that the test set metrics in the paper use more trick or TTA compare with Val set of Scannet. In order to further replicate PTv3, I'm wondering if it would be possible to provide a list of tricks for the test set using in the paper, as well as the associated code implementation?

Hi, I think exactly the same as what I mentioned in the issue you pointed out.