[BUG] 2.3.0-A with --psy-rd >0 produces artifacts and inconsistent results with identical source and settings.

[igorbaryshev]

Overview

• Random block-shaped artifacts at the exact same frames:
  
  
  
  

• A screenshot from another encode to make sure it's not just one video issue:
  
   

• File sizes are inconsistent as well, obviously.
  

• Presets 5, 6, 7 are the most affected. The slowest I tried was 3 and it wasn't as noticeable there. Preset 10 I couldn't really tell.

• Starts to become visible at --psy-rd values around 0.5 and up, but file size inconsistencies are present with any non-0 value at any preset I tried.

• CRF's tried were 35 and 22.5

Branch
In which branch does the issue appear to be occurring?

• master
• testing

Reproduction
Steps to reproduce the behavior:

1. Encode with --psy-rd 6.0 and --preset 6
2. Watch the result

Expected behavior
No obvious artifacts, consistent encoding results.

Platform (please complete the following information):

• Windows 11 24H2
• Clang 19.1.6, and another build by @Andarwinux with Clang 20.0.0git
• Build command for my Clang 19.1.6 build:

cmake -B svt_build --fresh -T ClangCL -DBUILD_SHARED_LIBS=OFF -DENABLE_AVX512=OFF -DLIBDOVI_FOUND=1 -DLIBHDR10PLUS_RS_FOUND=1 -DCMAKE_CXX_FLAGS_RELEASE="/DNDEBUG /clang:-O2 -flto -march=znver1 -fprofile-use=D:\PGO\IR+CSIR.profdata" -DCMAKE_C_FLAGS_RELEASE="/DNDEBUG /clang:-O2 -flto -march=znver1 -fprofile-use=D:\PGO\IR+CSIR.profdata" && cmake --build svt_build --config Release

Version String (please complete the following information):

• Clang 19.6.1 build

SVT-AV1-PSY v2.3.0-A-dirty (release)
PSY Release: A

• Clang 20.0.0git build

SVT-AV1-PSY v2.3.0-A (release)
PSY Release: A

Additional context / Relevant Files

• Was only able to test 10-bit encoding, because 8-bit encodes crash at the very beginning after random number of frames, like up to 400, sometimes as low as 44, average ~120.
• I did several encodes with my Clang 19.1.6 build to verify that it's not just a build difference when it comes to resulting file size and artifact randomness.
• With --psy-rd 0 files seem to be identical from encode to encode.

[BlueSwordM]

Thank you for the bug report. I will try to reproduce it on my end (CachyOS Linux).

Did you try weaker settings (0.5 to 4.0)? Perhaps the calculated distortion is too high, because while I can't reproduce the delerious transform bug, I can see artifacts that never appear at more reasonable strengths (<2.0).

[igorbaryshev]

Did you try weaker settings (0.5 to 4.0)?

Yep, tried 3.0 and 4.0, same story. Just like I said, anything at 0.5 and up becomes pretty visible on my side.

[igorbaryshev]

--psy-rd 1.0

[kuzon3]

im getting the same bug even at 0.5.
built my own svt using the build instructions for windows.
Edit: Ok after some testing found out that if you use the default crf settings. --psy-rd works fine (did my testing with --psy-rd 3.0) but if you use a lower crf setting like crf 18. artifacts start appearing.
tests where made using svts defaults settings (i did switch to slow once i saw no artifacts where appearing at this setting. crf was still at 35(default))

[Ironclad17]

I saw the same occasionally in very noisy, heavily filtered scenes with --psy-rd 1.0. Lowering to crf 10.0 and preset -2 appeared to resolve it, but it may have been random. This occurred in both 8-bit & 10-bit encodes of the same scene.
call ".\vcvars64.bat" && cmake --fresh -B svt_build -G Ninja -DCMAKE_BUILD_TYPE=Release -DBUILD_SHARED_LIBS=OFF -DBUILD_APPS=ON -DSVT_AV1_LTO=OFF -DLIBDOVI_FOUND=ON -DLIBHDR10PLUS_RS_FOUND=ON -DCMAKE_CXX_FLAGS_RELEASE="-Qunused-arguments -Wno-deprecated /DNDEBUG /clang:-O2 -flto -march=znver2 -fprofile-use=".\PGO\IR+CSIR.profdata"" -DCMAKE_C_FLAGS_RELEASE="-Qunused-arguments -Wno-deprecated /DNDEBUG /clang:-O2 -flto -march=znver2 -fprofile-use=".\PGO\IR+CSIR.profdata""

The scene in question was: --crf 15.0 --preset 4 --tune 3 --sharpness 2 --frame-luma-bias 100 --noise-norm-strength 4 --variance-boost-strength 4 --variance-octile 4 --psy-rd 1.0 --enable-dlf 2 --enable-cdef 0 --enable-restoration 0 --enable-tf 0 --fast-decode 1 --color-range 0 --matrix-coefficients 1 --transfer-characteristics 1 --color-primaries 1 --keyint 0 --lp 4
--crf 26.0 and --crf 44.5 had similar issues.

[gianni-rosato]

Hi,

Would you be able to provide a couple of things to assist in our diagnosis?

• A sample clip
• Encoding settings to reproduce the issues you describe
• Test conditions that reproduce the bug on Linux and/or macOS

I'm able to reproduce the issue with inconsistent run-to-run behavior on Linux, but I cannot reproduce any visual issues.

We should probably make this more clear in our documentation, but it is worth noting that SVT-AV1-PSY is not officially supported for Windows and there are no plans to change this any time soon. We're most interested in reproducing the visual inconsistencies you present here in Linux/macOS environments. We greatly appreciate your help.

[igorbaryshev]

• Sample clip ->friends.zip
  *.y4m file was over the size limit packed in a 7z inside zip, so I'm providing the clip that I made the .y4m from with this command: ffmpeg -i friends.mkv -strict -1 friends.y4m
• The simplest encoding settings successfully reproduce the issue on Ubuntu 24.10:
  SvtAv1EncApp --input friends.y4m --output friends.ivf --preset 6 --psy-rd 6.0
• Test conditions were just basic Ubuntu 24.10 and the app itself, built with build.sh -x from testing branch with latest commit being fbe4e20 (just wanted to see whether it got fixed somehow in testing vs master)

[BlueSwordM]

• Sample clip ->friends.zip
  *.y4m file was over the size limit packed in a 7z inside zip, so I'm providing the clip that I made the .y4m from with this command: ffmpeg -i friends.mkv -strict -1 friends.y4m

  • The simplest encoding settings successfully reproduce the issue on Ubuntu 24.10:
    SvtAv1EncApp --input friends.y4m --output friends.ivf --preset 6 --psy-rd 6.0

  • Test conditions were just basic Ubuntu 24.10 and the app itself, built with build.sh -x from testing branch with latest commit being fbe4e20 (just wanted to see whether it got fixed somehow in testing vs master)

Oh thank you, I am now able to reproduce the issue.
On my sources, even at P9, I wasn't able to reproduce this.

Thanks a lot ❤️