v2.3.0 - Community Builds Thread

[gianni-rosato]

Community Builds Thread

This is a place for the community to share unofficial tools not affiliated with the project — mainly consisting of binaries compiled by community members.

Trust

⚠️ Community-built binaries are not affiliated with the SVT-AV1-PSY project & are to be used at your own risk. If you encounter bugs while using a community-built binary, ensure that you double-check with the person responsible for providing you with the binary before submitting an issue report.

Architecture

When downloading pre-compiled binaries, you might see AVX, AVX2, AVX-512, x86-64-v3, etc. If you don't know exactly what ISA extensions your CPU supports, here is a chart to help you quickly understand your hardware's support:

Not Vendor-Specific

x86-64    : Processors lacking SSE4.2 and newer features
x86-64-v2 : Processors with features up to and including SSE4.2
x86-64-v3 : Processors with features up to and including AVX and/or AVX2
x86-64-v4 : Processors with features up to and including AVX512

ISA extensions like AVX, AVX2, & AVX512 are for vector processing which SVT-AV1-PSY relies heavily on for its fast multithreaded performance. Below are some helpful charts to help you narrow down the best options if you have popular x64 hardware:

AMD

Generation                                             Vectors  Build
Zen   (1xxx)                                           AVX2     x86-64-v3 / znver1
Zen+  (1xxxAF-2xxx-3xxxG/H/U/C)                        AVX2     x86-64-v3 / znver1
Zen2  (3xxx-4xxxS/G/H/U-5700U-5500U-5300U-7520U-7320U) AVX2     x86-64-v3 / znver2
Zen3  (5xxx-7330U-7530U-7730U)                         AVX2     x86-64-v3 / znver3
Zen3+ (6xxx-7xxxH/U)                                   AVX2     x86-64-v3 / znver3
Zen4  (7xxx-8xxxG)                                     AVX512   x86-64-v4 / znver4

Intel (Desktop)

Generation                Vectors  Build
Haswell     (4th gen)     AVX2     x86-64-v3 / haswell
Broadwell   (5th gen)     AVX2     x86-64-v3 / broadwell
Skylake     (6th gen)     AVX2     x86-64-v3 / skylake
Kaby Lake   (7th gen)     AVX2     x86-64-v3 / skylake
Coffee Lake (8-9th gen)   AVX2     x86-64-v3 / skylake
Comet Lake  (10th gen)    AVX2     x86-64-v3 / skylake
Rocket Lake (11th gen)    AVX512   x86-64-v4 / rocketlake
Alder Lake  (12th gen)    AVX2     x86-64-v3 / alderlake
Raptor Lake (13-14th gen) AVX2     x86-64-v3 / raptorlake

Included below is even more information about what is available when specifying -march & -mtune on x64 CPUs:

Known valid x64 arguments for -march=:

i386 i486 i586 pentium lakemont pentium-mmx winchip-c6 winchip2 c3 samuel-2 c3-2 nehemiah c7 esther i686 pentiumpro pentium2 pentium3 pentium3m pentium-m pentium4 pentium4m prescott nocona core2 nehalem corei7 westmere sandybridge corei7-avx ivybridge core-avx-i haswell core-avx2 broadwell skylake skylake-avx512 cannonlake icelake-client rocketlake icelake-server cascadelake tigerlake cooperlake sapphirerapids emeraldrapids alderlake raptorlake meteorlake graniterapids graniterapids-d arrowlake arrowlake-s lunarlake pantherlake bonnell atom silvermont slm goldmont goldmont-plus tremont gracemont sierraforest grandridge clearwaterforest knl knm intel geode k6 k6-2 k6-3 athlon athlon-tbird athlon-4 athlon-xp athlon-mp x86-64 x86-64-v2 x86-64-v3 x86-64-v4 eden-x2 nano nano-1000 nano-2000 nano-3000 nano-x2 eden-x4 nano-x4 lujiazui yongfeng k8 k8-sse3 opteron opteron-sse3 athlon64 athlon64-sse3 athlon-fx amdfam10 barcelona bdver1 bdver2 bdver3 bdver4 znver1 znver2 znver3 znver4 znver5 btver1 btver2 generic native

Known valid x64 arguments for -mtune=:

generic i386 i486 pentium lakemont pentiumpro pentium4 nocona core2 nehalem sandybridge haswell bonnell silvermont goldmont goldmont-plus tremont sierraforest grandridge clearwaterforest knl knm skylake skylake-avx512 cannonlake icelake-client icelake-server cascadelake tigerlake cooperlake sapphirerapids alderlake rocketlake graniterapids graniterapids-d arrowlake arrowlake-s pantherlake intel lujiazui yongfeng geode k6 athlon k8 amdfam10 bdver1 bdver2 bdver3 bdver4 btver1 btver2 znver1 znver2 znver3 znver4 znver5

• Note: -march=foo implies -mtune=foo unless you also specify a different -mtune. This is one reason why using -march is better than just enabling options like -mavx without doing anything about tuning.

Antivirus

🛑 Be wary of antivirus software on Windows detecting EXEs distributed here as malicious software. While they may not always legitimately be malicious, it is important to maintain a healthy level of skepticism when running code that someone else has compiled.

[gitoss]

If anyone wants to experiment with speeding up encoding with -ffast-math ... it sets -fno-honor-nans and generates a nan warning in enc_dec_process.c.

This can be cirumenvented by using -Xclang -funsafe-math-optimizations which still sets a lot of fp optimizations, but not finite math only. I'm using Visual Studio with LLVM 19.1.3 https://releases.llvm.org/19.1.0/tools/clang/docs/UsersManual.html

I still have no idea if this speed bump produces visible artifacts, or what other non-intended consequences there could be. Works just fine for me so far though.

[ItachiUchiha-IU]

I have made this from my smartphone while using my pc on remote, so if I made something wrong and anything doesn't work pls comment below.
Mainline changed a default cmake flag so we had to turn it off because it would've overwritten the default LTO setting (which is fat) and built with LTO thin which is slower.

The zip includes the generic builds: x86-64, x86-64-v2, x86-64-v3, x86-64-v4
and the march specific builds: znver1, znver2, znver4, haswell, broadwell, skylake, rocketlake, alderlake, raptorlake

cmake --fresh -B svt_build -T ClangCL -DBUILD_SHARED_LIBS=OFF -DSVT_AV1_LTO=OFF -DENABLE_AVX512=$ONOFF -DCMAKE_CXX_FLAGS_RELEASE="/DNDEBUG /clang:-O2 -flto -march=$allmarch" -DCMAKE_C_FLAGS_RELEASE="/DNDEBUG /clang:-O2 -flto -march=$allmarch"

The zip is a Matryoshka cause github doesnt support 7z
SvtAv1EncApp-windows-all-march-but-znver3.zip
For more infos, findings and discussions about the builds (the znver3 case, the parameters used and tested, w10-11, clang 17-19, ninja, PGO, BOLT), check the Community Builds Thread for v2.2.1-A

A PGO optimized binary for znver4 is below.
For a fast info on my R9 7900 it gives ~5% speed gains in presets that are not too low like p-1 (didn't test p0 nor p1; p2 and 4 are ~5% faster); if it crashes on u, pls comment below about it and use the non-pgo optimized binary

Everyone is especially invited to contribuite about PGO and BOLT on Windows!! (several threads and comments exist on the Community Builds Thread for v2.2.1-A linked above)

[ItachiUchiha-IU]

@ItachiUchiha-IU Do you plan for v2.3.0-A build?

I was planning to release them in the community build for v2.3.0-A but, being that it's not up yet, I'll start putting them here.
But there's some informations that need to be given, because this new version ends up in non-stable binaries. (fast info: it's strongly advised to use av1an to keep the encoding running even if a chunk crashes)
Coincidentally I'm away from home during this new version too, so I'm posting them from phone in remote again, and later (this afternoon/evening EU) I'll update this message (and reply to gitoss too).

OK here's the infos we've gathered so far:

1. v2.3.0-A is slower than the previous version, especially if not using --psy-rd (or using very low values); as confirmed with the "PSY guys" (the devs that work on this fork), this is the expected behavior as the --psy-rd code runs and it's called quite often; thus, with it off or with such low values, from tests made on different machines on both linux and windows, it can be slower by 10-20%

• "solution" : use --psy-rd 0.1 or higher, and the speed difference will go down a lot (in some of my tests on windows I got down to 2-3% speed diff, while on linux it seems to be working a little better in some cases)

2. the binaries for v2.3.0-A built on windows crash; the cross-compiled binary doesn't seem to crash but it's quite slower than the windows builds: some got an 8% speed diff, some way more (I can't test myself if it crashes or what speed diff I get cause of the reasons mentioned above)

• "solution" : use av1an. av1an splits the encode per scene before encoding them, and if a chunk crashes, it simply re-encodes that single chunk and the encoding continues without any major problem
• the psy guys already know about the binaries crashing on windows cause the only ones reporting these crashes are the ones of us that are on windows. we have already started looking into it together with them but it could still end up being something not easy to fix, so it may be something that will get worked on tog with mainline after v3 is released.

3. I don't remember if I said everything I wanted to say, so if something else comes to mind later, or something else comes out, I'll edit it and/or add points.

svt-av1-psy_2024-12-25_20.29.49_MULTI-MARCH_clang19.1.5_2.zip
svt-av1-psy_2024-12-25_20.42.02_MULTI-MARCH_av1an-pgo_clang19.1.5.zip
svt-av1-psy_2024-12-25_21.08.08_MULTI-MARCH_dovi+hdr10plus_clang19.1.5_2.zip
svt-av1-psy_2024-12-25_23.13.06_MULTI-MARCH_dovi+hdr10plus_av1an-pgo_clang19.1.5_2.zip

[igorbaryshev]

some speed regression in w11's 24H2 update

This one is probably due to Memory integrity feature being on by default. I don't recall it being enabled in prior versions, and disabling it netted almost 2% performance gain.

[ItachiUchiha-IU]

I did find a way to boost p-1 too

Just for completeness' sake, could you please say how you boosted preset 1, too? Thanks.

For p-1/2/3 we mean the negative presets, thus for p-1 I meant --preset -1.
Basically I tried something random and it worked: I simply did PGO training with a 6 min clip (through av1an) and I got the previously mentioned speed gains in most contents, but the binary kept crashing. So I ended up trying several tricks and in the end I fixed it but merging the p-1 profile with a p10 profile.
The real problem tho is that training is a lot slower than encoding, thus resulting in it being basically useless considered the nearly negligible gain (using w11 and/or --ffastmath have a bigger impact). I even wanted to test on a 24min clip but the training would've been days long and I just gave up.

[gitoss]

2. the binaries for v2.3.0-A built on windows crash; the cross-compiled binary doesn't seem to crash but it's quite slower than the windows builds: some got an 8% speed diff, some way more (I can't test myself if it crashes or what speed diff I get cause of the reasons mentioned above)

I am now able to verify this - your native build crashes, Andarwinux' cross-compibled build doesn't, and it's exactly the same with my own native and cross builds.

The crashes cease if --lp is lowered, using a mimalloc-injected exe doesn't help, setting --psy-rd 0 doesn't help.

This is annoying because up to now compiling native pgo builds were just fine, and I have no idea what commit between the 2.3.0 and 2.3.0-A release is the culprit @gianni-rosato

[Patman86]

Have a look here:

#114

The commit that causes the crash ist mantioned here.

[ItachiUchiha-IU]

I have made this from my smartphone while using my pc on remote, so if I made something wrong and anything doesn't work pls comment below.
More infos in the comment above.

Znver4 optimized with PGO with Stefan Edberg clip

cmake --fresh -B svt_build -T ClangCL -DBUILD_SHARED_LIBS=OFF -DSVT_AV1_LTO=OFF -DENABLE_AVX512=ON -DCMAKE_CXX_FLAGS_RELEASE="/DNDEBUG /clang:-O2 -flto -march=znver4" -DCMAKE_C_FLAGS_RELEASE="/DNDEBUG /clang:-O2 -flto -march=znver4"

SvtAv1EncApp znver4 with PGO.zip

On my R9 7900 it gives ~5% speed gains in presets that are not too low like p-1 (didn't test p0 nor p1; p2 and 4 are ~5% faster); if it crashes on u, pls comment below about it and use the non-pgo optimized binary

[gitoss]

For Intel cpu, using their 2025.0 compiler could be beneficial: https://www.intel.com/content/www/us/en/developer/articles/release-notes/intel-oneapi-dpc-c-compiler-release-notes.html

"Hardware Profile Guided Optimization (HWPGO): Key improvements include enhanced profile propagation for better accuracy, additional profile-driven optimizations to further boost performance, and early support for "pseudo probes" on Windows as an alternative to DWARF for profiling. Additionally, HWPGO has introduced selective function outlining, allowing for specific functions to be optimized based on profiling data, further enhancing runtime efficiency.​"

[Uranite]

Full LTO build for Termux
Contains arm and aarch64
Pick one according to your arch, and install via dpkg -i, afterwards, you can encode via libsvtav1 in FFmpeg

termux-svt-av1-psy-v2.3.0.zip

[igorbaryshev]

I'm not very experienced with building/tuning stuff, but I finally managed to make a clang 17 znver2 -O2 LTO+PGO with Dolby Vision support build that was "trained" on 8 clips of various resolutions (from 540p to 2160p), that include Dolby Vision, regular HDR, and just general 8bit-to-10bit converted clips, with preset 6 and added film grain, --dolby-vision-rpu and HDR with other related settings provided by StaxRip automatically where applicable, and some other custom settings that I usually use for encoding. The resulting LTO+PGO build is ~4%+ faster than just LTO.
Unfortunately, it didn't want to compile with HDR10+ support on clang, so I just dropped it 😔as I don't have anything with HDR10+ that needs converting, yet.
To measure performance, I used chunked encoding (2 chunks) in StaxRip for full saturation
LTO-only time 00:06:07 + 00:06:16
LTO+PGO time 00:05:53 + 00:06:00
The resulting build is surprisingly slim, just 5.45 MiB for LTO+PGO, +353 KiB for DoVi.
Test for yourself!
svt-av1-psy_znver2_LTO[+PGO].zip
If there're any tips, suggestions, requests, please chime in 🤗

[Andarwinux]

I reused mpv-winbuild to provide a fully featured (with libdovi and libhdr10plus) statically linked Windows daily build of svtav1-psy.
https://github.com/Andarwinux/mpv-winbuild/releases

This build is cross-compiled on GHA using trunk clang (20.0.0git) and mingw-w64, with ThinLTO+IRPGO+Polly enabled and mimalloc integrated. Currently available in x86-64-v3 and cortex-a76 (arm64).

PGO training is done via ffmpeg and is fairly arbitrary, so it may not provide performance comparable to other PGO builds.

[Andarwinux]

Are large pages enabled on your system? If you build your own mimalloc, you may need to set MIMALLOC_ALLOW_LARGE_OS_PAGES=1. Also MIMALLOC_ARENA_EAGER_COMMIT=1 may gain extra performance, but may lead to crazy VM usage, so it is not enabled by default.

[igorbaryshev]

Are large pages enabled on your system? If you build your own mimalloc, you may need to set MIMALLOC_ALLOW_LARGE_OS_PAGES=1. Also MIMALLOC_ARENA_EAGER_COMMIT=1 may gain extra performance, but may lead to crazy VM usage, so it is not enabled by default.

I'm still getting the exact same ~1% performance improvement after adding those variables, adding myself to the list in gpedit that allows to lock pages in memory and rebooting.
idk if there's anything needed to be done to make mimalloc be active, but when I set MIMALLOC_VERBOSE=1 and run the app, I get the output with mimalloc stuff. The app also uses more memory, which probably indicates that mimalloc is in fact doing something. That was without setting any variables you mentioned or allowing myself to lock pages in memory.

[Andarwinux]

Maybe the improvement is platform related. In my case, mimalloc+large page gives 8% improvement.

[gitoss]

Are large pages enabled on your system? If you build your own mimalloc, you may need to set MIMALLOC_ALLOW_LARGE_OS_PAGES=1. Also MIMALLOC_ARENA_EAGER_COMMIT=1 may gain extra performance, but may lead to crazy VM usage, so it is not enabled by default.

These seem to be just env vars - it seems they don't require bulding one's own mimalloc (though I did that because of -march)? https://microsoft.github.io/mimalloc/environment.html

As for cmake vars not enabled by default MI_WIN_USE_FIXED_TLS seeems interesing (if using cmake and not the pre-made .sln for VS) https://github.com/microsoft/mimalloc/blob/dev/CMakeLists.txt

[Andarwinux]

Are large pages enabled on your system? If you build your own mimalloc, you may need to set MIMALLOC_ALLOW_LARGE_OS_PAGES=1. Also MIMALLOC_ARENA_EAGER_COMMIT=1 may gain extra performance, but may lead to crazy VM usage, so it is not enabled by default.

These seem to be just env vars - it seems they don't require bulding one's own mimalloc (though I did that because of -march)? https://microsoft.github.io/mimalloc/environment.html

mimalloc can change some of the default settings, which is what I did with my build.

As for cmake vars not enabled by default MI_WIN_USE_FIXED_TLS seeems interesing (if using cmake and not the pre-made .sln for VS) https://github.com/microsoft/mimalloc/blob/dev/CMakeLists.txt

This is equivalent to the ELF local-exec TLS, which does not support dynamic linking (DLL).