v2.3.0-A – Community Builds Thread

[gianni-rosato]

Community Builds Thread

This is a place for the community to share unofficial tools not affiliated with the project — mainly consisting of binaries compiled by community members.

Trust

⚠️ Community-built binaries are not affiliated with the SVT-AV1-PSY project & are to be used at your own risk. If you encounter bugs while using a community-built binary, ensure that you double-check with the person responsible for providing you with the binary before submitting an issue report.

Architecture

When downloading pre-compiled binaries, you might see AVX, AVX2, AVX-512, x86-64-v3, etc. If you don't know exactly what ISA extensions your CPU supports, here is a chart to help you quickly understand your hardware's support:

Not Vendor-Specific

x86-64    : Processors lacking SSE4.2 and newer features
x86-64-v2 : Processors with features up to and including SSE4.2
x86-64-v3 : Processors with features up to and including AVX and/or AVX2
x86-64-v4 : Processors with features up to and including AVX512

ISA extensions like AVX, AVX2, & AVX512 are for vector processing which SVT-AV1-PSY relies heavily on for its fast multithreaded performance. Below are some helpful charts to help you narrow down the best options if you have popular x64 hardware:

AMD

Generation                                             Vectors  Build
Zen   (1xxx)                                           AVX2     x86-64-v3 / znver1
Zen+  (1xxxAF-2xxx-3xxxG/H/U/C)                        AVX2     x86-64-v3 / znver1
Zen2  (3xxx-4xxxS/G/H/U-5700U-5500U-5300U-7520U-7320U) AVX2     x86-64-v3 / znver2
Zen3  (5xxx-7330U-7530U-7730U)                         AVX2     x86-64-v3 / znver3
Zen3+ (6xxx-7xxxH/U)                                   AVX2     x86-64-v3 / znver3
Zen4  (7xxx-8xxxG)                                     AVX512   x86-64-v4 / znver4

Intel (Desktop)

Generation                Vectors  Build
Haswell     (4th gen)     AVX2     x86-64-v3 / haswell
Broadwell   (5th gen)     AVX2     x86-64-v3 / broadwell
Skylake     (6th gen)     AVX2     x86-64-v3 / skylake
Kaby Lake   (7th gen)     AVX2     x86-64-v3 / skylake
Coffee Lake (8-9th gen)   AVX2     x86-64-v3 / skylake
Comet Lake  (10th gen)    AVX2     x86-64-v3 / skylake
Rocket Lake (11th gen)    AVX512   x86-64-v4 / rocketlake
Alder Lake  (12th gen)    AVX2     x86-64-v3 / alderlake
Raptor Lake (13-14th gen) AVX2     x86-64-v3 / raptorlake

Included below is even more information about what is available when specifying -march & -mtune on x64 CPUs:

Known valid x64 arguments for -march=:

i386 i486 i586 pentium lakemont pentium-mmx winchip-c6 winchip2 c3 samuel-2 c3-2 nehemiah c7 esther i686 pentiumpro pentium2 pentium3 pentium3m pentium-m pentium4 pentium4m prescott nocona core2 nehalem corei7 westmere sandybridge corei7-avx ivybridge core-avx-i haswell core-avx2 broadwell skylake skylake-avx512 cannonlake icelake-client rocketlake icelake-server cascadelake tigerlake cooperlake sapphirerapids emeraldrapids alderlake raptorlake meteorlake graniterapids graniterapids-d arrowlake arrowlake-s lunarlake pantherlake bonnell atom silvermont slm goldmont goldmont-plus tremont gracemont sierraforest grandridge clearwaterforest knl knm intel geode k6 k6-2 k6-3 athlon athlon-tbird athlon-4 athlon-xp athlon-mp x86-64 x86-64-v2 x86-64-v3 x86-64-v4 eden-x2 nano nano-1000 nano-2000 nano-3000 nano-x2 eden-x4 nano-x4 lujiazui yongfeng k8 k8-sse3 opteron opteron-sse3 athlon64 athlon64-sse3 athlon-fx amdfam10 barcelona bdver1 bdver2 bdver3 bdver4 znver1 znver2 znver3 znver4 znver5 btver1 btver2 generic native

Known valid x64 arguments for -mtune=:

generic i386 i486 pentium lakemont pentiumpro pentium4 nocona core2 nehalem sandybridge haswell bonnell silvermont goldmont goldmont-plus tremont sierraforest grandridge clearwaterforest knl knm skylake skylake-avx512 cannonlake icelake-client icelake-server cascadelake tigerlake cooperlake sapphirerapids alderlake rocketlake graniterapids graniterapids-d arrowlake arrowlake-s pantherlake intel lujiazui yongfeng geode k6 athlon k8 amdfam10 bdver1 bdver2 bdver3 bdver4 btver1 btver2 znver1 znver2 znver3 znver4 znver5

• Note: -march=foo implies -mtune=foo unless you also specify a different -mtune. This is one reason why using -march is better than just enabling options like -mavx without doing anything about tuning.

Antivirus

🛑 Be wary of antivirus software on Windows detecting EXEs distributed here as malicious software. While they may not always legitimately be malicious, it is important to maintain a healthy level of skepticism when running code that someone else has compiled.

[ItachiUchiha-IU]

Hi, coincidentally I happen to not be at home again during a new release, so I'm gonna post the builds in remote from my phone again.

This time there's some informations that need to be given, because this new version ends up in non-stable binaries for windows:

1. v2.3.0-A is slower than the previous version, especially if not using --psy-rd (or using very low values); as confirmed with the "PSY guys" (the devs that work on this fork), this is the expected behavior as the --psy-rd code runs and it's called quite often; thus, with it off or with such low values, from tests made on different machines on both linux and windows, it can be slower by 10-20%

• "solution" : use --psy-rd 0.1 or higher, and the speed difference will go down a lot (in some of my tests on windows I got down to 2-3% speed diff, while on linux it seems to be working a little better in some cases); important info about --psy-rd > 0 at point 3

2. the binaries for v2.3.0-A built normally on windows crash; injecting the binary to redirect the calls to an external mimalloc seems to be fixing windows builds (some users are reporting crashes with their injected builds on 8-bit encodes only, but idk if this affects mine too, as 8-bit av1 is illegal anyway), thus, I have now injected all my builds; u'll need to keep the mimalloc files in the same folder as the svtav1encapp.exe binary; the cross-compiled binary doesn't seem to be crashing for 8-bit encoding either but it's quite slower than the windows builds: some got an 8% speed diff, some way more (I can't test myself if it crashes or what speed diff I get cause of the reasons mentioned above)

• "solution" : use av1an. av1an splits the encode per scene before encoding them, and if a chunk crashes, it simply re-encodes that single chunk and the encoding continues without any major problem this point doesn't stand in the cases, with specific settings, where the encoder crashes instantly cause that means it keeps crashing before even completing a single chunk and thus av1an can't save da day; so I'll just cross out this point
• av1an is always recommended as it is able to keep encoding even if the encoder crashes occasionally (and for many other reasons like full cpu usage, zoning, better scene detection and keyframe choices, stop and resume encoding)
• the psy guys already know about the binaries crashing on windows cause the only ones reporting these crashes are the ones of us that are on windows. we have already started looking into it together with them but it could still end up being something not easy to fix, so it may be something that will get worked on tog with mainline after v3 is released.

3. --psy-rd ON (which means any value > 0) is currently broken on windows and it will create artifacts

• "solution" : turn --psy-rd OFF (0) or use p-1 as it seems to be fixing the artifacts (actually idk if p0 or p1 fix it too, as I only tested p-1 and p2+, or if higher crf values than the 18 I tested create artifacts again, so be careful about it; u can use video-compare or vspreview to compare two videos easily and search for eventual artifacts)

4. I don't remember if I said everything I wanted to say, so if something else comes to mind later, or something else comes out, I'll edit it and/or add points.

More discussions regarding this are in the v2.3.0 discussion -> #88
Infos about the builds are at #88 and #69 for now, I'll edit this message when I have the time.
svt-av1-psy_2024-12-25_20.29.49_MULTI-MARCH_clang19.1.5_2.zip
svt-av1-psy_2024-12-25_20.42.02_MULTI-MARCH_av1an-pgo_clang19.1.5.zip
svt-av1-psy_2024-12-25_21.08.08_MULTI-MARCH_dovi+hdr10plus_clang19.1.5_2.zip
svt-av1-psy_2024-12-25_23.13.06_MULTI-MARCH_dovi+hdr10plus_av1an-pgo_clang19.1.5_2.zip
Injecting the binary with mimalloc (.\minject.exe -i .\SvtAv1EncApp.exe?) seems to fix the problem on windows builds (credits to @Andarwinux), so I invite u to try the following injected builds (u need the malloc files in the same folder as svtav1encapp.exe) (some users r still reporting crashes or their injected builds for 8-bit encodes but idk if this affects mine too):
svt-av1-psy_2024-12-25_20.29.49_MULTI-MARCH_clang19.1.5_injected_2.zip
svt-av1-psy_2024-12-27_12.20.00_MULTI-MARCH_av1an-pgo_clang19.1.5_injected_2.zip
svt-av1-psy_2024-12-25_21.08.08_MULTI-MARCH_dovi+hdr10plus_clang19.1.5_injected_2.zip
svt-av1-psy_2024-12-25_23.13.06_MULTI-MARCH_dovi+hdr10plus_av1an-pgo_clang19.1.5_injected_2.zip
Any report would be appreciated.

[Ironclad17]

These were built with ninja and clang-cl: /clang:-O2 -flto. IR+CSIR PGO
2.3.0-A.svt-av1-psy.dovi.hdr10plus.mimalloc.znver2.clang19.1.6.zip
2.3.0-A.svt-av1-psy.dovi.hdr10plus.mimalloc.pgo.znver2.clang19.1.6.zip
Update: Mimalloc resolved all issues for me. 8bit and 10bit clips encoded fine in both versions. All dlls included.

[ItachiUchiha-IU]

I just had time to inject my builds and they seem to be working now (tested znver4 pgo and non-pgo).
If anyone can test them and report, I would appreciate.
I also thank everyone who participated in trying to find a solution to the windows crashes.
I have just read that ppl are still experiencing crashes with their injected builds with 8-bit content(or encoding in 8-bit? why would anyone encode in 8-bit with av1 tho).

[tuanden0]

@ItachiUchiha-IU

I used your "svt-av1-psy_2024-12-25_20.29.49_MULTI-MARCH_clang19.1.5_injected_2.zip", and try to encode with 4 instances and it printed out "out of memory" on the console but used only 15% on my machine. I rollback to v2.3.0 and can run 4 instances without any problem.

[ItachiUchiha-IU]

tried to encode with 4 instances and it printed out "out of memory" on the console but used only 15% on my machine. I rollback to v2.3.0 and can run 4 instances without any problem.

You mean u're contemporary using 4 instances of the encoder to saturate your CPU because with one instance it works correctly but has very low CPU usage?
If I understood it correctly, I'd strongly recommend to use Av1an instead.

If u need guidance to install and use av1an, don't worry and ask here.

[tuanden0]

@ItachiUchiha-IU

Yes, I run 4 instances of the encoder to saturate my CPU.

But on my report, the error is printed out it "out of memory" and exit right away but my RAM usage is about 15%.

Thank for your recommend, I will try to use Av1an.

[Uranite]

Termux build for arm and aarch64 with flto
Pick one according to your arch, and install via dpkg -i
Afterwards, you can encode via libsvtav1 in ffmpeg
Disclaimer: the build is actually f560b40 instead of v2.3.0-A, but I don't feel like rebuilding it
termux-svt-av1-psy-v2.3.0-A.zip

[igorbaryshev]

I found that znver3 is not slower than znver2 when building with IR+CSIR PGO, binary has a comparable footprint as well.
Also, for the first time I'm seeing a slight improvement instead of slowdown when using -O3 over -O2.
Will be sharing multiple builds for various architectures that use profiling data generated for O3 znver3.

Edit: here are all the builds, many marches, including generic.
I used the current state testing branch because I was lazy to switch back to master when I was generating profiling data 😁
Also includes statically dynamically linked mimalloc and optional Dolby Vision + HDR10+ support.
2.3.0-A(testing).IR+CSIR.PGO.-O3.all.marches.mimalloc.[dovi+hdr10+].zip
2.3.0-A(testing).IR+CSIR.PGO.-O3.all.marches.dyn.mimalloc.[dovi+hdr10+].Clang.19.1.6.zip

Dynamically linked mimalloc lib gave me a massive boost of ~2.5%❗ over no mimalloc/failed statically linked.

[igorbaryshev]

@Andarwinux looks like mimalloc just doesn't do much when doing a simple static link, I got it to show some stats, but it's still not doing anything really, so I just linked dynamically instead. I think it wasn't really meant to work on Windows that way.

[igorbaryshev]

Pitted my znver3 -O3 mimalloc build against @ItachiUchiha-IU's znver2 -O2 mimalloc build.
Mine got a 5.5% speed advantage.
Now I gotta test some completely random clip.
Edit: the random unrelated clips and settings aren't doing as great, ~2.5-3.5% increase.

[ItachiUchiha-IU]

I hope u're compiling the binaries with the same params apart from the ones u mentioned (no -ffastmath etc) when comparing speeds, and using av1an or a method that assures real 100% cpu utilization

[igorbaryshev]

I hope u're compiling the binaries with the same params apart from the ones u mentioned (no -ffastmath etc) when comparing speeds, and using av1an or a method that assures real 100% cpu utilization

From my testing -ffast-math doesn't really do anything, maybe makes a binary just a tiny bit smaller. I throw it in just because, but it's not really helping with performance. I just compared on vs off, and there was no difference.
I also used chunked encoding to compare performance.

[igorbaryshev]

@ItachiUchiha-IU actually, I just noticed an increase in performance from --ffast-math of ~2% when using more than 2 chunks 🤔
I hope StaxRip implements chunked RPU's soon, so I can actually take advantage of that, cause right now when encoding with DoVi you'd have to encode the whole file at once in order to properly inject the RPU. That is unless you're willing to manually split the RPU and then separately schedule all those chunks for encoding with respective source chunks.