Should we consider building backends in their own repositories / pipelines?

[dave-gray101]

While I was looking at #2748, I was thinking about other ways we could potentially speed up things on our CI builds. The LocalAI binary itself compiles quite quickly, as one would expect - the slow part is compiling each backend for each architecture. I've got an idea that would add some complexity to our build process, so I'm curious if there is interest before I run any full experiments.

Essentially, I propose to kick all of our backends outside this repository into their own repos. Each of those repositories would have a CI pipeline responsible for building when there's a change - the llama and whisper pipelines being particularly active compared to some of our other ones - and then tagging a release for each platform. The LocalAI makefiles, dockerfiles and pipelines would be updated to download the correct backend binary by default.

I can see some pros and cons with this approach. I expect that avoiding rebuilding all of our backends for minor golang changes would be a huge CI speedup. On the other hand, it'll be slightly more annoying for you and me to update the gRPC definitions (when they do change) and I can't say I'm a huge fan of dockerfiles that download binaries... even if they are our own repositories and build systems.

At the end of the day, it still seems crazy to wait ~4 hours for hipblas builds on changes that don't even touch a single backend - it might be worth setting up the web of pipeline action triggers required to keep everything in sync.

[gymnae]

As a newb, I'd like it. Issues would be focused on certain backends. For example, getting the ROCm GPU images to work like their CUDA counterparts