Optional integration of LuaJIT instead of Lua

[KageKirin]

Hi,

I'm currently still testing this, hence making it a draft PR.
It's stable so far on MacOSX, but I have yet to build it on Windows and Linux.

This PR adds the option to build genie with LuaJIT instead of regular Lua.
For this, it adds the option --with-luajit to genie.lua and the variable GENIE_OPTIONS to the projgen makefile.

To build with LuaJIT, re-generate the projects with this:

make projgen GENIE_OPTIONS=--with-luajit

and then build as usual with make or make release.

[beevik]

It seems like this change adds a lot of code to genie, but I'm curious what the benefit of using luajit over lua is. Genie isn't something I normally think of as needing faster performance, but maybe I've just never encountered the right use case.

[KageKirin]

I'm still checking if the performance gain is worth the change.

As for the usefulness, in the probably general case of generating just a few projects (<10, or a bit more generously <50), there's nothing much to be gained as both Lua 5.3 and LuaJIT are pretty fast by themselves.

However, what brought me to test this, was actually a rather extreme use-case, with ~100-500 projects to generate. (Long story short: about 400 of these projects belong to the same lib, which I divided in order to circumvent a linker issue with too many object files). In this use-case, being able to save a few seconds per projects accumulates to save several minutes of project generation time, which in turn accumulates to save even more minutes per platform/project type.
In other words, if it allows me to reduce 30 minutes of project generation time to still long 5 minutes, it might be worth the effort.

Now for this PR, I made it optional to use either, keeping Lua 5.3 as default.
I still need to test if using LuaJIT breaks anything else, but so far I haven't had much breakage, and I don't expect much breakage either, since GENie is not relying on Lua5.3's integer handling afair.

Using LuaJIT, I see ways to optimize the project generation part to be multi-threaded using LuaJIT's coroutines, but I haven't implemented that part yet.

[beevik]

However, what brought me to test this, was actually a rather extreme use-case, with ~100-500 projects to generate.

Makes sense. If this turns out to be a big perf win, would there be any reason to keep the interpreted lua code path around?

[KageKirin]

Makes sense. If this turns out to be a big perf win, would there be any reason to keep the interpreted lua code path around?

Probably not, but I'd rather fade out the support over time than losing potential compatibility with one more exotic target platform.

[matthargett]

what I'd love to see is for this to be built-time switchable, for Lua runtimes that support the more-or-less de facto ABI for numerous Lua-based runtimes:

• luau, a drop in replacement runtime for 'C Lua'
• C Lua (the OG)
• OpenResty's high-durability version of LuaJIT
• LuaJIT

For most game consoles and mobile platforms, deploying an atypical client-side JIT is not allowed for security attack surface reasons. For that reason, I might recommend generalizing to an alternative Lua runtime that can operate in a very fast interpreter first. Folks using the darklua transpiler to have shared common code across different VMs (e.g. Luau on the client and OpenResty LuaJIT on the server side) would see the biggest boost.