Flixel rendering overhaul

[ninjamuffin99]

https://haxeflixel.com/blog/08-openfl-5-compatibility-status-update/

Around 2017, Beeblrox posted some progress he made in redoing the way HaxeFlixel does things like shaders, and FlxStrip rework. By the title of that blog, that was on OpenFL 5. We are now on OpenFL 9!

https://github.com/AustinEast/flixel/tree/Beeblerox-FlxMaterial-and-camera-scaling

Sometime around a year ago, I was getting Austin East to work on getting beeble's work running in modern flixel/haxe/openfl/lime

According to messages between me and austin:

2022/10/20
ninjamuffin99: so where was the things with all the polygons / tris or whatev, thats with all the openGL rendering right?
that includes like, making FlxStrip play nicer and all that?

austin: yeah all that high level API stuff is in :) like the stuff from the famous beeble blog post on the flixel website lol

and another conversation with austin about the project

okay so ive gone over everything and i want to break down all the big items and where theyre at. this will also be helpful for me since there is a lot going on here lol
i originally started this as a port of beeble's branch, but things sorta evolved from there, based on conversations i had w him and looking at other renderers out there
austineast — 05/29/2022 10:35 PM
beeble's branch featured that fancy opengl renderer, but it was heavy tied to openfl, which has changed quite a bit since the branch was worked on and everything is basically broken. lol. at first i tried to fix it as-is, but after talking to beeble about it, he recommended rewriting it in a way that would be a bit closer to the metal (ie more opengl code through lime vs openfl) and pointed me to haxepunk's renderer, which ended up being a good resource
austineast — 05/29/2022 10:42 PM
i noticed haxepunk had a nice backend abstraction setup, which really helped keep the haxepunk codebase/renderer clean and easy to parse. its written in a way where any lime/openfl/nme specific code is separated out to a backends/ directory. and since i was in the process of planning out the flixel renderer rewrite, it seemed like a good idea to start doing that with flixel's codebase to help me really learn the internals and really future proof this new renderer
austineast — 05/29/2022 11:06 PM
with all that being said, heres the general statuses of the different pieces:

1. merge beeble's branch into the latest flixel and get it compiling : complete (once it was compiling, i discovered that while the renderer abstraction was theoretically working well, openfl changes had really broken all of them hard. that lead me to the next task)

2. port flixel's current working renderers (tiled mode/quad mode) to get them running with the new renderer abstraction: complete. ive tested it with a couple of my own projects and it seems to run fine with little-to-no breaking changes (it depended on whether the project was one of the ones where i do crazy graphics things or not lmao)

3. separate openfl/lime code out into a backends directory: most (if not all) lime code has been moved out. there is still a bunch of openfl code. ive been porting things as i touch them, but there are just a lot of references to things like openfl's Bitmap and BitmapData lol. luckily this backend stuff doesnt have to be done all at once, since its basically abstracting directly over openfl stuff like typedef Sprite = flash.display.Sprite; so it doesnt cause any compile errors. its not vital to get done immediately, but it will probably want to be finished before this gets merged into flixel 5.0+ though

4. write a new opengl renderer: in progress. i have the basic architecture figured out (ie how to structure everything like vertex butffers, drawcalls for quads and tris, etc), but now im actually implementing the low level opengl part (ie managing the buffers, creating shaders and sending data to buffers correctly, sending it all to the gpu correctly etc)
   a lot of time was spent on planning and figuring out the backend abstraction, but that was all on my own accord without really consulting you so im not factoring that into charged hours
   itll be really cool to have tho
   this opens the door for a heaps/kha/whatever backend in the future

THE TLDR

• HaxeFlixel has less than stellar support for shaders (no multipass, cumbersome to toss in multiple textures, no camera buffer)
• In 2017, Beeblrox worked on fixing these things up, however OpenFL was moving a lot and making lots of changes
• In 2022, AustinEast worked off Beeblrox's old code and brought it up to modern standards of sorts, but needs work to bring it across the finish line.

Where to go from here?

My gang at The Funkin' Crew can and will pay someone to put this work forward. Ideally.... it can be austin!! However he may be unavailable, so I suppose first course of action can be finding SOMEONE for me to throw money to.

[ACrazyTown]

I wonder if it's worth it to build an OpenGL renderer from scratch nowadays? It has been deprecated on Apple devices since 2018 and Android will soon support it only through a compatibility layer.
Would having a custom renderer also make it harder to port to consoles? Could Flixel just use OpenFL's Stage3D/Context3D apis and leave the low level rendering parts to OpenFL/Lime?