Improve The De-Amalgamated Lemon Version

[tajmone]

Task Goal

Manually edit the de-amalgamated Lemon sources in lemon/sliced/ so that each Lemon module can be compiled independently.

This would greatly simplify the task of porting Lemon to other languages, allowing authors to port one module at the time, relying on interfaces to the original C modules for the unported parts.

Rationale

Currently, the de-amalgamated Lemon sources in lemon/sliced/ rely on #include directives being injected at the end of "lemon.c", so that the split contents may be included back in the original order to allow compiling "lemon.c".

Although this solution works, it's more of a quick-fix expedient rather than a true reversal of the amalgamation process, for the SQLite amalgamator stripped away the original #include directives from each module, and merged them all together (possibly removing some unneeded ones) into a single block in the amalgamated "lemon.c" output.

There's no way for the Lemon Slicer tool to reconstruct those lost #include directives, except by having access to a manually fixed version of the de-amalgamated sources — in this case, the tool could be programmed to inject the required code blocks at the beginning (or/and end) of each module by means of a dictionary of strings snippets associated which each module name.

If anyone can manually fix the automatically de-amalgamated Lemon sources in lemon/sliced/, by adding all the required directives, so that Lemon could be compiled as individual modules, I'll then be able to improve the Lemon Slicer tool to generate a truely de-amalgamated Lemon version.

This would allow porting Lemon to other languages, one module at the time. Languages which can interface with C modules, and support FFI, could then rely on them for the unported parts, allowing the porter to work in gradual steps — e.g., one might start porting the code that interprets the Lemon grammar, first, and rely on the C modules for the other tasks.

This can't be done with the current de-amalgamated version, for it's really just a split-up version of "lemon.c", where removal of a single module would break the whole build.

I'm not sure that this can be even done — it's been so many years since Lemon is no longer being amalgamated from multiple sources, and the code has been updated and refactored many times over in the current single-file incarnation that chances are that this might no longer be doable.

If the task requires tweaking the actual source code beyond mere injection of code at the beginning or the end of the file, then the effort might not be worth the hassle — because we want to be able to automatically re-generate a de-amalgamated version from the upstream "lemon.c" source, in order to keep the code always up to date.

But it's definitely worth a try, for it would be really great to see Lemon ported to modern languages like Nim, D, etc. And providing a version of Lemon split into independent and fully compilable modules would greatly simplify this task.

Reference Links

• Lemon Slicer — GitHub repository.
• The SQLite Amalgamation — for more info about amalgamation.
• SQLite repository README — for details on the amalgamation process and scripts.