Add plugin entry points

[ewels]

Allow other packages to customise both the formatters and the FormatParser class by using custom entrypoints.

Proposing as an alternative to #45 - these entrypoints allow me to put that custom highlighter code in another Python package. It can be maintained in a separate release cycle there, and means that it doesn't pollute the main toolong code base for all other users.

[ewels]

I'm nearly at the point now where I'm happy / done with this functionality now. However, there is one sticking point that I'm struggling with. The log files I'm parsing have DEBUG/INFO/WARN/ERROR log statements. But they can sometimes be multi-line. I want to propagate that log status to the subsequent lines.

I tried adding a status variable in the FormatParser class __init__ which I can update on each line. This works fairly well. But because Toolong is clever and doesn't necessarily parse lines in order, the output frequently (and randomly) jumbles up the log status. For example:

I wondered about doing a pre-run through the file somehow to add markers to the start of each line, which I could then use in the highlighting. But again Toolong is too smart and doesn't fetch the entire file into memory. The best I've got is to somehow hijack the reading, copy to a tmpfile and edit in place, then continue with regular loading. But that feels very hacky, and would break file tailing. So I'm a bit stuck and out of ideas.

@willmcgugan any suggestions? Or should I give up on this front? 😬 😅

[willmcgugan]

My only concern is that I think this approach may not work well with some installation methods. If you install with pipx or brew, then you could only add entry points if you install them in to the same virtual environment.

It is a good idea to allow third-party formatters. But maybe this should be a config file that references a plugin. Ideally the plugin would be something declarative, like a regex styles.

Haven't considered follow on lines. I think that would require an extension to the format parser system.

[ewels]

Right, good point about the installation methods 👍🏻 I've actually been a little cheeky in this instance and added a new nf-core log command to our nf-core/tools package that simply calls tl in a subprocess shell. We already have rich and textual as dependencies, so adding toolong to the list is a minimal addition. Then the two packages will always be installed together and users don't need to memorise a new command (I did add a log message extolling the virtues of Toolong, in my defence 😅 ). But this is a bit of a weird edge case in that regard.

I agree that a config based / declarative plugin system would be nicer for most users. I'm not really sure where I'd start with that though.