TUI windowing API

[jaxter184]

I've got a TUI-based host and accompanying plugin suite, and I was planning to create a custom extension for it, but it occurred to me that I could maybe use the gui extension. I understand that my use case is extremely fringe, and I would be extremely surprised if anyone else wanted to use it, but there's a comment in gui.h that says:

// If your windowing API is not listed here, please open an issue and we'll figure it out.
// https://github.com/free-audio/clap/issues/new

so I figured I'd leave a comment here just in case.

My understanding is that I can just define my own CLAP_WINDOW_API_VT100 constant and use the void *ptr; // for anything defined outside of clap union field, and I can implement it using the gui extension without making any changes to the clap headers. Is this understanding correct? Is there any meaningful benefit to making an issue to add my weird edge case?

[defiantnerd]

Yes, this is absolutely possible and I admit I am totally thrilled by the idea of a text based host.

You can define your own string and I would recommend a reverse-url-based one. And then, the void* is yours.

Is this an OSS project?

[jaxter184]

You can define your own string and I would recommend a reverse-url-based one. And then, the void* is yours.

Cool, thanks for confirming, I'll probably start out with a reverse url just to be safe, but I think ANSI/VT100 is a standard with wide enough adoption to justify a regular string. Would also feel weird to make people use net.jaxter184.vt100 instead of just vt100

Is this an OSS project?

The project is here: https://git.sr.ht/~jaxter184/tlature (audioless demo) but its in a weird place right now because I'm shuffling a bunch of stuff around. Feel free to reach out if you have questions or suggestions!

[defiantnerd]

in case you open a plugin that is not supporting your GUI type, you can also check out the remote-controls extensions: https://github.com/free-audio/clap/blob/main/include/clap/ext/remote-controls.h

This would also be easily displayed in a text based audio host.

[baconpaul]

I also think this is a super cool idea

Would “vt100” be right for the api name? You could also use something a bit more terminal generic. Unless you really are just sending a vt100 stream

And then all us olds get to tell the Younguns what a vt100 was.

You may still want an extension of your own or a helper library or something but agree with timo the gui library is meant to do this. The only risk you run is since today Mac is always cocoa Lin always x11 and win win32 you may find badly written plugins which always return true for api supported (he and I found people skipping factory checks similarly when we worked on the wrappers)

[baconpaul]

Oh also I tried building your project and got a suite of errors using latest rust on my mac. Where's the best place to report them? Is there an issue tracker or some such you use at your git host?

[jaxter184]

thanks for trying; I'm mostly working with Linux and don't get a chance to test other builds often. If you could leave a report at https://todo.sr.ht/~jaxter184/tlature id appreciate it. (there should be a little text entry on the left side of that page that you can submit to)

[baconpaul]

OK. They really don't seem macOS dependent. Dropped comments https://todo.sr.ht/~jaxter184/tlature/57#event-376295 here

sorry for the half post etc.... i wasn't used to the UI!

[abique]

Hi,
I haven't followed up with this topic, but just a word to say that this is a very interesting idea.
I'm looking forward the PR 👍
Cheers,
Alex

[baconpaul]

@jaxter184 if you wanted to do a PR it would involve the following

clap/include/clap/ext/gui.h

Line 62 in c2c1dea

static const CLAP_CONSTEXPR char CLAP_WINDOW_API_WAYLAND[] = "wayland";

after that add something like CLAP_WINDOW_ANSI_CONSOLE (probably).

Add a comment explaining what you use the *ptr for. My guess it could be something like "the *ptr can be cast to a C FILE* to which you can write ANSI terminal commands". I know that may be inconvenient for rust, but we need some semantic which matches a C99 spec for the ptr.

Write a tiny demo which uses this somewhere in a CLAP. Or draft one of us to help do that. I think it would be super fun, for instance, to write an ansi terminal spectral viewer

Roll it up and make a PR against next

(optional but handy) send a PR to clap-info https://github.com/free-audio/clap-info/blob/d3608ac3d89a6cb349160b979b1d0cc3ae68f4ac/src/clap-info/info-other-extensions.cpp#L68 to detect your feature or open an issue reminding us to do so

I think the only hard part is what's the C99-compatible ABI stable meaning of the *ptr

wdyt?

[defiantnerd]

just a FILE* would be insufficient, I think. There must be an API that identifies the "Window" and allows to work in multiple ways with it. Not just rendering ANSI, but probably also receive Text input and additional events like hide/show. As paul mentioned, it must also be able to take part in an event loop.

[baconpaul]

Yeah that’s why I thought of an ncurses window

but the event loop is you call getch right? I guess it depends how main is structured

[defiantnerd]

Right - it must be "as always": the host maintains the run loop for the "UI" thread. It must either have a function to pass keystrokes to the TUI based plugin or it must pass the runloop responsibility. The later is problematic since the host might maintain other displays as well.

So there must be a bit of an ext based additional communication and the plugins user input must be maintained by the host.

I know we had the decision not to pass events via the gui ext, but in this case it might be necessary, either via an extension or via an clap-external mechanism.

On win32/macOS etc. this happens by callbacks to the winproc or to the associated delegate, but I assume there is nothing specific here yet.

But it would be fun to design.

[jaxter184]

I've got a rough design now where I pass the plugin a keycode channel reader handle for inputs and a byte channel writer handle for display, and vice-versa to the host (another audioless demo). Right now it sends Rust-specific structures and doesn't have a C version, but converting it to be C-flavored is probably the first thing I'll do after I actually get input handling working.

I know we had the decision not to pass events via the gui ext

I'm glad you brought this up, because I was unaware of this decision. I'm not super familiar with the lower levels of GUI programming and assumed that they just received the input information through the window pointer. You mentioned winproc, if that's not passed through the window pointer, then where does the plugin get that information? Is it part of a library/API completely separate from CLAP?

Another approach would be CLAP_WINDOW_NCURSES

I don't think I'll go with this, mainly because I'm not using ncurses for my TUI rendering. That being said, I'll definitely read up on it, particularly the part y'all are talking about with getch, because I imagine if anyone other than me decides to make a TUI plugin, it is likely to be with ncurses.

In my case, I was able to get away with just a channel of bytes, which has the added benefit that I can just write characters and escape codes directly to the channel, and the host is responsible for converting that stream of bytes into a parsed virtual terminal UI (which is what I've done in the demo linked above). But there are probably some more considerations to make when figuring out what the API of that channel will be once it's in C. My current plan is to just adapt my current channel approach into the most library-agnostic, C-compliant way possible (probably something vaguely similar to the event buffer), and that's what I'll probably submit as a PR unless a better suggestion comes up.

And speaking of the PR, does it really make sense for this to be included in gui.h? As much as I'd love to have first-class support for terminal UIs in CLAP, is this sort of esoteric usage not the exact reason that the void *ptr; union field exists? I'm happy to make the PR, but I just want to make sure that this is actually something that makes sense and I'm not just pushing unnecessary complexity into the plugin format. Especially since, unlike with win32/cocoa/x11, there is no "official" API for terminal UIs, so whatever structs I decide to put into the pointer are going to also have to be defined in the CLAP headers.

[baconpaul]

I think you are correct @jaxter184

you can use the GUI mechanism without a PR to gui.h through an extension you provide which

1. Has the define for the GUI name you want and
2. Has any additional classes or helpers you need

Then folks can choose that extension or not.

Sounds like the API you want is (roughly) FILE *[2] for your ptr. An in and an out pipe. But you also need to call the refresh on the element so you may need a third thing which is a refresh ptr. So perhaps you could do something like

jaxter-code/ext/clap-tui-gui.h

#define TUI_GUI "jaxter.thisthat.tuigui"

// This is what the PTR is
struct TUIGUIWindow
{
   FILE *guiToTerm;  
   FILE *eventsToGui;
}

struct TUIGUIEventHandler
{
   // if you get someone who supports TUIGUI flavor and doesn't implement this it is an error
   bool(CLAP_ABI *processNextEvent)(const clap_plugin_t *plugin);
}

You could also, of course, have the Window just be an int and then make processNextEvent have signature plugin, in pipe, outpipe

but I think you are headed towards something like that

(If I was writing an nurses based gui I would do it as above, just replace the file pair with an ncurses WINDOW *