Webview extension (draft)

[geraintluff]

This draft lets CLAP plugins straightforwardly use a web-page for their UI.

Messages are then exchanged in both directions as opaque blocks of bytes. The webpage receives and sends messages using standard web APIs.

The host is responsible for storing messages in suitable (lock- and allocation-free) queues.

Closes #416

[CLAassistant]

All committers have signed the CLA.

[geraintluff]

Jules at ADC (Javascript, WebViews and C++)

At some point we're going to need to have a way for the DAW to ask the plugin: look, is the window you're giving me going to have a webview in it? In which case just give me the HTML and we can put it in, we can actually embed it or put it in our own webview, or do something a bit less layer-y.

[baconpaul]

I'm not a web tech person so this may be not useful.

shouldn't the file be called webview.h not web.h and similarly clap_plugin_webview etc....

all the functions are marked [thread-safe] which means they can be called from any function with any degree of concurrency. I'm surprised to see that vs [main-thread]. Why would the audio thread ever need to directly message the web view?

Shouldn't get_start have a more descriptive name like provide_starting_uri

the protocol for get start and is open are unclear. Is it possible to write a state diagram like the one in gui.h?

[abique]

Maybe you could add this extension to the web-clap repository for now?
So we could iterate there quickly without having to ask for review.
And when we have a made some more progress, we can get this extension back in clap.
What do you think?

[baconpaul]

I think that's pretty smart, @abique

this space is nascent enough that you want a fast iteration sandbox and then bring it to us less-web-enlightened folks for reviews

I really would focus on naming and protocol/state machine early on though. It's the kind of thing which is painful to fix, and long descriptive names are fine. That was basically my comment set.

[geraintluff]

This extension isn't specific to WCLAP (although it's useful there), and doesn't depend on any of the WASM-specific decisions.

If you'd like this to block on the WCLAP stuff for other reasons, then moving it makes sense.

[geraintluff]

shouldn't the file be called webview.h not web.h and similarly clap_plugin_webview etc....

✅ I agree with this. web.h was initially suggested by @abique so I used that, but "webview" would be more accurate.

Why would the audio thread ever need to directly message the web view?

My reasoning was: the UI should be updated about both state save/load (main thread) and parameter update events (audio thread).

The plugin could instead run its own message queue, post all the parameter updates there, request a main-thread callpack, and then forward them to the webview. This way was easier for the plugin, the other way would be easier for the host. 🤷

Shouldn't get_start have a more descriptive name like provide_starting_uri

✅ Makes sense.

the protocol for get start and is open are unclear. Is it possible to write a state diagram like the one in gui.h?

It should be possible to call either of them at any time. I wasn't imagining either of them would trigger a state-change.

is_open() would return true if there was a webview UI running, so likely to receive messages sent to it. So far, I've checked this on the state/parameter changes to see whether it should bother, which I guess means it'd only be called when the plugin is processing stuff?

[baconpaul]

My reasoning was: the UI should be updated about both state save/load (main thread) and parameter update events (audio thread).

The plugin could instead run its own message queue, post all the parameter updates there, request a main-thread callpack, and then forward them to the webview. This way was easier for the plugin, the other way would be easier for the host. 🤷

every plugin already does this to update their native GUI. We don't ever want to encourage gui writes from the audio thread vs 'get the message to a lock free queue' approach. (the reason is that every [thread-safe] function has to meet the full realtime constraints of no allocation, no network traffic, no disc i/o etc... and i don't think you can meet that here)

It should be possible to call either of them at any time. I wasn't imagining either of them would trigger a state-change.

So can a host call receive on a plugin before it calls get_start? Or if is_open would return false? Is the plugin then supposed to preserve those messages until a start? (assuming these have better names in final)

[geraintluff]

So can a host call receive on a plugin before it calls get_start? Or if is_open would return false? Is the plugin then supposed to preserve those messages until a start? (assuming these have better names in final)

Really great points - I'll improve this and sketch out a diagram.

The plugin could instead run its own message queue [...]

every plugin already does this to update their native GUI. We don't ever want to encourage gui writes from the audio thread vs 'get the message to a lock free queue' approach. [...]

From the perspective of a (occasionally lazy 😛) plugin developer, that seemed like wasted duplicate effort. The host will also have lock-free queues in its codebase already, and if the host got the message to its lock-free queue (instead of the plugin) then I thought perhaps the plugin wouldn't need to run such a queue at all, if it only supported the webview UI. (And the "web" GUI type for sizing info)

However, I'm doubting that now! The host can provide stronger guarantees about delivery if called on the main thread, and the plugin is also much more likely to know how big the queue should be, what to do if the queue is full, etc.

Thanks for the feedback!

[baconpaul]

Yeah clap helpers has a lock free queue as does juce and other packages. It’s already a paradigm plugins have. And frameworks wrap them if you want to think at a higher level. Also having a non allocating non blocking lock free data structure for a message of arbitrary size with multiple producers is …. Hard. And your api implies that a host has to implement that. Whereas the spsc fixed sized queues folks use to do interoperation between main and ui threads are well trodden and in almost every plugin codebase already

great stuff. Very interesting to see this evolve.

[geraintluff]

@baconpaul I've updated things based on your comments.

The only method I left as [thread-safe] is is_open(), because (a) it's almost certainly just querying a boolean, and (b) I've clarified that when it should be used as a hint for non-essential stuff like metering, with no guarantees about synchronisation.

I personally think is_open() should either stay [thread-safe] or be removed completely. I found it convenient to call from the audio thread for deciding whether to spend time collating/posting data for a main-thread metering message.

However, we could also tell plugins to support clap.gui and manage their own flag (based on create()/destroy()) if they want this information.