Open Sound Control support

[jaxter184]

Has there been any consideration into supporting OSC? I would have liked for it to have the same level of "native"ness as MIDI, but I guess having an extension for it might be a little more reasonable given that it's not a super common feature for a plugin to have (the only example I know of is LV2, and even there, it's an extension). Not many DAWs have support for it either, but ossia is built pretty heavily on top of OSC, and I believe Ardour has some support for it.

The specific use case I'm targeting is text input for voice synthesizers. Right now, most voice synths (plogue alterego, vocaloid, synthv) have separate editors for declaring words/phonemes, but I've been using an OSC-based approach with a standalone voice synth to sequence them as strings and UTF-8 characters in the host, and it feels a lot more ergonomic, efficient, and enjoyable. Of course, OSC support would go far beyond just this use case.

The way CLAP is designed, it seems like in my particular case, I could just make a proprietary extension since I have control over both the host and the plugin, but since OSC is an open standard, I think it would make sense to add to the extensions in this repository. I'd be curious to hear whether y'all agree or whether I'm just a deranged OSC stan.

[abique]

Hi,

For text input, we don't need OSC, but just an event with some text. This is quite easy actually.

Cheers,
Alex

[jaxter184]

A text event takes care of 90% of what I'm looking for, but there are some other things about OSC that I think make it nice to work with, like blobs, bundling and multiple arguments in a message. There's also stuff like pattern matching, but I'm on the fence as to whether the benefit of that particular feature is worth the additional complexity, and it could maybe be done on the host side.

Also, how would this event system interact with the ports? There are a lot of related controls that would go between, for example, a "text to phoneme" plugin and a "phoneme to audio" plugin, and I'm not sure how the host would be aware of a text event output. Would I just use a note-port with the dialect flag set to 0?

Anyway, a text event sounds like a good start, and would be very helpful for my use case. Thanks for your quick response!

[7890]

Hi, could we do this:

diff --git a/include/clap/events.h b/include/clap/events.h
index 5086b4d..34f9efb 100644
--- a/include/clap/events.h
+++ b/include/clap/events.h
@@ -114,8 +114,23 @@ enum {
    CLAP_EVENT_MIDI = 10,       // raw midi event; clap_event_midi
    CLAP_EVENT_MIDI_SYSEX = 11, // raw midi sysex event; clap_event_midi_sysex
    CLAP_EVENT_MIDI2 = 12,      // raw midi 2 event; clap_event_midi2
+
+   CLAP_EVENT_TEXT_PLAIN = 13, //plain text
+   CLAP_EVENT_OPEN_SOUND_CONTROL = 14, //raw OSC 
 };
 
+typedef struct clap_event_text_plain {
+   clap_event_header_t header;
+   char *buffer;
+   uint32_t size;
+} clap_event_text_plain_t;
+
+typedef struct clap_event_open_sound_control {
+   clap_event_header_t header;
+   void *buffer;
+   uint32_t size;
+} clap_event_open_sound_control_t;
+
 // Note on, off, end and choke events.
 //
 // Clap addresses notes and voices using the 4-value tuple

Does this fit the foreseen way to add new events?
Adding host and plugin extensions for convenience would be an option.

[7890]

Here is another more generic approach for text and binary buffer events. Thoughts?

diff --git a/include/clap/events.h b/include/clap/events.h
index 5086b4d..b9a9b39 100644
--- a/include/clap/events.h
+++ b/include/clap/events.h
@@ -114,8 +114,27 @@ enum {
    CLAP_EVENT_MIDI = 10,       // raw midi event; clap_event_midi
    CLAP_EVENT_MIDI_SYSEX = 11, // raw midi sysex event; clap_event_midi_sysex
    CLAP_EVENT_MIDI2 = 12,      // raw midi 2 event; clap_event_midi2
+
+   CLAP_EVENT_GENERIC = 13,    // generic (byte) buffer event; clap_event_generic_buffer
+};
+
+enum clap_event_generic_flags {
+  CLAP_EVENT_GENERIC_BINARY = 1 << 0, // anything that's not a human readable string
+  CLAP_EVENT_GENERIC_TEXT = 1 << 1,   // plain text, in different formats
+
+  CLAP_EVENT_GENERIC_BINARY_WITH_MAGIC = 1 << 2, // buffer has magic starting bytes
+  CLAP_EVENT_GENERIC_BINARY_OSC = 1 << 3, // buffer is OSC message '/...'
+  CLAP_EVENT_GENERIC_TEXT_JSON = 1 << 4,  // buffer is JSON string '{...}'
 };
 
+typedef struct clap_event_generic_buffer {
+   clap_event_header_t header;
+   uint32_t flags; // see clap_event_generic_flags
+   const uint8_t *buffer;
+   uint32_t size;
+} clap_event_generic_buffer_t;
+
 // Note on, off, end and choke events.
 //
 // Clap addresses notes and voices using the 4-value tuple

[baconpaul]

Bit stream of consciousness, apologies. But

If we go that route, one thing to document is: What's the lifetime of that buffer *? The answer I think is "the calling member has to make sure the buffer is valid throughout the process call, but the plugin should not assume the pointer is valid beyond the process call, and if it needs the value, it should copy it

The flags here are not additive (you can't have BINARY_OSC and TEXT_JSON) so they don't need to be1<<3 bit mask values but instead can just be actual values.

These types of APIs (clipboards, information interchange, etc...) often use a mime type to describe the data. I wonder if that could help here somehow?

BINARY_WITH_MAGIC seems a bit odd.

[defiantnerd]

Probably GENERIC is too generic. I can understand the usage for text-to-speech synthesizers, although I don't know how good DAWs handle text events.

Adding generic buffers without a specific use case could lead to a lot of different implementations without compatibility. I would wait for a real use-case before adding such a thing. TEXT_JSON is so generic since it does not even define what such an event should be able to do.

In case it is a special relationship between a specific host and specific plugins, you might better implement a private extension?

[jaxter184]

For what it's worth, this is what I ended up with for OSC messages (apologies for the Rust). It's pretty much identical to what you've come up with (minus the other event types, plus a port index, which maybe doesn't make sense in OSC). I based my design off the MIDI SysEx event, which I believe has the same lifetime considerations as what baconpaul described.

I also agree that it makes more sense implemented in an extension than in the primary CLAP repository (I've had very good experiences so far making extensions), though it'd be nice to coordinate and gather use cases so that we don't just end up with 8 non-intercompatible extensions.

[7890]

Interesting points, all true. If it could be just another event type, all the existing infrastructure can be used (and it's just another type that some plugins will not support). The lifetime of the buffer would be as described, the caller must care about it.
For buffers with type OSC, the dialect is left open so there's indeed a high probability of incompatibilities.
Another approach that came to mind was to introduce a new event space, just for custom buffers or as such OSC. Host and plugin could then further negotiate what's available. There could be an extension to query single message types or whole sets of messages with described semantics.
The main usecase I see is that hosts could receive custom OSC messages and just pass them on to a plugin without any introspection, mapping or other work. The plugin would be the main OSC processor and eventually hand OSC events back. Or the plugin could register for messages with a specific syntax on the host, while the host must not know the semantics but pass matching messages to the plugin. In the latter scenario, a plugin would be not unlike a message handler in liblo, and the host would be the server.
If OSC messages should be bound to a port is unclear. In theory, if the message relates to a port, it could be expressed in the message.
Introducing OSC message event type could overcome the otherwise mostly static nature of the event structures but puts a burden on semantics & compatibility. More thinking needed.

[7890]

@jaxter184 using the MIDI sysex event type's buffer to put OSC also was my first thought :)