Proposal for a solution to improve integration of hardware controllers with software plugins using CLAP

[safr-world]

Introduction

I would like to work to improve the experience for both users and developers in regard to how hardware controllers integrate with software plugins.

I find that the current solutions are not in anyway ideal and lead to frustrations from both parties.

It is my belief that the relationship between CLAP, as a relatively new plugin format and Bitwig, as a forward thinking DAW, provides a unique a unique opportunity to resolve this problem.

I am not a developer myself, but I am writing this in the hope of starting some discussion.

The Current Situation and Problems With It

At the moment, the relationship between plugin, controller and DAW is quite simple and inherently limited.

If I, as a user, would like to a controller a plugin from my controller, then the signal flow will operate like this:

Controller ====>> DAW ====>> Plugin

After recording automation, the playback signal flow then works like this (I think - please correct me if I’m wrong):

DAW ====>> Plugin

The plugin reports which parameters are available for mapping, but because hardware controller mapping happens at the DAW level, this means that some manual mapping has to be done.

Currently, this is achieved in the following ways:

• The user must map the plugin. This is time consuming for the user and detracts from making music.
• The plugin developer must (in effect) provide mappings for each DAW. This is impractical for the developer.
• Hardware controller manufacturers provide their own software solutions. This has had some success but again, is ultimately impractical for plugin developers to maintain.

My Solution

My solution proposes to change the signal flow and make other improvements in the process.

In this solution, the signal flow would be:

Controller <<====>> Plugin <<====>> DAW

By making the plugin the central point, there can be some advantages:

• An alternative to MIDI can be used that has a higher resolution and can report back parameter names as strings to controllers.
• Bidirectional control across the entire chain.
• Plugin developers can dictate mappings to controllers. No more requirement from users to create their own mapping. Think about hardware. Manufacturers have to make choices that influence the design of the hardware, and therefore the use by the user. A similar (although not exact) experience could be achieved.
• Because the mappings are embedded at plugin level, the plugin developer only has to do it once to achieve parity across any DAW supporting CLAP.
• A hardware controller manufacturer could make controllers designed to operate certain types of plugins. For example, a controller designed to control synthesiser plugins could be laid out in a certain way, but could easily receive the mappings for any synthesiser plugin and know where to place the controls. To achieve this parameter groups should be reported to the controller. The controller should know what a Filter is, an Oscillator etc.

The Challenges

There are several challenges to overcome to achieve this:

• The first challenge is acceptance of the proposition. Because of what’s being proposed, changes would have to be made to both CLAP and supporting DAWs. This requires thought and development time.
• Hardware manufacturing. At least one hardware manufacturer should be involved to work through the practicalities from their side.
• Integration to other DAWs. To really make this work, other DAWs would need to accept CLAP as a standard. This is difficult in a field that already has several standards, but CLAP has the advantage (and I think the wisdom) of being opensource and thereby removing the biggest hurdle to adoption.

Closing

Hopefully this is some food for thought. I’d really love to see this area improved but I can’t do it by myself!
I look forward to any feedback.

[abique]

Hi,

I think the current situation is not as bad as you may think.
We have the remote controls extension which lets the plugin define the "remote controls" as you know them in Bitwig, and I am happy with the current signal flow: hw -> daw -> plugin.

It is possible to have hw <-> plugin, see NI Maschine for example.

It is a bit of a chicken and egg problem to introduce new hw controller integration concept and requires the ability to build the hw and software in both host and plugins and distribute that in a large amount. This currently isn't a goal for CLAP nor even possible with our resources.

Cheers,
Alex

[safr-world]

Hi Alexandre,

I really like Bitwig's mappings. I think that it's the best attempt at a solution for this and has the added benefit that it's possible to save the files and transport then to other machines.

Unfortunately, it doesn't get around the initial hurdle of having to map devices.
I use Mossgruber's script and that does a really good job with making my SLmkIII work with Bitwig devices. But as soon as I want to use a 3rd party plugin, I've got to sit there and work it out before I get to make any music.
It can have quite a negative impact on creativity particularly because I have a full time job and really only have time to play around in the evenings.
But I think that if you ask around, you'll find a lot of people with the same frustration.

It's not a Bitwig problem per se, because all DAWs are in the same position. For example, Push + Ableton has the same problem.

What I think is unique here, is that the way that Bitwig mappings work is the closest I've seen to a solution. Also, Clap is a brand new plugin format which positions you guys in aa on source sandbox to fix some longstanding creative problems.

Cheers ..
Ed

[defiantnerd]

You could easily provide a connection to any hardware from your plugin. There are existing solutions to this problem, independent from CLAP. And you are right, in this case the plugin needs to be in the center of the network/communication.

What CLAP could benefit from: an interface where a connected hardware surface would be defined and offer it to the plugin which can adapt to the offered actual hardware interfaces. A rough idea would be that the plugin could enumerate available buttons, knobs and displays together with hinting ("Knob 14, labelled with 'Select', Position -20,50 "). There is a similar concept in the gaming world, where controllers define such a layout scheme, so games can display a sketch of the controller for function assignments.

When I have some time, I try to write a more precise proposal.

[safr-world]

Thanks for giving it some thought.

What I have in mind is to standardise the controller information being reported by plugins that could then be utilised by hardware manufacturers. I think your idea might be the other way round? I understand what you're saying when you're comparing with game controllers but I think that the amount of parameters ultimately leads to the same issue.

So if it were to be defined by a standard in the plugin, perhaps it looks something like this:

Type of plugin: Either Instrument=1) or Effect=2
Parameter groups (modules): For example with a synth, Oscillator=1, Filter=2, Mixer=3 etc. If you have multiple groups, Oscillator1=1.1, Oscillator2=1.2 etc.
Parameters: For example an Oscillator: Wave type=1, Pitch=2, Fine pitch=3 etc.

You end up with a hierachy:

• Instrument=1 (An example of a synth)
  • Oscillator=1.1
    • Wave type=1
    • Pitch=2
    • Fine pitch=3
  • Oscillator=1.2
    • Wave type=1
    • Pitch=2
    • Fine pitch=3
  • Filter=2
    • Filter type=1
    • Cutoff=2
    • Resonance=3
• Effect=2 (An example of a multieffect)
  • Filter=1
    • Filter type=1
    • Cutoff=2
    • Resonance=3
  • Reverb=2
    • Dry/Wet=1
    • Size=2
    • Damping=3

As a hardware manufacturer, the above layout specs could then be used to determine where parameters are placed on a control surface. The idea is that it's then possibly to break out of the 8 knob/fader/button paradigm which most controllers follow to allow the creation of more interesting layouts.

It would also be possible to use a "template mode" where the plugin could override the hardware's default layout. So if a plugin manufacturer wanted their plugins to be laid out in a particular way on particular controllers, it could do so.