Cloud Events Message Format

[mlagally]

Repurposing the cloud events message format as an informative payload format for WebHooks.
This does not prevent other payload formats being defined and used.

---

Preview | Diff

[benfrancis]

In my opinion this should be a payload binding template for the application/cloudevents+json media type, not a profile.

Profiles are meant to provide out-of-the-box interoperability for greenfield implementations, not compatibility with brownfield implementations.

Specifying multiple payload formats for the HTTP Webhook Profile will harm interoperability, not help it.

[egekorkan]

I totally agree with @benfrancis . I have even proposed it at https://github.com/w3c/wot/blob/main/proposals/deliverable-proposals/binding-templates.md

It is a bad idea to propose a format for a greenfield application where the format is inherently incompatible with TDs (redundant fields, bad metadata/data separation). Plus, it is not a standard so we have to maintain it.

[mlagally]

@egekorkan, we have to be pragmatic on the fact that there is no binding template specification for cloud events.
The proposed text does not preclude to define other payload formats, if needed, but if a deployment scenario already uses cloud events for notification messages, why not give guidance in this informative section ?

If there are other ideas about message formats, please contribute your ideas in a PR.

[benfrancis]

I strongly object to this being a profile for all the reasons already discussed.

I have proposed a prescriptive payload format for the HTTP Webhook Profile in #258 (comment)

[egekorkan]

if a deployment scenario already uses cloud events for notification messages, why not give guidance in this informative section ?

This is not a motivation for a profile. Also, if other payloads can be defined, what does a Profile consumer do? Try to implement parsers for all of them? How would the consumer distinguish the two payloads?

[mlagally]

I strongly object to this being a profile for all the reasons already discussed.

I have proposed a prescriptive payload format for the HTTP Webhook Profile in #258 (comment)

@benfrancis
We discussed the topic extensively in today's Profile call.

There were several supporters for defining two Webhook payload formats:

1. To drive WoT adoption in deployments to real world cloud systems that are already in the market and use CloudEvents

2. A format for green field devices that can be implementing your proposal from #258 (comment)

The IoT market has evolved since multiple years and real world IoT cloud systems are deployed in the market. To keep the interoperability promise of WoT, we should not neglect but rather support these systems.

Please create a PR with your proposed payload format, which is an equivalent representation for events and can coexist with the CloudEvents profile.

[mlagally]

if a deployment scenario already uses cloud events for notification messages, why not give guidance in this informative section ?

This is not a motivation for a profile. Also, if other payloads can be defined, what does a Profile consumer do? Try to implement parsers for all of them? How would the consumer distinguish the two payloads?

We discussed the topic extensively in today's profile call:
If the profile with the cloud events payload has a dedicated identifier, consumers can support these things out of the box.
A separate Webhook profile that adopts the payload format proposed by @benfrancis can be used for green field devices.
Each has its own profile identifier and can easily be distinguished.

[benfrancis]

We already have two separate HTTP-based profiles for delivering events and now you want to add a third?

I don't think we are on the same page about what profiles are for.

Profiles can only provide interoperability within a profile, therefore the number of profiles needs to be as small as possible because the more profiles there are the more fragmented the Web of Things will be.

Ideally there would only be a single HTTP-based profile, but we couldn't agree on a single eventing mechanism so we agreed to have two separate profiles for events on the basis that SSE and Webhooks fit two different deployment models.

Now we can't agree on a single payload format for Webhooks so you're suggesting there should be two separate Webhook profiles. This will further fragment the Web of Things.

We need to choose a single payload format for Webhooks. @egekorkan and I have explained in detail why CloudEvents are not a good fit for the Web of Things data model, and that direct compatibility with brownfield systems is a non-goal for profiles.

We need to define our own single simple payload format for Webhooks which fits the Web of Things data model, and it doesn't need to imitate CloudEvents, which is not even a standard.

If we keep splitting profiles every time we have a disagreement on the details then I'm afraid the whole concept of profiles is going to fall apart.

[benfrancis]

@mlagally I've just read the draft minutes from the meeting, including @egekorkan's objections to your proposed direction which I note you didn't mention here.

Having read over the notes I'd like to again reiterate that there are currently two separate and complimentary approaches to interoperability on the Web of Things:

• Descriptive protocol binding templates for one-to-one integrations with brownfield systems
• Prescriptive profiles for plug-and-play interoperability between greenfield systems

Existing systems which use CloudEvents fall into the former category and are not a good fit for the latter category for all of the reasons already discussed (which Ege also reiterated in today's meeting). If you have an application which needs to integrate with existing systems using CloudEvents but can not be described using an existing binding template, then I suggest you contribute a binding template specifically for describing CloudEvents and don't try to shoehorn all of Oracle's specific requirements into the Profile specification.

[mlagally]

@benfrancis

We need to choose a single payload format for Webhooks. @egekorkan and I have explained in detail why CloudEvents are not a good fit for the Web of Things data model, and that direct compatibility with brownfield systems is a non-goal for profiles.

If the Web of Things data model is not a good fit for CloudEvents, the TD specification has a bigger problem.

I think we should not ignore the fact, that there are existing standards and de-facto standards (e.g. WebHooks), which came to the world to solve the eventing problem for large scale systems.

If we go down that route to require a specification of a dedicated (sub-) protocol each time we need a well-defined payload format (e.g. for action status responses, error responses etc.) we are fragmenting the problem space significantly.

@egekorkan The profile specification should be pragmatic, and I do not see a real problem with parsing different event payload formats. A JSON parser can parse the object independently from the actual keys the object contains.

Cloud Events are common ground in today's Cloud systems. Let's make sure we address these market needs with the WoT profiles specification.

[egekorkan]

If the Web of Things data model is not a good fit for CloudEvents, the TD specification has a bigger problem.

We did not say that. TD can perfectly describe CloudEvents format. The problem is recommending it for a greenfield WoT application. You are failing to understand our technical comments, which there are many of at this point.

Cloud Events are common ground in today's Cloud systems. Let's make sure we address these market needs with the WoT profiles specification.

That might be true (please provide references) but this is not the goal of the profile spec from my point of view.

[benfrancis]

@mlagally wrote:

If the Web of Things data model is not a good fit for CloudEvents, the TD specification has a bigger problem.

That's not what I said. It's perfectly possible to describe CloudEvents using the Web of Things data model, but prescribing the use of the CloudEvents payload format for new implementations introduces a lot of redundant metadata in every event payload which is already provided in a Thing Description.

CloudEvents is one way to deliver events over the Web of Things, but it is not the most optimal way and therefore we should not require it for greenfield implementations.

I think we should not ignore the fact, that there are existing standards and de-facto standards (e.g. WebHooks), which came to the world to solve the eventing problem for large scale systems.

I agree, which is why we are following the Webhooks pattern in the Webhook profile, and re-using the EventSource standard in the HTTP SSE Profile.

If we go down that route to require a specification of a dedicated (sub-) protocol each time we need a well-defined payload format (e.g. for action status responses, error responses etc.) we are fragmenting the problem space significantly.

Prescribing a specific sub-protocol is exactly what profiles are for. We have to prescribe a single payload format in order to guarantee interoperability, the question is just which one. This is the opposite of fragmentation.

The profile specification should be pragmatic, and I do not see a real problem with parsing different event payload formats. A JSON parser can parse the object independently from the actual keys the object contains.

The problem is that it allowing multiple payload formats "fragments the problem space" as you put it, unless we require that all consumers implement all of the payload formats.

Cloud Events are common ground in today's Cloud systems. Let's make sure we address these market needs with the WoT profiles specification.

Maybe that is true (I'm not sure), but even if we design the Webhook Profile around the CloudEvents specifications it still won't guarantee interoperability with existing CloudEvents consumers. The CloudEvents specifications leave a lot unspecified, such as HTTP verbs and payloads for subscribing to events. We will have to specify these details, which means it will still require a greenfield Consumer implementation in order to fully conform with the profile.