Regex Patterns to Match Topics?

[derekrprice]

Just wondering if the functions kafka trigger supports regex patterns in lieu of a topic name like most kafka consumers. I can't find anything in the docs on what that syntax might look like. Thanks!

[shrohilla]

@derekrprice, could you provide more details what exactly are you looking for ??

[derekrprice]

All the Kafka clients I have looked up support subscribing to all topics matching a user-specified regular expression (see the docs for the JS client and the .NET client). For example, we have topics under devices/<id>/state/<parameter> and I would like a single functions consumer to subscribe to all state changes across all devices, so something matching a Javascript regex pattern like /^devices\/\d+\/state\/\w+$/. It would also be convenient if I could add capture groups for the ID & Parameter fields and have those passed in as part of the function context or parameters as well so that I didn't have to reparse the matched topic to gather them myself.

For example, something like this in a functions.json would hopefully accomplish all of the above:

{
    "bindings": [
        {
            "type": "kafkaTrigger",
            "name": "event",
            "direction": "in",
            "dataType" : "json",
            "topicRegex" : "/^devices\\/(\\d+)\\/state\\/(\\w+)$/",
            "consumerGroup" : "mongo-sync",
            "username": "%ConfluentCloudUserName%",
            "password": "%ConfluentCloudPassword%",
            "protocol": "saslSsl",
            "authenticationMode": "plain",
            "brokerList": "%BrokerList%"
        }
    ]
}

Not sure if this is a separate ask, but I would need the matched topic and key of the matched event included in either the event or the context as well. At the moment, I believe that only the event text is being included.

Thanks!