From 4536651e54bf82bb09d7804823f34d3ec311a570 Mon Sep 17 00:00:00 2001 From: Anca Lita <27920906+ancalita@users.noreply.github.com> Date: Mon, 7 Aug 2023 11:13:09 +0100 Subject: [PATCH 1/3] update PII docs --- docs/docs/pii-management.mdx | 32 ++++++++++++++++++++++++++++++++ 1 file changed, 32 insertions(+) diff --git a/docs/docs/pii-management.mdx b/docs/docs/pii-management.mdx index 1791226b4752..4f144c313252 100644 --- a/docs/docs/pii-management.mdx +++ b/docs/docs/pii-management.mdx @@ -240,6 +240,33 @@ The `anonymization_topics` section contains a list of Kafka topics to which the Each Kafka topic must have a `name` field and an `anonymization_rules` field. The `name` field specifies the name of the Kafka topic. The `anonymization_rules` field specifies the `id` of the anonymization rule list to be used for the Kafka topic. +### Streaming anonymized events to Rasa X/Enterprise with Kafka + +Streaming anonymized events to Rasa X/Enterprise is only supported for Rasa X/Enterprise versions `1.3.0` and above. +In addition, you must use the Kafka event broker, other event broker types are not supported. + +You can stream anonymized events to Rasa X/Enterprise via Kafka by adding the `rasa_x_consumer: true` key-value pair to +the `anonymization_topics` section: + +```yaml +event_broker: + type: kafka + partition_by_sender: True + url: localhost + anonymization_topics: + - name: topic_1 + anonymization_rules: rules_1 + rasa_x_consumer: true + - name: topic_2 + anonymization_rules: rules_2 +``` + +If multiple Kafka anonymization topics contain the `rasa_x_consumer` key-value pair, the anonymized events will be streamed +to the Kafka topic that is mapped to the first Kafka anonymization topic that contains the `rasa_x_consumer` key-value pair. + +Note that the `rasa_x_consumer` key-value pair is optional. If it is not specified, the anonymized events will be published +to the Kafka topic, but they will not be streamed to Rasa X/Enterprise. + ## How to enable anonymization of PII in logs You can enable anonymization of PII in logs by filling the `logger` section in the `endpoints.yml` file. @@ -257,3 +284,8 @@ The `anonymization_rules` field specifies the `id` of the anonymization rule lis We strongly recommend to run with log level INFO in production. Running with log level DEBUG will increase the assistant's response latency because of processing delays. ::: + +Note that running `rasa shell` in debug mode with a Kafka event broker might result in logs related to the event publishing +to be printed to console **after** the bot message. This behaviour is expected because the event anonymization and publishing +is done asynchronously as a background task and completes after the assistant has already predicted and executed the +bot response. From 715258e2dc5f14bd79b8cf024dac67078692d8a9 Mon Sep 17 00:00:00 2001 From: Anca Lita <27920906+ancalita@users.noreply.github.com> Date: Mon, 7 Aug 2023 11:28:27 +0100 Subject: [PATCH 2/3] add changelog entry --- changelog/12712.doc.md | 2 ++ 1 file changed, 2 insertions(+) create mode 100644 changelog/12712.doc.md diff --git a/changelog/12712.doc.md b/changelog/12712.doc.md new file mode 100644 index 000000000000..012693eda568 --- /dev/null +++ b/changelog/12712.doc.md @@ -0,0 +1,2 @@ +Updated PII docs with new section on how to use Rasa X/Enterprise with PII management solution, and a new note on debug +logs being displayed after the bot message with `rasa shell`. From 5214cb2b58d8e18698543f5cefab255ea94b4af0 Mon Sep 17 00:00:00 2001 From: Anca Lita <27920906+ancalita@users.noreply.github.com> Date: Tue, 8 Aug 2023 10:43:13 +0100 Subject: [PATCH 3/3] address review comments --- docs/docs/pii-management.mdx | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/docs/docs/pii-management.mdx b/docs/docs/pii-management.mdx index 4f144c313252..e65e5ecc90cf 100644 --- a/docs/docs/pii-management.mdx +++ b/docs/docs/pii-management.mdx @@ -262,7 +262,8 @@ event_broker: ``` If multiple Kafka anonymization topics contain the `rasa_x_consumer` key-value pair, the anonymized events will be streamed -to the Kafka topic that is mapped to the first Kafka anonymization topic that contains the `rasa_x_consumer` key-value pair. +to the Kafka topic that is mapped to the first topic in the `anonymization_topics` list that contains the `rasa_x_consumer` +key-value pair. Note that the `rasa_x_consumer` key-value pair is optional. If it is not specified, the anonymized events will be published to the Kafka topic, but they will not be streamed to Rasa X/Enterprise. @@ -287,5 +288,5 @@ Running with log level DEBUG will increase the assistant's response latency beca Note that running `rasa shell` in debug mode with a Kafka event broker might result in logs related to the event publishing to be printed to console **after** the bot message. This behaviour is expected because the event anonymization and publishing -is done asynchronously as a background task and completes after the assistant has already predicted and executed the +is done asynchronously as a background task, so it will complete after the assistant has already predicted and executed the bot response.