Creating Index patterns via role

README.md

@@ -23,6 +23,10 @@ This ansible playbook supports the following,
 - Configure TLS/SSL for OpenSearch transport layer(Nodes to Nodes communication) and REST API layer
 - Generate self-signed certificates to configure TLS/SSL for opensearch
 - Configure the Internal Users Database with limited users and user-defined passwords
+- Configuration of authentication and authorization via OpenID
+- Overriding default settings with your own
+- Creation/Updating ISM Policies
+- Creating Index patterns
 - Install and configure the Apache2.0 opensource OpenSearch Dashboards
 
 ### Prerequisite
@@ -87,13 +91,73 @@ cluster_type: single-node
 
 You should set the reserved users(`admin` and `kibanaserver`) password using `admin_password` and `kibanaserver_password` variables.
 
+If you define your own internal users (in addition to the reserved `admin` and `kibanaserver`) in custom configuration 
+files, then passwords to them should be set via variables on the principle of `<username>_password`
+
 It will install and configure the opensearch. Once the deployment completed, you can access the opensearch Dashboards with user `admin` and password which you provided for variable `admin_password`.
 
     # Deploy with ansible playbook - run the playbook as non-root user which have sudo privileges,
     ansible-playbook -i inventories/opensearch/hosts opensearch.yml --extra-vars "admin_password=Test@123 kibanaserver_password=Test@6789" --become
 
 **Note**: Change the user details in `ansible_user` parameter in `inventories/opensearch/hosts`  inventory file.
 
+### OpenID authentification
+To enable authentication via OpenID, you need to change the `auth_type` variable in the inventory file 
+`inventories/opensearch/group_vars/all/all.yml` by setting the value `oidc` and prescribe the necessary settings 
+in the `oidc:` block.
+
+### Custom configuration files
+
+To override the default settings files, you need to put your settings in the `files` directory. The files should be 
+named exactly the same as the original ones (internal_users.yml, roles.yml, tenants.yml, etc.)
+
+Especially note the file `files/internal_users.yml`. If it exists and the `copy_custom_security_configs: true` setting is enabled, 
+then only in this case the task of setting passwords for internal users from variables is started. If the file `internal_users.yml` 
+is not located in the `files` directory, but, for example, in one of its subdirectories, then playbook will not work correctly
+
+### IaC (Infrastructure-as-Code)
+
+If you want to use the role not only for the initial deployment of the cluster, but also for further management of it, 
+then set the `iac_enable` parameter to `true`. 
+
+By default, if the /tmp/opensearch-nodecerts directory with certificates exists on the server from which the playbook 
+is launched, it is assumed that the configuration has not changed and some settings are not copied to the target servers.
+
+Conversely, if the /tmp/opensearch-nodecerts directory does not exist on the server from which the playbook is launched, 
+then new certificates and settings are generated and they are copied to the target servers.
+
+If you use this repository not only for the initial deployment of the cluster, but also for its automatic configuration 
+via CI/CD, then new certificates will be generated every time the pipeline is launched, overwriting existing ones, which 
+is not always necessary if the cluster is already in production.
+
+When iac_enable enabling, and all the cluster servers have all the necessary certificates, they will not be copied again. 
+If at least on one server (for example, when adding a new server to the cluster) if there is not at least one certificate 
+from the list, then all certificates on all cluster servers will be updated
+
+Also, if the option is enabled, the settings files will be updated with each execution (previously, the settings were 
+updated only if the /tmp/opensearch-nodecerts directory was missing on the server from which the playbook was launched 
+and new certificates were generated)
+
+### ISM Policies
+
+OpenSearch uses the ISM (Index State Management) plugin to manage the lifecycle of indexes. With the help of policies, 
+you can, for example, change the number of replicas for indexes, when certain conditions occur, or delete them.
+
+If you want to manage policies using the opensearch role, set the `apply_custom_ism: yes` parameter, and create json 
+files with policies in the `files/ism/policy` directory. Examples of policies can be found in the same directory.
+
+### Index patterns
+
+To search for indexes in Dashboards, you need to create an index pattern (`Stack Management` -> `Index patterns`). 
+If there are a lot of indexes and they are in different tenants, then manually creating them can be quite time-consuming, 
+especially if one pattern needs to be created in several tenants at the same time.
+
+If you want to create a large number of index patterns using the `dashboards` role, then set the `iac_enable: yes` 
+parameter and fill in the `create_index_patterns` list.
+
+When performing the role, index patterns will be created. When creating, the "overwrite=true" parameter is used, 
+which prevents the creation of identical objects when running multiple times.
+
 ## Contributing
 
 See [developer guide](DEVELOPER_GUIDE.md) and [how to contribute to this project](CONTRIBUTING.md).

files/internal_users.yml

@@ -0,0 +1,26 @@
+---
+# This is the internal user database
+# The hash value is a bcrypt hash and can be generated with plugin/tools/hash.sh
+
+_meta:
+  type: "internalusers"
+  config_version: 2
+
+# Define your internal users here
+
+admin:
+  hash: "{{ admin_password }}"
+  reserved: true
+  backend_roles:
+  - "admin"
+  description: "admin user"
+
+kibanaserver:
+  hash: "{{ kibanaserver_password }}"
+  reserved: true
+  description: "kibanaserver user"
+
+logstash:
+  hash: "{{ logstash_password }}"
+  reserved: true
+  description: "logstash user"

files/ism/policy/delete_after_30d.json

@@ -0,0 +1,42 @@
+{
+  "policy": {
+    "description": "delete after 30d workflow",
+    "default_state": "hot",
+    "schema_version": 1,
+    "ism_template": {
+      "index_patterns": [
+        "ingress-nginx-*",
+        "mywebapp-*"
+      ],
+      "priority": 100
+    },
+    "states": [
+      {
+        "name": "hot",
+        "actions": [
+          {
+            "replica_count": {
+              "number_of_replicas": 1
+            }
+          }
+        ],
+        "transitions": [
+          {
+            "state_name": "delete",
+            "conditions": {
+              "min_index_age": "30d"
+            }
+          }
+        ]
+      },
+      {
+        "name": "delete",
+        "actions": [
+          {
+            "delete": {}
+          }
+        ]
+      }
+    ]
+  }
+}

files/ism/policy/hot7_warm30_delete.json

@@ -0,0 +1,60 @@
+{
+  "policy": {
+    "description": "hot (7d) warm (30d) delete (after 30d) workflow",
+    "default_state": "hot",
+    "schema_version": 1,
+    "ism_template": {
+      "index_patterns": [
+        "kube-apiserver-audit-",
+        "syslog-*"
+      ],
+      "priority": 100
+    },
+    "states": [
+      {
+        "name": "hot",
+        "actions": [
+          {
+            "replica_count": {
+              "number_of_replicas": 1
+            }
+          }
+        ],
+        "transitions": [
+          {
+            "state_name": "warm",
+            "conditions": {
+              "min_index_age": "7d"
+            }
+          }
+        ]
+      },
+      {
+        "name": "warm",
+        "actions": [
+          {
+            "replica_count": {
+              "number_of_replicas": 0
+            }
+          }
+        ],
+        "transitions": [
+          {
+            "state_name": "delete",
+            "conditions": {
+              "min_index_age": "30d"
+            }
+          }
+        ]
+      },
+      {
+        "name": "delete",
+        "actions": [
+          {
+            "delete": {}
+          }
+        ]
+      }
+    ]
+  }
+}

files/roles.yml

@@ -0,0 +1,54 @@
+---
+_meta:
+  type: "roles"
+  config_version: 2
+
+
+indexes_full_access:
+  reserved: false
+  index_permissions:
+    - index_patterns:
+        - "*"
+      allowed_actions:
+        - "*"
+  tenant_permissions:
+  - tenant_patterns:
+    - "*"
+    allowed_actions:
+    - "kibana_all_write"
+# ----------------------------------------------------
+indexes_security_search_full_access:
+  reserved: true
+  index_permissions:
+    - index_patterns:
+        - "kube-apiserver-audit-*"
+        - "syslog-*"
+      allowed_actions:
+        - "indices:data/read/search*"
+        - "read"
+        - "view_index_metadata"
+  tenant_permissions:
+  - tenant_patterns:
+    - "SECURITY"
+    allowed_actions:
+    - "kibana_all_write"
+# ----------------------------------------------------
+indexes_web_search_full_access:
+  reserved: true
+  index_permissions:
+    - index_patterns:
+        - "ingress-nginx-*"
+        - "mywebapp-*"
+      allowed_actions:
+        - "indices:data/read/search*"
+        - "read"
+        - "view_index_metadata"
+  tenant_permissions:
+  - tenant_patterns:
+    - "WEB"
+    allowed_actions:
+    - "kibana_all_write"
+# ----------------------------------------------------
+# Restrict users so they can only view visualization and dashboard on OpenSearchDashboards
+kibana_read_only:
+  reserved: true

files/roles_mapping.yml

@@ -0,0 +1,59 @@
+---
+# In this file users, backendroles and hosts can be mapped to Security roles.
+# Permissions for OpenSearch roles are configured in roles.yml
+
+_meta:
+  type: "rolesmapping"
+  config_version: 2
+
+kibana_server:
+  reserved: true
+  users:
+  - "kibanaserver"
+
+logstash:
+  reserved: true
+  users:
+  - "logstash"
+
+# Define your roles mapping here
+all_access:
+  reserved: false
+  backend_roles:
+  - "admin"
+  - "opensearch_admin"
+  description: "Maps admin to all_access"
+# ----------------------------------------------------
+indexes_full_access:
+  reserved: false
+  backend_roles:
+  - "opensearch_admin"
+  description: "Maps admin to indexes_full_access"
+# ----------------------------------------------------
+own_index:
+  reserved: false
+  users:
+  - "*"
+  description: "Allow full access to an index named like the username"
+# ----------------------------------------------------
+readall:
+  reserved: false
+  backend_roles:
+  - "opensearch_index_read_all"
+# ----------------------------------------------------
+indexes_security_search_full_access:
+  reserved: true
+  backend_roles:
+  - "opensearch_index_read_all"
+  - "opensearch_index_read_security"
+  description: "Maps users to indexes_security_search_full_access"
+# ----------------------------------------------------
+indexes_web_search_full_access:
+  reserved: true
+  backend_roles:
+  - "opensearch_index_read_all"
+  - "opensearch_index_read_web"
+  description: "Maps users to indexes_web_search_full_access"
+
+
+

files/tenants.yml

@@ -0,0 +1,12 @@
+---
+_meta:
+  type: "tenants"
+  config_version: 2
+
+# Define your tenants here
+SECURITY:
+  reserved: false
+  description: "Tenant for security logs (e.g. kubernetes audit or opensearch audit)"
+WEB:
+  reserved: false
+  description: "Tenant for web-app logs"