Skip to content

Commit

Permalink
General purpose tenant contact tool
Browse files Browse the repository at this point in the history
To use this contact tool from the tools directory, you will
need to create a message file, e.g. /tmp/message, with content
example:

  --- snip ---

Regarding your allocation

Description: {{description}}
Allocation: {{cloud_name}}
Ticket: {{ticket}}

This message was composed using a simple jinja template.  The available variables are:

    description (taken from allocation object)
    cloud_name (taken from the cloud name)
    ticket (taken from allocation object)

This is just a test message while testing the notification tool.

  --- snip ---

and then call the tool with one of the following forms:

(rack based)
   ./notify_tenant.py --message /tmp/message  --subject "testing - outage impacting racks F18 and F22" --rack "f18 e22"

(allocation based)
    ./notify_tenant.py --message /tmp/message  --subject "testing - outage in CLOUD02 CLOUD03'" --cloud 'cloud02 cloud03'

(all allocations)
    ./notify_tenant.py --message /tmp/message  --subject "testing - ALL environments notification" --all

Fixes: #538
Change-Id: I58a7b076b019db05925885b41f57b3f8a4a17078
  • Loading branch information
kambiz-aghaiepour committed Dec 4, 2024
1 parent 2287196 commit 39d90e1
Show file tree
Hide file tree
Showing 3 changed files with 379 additions and 9 deletions.
62 changes: 53 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -84,6 +84,7 @@ QUADS automates the future scheduling, end-to-end provisioning and delivery of b
* [Find Available Web Interface](#find-available-web-interface)
* [Find a System by MAC Address](#find-a-system-by-mac-address)
* [Find Systems by Switch IP Address](#find-systems-by-switch-ip-address)
* [Tenant Notifications via Email or Ticketing system](#tenant-notifications-via-email-or-ticketing-system)
* [Using JIRA with QUADS](#using-jira-with-quads)
* [Backing up QUADS](#backing-up-quads)
* [Restoring QUADS from Backup](#restoring-quads-from-backup)
Expand Down Expand Up @@ -973,29 +974,29 @@ Resource properly removed
* To validate a clouds network config:

```bash
/opt/quads/quads/tools/verify_switchconf.py --cloud cloud10
python3 $PYTHONDIR/site-packages/quads/tools/verify_switchconf.py --cloud cloud10
```

* To validate and fix a clouds network config use `--change`

```bash
/opt/quads/quads/tools/verify_switchconf.py --cloud cloud10 --change
python3 $PYTHONDIR/site-packages/quads/tools/verify_switchconf.py --cloud cloud10 --change
```

* To validate a singular hosts network switch configuration:
```
/opt/quads/quads/tools/verify_switchconf.py --host host01.example.com
python3 $PYTHONDIR/site-packages/quads/tools/verify_switchconf.py --host host01.example.com
```

* To validate and fix a single hosts network config use `--change`

```
/opt/quads/quads/tools/verify_switchconf.py --host host01.example.com --change
python3 $PYTHONDIR/site-packages/quads/tools/verify_switchconf.py --host host01.example.com --change
```

* To straddle clouds and place a single host into a cloud it does not belong in (rare use case):
```bash
/opt/quads/quads/tools/verify_switchconf.py --host host01.example.com --cloud cloud10
python3 $PYTHONDIR/site-packages/quads/tools/verify_switchconf.py --host host01.example.com --cloud cloud10
```

Note, if host01.example.com is not in cloud10, but rather cloud20, you will see the following output:
Expand All @@ -1016,14 +1017,14 @@ WARNING -
* Passing the `--change` argument will make the changes effective in the switch. Not passing this will only verify the configuration is set to the desired.

```bash
/opt/quads/quads/tools/modify_switch_conf.py --host host01.example.com --nic1 1400 --nic2 1401 --nic3 1400 --nic4 1402 --nic5 1400
python3 $PYTHONDIR/site-packages/quads/tools/modify_switch_conf.py --host host01.example.com --nic1 1400 --nic2 1401 --nic3 1400 --nic4 1402 --nic5 1400
```
* All `--nic*` arguments are optional so this can be also done individually for all nics.

#### Mapping Interface to VLAN ID
* An easy way to figure out what VLAN corresponds to what generic `em` interface in the QUADS `--ls-interfaces` information we now include the following tool:
```bash
./opt/quads/quads/tools/ls_switch_conf.py --cloud cloud32
python3 $PYTHONDIR/site-packages/quads/tools/ls_switch_conf.py --cloud cloud32
INFO - Cloud qinq: 1
INFO - Interface em1 appears to be a member of VLAN 1410
INFO - Interface em2 appears to be a member of VLAN 1410
Expand Down Expand Up @@ -1141,6 +1142,49 @@ quads --ls-available --schedule-start "2019-12-05 08:00" --schedule-end "2019-12
quads --ls-available --schedule-end "2019-06-02 22:00"
```

### Tenant Notifications via Email or Ticketing system

* With the `notify_tenant.py` tenants can be easily emailed with important messages regarding their environment.
* Common use cases are to inform users of outages that may impact them. The `notify_tenant.py` can be called with various options.
* The contents of the messages sent should be crafted in a temporary file (which is a simple jinja template that interprets 3 possible variables).

- description (taken from allocation object)
- cloud_name (taken from the cloud name)
- ticket (taken from allocation object)

* For example, you can use a message template file (e.g. stored in `/tmp/message`) such as:

```
Regarding your allocation
Description: {{description}}
Allocation: {{cloud_name}}
Ticket: {{ticket}}
We are informing you of an upcoming outage, etc.
```

* Rack based notifications
```bash
python3 $PYTHONDIR/site-packages/quads/tools/notify_tenant.py --message /tmp/message --subject "Upcoming outage notification" --rack "f18 e22" --email --post
```
- To ensure email is sent, use the `--email` flag.
- To ensure message is posted to your ticketing system, use the `--post` flag.
- Omitting both `--email` and `--post` means no notification will get sent or posted.

* Cloud based notifications
```bash
python3 $PYTHONDIR/site-packages/quads/tools/notify_tenant.py --message /tmp/message --subject "Upcoming outage notification" --cloud "cloudXX cloudYY" --email --post
```
- The above will use the template message in `/tmp/message` and send it to the owners and cc-users of cloudXX and cloudYY.
- The message will also be posted to your ticketing system.

* Notifications to all users.
```bash
python3 $PYTHONDIR/site-packages/quads/tools/notify_tenant.py --message /tmp/message --subject "Upcoming outage notification" --all --email --post
```
- The above sends notifications to all active environments using your template message file in `/tmp/message`

#### Find Available Hosts based on Hardware or Model

* In QUADS `1.1.4` and higher you can now filter your availability search based on hardware capabilities or model type.
Expand Down Expand Up @@ -1335,15 +1379,15 @@ quads --validate-env --skip-network
* In `QUADS 1.1.8` you can skip past systems and host validation (Foreman) via:

```
/opt/quads/quads/tools/validate_env.py --skip-system
python3 $PYTHONDIR/site-packages/quads/tools/validate_env.py --skip-system
```

### Skipping Past Network and Systems Validation per Host

* In `QUADS 1.1.8` you can skip past both systems and network checks per host via:

```
/opt/quads/quads/tools/validate_env.py --skip-hosts host01.example.com host02.example.com
python3 $PYTHONDIR/site-packages/quads/tools/validate_env.py --skip-hosts host01.example.com host02.example.com
```

* Effectively, any host listed with `--skip-hosts` will pass it completely through validation.
Expand Down
232 changes: 232 additions & 0 deletions src/quads/tools/notify_tenant.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,232 @@
#!/usr/bin/env python3

import argparse
import asyncio
import logging
import os
import requests

from datetime import datetime, timedelta
from enum import Enum
from jinja2 import Template

from quads.config import Config
from quads.quads_api import QuadsApi, APIServerException, APIBadRequest
from quads.tools.external.netcat import Netcat
from quads.tools.external.postman import Postman
from quads.tools.external.jira import Jira, JiraException

logger = logging.getLogger(__name__)
logging.basicConfig(level=logging.INFO, format="%(message)s")


def post_message(args, ticket, description, cloud_name):
quads = QuadsApi(Config)
logger.info(
f"Posting message. Message file = {args.message}, "
+ f"Subject = {args.subject}, "
+ f"ticket = {ticket}, description = {description}"
)
try:
with open(os.path.join(args.message)) as _file:
template = Template(f"Subject: {args.subject}\n\n" + _file.read())
except Exception as e:
logger.info(f"{e}")
return False
content = template.render(
description=description,
ticket=ticket,
cloud_name=cloud_name,
)
loop = asyncio.get_event_loop()
try:
jira = Jira(
Config["jira_url"],
Config["jira_username"],
Config["jira_password"],
loop=loop,
)
except JiraException as ex: # pragma: no cover
logger.error(ex)
return False

_ass = quads.filter_assignments({"active": True, "validated": True, "cloud": cloud_name})[0]
result = loop.run_until_complete(jira.post_comment(_ass.ticket, content))
if not result:
logger.warning("Failed to update Jira ticket")

return result


def send_message(args, owner, ccuser, ticket, description, cloud_name):
logger.info(
f"Sending message. Message file = {args.message}, "
+ f"Subject = {args.subject}, owner = {owner}, ccuser = {ccuser}, "
+ f"ticket = {ticket}, description = {description}"
)
cc_users = [_cc_user.strip() for _cc_user in Config["report_cc"].split(",")]
for user in ccuser:
cc_users.append("%s@%s" % (user, Config["domain"]))
try:
with open(os.path.join(args.message)) as _file:
template = Template(_file.read())
except Exception as e: # pragma: no cover
logger.info(f"{e}")
return False
content = template.render(
description=description,
ticket=ticket,
cloud_name=cloud_name,
)
postman = Postman(
"INFO: [%s] %s" % (cloud_name, args.subject),
owner,
cc_users,
content,
)
result = postman.send_email()
return result


def determine_action(args):
quads = QuadsApi(Config)
results = []
if args.cloud:
cloud_names = args.cloud.split()
for _cloud_name in cloud_names:
_cloud = None
try:
_cloud = quads.get_cloud(_cloud_name)
except (APIServerException, APIBadRequest) as ex:
logger.debug(str(ex))
if not _cloud:
logger.error(f"Cloud: {_cloud_name} not found")
else:
logger.info(f"Cloud: {_cloud_name}")
if _cloud_name == "cloud01":
logger.info(f"Skipping notification for {_cloud_name}. This is used for available hosts.")
else:
hosts = quads.filter_hosts({"cloud": _cloud_name, "retired": False})
host_count = len(hosts)
if host_count > 0:
_ass = quads.filter_assignments({"active": True, "validated": True, "cloud": _cloud_name})[0]
logger.info(
f"Sending notification for {_cloud_name}, with {host_count} hosts. Owner = {_ass.owner}, cc_users = {_ass.ccuser}"
)
if args.email:
send_message(args, _ass.owner, _ass.ccuser, _ass.ticket, _ass.description, _cloud_name)
if args.post:
result = post_message(args, _ass.ticket, _ass.description, _cloud_name)
results.append(result)
else:
logger.info(f"Skipping notification for {_cloud_name}, no hosts found")

if args.all:
_clouds = quads.get_clouds()
for _cloud_obj in _clouds:
logger.info(f"Cloud: {_cloud_obj.name}")
hosts = quads.filter_hosts({"cloud": _cloud_obj.name, "retired": False})
if _cloud_obj.name == "cloud01":
logger.info(f"Skipping notification for {_cloud_obj.name}. This is used for available hosts.")
else:
host_count = len(hosts)
if host_count > 0:
_ass = quads.filter_assignments({"active": True, "validated": True, "cloud": _cloud_obj.name})[0]
logger.info(
f"Sending notification for {_cloud_obj.name}. with {host_count} hosts. Owner = {_ass.owner}, cc_users = {_ass.ccuser}"
)
if args.email:
send_message(args, _ass.owner, _ass.ccuser, _ass.ticket, _ass.description, _cloud_obj.name)
if args.post:
result = post_message(args, _ass.ticket, _ass.description, _cloud_obj.name)
results.append(result)
else:
logger.info(f"Skipping notification for {_cloud_obj.name}, no hosts found")

if args.rack:
rack_names = args.rack.split()
hosts = quads.filter_hosts({"retired": False})
clouds_in_racks = []
for _host_obj in hosts:
if _host_obj.name.startswith(tuple(rack_names)):
clouds_in_racks.append(_host_obj.cloud.name)
for cloud_name in sorted(set(clouds_in_racks)):
logger.info(f"Cloud: {cloud_name}")
_ass = quads.filter_assignments({"active": True, "validated": True, "cloud": cloud_name})[0]
logger.info(f"Sending notification for {cloud_name}. Owner = {_ass.owner}, cc_users = {_ass.ccuser}")
if args.email:
send_message(args, _ass.owner, _ass.ccuser, _ass.ticket, _ass.description, cloud_name)
if args.post:
result = post_message(args, _ass.ticket, _ass.description, cloud_name)
results.append(result)

return results


def verify_argparse(args):
if not any([getattr(args, option) for option in ["all", "cloud", "rack"]]):
logger.error("Please select at least one option from --all, --cloud, or --rack")
exit(1)
if args.rack and (args.all or args.cloud):
logger.error("Argument --rack cannot be used with either --all or --cloud")
exit(1)


def main(): # pragma: no cover
parser = argparse.ArgumentParser(description="Notification tool to send messages to tenants")
parser.add_argument(
"--message",
dest="message",
type=str,
default=None,
help="Path to file containing message contents",
required=True,
)
parser.add_argument(
"--cloud",
dest="cloud",
type=str,
default=None,
help="List of allocations to target, e.g. 'cloud02 cloud03 cloud07'",
)
parser.add_argument(
"--all",
dest="all",
action="store_true",
help="Determine whether notification should be sent to all active allocations",
)
parser.add_argument(
"--rack",
dest="rack",
type=str,
default=None,
help="Racks to consider when sending notification, e.g. 'd30 d31 d32'",
)
parser.add_argument(
"--email",
dest="email",
action="store_true",
help="Determine whether email should be sent",
)
parser.add_argument(
"--post",
dest="post",
action="store_true",
help="Determine whether message should be posted to ticketing system",
)
parser.add_argument(
"--subject",
dest="subject",
type=str,
default=None,
help="Notification subject line",
required=True,
)

args = parser.parse_args()
verify_argparse(args)
determine_action(args)


if __name__ == "__main__": # pragma: no cover
main()
Loading

0 comments on commit 39d90e1

Please sign in to comment.