Easily setup a service level objective using prometheus, based on lessons from the SRE workbook.
slo-generator -slo.path=slo_example.yml -rule.output rule.yml
Look the file slo_example.yml to see how to parametrize SLOs and generate Prometheus rules by running the following command:
- 1. Target Error Rate ≥ SLO Threshold, using
alertMethod: simple - 2. Increased Alert Window, using
alertMethod: simple - 3. Incrementing Alert Duration, using
alertMethod: simple - 4. Alert on Burn Rate
alertMethod: simple and burnRate: <rate> - 5. Multiple Burn Rate Alerts, using
alertMethod: multi-window and shortWindow: false - 6. Multiwindow, Multi-Burn-Rate Alerts, using
alertMethod: multi-window
Look the file slo_simple_example.yml to see a full example of usage. the simple alert method require two params:
- alertWindow: how far back in time will used to alerting. supported values: 5m, 30m, 1h, 2h, 6h, 1d and 3d.
- alertWait: for long time will begin fire an alert.
The philosofy of this alert is described on the section of book: (https://landing.google.com/sre/workbook/chapters/alerting-on-slos#6-multiwindow-multi-burn-rate-alerts)
The Workbook suggests to create classes to simplify how to set a SLO for your services, read details about concepts here
Look at slo_example_with_classes.yml and slo_classes.yml to see how to define classes and associate with your services.
After that, you can run the command specifying the classes:
slo-generator -slo.path=slo_example_with_classes.yml -classes.path slo_classes.yml -rule.output rule.yml
We support to export SLOs as a well known PrometheusRule resources managed by prometheus-operator, just use -kubernetes flag, example:
slo-generator -kubernetes -slo.path=slo_example.yml > slo_manifest.yml
cat slo_manifest.yml
kubectl apply -f slo_manifest.yml
All generated SLOs are visible by grafana:
Import dashboard using following JSON files