Common Rule Filters: Apply PromQL label filter conditions across indicators of an Alert Group

Common Rule Filters: Apply PromQL label filter conditions across indicators of an Alert Group

Common Rule Filters for Alert Groups allow users to specify PromQL filter conditions that will get injected into every Indicator that is part of the group. Since these filters are injected at the indicator level, they are also honored in the alert evaluations.

A demo for Common Rule Filters

Common Rule Filters can be used when an Alert Group represents some component in your system, and the component is identified by a specific PromQL label filter. For example, if your alert group represents a particular Kubernetes deployment called “login”, the PromQL of every indicator will have deployment="login" in the filters.

Common Rule Filters allow you to specify this condition only once at a group level, instead of adding a label filter for every indicator or alert rule. It is also a good way to show the intent of scope of an alert group. The above deployment filter, when configured as a Common Rule Filter prominently signals that the alert group represents a particular deployment.

They can also be maintained by declarative alerting via IaC. The updated YAML structure with the new adhoc_filters key would look something like this:

entities:
  - name: quant-services
    external_ref: quant-services
    type: service
    entity_class: alert-manager
    namespace: test-namespace
    indicators:
     - name: throughput
       query: sum(up)
       unit: count
    alert_rules:
      - indicator: throughput
        name: throughput_2000
        greater_than: 2000
        bad_minutes: 4
        total_minutes: 10
    adhoc_filter:
      data_source: last9-legacy-cluster - ap-south-1 (system)
      labels:
        cluster: alpha.last9.io
💡
Sign up on Levitate and unlock High Cardinality monitoring that doesn’t die on you.