Why Your PromQL Availability Query Returns Nothing When Services Are Healthy

Your SLI query shows 100% availability as No Data. Here is why PromQL returns empty results instead of zero, and how to fix sums, ratios and alerts that go blank.

Boxy analog meter with a lime face, knobs and a cable

Contents

Your availability dashboard looks great under load. The moment your services run clean (zero errors, everything healthy), the metrics disappear. Gaps in the chart. “No Data” where you expect 100%.

This is not a configuration problem. It is how PromQL works, and the same behavior blanks out any sum, ratio or alert that depends on a series that may not exist. Three things cause it: a missing series, an instant evaluation that misses a sparse sample, and or binding looser than you expect. This post fixes each one.

The Setup

Say you are tracking availability for a set of microservices using trace-derived metrics from OpenTelemetry. A standard SLI approach computes availability as:

Availability = (1 - error_rate) × 100

Where error rate is the ratio of 5xx responses to total requests. A typical first attempt:

(1 - (
sum by (service_name) (
trace_endpoint_count{
service_name=~"auth-service|billing-service|api-gateway|user-service",
env="prod",
span_kind="SPAN_KIND_SERVER",
http_status_code!="",
http_status_code=~"5.*"
}
)
/
(
sum by (service_name) (
trace_endpoint_count{
service_name=~"auth-service|billing-service|api-gateway|user-service",
env="prod",
span_kind="SPAN_KIND_SERVER"
}
) + 0.0000001
)
)) * 100

The + 0.0000001 in the denominator avoids division by zero. Looks reasonable.

The Problem

This query works when services are throwing 5xx errors. The moment a service has zero 5xx responses, which is the good scenario, the numerator returns no data. Not 0. An empty instant vector.

In PromQL, when no time series matches a selector, the result is an empty set. Dividing an empty vector by anything produces another empty vector. The entire expression for that service evaluates to nothing, and your dashboard shows a gap.

The same thing happens in sums. If sum(a) + sum(b) has no series for b, the result is empty, not the value of sum(a). One missing operand blanks the whole expression.

This is especially painful on SLI dashboards, where a missing data point can trigger alerts or unsettle stakeholders precisely when the service is healthiest.

Why PromQL Works This Way

PromQL is a set-based language. Every selector returns a set of time series. Arithmetic operators work on matching series across sets. If a series does not exist in one operand, there is nothing to match, so no result is produced.

This differs from SQL, where COUNT(*) on an empty result set returns 0. In PromQL, no matching series means no output. See the PromQL cheat sheet for a full reference on how vector matching works.

The Fix for Grouped Ratios: The * 0 Fallback Pattern

Make sure the 5xx selector always returns a series, even when there are no 5xx errors. Use or with a zero-valued version of a series you know exists (total requests):

(1 - (
sum by (service_name) (
trace_endpoint_count{
service_name=~"auth-service|billing-service|api-gateway|user-service",
env="prod",
span_kind="SPAN_KIND_SERVER",
http_status_code!="",
http_status_code=~"5.*"
}
or
trace_endpoint_count{
service_name=~"auth-service|billing-service|api-gateway|user-service",
env="prod",
span_kind="SPAN_KIND_SERVER"
} * 0
)
/
(
sum by (service_name) (
trace_endpoint_count{
service_name=~"auth-service|billing-service|api-gateway|user-service",
env="prod",
span_kind="SPAN_KIND_SERVER"
}
) + 0.0000001
)
)) * 100

The key addition is:

or
trace_endpoint_count{...all services, env="prod", span_kind="SPAN_KIND_SERVER"} * 0

Step by step:

  1. 5xx errors exist: The first selector returns matching series. or keeps every series from the left side and adds only right-side series whose label sets are not already there. Normal path.
  2. No 5xx errors: The first selector returns nothing, so or returns the fallback: total request count multiplied by zero. That is a time series with the correct service_name label and a value of 0.
  3. sum by (service_name) collapses correctly, giving 0 for the error count.
  4. Final result: (1 - 0/total) * 100 = 100%, exactly right for a healthy service.

When or vector(0) Is the Right Fix

vector(0) returns a single-element vector with value 0 and no labels. That makes it the shortest fix when your expression has no grouping:

sum(rate(orders_completed_total[5m])) or vector(0)

If orders_completed_total has no series, sum() returns nothing and or falls back to a single {} series with value 0. If it has series, the ungrouped sum() already returns a {} series, so or keeps it and drops the fallback.

It stops working once the expression is grouped. Take sum by (service_name) (errors or vector(0)) / sum by (service_name) (total). When errors is empty, the aggregation returns a {} series. The division matches on label sets, {} matches no {service_name="..."} series on the right, and the element is dropped. You get no error, just an empty result.

When only some groups are missing, the groups that have data come through and the missing ones stay blank. You might expect a many-to-one matching error here, but PromQL raises none. The label-less zero has nothing to pair with.

So the rule is short. Use or vector(0) when the result is a single ungrouped value. Use the * 0 pattern from the previous section when the result carries labels.

Sparse business metrics

The case where or vector(0) earns its place is a dashboard that adds several event-driven series together. A team we worked with had a panel summing completed events across product lines, written as one block per line (metric and label names here are illustrative):

sum(rate(events_completed_total{line="a"}[5m]))
+
sum(rate(events_completed_total{line="b"}[5m]))

Whenever line b had no events in the window, its block returned nothing, and the + blanked the whole expression. The panel dropped to blank for all lines, even though line a was busy.

The fix was to end every block with or vector(0), so each operand always exists:

(sum(rate(events_completed_total{line="a"}[5m])) or vector(0))
+
(sum(rate(events_completed_total{line="b"}[5m])) or vector(0))

The parentheses matter, and the next section explains why.

Operator Precedence in Ratios

In PromQL, or, and and unless have the lowest precedence of all binary operators. Arithmetic and comparisons bind tighter. So this:

sum(errors) / sum(total) or vector(0)

is parsed as (sum(errors) / sum(total)) or vector(0). The fallback wraps the whole ratio. If errors is missing, the ratio is empty and you get 0, which reads as “zero error rate” even when total is also missing. That is a silent wrong answer.

Put the fallback on each operand instead:

(sum(errors) or vector(0)) / clamp_min(sum(total) or vector(0), 0.0000001)

Now a missing errors becomes 0 and the numerator is always defined. The denominator is floored so you never divide by zero. In Prometheus, x / 0 gives +Inf and 0 / 0 gives NaN, and neither renders as a clean number on a dashboard. The 0.0000001 floor is the same guard the earlier queries use as + 0.0000001; clamp_min does the same job without shifting a non-zero denominator.

Notice what the fix cannot do. If total is missing, you now get 0 / 0.0000001 = 0. For an error ratio that is fine. For an availability SLI, (1 - 0) * 100 = 100% while no traffic is being measured at all. That is the reason to pair these queries with an alert on missing data, covered below.

Instant vs Range Queries for Sparse Metrics

An instant query evaluates the expression at one timestamp. A range query evaluates it at every step between a start and an end. The Prometheus HTTP API exposes both, and dashboards pick one per query.

For each evaluation timestamp, Prometheus takes the newest sample that is less than the lookback period old, 5 minutes by default. A sparse metric that last reported 6 minutes ago has no value at that instant, so the instant query returns nothing for it.

One nuance keeps this from being a blanket 5-minute grace period. A scraped series that stops appearing is marked stale right away, so it disappears at the next evaluation. The 5-minute window mostly matters for pushed or remote-written metrics, which is typical for event-driven business metrics. Exporters that attach their own timestamps behave differently again, per the staleness rules.

What Last9 dashboards do with it

Each Last9 dashboard query runs in Range or Instant mode, and Stat panels are where sparse metrics usually bite. Our dashboard docs are explicit: a Stat panel shows the value at the last evaluated step, even in Range mode. The time range only changes the chart axis.

So switching a Stat panel to Range mode does not make it cover the window. Aggregate over the window instead:

sum(sum_over_time(events_completed[$__range])) or vector(0)

Pick the aggregator that matches what the number should mean: avg_over_time for averages, max_over_time for peaks, sum_over_time for totals. If no sample exists anywhere in the window, the range vector is empty and the result is still blank, which is why the or vector(0) stays. Our post on how sum_over_time works covers the gap behavior in more detail.

Alert on Silent Gaps With absent()

Everything above makes missing data look like zero. That is right for a dashboard and risky for an alert. A threshold rule such as errors > 0 or availability < 99 has nothing to compare when the series is gone, so it stays silent. Fill the gap with zero and it stays silent too, while the dashboard now shows a confident number.

A threshold rule cannot tell “no errors” from “no telemetry”. Add a second rule that only watches for the gap:

absent_over_time(events_completed_total{env="prod"}[10m])

This returns a single series with value 1 when no sample exists for the whole 10 minutes, and an empty vector otherwise. A single missed sample does not trigger it, because the window is 10 minutes, twice the default lookback.

Plain absent() is the instant form. It checks one evaluation timestamp and only looks back as far as the default 5-minute lookback. The difference with absent_over_time() is the window: it is explicit, and you can make it longer than the lookback, so the rule fires only after the gap has lasted that long. See the function reference for both.

Two details trip people up:

  • Labels come only from equality matchers. absent(up{job="api"}) returns {job="api"}. A regex matcher such as job=~"api.*" adds no label, and absent(sum(...)) returns {}. Add static labels in the alert rule so the notification says what went missing.
  • It checks a selector, not a service. absent() tells you no series matched. Pair it with the specific metric that should always exist for that service.

Last9 alert rules let you write PromQL directly, so the same expression can be the rule’s query with a > 0 condition. For the Prometheus rule-file form, see our alerting examples.

Other Approaches

clamp_min(..., 0) sets a floor value on existing series. It does not help when the series does not exist at all, because there is nothing to clamp.

Recording rules pre-compute the error count with the zero case handled. They work, but add operational overhead and another artifact to maintain.

absent() inside the query returns a series with value 1 when its input is empty. You could build (your_query or absent(your_query) * 0), but absent() does not preserve labels across grouped queries, so it fails where the * 0 pattern works.

The General Pattern

Any time you compute a ratio in PromQL where the numerator might legitimately be empty:

sum by (label) (
metric_with_specific_filter{...}
or
metric_with_broader_filter{...} * 0
)
/
sum by (label) (
metric_with_broader_filter{...}
)

The broader filter should match a superset of what the specific filter matches: same labels, without the restrictive condition. * 0 gives you the right labels with a zero value, and or only uses the fallback when the primary selector has no series for those labels.

Which fix to use depends on the situation:

SituationFix
Ungrouped sum or stat that can be emptyexpr or vector(0)
Grouped ratio, numerator can be emptynumerator or broader_selector * 0 inside sum by
Several blocks added together(block or vector(0)) around each block
Ratio with a fallbackParentheses around each operand, not around the ratio
Denominator can be zeroclamp_min(denominator, 0.0000001)
Stat panel on a sparse metricAggregate over [$__range], then or vector(0)
Detect that data stoppedA separate absent_over_time(metric[10m]) rule

Build SLIs That Hold Up in Production

Availability queries are the foundation of any practical SLO implementation. The * 0 pattern and the parenthesised or vector(0) are query-correctness issues that only surface in production, when traffic patterns hit edge cases your staging environment never saw.

If you are using OpenTelemetry trace-derived metrics, Last9 stores trace_endpoint_count natively from your OTLP pipeline, so the query above works without recording rules or custom aggregations. Check the OpenTelemetry integration docs to wire it up.

FAQ

Why does my PromQL query return no data instead of 0?

PromQL works on sets of time series. When a selector matches no series, or the matching series has no sample inside the lookback window, the result is an empty vector, not a zero. Any arithmetic on an empty vector stays empty, so a sum, ratio or stat panel built on it shows no data.

How do I make PromQL return 0 when a series is missing?

Append or vector(0) to an ungrouped expression, for example sum(errors) or vector(0). For grouped expressions, use or with the same metric multiplied by 0 so the fallback series keeps the labels. Wrap each side of a ratio in parentheses so the fallback applies to the operand and not to the whole ratio.

Does or vector(0) work with sum by?

Not on its own. vector(0) has no labels, so after a sum by (label) it cannot match any labeled series on the other side of a division and is dropped. The result stays empty for the missing groups. Use a labeled fallback, such as the same metric multiplied by 0, inside the aggregation.

What is the difference between an instant query and a range query in Prometheus?

An instant query evaluates the expression at one timestamp and returns one value per series. A range query evaluates it at every step between a start and an end time and returns a series of values. Sparse metrics can look empty in an instant query because the latest sample may be older than the lookback window.

How do I alert when a metric stops reporting?

Write a separate rule with absent() or absent_over_time(). absent_over_time(metric[10m]) returns 1 only when no sample exists for the whole 10 minutes, so one missed sample does not flap the alert. Threshold rules cannot do this job, because a missing series produces no value to compare.

About the authors
Prathamesh Sonpatki

Prathamesh Sonpatki

Prathamesh works as an evangelist at Last9, runs SRE stories - where SRE and DevOps folks share their stories, and maintains o11y.wiki - a glossary of all terms related to observability.

Last9 logo and enter key

Start observing for free. No lock-in.

OpenTelemetry · Prometheus

Just update your config. Start seeing data on Last9 in seconds.

Datadog · New Relic · Others

We've got you covered. Bring over your dashboards & alerts in one click.

Built on Open Standards

100+ integrations. OTel native, works with your existing stack.