Alert Timing & Delivery Reference
Understand exactly when your alerts fire, repeat, and resolve across notification channels. Quick reference for alert timing behavior with threshold alerts, anomaly detection, and SLO violations.
Get alerts when you need them. Here’s exactly when notifications fire, repeat, and resolve across your channels.
Quick Reference
Threshold & Anomaly Alerts
- First alert: 2 minutes after threshold breach
- Repeat alerts: Every 61 minutes while active
- Resolution: 11 minutes after condition clears
SLO Alerts
- First alert: 4 minutes after SLO violation
- Repeat alerts: Every 16 minutes while active
- Resolution: 31 minutes after SLO recovers
How Alert Timing Works
When something goes wrong at timestamp t-1
, here’s the timeline:
Threshold & Anomaly Alerts
t-1: Issue occurs (CPU spikes, error rate jumps)t+1: Last9 generates the alertt+2: Notification hits your channelt+61: Reminderrecovery+11: Resolution
SLO Alerts
t-1: SLO violation begins (error budget burned)t+3: Last9 generates the alertt+4: Notification hits your channelt+16: Reminderrecovery+31: Resolution
Notification Channels
Threshold & Anomaly Alerts: All channels (Slack, PagerDuty, OpsGenie, Webhook, Email)
SLO Alerts: Slack, PagerDuty, OpsGenie, Webhook only (Email not supported)
All supported channels follow the same timing behavior:
Alert Type | First Notification | Repeat Frequency | Resolution Delay |
---|---|---|---|
Threshold/Anomaly | t+2 minutes | Every 61 minutes | 11 minutes |
SLO Violation | t+4 minutes | Every 16 minutes | 31 minutes |
Why These Delays?
- Processing time: Last9 needs 1-3 minutes to analyze metrics and confirm alert conditions
- Delivery buffer: 1-minute buffer accounts for network latency and channel processing
- Resolution delays: Prevents flapping alerts when conditions briefly recover then fail again
Examples
Scenario: API response time spikes at 1:59 PM
- 2:01 PM: Last9 confirms threshold breach
- 2:02 PM: Webhook fires, Slack notification arrives
- 3:01 PM: Reminder if still alerting
- When fixed + 11 min: Resolution notification
Scenario: Error budget burns through at 3:14 PM
- 3:18 PM: Last9 confirms SLO violation
- 3:19 PM: Slack/PagerDuty/Webhook notifications sent (Email not supported for SLO)
- 3:31 PM: First reminder if still violating
- When recovered + 31 min: Resolution notification
Troubleshooting
Please get in touch with us on Discord or Email if you have any questions.