Editor support for Streaming Aggregation

When we launched the support for streaming aggregations to tame high cardinality data, the workflow involved using a GitOps based CI/CD pipeline.

The Last9 team will provide the GitHub repository with predefined CI/CD workflow for streaming aggregation.

The flow is involves 4 easy steps.

Define the streaming aggregations using PromQL in a per cluster file.
Send a pull request.
Review and merge the pull request and that's it the streaming aggregation rule will get applied via the CI/CD pipeline.
Explore in the embedded Grafana, to verify the streaming aggregation has taken effect.
Repeat!

While this flow is ideal for collaboration between different stakeholders and teams as the concerns for performing streaming aggregations can cut across different organizational departments considering the importance of high cardinality data with respect to cost, business importance and infrastructure, we also realized the need of a workflow via the Last9 dashboard itself to handle the streaming aggregation pipelines.

With today's release, we have the Streaming Aggregation Editor available for each Levitate cluster. Just head over to your cluster, click on Settings and you will see the editor in action.

Everything else remains the same, refer to following document to know in detail about how to define streaming aggregations.

🚿 Streaming Aggregation

This document explains how to setup streaming aggregation pipelines in Levitate.

Last9 Documentation

💡

If you are already using a GitOps based pipeline for streaming aggregations, you can continue using it. The editor will be disabled for you that scenario to avoid accidental overwrites. You can use either the editor or GitOps pipeline to manage streaming aggregations rules.

This release also has tons of performance improvements to make Levitate more stable and more reliable as a managed time series data warehouse.