May 13th, ‘23
Editor support for Streaming Aggregation
When we launched the support for streaming aggregations to tame high cardinality data, the workflow involved using a GitOps based CI/CD pipeline.
The Last9 team will provide the GitHub repository with predefined CI/CD workflow for streaming aggregation.
The flow is involves 4 easy steps.
- Define the streaming aggregations using PromQL in a per cluster file.
- Send a pull request.
- Review and merge the pull request and that's it the streaming aggregation rule will get applied via the CI/CD pipeline.
- Explore in the embedded Grafana, to verify the streaming aggregation has taken effect.
While this flow is ideal for collaboration between different stakeholders and teams as the concerns for performing streaming aggregations can cut across different organizational departments considering the importance of high cardinality data with respect to cost, business importance and infrastructure, we also realized the need of a workflow via the Last9 dashboard itself to handle the streaming aggregation pipelines.
With today's release, we have the Streaming Aggregation Editor available for each Levitate cluster. Just head over to your cluster, click on
Settings and you will see the editor in action.
Everything else remains the same, refer to following document to know in detail about how to define streaming aggregations.
This release also has tons of performance improvements to make Levitate more stable and more reliable as a managed time series data warehouse.