Apr 11th, ‘23/Changelog
Tame High Cardinality Data using Streaming Aggregation
Levitate now allows streaming aggregation of metrics to deal with near infinite high cardinality metrics. Define the streaming aggregations rules using PromQL.
High Cardinality data is almost reality in today's cloud native environment world. Levitate has superior defaults for managing high cardinality.
| Quota Type | Limit | Reset Period | Possible Actions |
|---|---|---|---|
| Per Time Series Cardinality | 1,000,000 | Per Hour | Can be Raised on Request |
| Per Time Series Cardinality | 20,000,000 | Per Day | Can be Raised on Request |
Even with these defaults, the metrics data may overflow. Levitate supports advanced features such as streaming aggregations to tackle such cases.
Streaming aggregation has been a long-standing concept in the data processing world, providing new observability capabilities that may have otherwise been avoided due to concerns about performance or resource requirements to store the necessary data. This powerful concept can significantly boost SLO queries, resulting in enhanced customer experiences. Furthermore, it can effectively monitor system performance in real time and detect anomalous components of the infrastructure without requiring large amounts of resources behind the query and storage layers.
Levitate now supports managing high cardinality data using PromQL based streaming aggregations.
Here is a walkthrough on how to start using streaming aggregations today with Levitate.
We have also written a detailed blog post about the technical details and scenarios in which streaming aggregations can be used.
Last9 Documentation