Probo Cuts Monitoring Costs by 90% with Last9

With a growing user base, Probo’s monitoring stack was getting expensive for the engineering team to maintain. While traffic on a normal day would hover around 200K requests per minute, the platform would see jumps of >2.5x in short durations and reach 2.5M requests per minute during peak moments of an IPL cricket match. While New Relic’s volume-and-user-based pricing is predictable, it can go out of hand very quickly. And, AWS Cloudwatch’s per-query pricing would make them optimize their dashboards when they should ideally be focusing on their system health and scaling components like RDS, Elasticache, and EKS.

As they looked at options to optimize their costs, they were also keen to move to a platform built on Open Standards that allows them to have a single pane of observability in the future. As a small SRE team managing the infrastructure, making developers self-sufficient was another goal the Probo team was trying to achieve. This means the monitoring tool should be embraced by everyone in the team instead of being controlled by only a few.

Embracing Open Standards from day one and building Last9 Levitate to be compatible with Prometheus and OpenTelemetry allows us to not only be cost-effective but also support a large ecosystem of integrations and tools for our customers. Getting started with Levitate takes only 5 minutes with the OpenTelemetry integrations support.

Seeing engineering teams optimize for cost instead of granular monitoring pains us at Last9. This is why we have a simple single-metric billing model based on the number of events ingested. That’s it. No per-user charge. No per-query charge. We want you to be able to freely read the data that you’ve written and use it for alerting

Our existing customer base, including Disney+ Hotstar and Clevertap, which uses Levitate to monitor large-scale live-streaming events like the Indian Premier League and Cricket World Cup tournaments, also faces the same challenges of sudden spikes in traffic as seen by Probo’s platform. This gave their engineering team the confidence to go ahead with Levitate as their monitoring and alerting tool.

Their AWS services send data to Levitate via CloudWatch Metric Stream, while vmagent scrapes the self-hosted services and remote writes the metrics data to Levitate. A Prometheus Operator spins up and manages service discovery and ingestion from Kubernetes.

Each team uses Declarative Alerting via IaC to automate alerting and keep it updated via service discovery — a pull request is raised and changes are applied via GitHub Actions. As part of the automation YAML, the teams are also able to declare team-specific Slack and PagerDuty channels. One of the principles listed on their engineering blog is “high agency, always” and this allows them to democratize alerting across the engineering team and not be limited to the platform team.

With all relevant data coming into Last9 Levitate, the Probo engineering team now has a bird’s eye view of their system’s main components in a master dashboard, allowing them to get to the root cause faster. Maintaining a large set of alerting rules, especially in today’s ephemeral cloud environment, is improved by automating the rules to be created and deleted as the service discovery is updated.

Having replaced Probo’s current stack for infrastructure monitoring, Levitate’s compatibility with OpenTelemetry now unlocks application performance monitoring as well as the next phase of a single pane of observability.