Thanos vs. VictoriaMetrics

This article compares Thanos and VictoriaMetrics discussing what they are, their architectural components, and their differences.

What is Thanos?

Thanos, an open-source project, is an extension built for Prometheus that addresses some of the challenges associated with long-term storage and high availability.

Thanos provides a highly available Prometheus setup with extended storage capabilities, enabling organizations to store and query historical data efficiently. To achieve this, it's introduced additional components, such as the Thanos Sidecar and Thanos Store, which enhance the scalability and durability of Prometheus deployments. Thanos consists of several components, including Thanos Sidecar, Thanos Query, Thanos Store, and Thanos Compact, which work together to create a distributed, fault-tolerant, and scalable time series database.

By leveraging a distributed architecture and integrating with object storage systems like Amazon S3 or Google Cloud Storage, it allows for seamless horizontal scalability. Thanos enables federated queries across multiple Prometheus instances, making it ideal for handling massive volumes of time series data.

Thanos Components

• Thanos Sidecar: Acts as a sidecar component proxy for Prometheus instances, enabling long-term storage by pushing data to object storage and facilitating global query federation across multiple Prometheus servers.
• Thanos Querier: Serves as the central query engine in the Thanos architecture, allowing users to execute queries across multiple Prometheus instances, providing a unified view of time series data.
• Thanos Store: Stores time series data in object storage, such as Amazon S3 or Google Cloud Storage, and provides efficient read access for Thanos Querier. It handles data deduplication and compression, optimizing storage utilization.
• Thanos Compact: Performs compaction of time series data in object storage, reducing storage overhead and improving query performance by removing redundant data blocks and chunks.
• Thanos Ruler: Offers extended rule-based alerting capabilities for the Thanos ecosystem, allowing users to define and evaluate complex alerting rules across distributed Prometheus instances.
• Thanos Receiver: Provides an endpoint for ingesting data from remote Prometheus instances and stores it in object storage. It enables efficient and reliable ingestion of data for long-term storage and analysis.
• Thanos Compactor: Manages the compaction process by merging and downsampling data blocks, improving query efficiency, and reducing storage requirements in the long-term storage layer.
• Thanos Bucket: Allows data to be stored in a time-partitioned manner, improving query performance by organizing data into manageable buckets based on time ranges.
• Thanos Query Frontend: As the user-facing component, receiving query requests and distributing them to Thanos Querier instances. It provides a web-based interface for executing queries and visualizing the results.
• Thanos Sidecar Downsample: Offers downsampling capabilities for Thanos Sidecar, reducing the granularity of stored time series data to improve query performance for longer time ranges.

Levitate - Last9's managed time series data warehouse stores metrics for 1 year without any downsampling. Get started today.

These components collectively form the Thanos architecture, providing enhanced scalability, fault tolerance, long-term storage, and global querying capabilities to Prometheus deployments.

What is VictoriaMetrics

VictoriaMetrics, on the other hand, is a high-performance, cost-effective, and scalable time-series database that can be used as long-term remote storage for Prometheus. It boasts superior data compression and high-speed data ingestion, making it an appealing alternative for large-scale monitoring tasks. VictoriaMetrics source code can be found at: https://github.com/VictoriaMetrics/VictoriaMetrics

The GitHub repositories for these projects contain all the source code, docs, issues, pull requests, and documentation. These are great resources and tutorials to get started using Docker if you want to understand more about how these systems work, if you want to contribute to the projects, or if you need help troubleshooting an issue.

Levitate - Last9's managed time series data warehouse provides better high cardinality support than VictoriaMetrics with per metric cardinality limits. Get started today.

VictoriaMetrics components

Victoria Metrics, while maintaining a simpler architecture, also includes several core components:

• VictoriaMetrics Single-Node: This is the basic building block of VictoriaMetrics. It includes the time-series database and the HTTP server for ingesting and querying data.
• VictoriaMetrics Cluster: Designed for large-scale deployments, the cluster version of VictoriaMetrics includes additional components such as the VMSelect, VMInsert, and VMStorage nodes for query execution, data ingestion, and long-term storage, respectively.
• vmagent: Vmagent is a tiny yet powerful data scraper that can pull data from various sources and send it to VictoriaMetrics or any other remote storage which supports the InfluxDB or Prometheus remote write protocol.
• vmalert: It evaluates alerting and recording rules against VictoriaMetrics or any other compatible TSDB.
• vmctl: This CLI tool migrates data from different TSDBs to VictoriaMetrics.
• vmstorage: The Storage backend for VictoriaMetrics.
• vmui: The UI layer that comes out of the box with VictoriaMetrics.

Difference between Thanos and Victoria Metrics

Thanos and VictoriaMetrics are monitoring systems that operate at a large scale and aim to provide long-term storage solutions for time-series data, especially in environments using Prometheus in the observability landscape. However, there are several key differences between the two.

Origin & Purpose

Thanos: Born as a project to address the need for multi-cluster monitoring and long-term storage in Prometheus without resorting to a completely separate monitoring system. It extends Prometheus with a global query view, unlimited retention, and downsampling.

VictoriaMetrics: Started as an alternative long-term remote storage TSDB for Prometheus.

Architecture

Thanos: It's based on a modular system where you deploy separate components like Sidecar, Store Gateway, Querier, etc. Each component performs a specific function. The system can be a bit complex but provides flexibility.

VictoriaMetrics: It uses simpler single-node and cluster versions. VictoriaMetrics emphasizes simplicity and ease of deployment.

Storage Efficiency

Thanos: Uses the TSDB format, the same as Prometheus. While it does provide downsampling to save space, it may not be as storage-efficient as VictoriaMetrics under certain conditions.

VictoriaMetrics: Claims to have better storage efficiency due to its custom storage format and compression algorithms.

Query Language

Thanos: Uses PromQL, the same query language as Prometheus and Prometheus API.

VictoriaMetrics: Introduced MetricsQL, inspired by PromQL, and is backward compatible with PromQL.

High Availability and Reliability

Thanos: Achieves high availability by relying on object storage (like AWS S3, GCS, Azure, etc.) for long-term data and has replication features for its other components. This design choice brings some backup advantages.

VictoriaMetrics: Offers a cluster version for high availability and redundancy as it stores data on local storage.

Downsampling and Retention

Thanos: Has explicit downsampling features and allows you to define different retention periods for raw, 5-minute, and 1-hour downsampled data.

VictoriaMetrics also provides downsampling and customizable retention policies, but the details might vary.

Integration

Thanos: Built to integrate closely with Prometheus and the Prometheus ecosystem and can integrate with Grafana for visualization dashboards.

VictoriaMetrics: While it offers compatibility with the Prometheus ecosystem, VictoriaMetrics can also be used as a drop-in replacement for TSDB. It integrates with Grafana and can also be used with Thanos or Cortex.

Performance

Performance can vary based on the use case, hardware, data volume, expected high cardinality, latency, and query complexity. Some users find VictoriaMetrics faster and more resource-efficient, with better CPU usage and disk space, while others may prefer Thanos' modular architecture for specific setups. Both TSDBs promise to handle large workloads with ease. VictoriaMetrics also has a benchmark published.

Community & Development

Thanos: Has a robust and active community with contributions from many organizations, including a Slack community channel. It's also a CNCF incubating project.

VictoriaMetrics: It also has an active community on Slack and Telegram, and the project emphasizes performance and simplicity in its design principles.

Integration with Kubernetes

Both Thanos and VictoriaMetrics are designed to work seamlessly with Kubernetes, given that Kubernetes is one of the primary environments where large-scale monitoring solutions are needed. Here's a breakdown of how both integrate with Kubernetes:

Thanos and Kubernetes

• Deployment as Stateful/Stateless Sets: depending on their nature, Thanos components can be deployed as StatefulSets (for components needing persistent storage) or Deployments (for stateless components).
• Thanos Sidecar: For each Prometheus instance running in a Kubernetes cluster, you'd typically deploy a Thanos Sidecar alongside it (in the same pod). This sidecar is responsible for uploading Prometheus data to object storage and is a data source for the Thanos Querier.
• Service Discovery: Thanos components can use Kubernetes service discovery to find each other. For instance, the Thanos Querier can discover all the Sidecars and Thanos Stores.
• Configuration with ConfigMaps: Config for Thanos components can be stored in Kubernetes ConfigMaps and mounted into the pods as needed.
• Storage: Thanos uses object storage (like Amazon S3, Google Cloud Storage, etc.) for long-term storage. Persistent volumes in Kubernetes are typically used for short-term storage.
• Helm Charts: There are Helm charts available for Thanos, which make deploying and managing its components on Kubernetes more straightforward.

VictoriaMetrics and Kubernetes

• Deployment as StatefulSet: VictoriaMetrics, especially in its single-node version, can be deployed as a StatefulSet in Kubernetes when persistent storage is needed.
• vmagent: VictoriaMetrics provides a component called vmagent that is designed to be deployed alongside Prometheus or instead of Prometheus. It fetches data from service endpoints and sends it to VictoriaMetrics.
• Service Discovery: Like Prometheus, vmagent you can use Kubernetes service discovery to find services to scrape metrics from.
• Configuration with ConfigMaps: Configurations, like scraping configs, can be stored in Kubernetes ConfigMaps.
• Storage: VictoriaMetrics stores data on disk and can leverage Persistent Volumes in Kubernetes. You can also use EBS volumes in the case of AWS. VictoriaMetrics can distribute data across multiple nodes/pods for its cluster version.
• Helm Charts: There are Helm charts available for VictoriaMetrics, simplifying its deployment on Kubernetes.
• Vertical Pod Autoscaling: Given its efficient resource usage, VictoriaMetrics often recommends using vertical pod autoscaling in Kubernetes.

In essence, both Thanos and VictoriaMetrics are built with cloud-native principles and integrate well with Kubernetes. Choosing one often depends on specific features, operational preferences, or existing infrastructure.

Summary

With time series databases such as Thanos, VictoriaMetrics, Cortex, m3DB, and TimescaleDB, it is crucial to consider their differences to choose one. This article focuses on Thanos vs VictoriaMetrics comparison.

Check out our detailed article on comparing different time series databases.

Managed time series databases, such as Levitate, are built for high cardinality, long-term retention, and scale. Levitate provides an excellent alternative to both Thanos and VictoriaMetrics as it takes away the pain of managing the monitoring solution by yourself while being cost-effective and providing high performance.