What are OpenTelemetry Metrics? A Comprehensive Guide

OpenTelemetry, often abbreviated as OTel, is an open-source observability framework that provides a standardized way to collect and export telemetry data. It's a CNCF (Cloud Native Computing Foundation) project that aims to unify observability standards and practices across the industry.

OpenTelemetry metrics are a crucial component of this framework, providing a way to measure and track the performance and behavior of applications over time. OTel metrics offer a level of flexibility and power that makes them suitable for a wide range of use cases.

In this blog, we’ll explore OpenTelemetry (OTel) metrics, including their various types and best practices for implementation.

The OpenTelemetry Metrics API and SDK

At the core of OpenTelemetry metrics are two main components: the API and the SDK.

OpenTelemetry Metrics API

The OpenTelemetry API provides a set of language-specific interfaces for instrumenting code. It's designed to be lightweight and have minimal dependencies, making it easy to adopt without significantly impacting application performance.

Here's a simple example of using the OpenTelemetry Metrics API in Python:

from opentelemetry import metrics

# Get a meter
meter = metrics.get_meter("app_meter")

# Create a counter
request_counter = meter.create_counter("requests", description="Number of requests")

# Increment the counter
request_counter.add(1)

OpenTelemetry vs. Traditional APM Tools: A Comparative Analysis | Last9

This article compares OpenTelemetry and traditional APM tools with their strengths, weaknesses, and ideal use cases to help you choose the right solution for your application performance monitoring needs.

Book a Demo

OpenTelemetry Metrics SDK

The SDK is the implementation of the API. It's responsible for processing the telemetry data, applying aggregations, and exporting the data to various backends. The SDK is where the configuration for exporters, resource detection, and sampling occurs.

Here's how a basic SDK configuration might be set up in Java:

import io.opentelemetry.api.OpenTelemetry;
import io.opentelemetry.exporter.otlp.metrics.OtlpGrpcMetricExporter;
import io.opentelemetry.sdk.OpenTelemetrySdk;
import io.opentelemetry.sdk.metrics.SdkMeterProvider;
import io.opentelemetry.sdk.metrics.export.PeriodicMetricReader;

OtlpGrpcMetricExporter exporter = OtlpGrpcMetricExporter.builder().build();
PeriodicMetricReader reader = PeriodicMetricReader.builder(exporter).build();

SdkMeterProvider meterProvider = SdkMeterProvider.builder()
    .registerMetricReader(reader)
    .build();

OpenTelemetry openTelemetry = OpenTelemetrySdk.builder()
    .setMeterProvider(meterProvider)
    .buildAndRegisterGlobal();

Types of Metric Instruments

OpenTelemetry provides several types of metric instruments, each suited for different use cases. Choosing the right instrument is crucial for accurate and meaningful metrics.

Counter

A Counter is used for values that only increase. These are often used for counting events, like the number of HTTP requests or errors.

request_counter = meter.create_counter("requests", description="Number of requests")
request_counter.add(1)

UpDownCounter

An UpDownCounter can both increase and decrease. These are useful for tracking things like the number of active connections or items in a queue.

active_connections = meter.create_up_down_counter("active_connections", description="Number of active connections")
active_connections.add(1)  # Connection opened
active_connections.add(-1)  # Connection closed

Histogram

Histograms are ideal for measuring the distribution of values like request durations or response sizes.

response_size_histogram = meter.create_histogram("response_size", description="Size of HTTP responses")
response_size_histogram.record(256)  # Record a response size of 256 bytes

Asynchronous Instruments

OpenTelemetry also supports asynchronous instruments, which are great for capturing system metrics that need to be observed periodically, like CPU usage or memory consumption.

def get_cpu_usage():
    # Code to get CPU usage
    return cpu_usage

meter.create_observable_gauge(
    "cpu_usage",
    callbacks=[get_cpu_usage],
    description="Current CPU usage"
)

Metric Aggregation and Collection

One of the powerful features of OpenTelemetry metrics is its flexible aggregation system. The SDK applies default aggregations based on the instrument type, but custom aggregations can also be specified.

For example, it's possible to collect not just the total count of requests, but also the rate of requests per minute. The OpenTelemetry SDK can handle this kind of aggregation, which is useful for real-time monitoring dashboards.

Exporting Metrics

Once metrics are collected, they need to be sent somewhere for storage and analysis. OpenTelemetry supports various export formats and protocols, including:

• OTLP (OpenTelemetry Protocol)
• Prometheus
• StatsD

The OpenTelemetry Collector is often used as an intermediary. It can receive data in the OTLP format and then export it to various backends. Here's a simple configuration for the collector to export to Prometheus:

receivers:
  otlp:
    protocols:
      grpc:

exporters:
  prometheus:
    endpoint: "0.0.0.0:8889"

service:
  pipelines:
    metrics:
      receivers: [otlp]
      exporters: [prometheus]

Integration with Observability Tools

One of the strengths of OpenTelemetry is its wide ecosystem support. It can be used with various observability tools, including:

• Prometheus
• Last9
• Grafana
• Jaeger
• Zipkin

The ability to send metrics to multiple backends simultaneously is particularly useful in heterogeneous environments.

Best Practices and Common Use Cases

Here are some best practices for working with OpenTelemetry metrics:

• Use semantic conventions: OpenTelemetry defines semantic conventions for common metrics. Adhering to these makes metrics more interoperable and easier to understand.
• Be mindful of cardinality: High-cardinality metrics can cause performance issues. Use labels judiciously.
• Implement health checks: Use metrics to implement health checks for services. This can be invaluable for Kubernetes deployments.
• Monitor dependencies: Use OpenTelemetry to monitor dependencies, like databases or external APIs. This can help identify bottlenecks in distributed systems.
• Combine metrics with traces and logs: For a complete observability solution, use OpenTelemetry for all three pillars: metrics, traces, and logs.

Conclusion

OpenTelemetry metrics provide a powerful and flexible way to implement observability in modern software systems. The standardized approach, coupled with the flexibility and power of the framework, makes it an excellent choice for both small applications and large, distributed systems.

As the project continues to evolve, it's likely to play an increasingly important role in shaping the future of observability. For developers and operations teams looking to implement or improve their metrics collection and analysis, OpenTelemetry offers a robust and future-proof solution.

FAQs

What is the difference between OpenTelemetry metrics and OpenMetrics?

OpenTelemetry metrics and OpenMetrics are both open-source projects related to metrics, but they serve different purposes:

• OpenTelemetry metrics is part of the larger OpenTelemetry project, which provides a complete observability framework including traces, logs, and metrics. It offers a standardized way to collect and export telemetry data across multiple languages and platforms.
• OpenMetrics is a project focused on evolving the Prometheus exposition format into a standard. It's primarily about the format of metric data, rather than the collection and export process.

OpenTelemetry can export metrics in the OpenMetrics format, allowing for interoperability between the two standards.

What are telemetry metrics?

Telemetry metrics are measurements collected from software systems to monitor their performance, behavior, and health. These metrics can include things like:

• Response times
• Error rates
• Resource utilization (CPU, memory, disk)
• Business-specific measurements (e.g., number of orders processed)

Telemetry metrics are crucial for understanding system behavior, identifying issues, and making data-driven decisions about system performance and capacity.

What is OpenTelemetry data?

OpenTelemetry data refers to the telemetry data collected and processed using the OpenTelemetry framework. This includes:

• Metrics: Numerical measurements of system behavior and performance.
• Traces: Distributed traces that show the path of requests through a distributed system.
• Logs: Timestamped records of discrete events that happened in the system.

OpenTelemetry provides a standardized way to collect, process, and export this data, making it easier to implement comprehensive observability across different languages and platforms.

What is the use of OpenTelemetry?

OpenTelemetry is used to implement observability in software systems. Its main uses include:

• Performance Monitoring: Tracking system performance metrics to identify bottlenecks and optimize resources.
• Error Detection: Quickly identifying and diagnosing errors in distributed systems.
• Distributed Tracing: Following requests as they travel through microservices architectures.
• Capacity Planning: Using historical data to predict future resource needs.
• Business Intelligence: Tracking metrics that are important for business decisions.
• Debugging: Providing detailed information to help developers understand and fix issues.

What are the types of open telemetry metrics?

OpenTelemetry supports several types of metrics:

• Counter: A cumulative metric that only increases in value (e.g., number of requests).
• UpDownCounter: A metric that can both increase and decrease (e.g., number of active connections).
• Histogram: A metric that samples observations and counts them in configurable buckets (e.g., request durations).
• Gauge: A metric that represents a single numerical value that can arbitrarily go up and down (e.g., CPU usage).

Additionally, OpenTelemetry supports synchronous and asynchronous versions of these metric types.

What are some of the benefits of OpenTelemetry?

Some key benefits of OpenTelemetry include:

• Standardization: Provides a single, vendor-neutral standard for telemetry data.
• Language Support: Offers consistent APIs across multiple programming languages.
• Flexibility: Can export data to multiple backends and supports various data formats.
• Open Source: Backed by a large community and major industry players.
• Reduced Vendor Lock-in: This makes it easier to switch between different observability tools.
• Comprehensive: Covers metrics, traces, and logs in a single framework.
• Performance: Designed to have minimal performance impact on the systems it monitors.

What is the difference between a metric and an event?

While both metrics and events are types of telemetry data, they serve different purposes:

• Metrics:
  • Represent numerical measurements of system behavior over time.
  • Are typically aggregated (e.g., average, sum, count) over a time period.
  • Are used to understand trends and patterns in system performance.
  • Examples: CPU usage, request latency, error rate.
• Events:
  • Represent discrete occurrences at a specific point in time.
  • Contains detailed information about what happened at that moment.
  • Are used to understand specific actions or state changes in a system.
  • Examples: User login, order placed, error occurred.

In OpenTelemetry, metrics are handled by the metrics API, while events are typically captured as part of logging or tracing.