Getting Started with OpenTelemetry Logging: A Practical Guide

Logging is the unsung hero of observability. While metrics tell you what's happening and traces reveal the "how," logs provide the why. OpenTelemetry, the open-source powerhouse for observability, offers a unified approach to handle logs, metrics, and traces.

But how exactly can you leverage OpenTelemetry Logging to simplify and enhance your observability setup?

This guide will walk you through it—from concepts to implementation.

OpenTelemetry vs. Prometheus | Last9

OpenTelemetry vs. Prometheus - Difference in architecture, and metrics

Last9

What is OpenTelemetry Logging?

At its core, OpenTelemetry is an open-source framework that helps you generate, collect, and manage telemetry data (logs, metrics, traces). It standardizes observability for cloud-native applications, reducing vendor lock-in and implementation complexity.

When it comes to logging, OpenTelemetry simplifies the process of creating structured, consistent logs across your systems. Instead of siloing logging, OpenTelemetry aligns it with the rest of your telemetry data, allowing for a clearer view of what’s happening within your systems.sq

Key benefits include:

• Unified approach to observability (metrics, logs, and traces)
• Reduced complexity by standardizing log formats
• Interoperability with popular tools and logging systems (Fluentd, Elasticsearch, etc.)
• Lower operational overhead for distributed systems

The OpenTelemetry log data model ensures that logs maintain a standardized structure. Logs are enriched with metadata like instrumentationscope, timestamp, and trace context, making them far more useful for modern applications.

Getting Started with Host Metrics Using OpenTelemetry | Last9

Learn to monitor host metrics with OpenTelemetry. Discover setup tips, common pitfalls, and best practices for effective observability.

Last9Last9

How OpenTelemetry Approaches Logs

Logs in OpenTelemetry are defined and structured differently compared to traditional logging systems. The approach revolves around semantic conventions and consistent attributes.

Let’s break this down:

1. Semantic Conventions: OpenTelemetry uses a set of conventions to describe log attributes consistently. For example:
   • service.name: Name of the service emitting the logs
   • host.name: The machine where the service is running
   • log.severity: The log level (INFO, WARN, ERROR, etc.)
   • log.message: The actual log content
   • observedtimestamp: When the log was observed
2. Log Signal: Logs are treated as a telemetry signal, alongside metrics and traces. This ensures that logging data integrates seamlessly with other observability data streams.
3. Contextual Information: OpenTelemetry enables logs to carry additional context, such as trace IDs and span IDs, along with contexts. This makes correlating logs with traces much easier for debugging.
4. File-based Logs and Receivers: OpenTelemetry supports filelog receivers, which can parse log files and forward them to the OpenTelemetry Collector for processing. Logs stdout can be collected and processed efficiently.

How to Use Jaeger with OpenTelemetry | Last9

This guide shows you how to easily use Jaeger with OpenTelemetry for improved tracing and application monitoring.

Last9

How to Implement OpenTelemetry Logging

Now that we understand the basics, let’s dive into implementing OpenTelemetry logging. We’ll cover setup, best practices, and a step-by-step example.

Prerequisites

Before starting, ensure you have:

• An application running in a supported language (Go, Java, Python, etc.)
• OpenTelemetry SDK for your chosen language
• A log exporter (e.g., Fluentd, OTLP, or Elasticsearch)
• Basic knowledge of yaml configurations and parsers

Step 1: Install the OpenTelemetry SDK

For this example, let’s use Python. Install the necessary OpenTelemetry libraries:

pip install opentelemetry-api
pip install opentelemetry-sdk
pip install opentelemetry-exporter-otlp-proto-http

Step 2: Configure the OpenTelemetry Logger

Set up a logger that generates OpenTelemetry-compatible logs. Here’s an example:

from opentelemetry import logs
from opentelemetry.sdk.logs import LoggingHandler
from opentelemetry.exporter.otlp.proto.http.logs_exporter import OTLPLogExporter
from opentelemetry.sdk.logs.export import SimpleLogRecordProcessor
import logging

# Initialize logger provider
logs.set_logger_provider(logs.LoggerProvider())
logger_provider = logs.get_logger_provider()

# Set up log exporter
log_exporter = OTLPLogExporter(endpoint="http://localhost:4317")
logger_provider.add_log_record_processor(SimpleLogRecordProcessor(log_exporter))

# Set up standard logging
handler = LoggingHandler(level=logging.INFO)
logging.basicConfig(level=logging.INFO, handlers=[handler])
logger = logging.getLogger(__name__)

# Generate a log message
logger.info("OpenTelemetry logging is ready to roll!")

This code does a few key things:

• Configures OpenTelemetry to process logs
• Exports logs using the OTLP (OpenTelemetry Protocol)
• Integrates with Python's standard logging module

Step 3: Add Context to Logs

Context is where OpenTelemetry shines. By attaching trace and span IDs, you can correlate logs with traces. Here’s how you do it:

from opentelemetry.trace import get_current_span

current_span = get_current_span()
trace_id = current_span.get_span_context().trace_id
span_id = current_span.get_span_context().span_id

logger.info(f"Trace ID: {trace_id}, Span ID: {span_id}")

With this, your logs now carry valuable trace information for better debugging.

Identify Root Spans in Otel Collector | Last9

How to identify root spans in OpenTelemetry Collector using filter and transform processors

Last9

Step 4: Export Logs

Logs can be exported to various backends, such as:

• Fluentd
• Elasticsearch
• OTLP-compatible tools (Jaeger, Prometheus, etc.)
• Kubernetes environments and Docker containers

Update the OTLPLogExporter endpoint to send logs where you need them.

Best Practices for OpenTelemetry Logging

Here are some tips to get the most out of OpenTelemetry logs:

1. Stick to Semantic Conventions: Use OpenTelemetry’s semantic conventions to ensure consistency across logs.
2. Include Contextual Data: Attach trace and span IDs to logs to enable correlation between logs and traces.
3. Optimize Log Levels: Use appropriate log levels (INFO, DEBUG, ERROR) to prevent log overload.
4. Export Efficiently: Use batching and proper exporters to reduce overhead and improve performance.
5. Integrate with Other Signals: Correlate logs with metrics and traces for a holistic observability experience.

Conclusion:

OpenTelemetry Logging offers a powerful, standardized approach to collecting and processing logs, making it an essential tool for modern observability. Aligning logs with metrics and traces provides deeper insights into your systems and simplifies the debugging process.

FAQs

What is OpenTelemetry logging?

OpenTelemetry logging is the process of creating, processing, and exporting structured logs that are enriched with contextual data such as trace IDs and timestamps.

Is logging considered telemetry?

Yes, logging is a form of telemetry data, alongside metrics and traces.

What is the difference between OpenTelemetry log and event?

Logs are timestamped records of events with context, while events can represent actions within a span in tracing.

What is tracing logging?

Tracing logging refers to logs enriched with trace and span information for easier correlation during debugging.

Why OpenTelemetry Logs Stand Out?

OpenTelemetry logs stand out because of their standardization, correlation capabilities, and seamless integration with metrics and traces.

Do you guys use minimal API very often?

Minimal APIs are used depending on project complexity and requirements.

Is there something like standardized OpenTelemetry log format?

Yes, the OpenTelemetry Log Data Model ensures a standardized format for logs.

How Does OpenTelemetry Support Logging?

OpenTelemetry supports logging through SDKs, semantic conventions, and tools like the OpenTelemetry Collector.

Does Azure or AWS provide a universal solution for OpenTelemetry logs/metrics/tracing?

Yes, both Azure and AWS support OpenTelemetry via managed observability solutions.

What Are the Benefits of the OpenTelemetry Log Data Model?

It provides a structured, consistent format for logs, making observability easier and vendor-neutral.

How do you integrate OpenTelemetry logging with existing logging frameworks?

OpenTelemetry supports integration with popular logging libraries through appenders and auto-instrumentation.

How do you implement OpenTelemetry logging in a microservices architecture?

You can deploy OpenTelemetry SDKs in each service, aggregate logs via the OpenTelemetry Collector, and correlate data using trace context.