# How to Use Jaeger with OpenTelemetry > This guide shows you how to easily use Jaeger with OpenTelemetry for improved tracing and application monitoring. Source: https://last9.io/blog/how-to-use-jaeger-with-opentelemetry/ [Distributed tracing](https://last9.io/blog/challenges-of-distributed-tracing/), a key aspect of observability, enables developers to track and analyze requests spanning multiple services. [OpenTelemetry](https://last9.io/guides/opentelemetry/what-is-opentelemetry/) and Jaeger are comprehensive solutions for distributed tracing. But how do they complement each other? This article is a deep dive into how you can use Jaeger with OTel. ## **What is Jaeger?** [Jaeger](https://www.jaegertracing.io/) is a [Cloud Native Computing Foundation (CNCF)](https://www.cncf.io/) open-source, end-to-end distributed tracing system built for monitoring and troubleshooting microservices-based architectures. Usually incorporated as an OpenTelemetry backend, it is used to track and visualize user request behavior across the distributed components of complex systems. ### **Jaeger’s functions** To complement its distributed tracing functions, Jaeger also has the following. #### 1. **Rich Visualization** Jaeger provides a user-friendly web interface for analyzing distributed traces. It offers features like charts, waterfall-style dependency visualization graphs, flame graphs, and detailed service-level breakdowns. #### 2\. **Root Cause Analysis** Jaeger offers in-depth visualization of entire event chains, provides tracing and logging of request paths, and details when and how a series of related software issues began, as well as which problematic microservice triggered issues in others. #### 3\. **Scalability** Jaeger scales horizontally to handle large amounts of tracing data. It uses storage backends like [Elasticsearch](https://www.elastic.co/elasticsearch), [Cassandra](https://cassandra.apache.org/_/index.html), or other compatible databases. #### 4\. **Flexibility** Jaeger supports multiple instrumentation methods for various programming languages (e.g., native [OpenTracing](https://last9.io/blog/opentelemetry-vs-opentracing/) instrumentation, [Prometheus](https://last9.io/blog/prometheus-operator-guide/), and client libraries). ## **What is OpenTelemetry?** OpenTelemetry (OTel) is an open-source [observability framework](https://last9.io/blog/modern-observability-system/) used to collect telemetry data from applications, and export to prespecified backends. Also, a CNCF project, that provides a unified set of APIs, libraries, agents, and protocol specifications. OpenTelemetry can also be used for distributed tracing, context propagation (for integrating request metadata across distributed services), and semantic conventions (for ensuring consistency in metric data representation and interpretation). Read more about OTel in our dedicated article. ## **Jaeger vs. OpenTelemetry** Both Jaeger and OpenTelemetry are complementary rather than competing tools. Nonetheless, let's compare the two.

Differentiators	Jaeger	OpenTelemetry
Purpose	A distributed tracing system.	Provide a unified set of APIs, libraries and protocols for capturing and exporting multiple telemetry data types in distributed environments.
Origin	Originally created by Uber Technologies as an open-source project. Now a Cloud Native Computing Foundation (CNCF) project.	A collaborative effort by multiple industry leaders and a Cloud Native Computing Foundation (CNCF) project. Combines the functionalities of OpenTracing and OpenCensus into a standardized observability framework.
Integration	Compatible with OpenTelemetry. Has instrumentation libraries derived from the OpenTelemetry project.	Provides instrumentation libraries, which can be used to generate Jaeger-compatible traces. Integrates well with other observability tools.
Ecosystem	Has a strong and growing ecosystem of plugins, extensions, and integrations.	Rapidly gaining industry support and has a growing ecosystem of expert contributors.

## **Why integrate Jaeger with OTel?** Consider these three advantages: - **End-to-end distributed tracing**: Integrating both tools into one observability framework provides you with a complete solution: OpenTelemetry allows you to track and capture user requests as they traverse various microservices and application components, then Jaeger provides a web UI and a storage backend for analysis and storage of tracked user requests. - **Automatic Instrumentation**: OTel’s automatic instrumentation reduces the manual effort required to add tracing to your application code. Since you do not need to manually modify the codebase, the risk of inaccurate or missing traces is reduced; meaning that more (accurate) traces can be collected and exported to Jaeger for visualization. - **Performance Optimization**: The end goal of observability is to ensure optimal application function. With inputs from OpenTelemetry and Jaeger, you can use insights from traces and trace metadata collected and analyzed to pinpoint performance bottlenecks in specific microservices, optimize resource usage, and improve overall system efficiency. > Both Jaeger and OTel have a set of exporters that ease the process of using both for end-to-end distributed tracing of your application. These exporters function as alternatives to one another, and the decision is yours to make on which one to use to sync both systems. However, it is best practice to choose the exporter that best fits your requirements in terms of compatibility, performance, and ease of use. Check out our [Developer's Guide to Installing OpenTelemetry Collector](https://last9.io/blog/developers-guide-to-installing-opentelemetry-collector/) for step-by-step instructions on setting up your observability stack. OpenTelemetry provides three specific Jaeger-compatible exporters. Let us examine each. ### **Jaeger Trace Exporter** The Jaeger Trace Exporter is a built-in OpenTelemetry exporter that allows you to export traces to a Jaeger backend server. It supports Thrift over UDP only, as it uses protocols such as gRPC or Thrift HTTP. ### **OTLP Exporter** The OpenTelemetry Protocol (OTLP) Exporter is generic in function. It exports telemetry data to various backends and endpoints—Jaeger inclusive. You can configure it to send traces to a Jaeger endpoint using the Jaeger reception format. ### **Batch Span Processor** OpenTelemetry also provides a Batch Span Processor that can be used to buffer and batch multiple spans before exporting them to a Jaeger backend. This processor can be configured to send the accumulated spans to Jaeger at regular intervals or when specific conditions are met. > Now, what are the set-in-stone steps to use Jaeger and OTel in your application?Below is a 10-step sequence explaining how. ## **Using Open Telemetry and Jaeger in your application** Follow these steps. ### 1\. **Determine your Use Case** Identify the specific use case or problem you want to address with distributed tracing. You may want to monitor the performance of your application, troubleshoot issues, or optimize resource usage. ### 2\. **Understand OpenTelemetry Concepts** Familiarize yourself with the core concepts of OpenTelemetry, such as [spans](https://last9.io/blog/identify-root-spans-in-otel-collector/), [traces](https://last9.io/blog/convert-opentelemetry-traces-to-metrics-using-spanconnector/), and attributes. Understand how instrumentation works and how to propagate context between services. ### 3\. **Choose a Programming Language** OpenTelemetry supports multiple languages, including Java, Python, Go, JavaScript, and more. Choose the appropriate OpenTelemetry SDK and instrument your code accordingly. ### 4\. **Install Dependencies** Install the necessary OpenTelemetry and Jaeger dependencies for your chosen programming language. These dependencies include the OpenTelemetry SDK, Jaeger receiver, and any required language-specific libraries or plugins. ### 5\. **Configure the OpenTelemetry Collector and Jaeger Receiver** These configurations enable you to send traces to the Jaeger backend. Provide the necessary configuration options, such as the Jaeger endpoint, authentication credentials, and sampling strategies. This configuration can usually be done through environment variables or a configuration file. ### 6\. **Instrumentation** Identify the parts of your codebase that you want to instrument with distributed tracing. Use the OpenTelemetry APIs and libraries in your code to create spans and add attributes to capture relevant information. Instrumentation can be done manually by adding code snippets or by utilizing auto-instrumentation features provided by OpenTelemetry, if available for your programming language. ### 7\. **Export Traces to Jaeger** Configure OpenTelemetry to export captured traces to Jaeger. This involves setting the appropriate exporter options and endpoints in your OpenTelemetry configuration or code. Ensure that the Jaeger endpoint is correctly configured to receive and store exported trace data. ### 8\. **Verify Integration** Test and verify that your application is correctly capturing and sending traces to Jaeger. Generate some test requests that exercise the areas of your codebase you have instrumented. Use the Jaeger UI to visualize and validate that the traces are successfully reaching the backend. ### 9\. **Visualize and Analyze Traces** Once your application is instrumented and traces are exported to Jaeger, you can start visualizing and analyzing the distributed traces. Use the Jaeger user interface (UI) to filter traces, view latency distributions, identify bottlenecks, and understand microservices’ behavior. ### 10\. **Iterate and Optimize** Continuously monitor and analyze your distributed traces. Use insights gained from trace visualization to optimize your services for performance and efficiency. Fine-tune your sampling strategies, adjust instrumentation, and make improvements based on the data you collect. Read our [Guide to Optimizing Prometheus Remote Write Performance](https://last9.io/blog/optimizing-prometheus-remote-write-performance-guide/) for tips on improving efficiency and performance in your monitoring setup. Now, let us exemplify the steps above. ## **How to use OTel with Jaeger** ### **Step 1: Choose a Programming Language** OpenTelemetry supports various languages like Java, Python, Go, and .NET. Choose the language your application is built in and ensure that both OpenTelemetry and Jaeger support it. For this guide, we’ll focus on **Python**. ### **Step 2: Install Required Tools & Libraries** Use Docker to run a Jaeger instance. Make sure to avoid container name conflicts by renaming it if necessary: ``` docker run -d --name jaeger_instance \ -e COLLECTOR_ZIPKIN_HTTP_PORT=9411 \ -p 5775:5775/udp \ -p 6831:6831/udp \ -p 6832:6832/udp \ -p 5778:5778 \ -p 16686:16686 \ -p 14268:14268 \ -p 14250:14250 \ -p 9411:9411 \ jaegertracing/all-in-one:latest ``` **Install OpenTelemetry and Jaeger Libraries** For Python, create a `Dockerfile` to install the required libraries: ``` FROM python:3.8-slim # Install OpenTelemetry and Jaeger libraries RUN pip install --no-cache-dir \ opentelemetry-api \ opentelemetry-sdk \ opentelemetry-instrumentation-requests \ opentelemetry-exporter-jaeger \ flask requests # Set up the working directory WORKDIR /app COPY . /app # Run the Python application CMD ["python", "app.py"] ``` #### **Step 3: Configure the OTel Exporter** Set up the Jaeger exporter to send traces from your application to Jaeger. Here’s an example Python snippet: ``` import os from opentelemetry import trace from opentelemetry.exporter.jaeger.thrift import JaegerExporter from opentelemetry.sdk.resources import Resource from opentelemetry.sdk.trace import TracerProvider from opentelemetry.sdk.trace.export import BatchSpanProcessor # Get Jaeger host and port from environment variables jaeger_host = os.getenv("JAEGER_HOST", "localhost") jaeger_port = int(os.getenv("JAEGER_PORT", 6831)) # Set up the TracerProvider with resource attributes trace.set_tracer_provider( TracerProvider(resource=Resource.create({"service.name": "your-application-name"})) ) # Configure Jaeger Exporter jaeger_exporter = JaegerExporter( agent_host_name=jaeger_host, agent_port=jaeger_port ) # Add the exporter to the trace provider trace.get_tracer_provider().add_span_processor( BatchSpanProcessor(jaeger_exporter) ) ``` This configuration will export traces to the Jaeger agent running on the specified `jaeger_host` and `jaeger_port`. #### **Step 4: Enable Auto-Instrumentation** Enable OpenTelemetry’s auto-instrumentation to trace your application code automatically. For a Flask web application, you can do the following: ``` from opentelemetry.instrumentation.flask import FlaskInstrumentor from flask import Flask # Create a Flask app app = Flask(__name__) # Instrument the Flask application FlaskInstrumentor().instrument_app(app) @app.route("/") def hello_world(): return "Hello, OpenTelemetry!" ``` This automatically instruments all Flask endpoints for trace generation. #### **Step 5: Verify Tracing** 1. **Run Your Application** Build and run your application’s Docker container, linking it to the Jaeger container: ``` docker build -t your-app . docker run -d --name your-app --link jaeger_instance -p 8000:8000 your-app ``` 2. **Generate Traffic** Send requests to your application to generate traces: ``` curl http://localhost:8000/ ``` 3. **Check Jaeger UI** Open the Jaeger UI at `http://localhost:16686` to view and analyze the traces. ### **Step 6: Explore and Analyze Traces** Once traces are successfully captured and sent to Jaeger, use Jaeger's rich visualization UI to analyze them and gain insights into application performance. The Jaeger backend should be available at \`http://your-jaeger-host:your-jaeger-port\`. ### **Step 7: Fine-Tune Configuration** Adjust auto-instrumentation configuration options to meet your specific requirements. This may include tweaking sampling strategies, adding custom attributes to traces, or excluding certain parts of your codebase from instrumentation. ### **Step 8: Monitor and Optimize** Continuously monitor your application's traces to identify bottlenecks or areas for optimization. ## **OpenTelemetry to Jaeger Transformation** Jaeger accepts tracing information formats such as OpenTelemetry Protocol (OTLP), Thrift Batch, and Protobuf Batch. A range of attributes, including trace, parent and span IDs, start and end times, attributes, events, links, and status can be mapped from OpenTelemetry to Jaeger formats to enable Jaeger to understand and process the data for visualization. Convert OpenTelemetry spans into Jaeger formats by following the steps outlined below. ### **Step 1: Ingesting Spans from Jaeger Using OpenTelemetry** To ingest spans from Jaeger, set up the OpenTelemetry Collector with a Jaeger receiver. The Jaeger receiver acts as a bridge between Jaeger and OpenTelemetry. Here's an example configuration file for the OpenTelemetry Collector with Jaeger receiver. ```yaml receivers: jaeger: protocols: grpc: thrift_http: exporters: logging: processors: batch: service: pipelines: traces: receivers: [jaeger] processors: [batch] exporters: [logging] ``` ### **Step 2: OpenTelemetry Collector** Set up and run the OpenTelemetry Collector using the configuration file from the previous step. Here's an example of how to run the OpenTelemetry Collector. ``` bash docker run --rm -v /path/to/config.yaml:/etc/otel-collector-config.yaml otel/opentelemetry-collector-contrib:latest --config=/etc/otel-collector-config.yaml ``` Make sure to replace \`/path/to/config.yaml\` with the path to your actual configuration file. ### **Step 3: Jaeger Receiver** With the OpenTelemetry Collector running, you can now configure Jaeger Agent and HotRod to send spans to it. Here's an example of configuring the Jaeger Agent. ``` bash docker run --rm -p 6831:6831/udp -p 6832:6832/udp jaegertracing/jaeger-agent:latest --reporter.grpc.host-port=:14250 ``` **_Replace \`\\` with the hostname or IP address of the machine running the OpenTelemetry Collector._** ### **Step 4: Jaeger Agent and HotRod** To simulate sending spans to Jaeger, you can use the Jaeger Agent and HotRod. #### Start the Jaeger Agent Container: ```bash docker run --name jaeger-agent \ -p 5778:5778/udp \ -p 6831:6831/udp \ -p 6832:6832/udp \ -p 5775:5775/udp \ jaegertracing/jaeger-agent:latest ``` Start the HotRod: ```bash docker run --name hotrod \ -e JAEGER_AGENT_HOST=your_jaeger_agent_host \ -e JAEGER_AGENT_PORT=6831 \ -p 8080-8083:8080-8083 \ jaegertracing/example-hotrod:latest all ``` Replace \`your_jaeger_agent_host\` with the address of the Jaeger Agent container. ### **Step 5: Examples** Now that everything is set up, you can instrument your applications with OpenTelemetry and start generating and exporting traces. Here's an example of using OpenTelemetry in Python: ```python from opentelemetry import trace from opentelemetry.trace import SpanKind # Create a tracer tracer = trace.get_tracer(__name__) # Create a span with tracer.start_as_current_span("example_span", kind=SpanKind.SERVER) as span: # Add custom attributes or events if needed span.set_attribute("example_attribute", "value") span.add_event("example_event") # Perform your application logic # End the span span.end() ``` Once the spans are generated, the OpenTelemetry Collector configured with the Jaeger receiver will pick them up and forward them to downstream exporters. You should now see spans flowing from the Hot Rod to the Jaeger Agent, and from there to the OpenTelemetry Collector. Afterward, the OpenTelemetry Collector will transform and send the spans to Jaeger. You can then use Jaeger's user interface accessible at [http://localhost:16686](https://www.jaegertracing.io/docs/1.76/getting-started/) to visualize and analyze the collected spans. ## **Conclusion** Integrating Jaeger with OpenTelemetry provides you with a comprehensive distributed tracing solution that allows you to capture, export, and analyze traces, enabling you to gain valuable insights into your microservices ecosystem. Understanding the basics of OpenTelemetry and Jaeger, and following the steps provided for installing, instrumenting, exporting, and visualizing traces will help you leverage the benefits of and effectively utilize the powerful combination to ensure optimal application performance and improve user experience. If you're passionate about reliability, observability, or monitoring, be sure to join our [SRE Discord community](https://discord.com/invite/W8gMppQC4b) to connect and share ideas!