OpenLLMetry

Monitor your LLM applications with OpenLLMetry traces exported to Last9. Track token usage, latencies, error rates, and model performance across OpenAI, Anthropic, LangChain, LlamaIndex, and more.

What is OpenLLMetry?

OpenLLMetry is an open-source project by Traceloop that provides OpenTelemetry-based instrumentation for LLM applications. It automatically captures traces from popular AI frameworks and model providers, giving you visibility into every LLM call your application makes.

Key capabilities include:

Automatic instrumentation - Zero-code tracing for OpenAI, Anthropic, Cohere, Bedrock, and 20+ providers
Framework support - Native integration with LangChain, LlamaIndex, Haystack, and CrewAI
Rich telemetry - Captures prompts, completions, token counts, latencies, and costs
OpenTelemetry native - Exports standard OTLP data to any compatible backend

Prerequisites

Before setting up the integration:

Last9 Account - Sign up at app.last9.io
LLM Application - An existing Python or JavaScript/TypeScript application using LLM providers or frameworks

Integration Setup

OpenLLMetry provides SDKs for Python and JavaScript/TypeScript. Choose your language below to get started.

Python
JavaScript / TypeScript

Install the OpenLLMetry SDK

Install the Traceloop SDK along with the instrumentations you need:

pip install traceloop-sdk

For specific frameworks, install additional packages:

# For OpenAI
pip install opentelemetry-instrumentation-openai

# For LangChain
pip install opentelemetry-instrumentation-langchain

# For LlamaIndex
pip install opentelemetry-instrumentation-llamaindex

# For Anthropic
pip install opentelemetry-instrumentation-anthropic

# For AWS Bedrock
pip install opentelemetry-instrumentation-bedrock

Get Your Last9 OTel Credentials

Navigate to Integrations → OpenTelemetry in your Last9 dashboard. Copy the OTel Endpoint and Auth Header values.
Initialize OpenLLMetry with Your Last9 Credentials

Add the Traceloop SDK initialization at the start of your application, passing the endpoint and Auth Header directly:
```
from traceloop.sdk import Traceloop

Traceloop.init(
    app_name="my-llm-app",
    api_endpoint="https://otlp.last9.io",
    headers={"Authorization": "<your-auth-header>"},
)
```
Replace <your-auth-header> with the full Auth Header value from Step 2, including the Basic prefix. Pass it as a headers dict rather than the TRACELOOP_HEADERS environment variable — OpenTelemetry’s env-header parser rejects the space in Basic <token> and silently drops the header.

OpenLLMetry then instruments supported libraries automatically.

Verify the Integration

Make an LLM call in your application:

from openai import OpenAI

client = OpenAI()
response = client.chat.completions.create(
    model="gpt-4",
    messages=[{"role": "user", "content": "Hello, world!"}]
)
print(response.choices[0].message.content)

Check the Last9 Traces Explorer to see your LLM traces.

Install the OpenLLMetry SDK

Install the Traceloop SDK:
```
npm install @traceloop/node-server-sdk
```
Get Your Last9 OTel Credentials

Navigate to Integrations → OpenTelemetry in your Last9 dashboard. Copy the OTel Endpoint and Auth Header values.
Initialize OpenLLMetry with Your Last9 Credentials

Add the Traceloop SDK initialization at the start of your application (before importing other modules), passing the endpoint and Auth Header directly:
```
import * as traceloop from "@traceloop/node-server-sdk";

traceloop.initialize({
  appName: "my-llm-app",
  baseUrl: "https://otlp.last9.io",
  headers: { Authorization: "<your-auth-header>" },
});
```
For CommonJS:
```
const traceloop = require("@traceloop/node-server-sdk");

traceloop.initialize({
  appName: "my-llm-app",
  baseUrl: "https://otlp.last9.io",
  headers: { Authorization: "<your-auth-header>" },
});
```
Replace <your-auth-header> with the full Auth Header value from Step 2, including the Basic prefix. Pass it as a headers object rather than the TRACELOOP_HEADERS environment variable — the SDK’s env-header parser splits on every =, which truncates base64-padded tokens.

Verify the Integration

Make an LLM call in your application:

import OpenAI from "openai";

const openai = new OpenAI();

async function main() {
  const response = await openai.chat.completions.create({
    model: "gpt-4",
    messages: [{ role: "user", content: "Hello, world!" }],
  });
  console.log(response.choices[0].message.content);
}

main();

Check the Last9 Traces Explorer to see your LLM traces.

View Traces in Last9

After sending LLM requests through your application, traces appear in Last9:

Navigate to Traces Explorer
Filter by your service name (the app_name you configured)
Explore trace data including:
- LLM operations - Chat completions, embeddings, and other model calls
- Token metrics - Input and output token counts per request
- Latency breakdown - Time spent in each operation
- Model information - Which model and provider handled each request

Advanced Configuration

Selective Instrumentation

By default, OpenLLMetry instruments all supported libraries. To enable only specific instrumentations, list them in the same init() call that sends data to Last9:

Python
JavaScript / TypeScript

from traceloop.sdk import Traceloop
from traceloop.sdk.instruments import Instruments

Traceloop.init(
    app_name="my-llm-app",
    api_endpoint="https://otlp.last9.io",
    headers={"Authorization": "<your-auth-header>"},
    instruments={Instruments.OPENAI, Instruments.LANGCHAIN},
)

import * as traceloop from "@traceloop/node-server-sdk";
import OpenAI from "openai";

traceloop.initialize({
  appName: "my-llm-app",
  baseUrl: "https://otlp.last9.io",
  headers: { Authorization: "<your-auth-header>" },
  instrumentModules: {
    openAI: OpenAI,
    langchain: true,
  },
});

Add Custom Attributes

Enrich your traces with custom metadata for better filtering and analysis:

Python
JavaScript / TypeScript

from traceloop.sdk.decorators import workflow, task

@workflow(name="document-qa")
def answer_question(document: str, question: str):
    # Your LLM logic here
    pass

@task(name="summarize")
def summarize_text(text: str):
    # Your summarization logic here
    pass

import { withWorkflow, withTask } from "@traceloop/node-server-sdk";

async function answerQuestion(document: string, question: string) {
  return withWorkflow({ name: "document-qa" }, async () => {
    // Your LLM logic here
  });
}

async function summarizeText(text: string) {
  return withTask({ name: "summarize" }, async () => {
    // Your summarization logic here
  });
}

Prompt and Completion Logging

By default, OpenLLMetry captures prompts and completions. To disable this for privacy:

Python
JavaScript / TypeScript

Set the environment variable before your application starts:

export TRACELOOP_TRACE_CONTENT=false

Set the environment variable before your application starts:

export TRACELOOP_TRACE_CONTENT=false

Or pass traceContent: false to initialize():

traceloop.initialize({
  appName: "my-llm-app",
  baseUrl: "https://otlp.last9.io",
  headers: { Authorization: "<your-auth-header>" },
  traceContent: false,
});

Supported Libraries

OpenLLMetry automatically instruments these libraries when detected:

Category	Libraries
LLM Providers	OpenAI, Anthropic, Cohere, Google AI, Azure OpenAI, Bedrock, Mistral, Ollama, Replicate, Together AI, Vertex AI
Frameworks	LangChain, LlamaIndex, Haystack, CrewAI
Vector DBs	Pinecone, Chroma, Weaviate, Milvus, Qdrant

Use Cases

Production Monitoring - Track LLM request latencies, error rates, and token usage in real-time alongside your existing application observability.
Cost Management - Monitor token consumption per feature, user, or workflow. Identify expensive operations and optimize prompt engineering.
Performance Optimization - Identify slow LLM calls, compare model performance, and optimize your AI pipelines based on real production data.
Debugging - Trace issues from user-facing errors back to specific LLM calls, prompts, and responses.

Troubleshooting

Traces do not appear in Last9
- Verify api_endpoint (Python) or baseUrl (JS/TS) is set to https://otlp.last9.io
- Confirm the headers passed to init() contain the correct Authorization value
- Ensure the Traceloop SDK is initialized before importing LLM libraries
- Check that your application has network access to otlp.last9.io

Please get in touch with us on Discord or Email if you have any questions.