# OpenLLMetry

> Send OpenTelemetry traces from your LLM applications to Last9 using OpenLLMetry for AI observability

Source: https://last9.io/docs/integrations/openllmetry/

Monitor your LLM applications with OpenLLMetry traces exported to Last9. Track token usage, latencies, error rates, and model performance across OpenAI, Anthropic, LangChain, LlamaIndex, and more.

## What is OpenLLMetry?

[OpenLLMetry](https://github.com/traceloop/openllmetry) is an open-source project by Traceloop that provides OpenTelemetry-based instrumentation for LLM applications. It automatically captures traces from popular AI frameworks and model providers, giving you visibility into every LLM call your application makes.

Key capabilities include:

- **Automatic instrumentation** - Zero-code tracing for OpenAI, Anthropic, Cohere, Bedrock, and 20+ providers
- **Framework support** - Native integration with LangChain, LlamaIndex, Haystack, and CrewAI
- **Rich telemetry** - Captures prompts, completions, token counts, latencies, and costs
- **OpenTelemetry native** - Exports standard OTLP data to any compatible backend

## Prerequisites

Before setting up the integration:

1. **Last9 Account** - Sign up at [app.last9.io](https://app.last9.io)
2. **LLM Application** - An existing Python or JavaScript/TypeScript application using LLM providers or frameworks

## Integration Setup

OpenLLMetry provides SDKs for Python and JavaScript/TypeScript. Choose your language below to get started.

**Python**

1. **Install the OpenLLMetry SDK**

   Install the Traceloop SDK along with the instrumentations you need:

   ```bash
   pip install traceloop-sdk
   ```

   For specific frameworks, install additional packages:

   ```bash
   # For OpenAI
   pip install opentelemetry-instrumentation-openai

   # For LangChain
   pip install opentelemetry-instrumentation-langchain

   # For LlamaIndex
   pip install opentelemetry-instrumentation-llamaindex

   # For Anthropic
   pip install opentelemetry-instrumentation-anthropic

   # For AWS Bedrock
   pip install opentelemetry-instrumentation-bedrock
   ```

2. **Get Your Last9 OTel Credentials**

   Navigate to [**Integrations** → **OpenTelemetry**](https://app.last9.io/integrations?integration=OpenTelemetry) in your Last9 dashboard. Copy the **OTel Endpoint** and **Auth Header** values.

3. **Initialize OpenLLMetry with Your Last9 Credentials**

   Add the Traceloop SDK initialization at the start of your application, passing the endpoint and Auth Header directly:

   ```python
   from traceloop.sdk import Traceloop

   Traceloop.init(
       app_name="my-llm-app",
       api_endpoint="https://otlp.last9.io",
       headers={"Authorization": "<your-auth-header>"},
   )
   ```

   Replace `<your-auth-header>` with the full Auth Header value from Step 2, including the `Basic ` prefix. Pass it as a `headers` dict rather than the `TRACELOOP_HEADERS` environment variable — OpenTelemetry's env-header parser rejects the space in `Basic <token>` and silently drops the header.

   OpenLLMetry then instruments supported libraries automatically.

4. **Verify the Integration**

   Make an LLM call in your application:

   ```python
   from openai import OpenAI

   client = OpenAI()
   response = client.chat.completions.create(
       model="gpt-4",
       messages=[{"role": "user", "content": "Hello, world!"}]
   )
   print(response.choices[0].message.content)
   ```

   Check the [Last9 Traces Explorer](https://app.last9.io/traces) to see your LLM traces.

**JavaScript / TypeScript**

1. **Install the OpenLLMetry SDK**

   Install the Traceloop SDK:

   ```bash
   npm install @traceloop/node-server-sdk
   ```

2. **Get Your Last9 OTel Credentials**

   Navigate to [**Integrations** → **OpenTelemetry**](https://app.last9.io/integrations?integration=OpenTelemetry) in your Last9 dashboard. Copy the **OTel Endpoint** and **Auth Header** values.

3. **Initialize OpenLLMetry with Your Last9 Credentials**

   Add the Traceloop SDK initialization at the start of your application (before importing other modules), passing the endpoint and Auth Header directly:

   ```typescript
   import * as traceloop from "@traceloop/node-server-sdk";

   traceloop.initialize({
     appName: "my-llm-app",
     baseUrl: "https://otlp.last9.io",
     headers: { Authorization: "<your-auth-header>" },
   });
   ```

   For CommonJS:

   ```javascript
   const traceloop = require("@traceloop/node-server-sdk");

   traceloop.initialize({
     appName: "my-llm-app",
     baseUrl: "https://otlp.last9.io",
     headers: { Authorization: "<your-auth-header>" },
   });
   ```

   Replace `<your-auth-header>` with the full Auth Header value from Step 2, including the `Basic ` prefix. Pass it as a `headers` object rather than the `TRACELOOP_HEADERS` environment variable — the SDK's env-header parser splits on every `=`, which truncates base64-padded tokens.

4. **Verify the Integration**

   Make an LLM call in your application:

   ```typescript
   import OpenAI from "openai";

   const openai = new OpenAI();

   async function main() {
     const response = await openai.chat.completions.create({
       model: "gpt-4",
       messages: [{ role: "user", content: "Hello, world!" }],
     });
     console.log(response.choices[0].message.content);
   }

   main();
   ```

   Check the [Last9 Traces Explorer](https://app.last9.io/traces) to see your LLM traces.

## View Traces in Last9

After sending LLM requests through your application, traces appear in Last9:

1. Navigate to [**Traces Explorer**](https://app.last9.io/traces)
2. Filter by your service name (the `app_name` you configured)
3. Explore trace data including:
   - **LLM operations** - Chat completions, embeddings, and other model calls
   - **Token metrics** - Input and output token counts per request
   - **Latency breakdown** - Time spent in each operation
   - **Model information** - Which model and provider handled each request

## Advanced Configuration

### Selective Instrumentation

By default, OpenLLMetry instruments all supported libraries. To enable only specific instrumentations, list them in the same `init()` call that sends data to Last9:

**Python**

```python
from traceloop.sdk import Traceloop
from traceloop.sdk.instruments import Instruments

Traceloop.init(
    app_name="my-llm-app",
    api_endpoint="https://otlp.last9.io",
    headers={"Authorization": "<your-auth-header>"},
    instruments={Instruments.OPENAI, Instruments.LANGCHAIN},
)
```

**JavaScript / TypeScript**

```typescript
import * as traceloop from "@traceloop/node-server-sdk";
import OpenAI from "openai";

traceloop.initialize({
  appName: "my-llm-app",
  baseUrl: "https://otlp.last9.io",
  headers: { Authorization: "<your-auth-header>" },
  instrumentModules: {
    openAI: OpenAI,
    langchain: true,
  },
});
```

### Add Custom Attributes

Enrich your traces with custom metadata for better filtering and analysis:

**Python**

```python
from traceloop.sdk.decorators import workflow, task

@workflow(name="document-qa")
def answer_question(document: str, question: str):
    # Your LLM logic here
    pass

@task(name="summarize")
def summarize_text(text: str):
    # Your summarization logic here
    pass
```

**JavaScript / TypeScript**

```typescript
import { withWorkflow, withTask } from "@traceloop/node-server-sdk";

async function answerQuestion(document: string, question: string) {
  return withWorkflow({ name: "document-qa" }, async () => {
    // Your LLM logic here
  });
}

async function summarizeText(text: string) {
  return withTask({ name: "summarize" }, async () => {
    // Your summarization logic here
  });
}
```

### Prompt and Completion Logging

By default, OpenLLMetry captures prompts and completions. To disable this for privacy:

**Python**

Set the environment variable before your application starts:

```bash
export TRACELOOP_TRACE_CONTENT=false
```

**JavaScript / TypeScript**

Set the environment variable before your application starts:

```bash
export TRACELOOP_TRACE_CONTENT=false
```

Or pass `traceContent: false` to `initialize()`:

```typescript
traceloop.initialize({
  appName: "my-llm-app",
  baseUrl: "https://otlp.last9.io",
  headers: { Authorization: "<your-auth-header>" },
  traceContent: false,
});
```

## Supported Libraries

OpenLLMetry automatically instruments these libraries when detected:

| Category          | Libraries                                                                                                       |
| ----------------- | --------------------------------------------------------------------------------------------------------------- |
| **LLM Providers** | OpenAI, Anthropic, Cohere, Google AI, Azure OpenAI, Bedrock, Mistral, Ollama, Replicate, Together AI, Vertex AI |
| **Frameworks**    | LangChain, LlamaIndex, Haystack, CrewAI                                                                         |
| **Vector DBs**    | Pinecone, Chroma, Weaviate, Milvus, Qdrant                                                                      |

## Use Cases

- **Production Monitoring** - Track LLM request latencies, error rates, and token usage in real-time alongside your existing application observability.
- **Cost Management** - Monitor token consumption per feature, user, or workflow. Identify expensive operations and optimize prompt engineering.
- **Performance Optimization** - Identify slow LLM calls, compare model performance, and optimize your AI pipelines based on real production data.
- **Debugging** - Trace issues from user-facing errors back to specific LLM calls, prompts, and responses.

---

## Troubleshooting

- **Traces do not appear in Last9**

  - Verify `api_endpoint` (Python) or `baseUrl` (JS/TS) is set to `https://otlp.last9.io`
  - Confirm the `headers` passed to `init()` contain the correct `Authorization` value
  - Ensure the Traceloop SDK is initialized before importing LLM libraries
  - Check that your application has network access to `otlp.last9.io`

Please get in touch with us on [Discord](https://discord.com/invite/Q3p2EEucx9) or [Email](mailto:support@last9.io) if you have any questions.