# Express.js

> Instrument your Express.js application with OpenTelemetry to send traces, metrics, and logs to Last9

Source: https://last9.io/docs/integrations/frameworks/javascript/expressjs/

Use OpenTelemetry to instrument your JavaScript Express application and send telemetry data to Last9. This integration provides automatic instrumentation for HTTP requests, middleware operations, and other Express.js functionality.

You can either run OpenTelemetry Collector as a separate service or send telemetry directly from your application to Last9.

## Prerequisites

Before setting up Express.js monitoring, ensure you have:

- Node.js 16.0 or higher installed
- Express.js application running
- Last9 account with integration credentials
- npm or yarn package manager available

1.  **Install OpenTelemetry Packages**

    Install the required OpenTelemetry packages for Express.js instrumentation:

    ```bash
    npm install \
      @opentelemetry/api@1.9.0 \
      @opentelemetry/auto-instrumentations-node@0.59.0 \
      @opentelemetry/core@2.0.1 \
      @opentelemetry/exporter-trace-otlp-grpc@0.201.1 \
      @opentelemetry/exporter-trace-otlp-http@0.201.1 \
      @opentelemetry/instrumentation@0.201.1 \
      @opentelemetry/resources@2.0.1 \
      @opentelemetry/sdk-node@0.201.1 \
      @opentelemetry/sdk-trace-base@2.0.1 \
      @opentelemetry/sdk-trace-node@2.0.1 \
      @opentelemetry/semantic-conventions@1.34.0
    ```

    These packages provide comprehensive instrumentation for Express.js applications and integrate with the Last9 platform.

2.  **Set Environment Variables**

    Configure OpenTelemetry environment variables for your Express.js application. Replace the placeholder values with your actual Last9 credentials:

    ```bash
    export OTEL_SERVICE_NAME="<your_service_name>"
    export OTEL_EXPORTER_OTLP_ENDPOINT="{{ .Logs.WriteURL }}"
    export OTEL_EXPORTER_OTLP_HEADERS="Authorization={{ .Logs.AuthValue }}"
    export OTEL_TRACES_SAMPLER="always_on"
    export OTEL_RESOURCE_ATTRIBUTES="deployment.environment=local"
    export OTEL_LOG_LEVEL=error
    ```

    **Environment Variables Explained:**

    - `OTEL_SERVICE_NAME`: Name of your Express.js service
    - `OTEL_EXPORTER_OTLP_ENDPOINT`: Last9 endpoint URL for telemetry data
    - `OTEL_EXPORTER_OTLP_HEADERS`: Authentication header for Last9
    - `OTEL_RESOURCE_ATTRIBUTES`: Service metadata including environment

3.  **Create Instrumentation File**

    Create an instrumentation file that sets up OpenTelemetry for your Express.js application. Choose the appropriate version based on your project setup:

**TypeScript**

    Create a file named `instrumentation.ts` in your project root:

    ```typescript
    import {
      NodeTracerProvider,
      TracerConfig,
    } from "@opentelemetry/sdk-trace-node";
    import {
      BatchSpanProcessor,
      SpanProcessor,
      Span,
    } from "@opentelemetry/sdk-trace-base";
    import { OTLPTraceExporter } from "@opentelemetry/exporter-trace-otlp-http";
    import { registerInstrumentations } from "@opentelemetry/instrumentation";
    import { getNodeAutoInstrumentations } from "@opentelemetry/auto-instrumentations-node";
    import { resourceFromAttributes } from "@opentelemetry/resources";
    import * as api from "@opentelemetry/api";
    import {
      W3CTraceContextPropagator,
      W3CBaggagePropagator,
      CompositePropagator,
    } from "@opentelemetry/core";

    // For troubleshooting, uncomment the following lines to enable OpenTelemetry debug logging:
    // import { diag, DiagConsoleLogger, DiagLogLevel } from "@opentelemetry/api";
    // diag.setLogger(new DiagConsoleLogger(), DiagLogLevel.DEBUG);

    // BaggageSpanProcessor - propagates W3C Baggage entries as span attributes for RUM correlation
    class BaggageSpanProcessor implements SpanProcessor {
      onStart(span: Span, parentContext: api.Context): void {
        const baggage = api.propagation.getBaggage(
          parentContext || api.context.active(),
        );
        if (!baggage) return;
        for (const [key, entry] of baggage.getAllEntries()) {
          span.setAttribute(key, entry.value);
        }
      }
      onEnd(): void {}
      forceFlush(): Promise<void> {
        return Promise.resolve();
      }
      shutdown(): Promise<void> {
        return Promise.resolve();
      }
    }

    const providerConfig: TracerConfig = {
      resource: resourceFromAttributes({
        ["service.name"]: process.env.OTEL_SERVICE_NAME,
        ["deployment.environment"]: process.env.NODE_ENV,
      }),
      spanProcessors: [
        new BaggageSpanProcessor(),
        new BatchSpanProcessor(new OTLPTraceExporter()),
      ],
    };

    // Initialize and register the tracer provider
    const provider = new NodeTracerProvider(providerConfig);
    provider.register({
      propagator: new CompositePropagator({
        propagators: [
          new W3CTraceContextPropagator(),
          new W3CBaggagePropagator(),
        ],
      }),
    });

    // Automatically instrument HTTP and Express (additional instrumentations can be added similarly)
    registerInstrumentations({
      instrumentations: [
        getNodeAutoInstrumentations({
          // instrumentation-fs is disabled to reduce the noise of spans related to file operations
          "@opentelemetry/instrumentation-fs": {
            enabled: false,
          },
        }),
      ],
    });
    ```

**JavaScript**

    Create a file named `instrumentation.js` in your project root:

    ```javascript
    const { NodeTracerProvider } = require("@opentelemetry/sdk-trace-node");
    const { BatchSpanProcessor } = require("@opentelemetry/sdk-trace-base");
    const {
      OTLPTraceExporter,
    } = require("@opentelemetry/exporter-trace-otlp-http");
    const {
      registerInstrumentations,
    } = require("@opentelemetry/instrumentation");
    const {
      getNodeAutoInstrumentations,
    } = require("@opentelemetry/auto-instrumentations-node");
    const { resourceFromAttributes } = require("@opentelemetry/resources");
    const api = require("@opentelemetry/api");
    const {
      W3CTraceContextPropagator,
      W3CBaggagePropagator,
      CompositePropagator,
    } = require("@opentelemetry/core");

    // For troubleshooting, uncomment the following lines to enable OpenTelemetry debug logging:
    // const { diag, DiagConsoleLogger, DiagLogLevel } = require('@opentelemetry/api');
    // diag.setLogger(new DiagConsoleLogger(), DiagLogLevel.DEBUG);

    // BaggageSpanProcessor - propagates W3C Baggage entries as span attributes for RUM correlation
    class BaggageSpanProcessor {
      onStart(span, parentContext) {
        const baggage = api.propagation.getBaggage(
          parentContext || api.context.active(),
        );
        if (!baggage) return;
        for (const [key, entry] of baggage.getAllEntries()) {
          span.setAttribute(key, entry.value);
        }
      }
      onEnd() {}
      forceFlush() {
        return Promise.resolve();
      }
      shutdown() {
        return Promise.resolve();
      }
    }

    const providerConfig = {
      resource: resourceFromAttributes({
        "service.name": process.env.OTEL_SERVICE_NAME,
        "deployment.environment": process.env.NODE_ENV,
      }),
      spanProcessors: [
        new BaggageSpanProcessor(),
        new BatchSpanProcessor(new OTLPTraceExporter()),
      ],
    };

    const provider = new NodeTracerProvider(providerConfig);
    provider.register({
      propagator: new CompositePropagator({
        propagators: [
          new W3CTraceContextPropagator(),
          new W3CBaggagePropagator(),
        ],
      }),
    });

    registerInstrumentations({
      instrumentations: [
        getNodeAutoInstrumentations({
          "@opentelemetry/instrumentation-fs": { enabled: false },
        }),
      ],
    });

    console.log("[OpenTelemetry] Express instrumentation initialized");
    ```

4.  **Import Instrumentation in Your Application**

    Import the instrumentation file at the very top of your main application file, before any other imports. This is crucial for proper instrumentation setup.

**TypeScript**

    If your main file is `server.ts`, import the instrumentation at the top:

    ```typescript
    import "./instrumentation";
    import express from "express";
    // ... other imports

    const app = express();
    // ... rest of your application code
    ```

**JavaScript**

    If your main file is `server.js`, require the instrumentation at the top:

    ```javascript
    require("./instrumentation");
    const express = require("express");
    // ... other requires

    const app = express();
    // ... rest of your application code
    ```

    **Important**: Make sure to import the instrumentation script before any `express` package is imported.

5.  **Run Your Express Application**

    Start your Express.js application normally. The OpenTelemetry instrumentation will automatically begin collecting and sending telemetry data to Last9:

    ```bash
    # For development
    npm start

    # Or with nodemon for development
    nodemon server.js

    # For production
    NODE_ENV=production npm start
    ```

## Understanding the Instrumentation

The instrumentation code performs the following steps:

1. **Sets up Trace Provider** with your application's name as the service name
2. **Configures OTLP Exporter** with Last9 endpoint and authentication
3. **Registers Auto-Instrumentation** for Express.js and common Node.js libraries
4. **Optimizes Performance** by using batch processing and disabling noisy file system instrumentation

## What Gets Instrumented

When you use automatic instrumentation with Express.js, OpenTelemetry automatically captures:

### HTTP Requests

- Request method, URL, and headers
- Response status codes and timing
- Route parameters and query strings
- Request and response body sizes

### Express Middleware

- Middleware execution order and timing
- Custom middleware operations
- Route handler performance
- Error handling middleware

### Database Operations

- SQL queries and execution times (MySQL, PostgreSQL, etc.)
- MongoDB operations
- Redis commands
- Database connection pooling

### External API Calls

- HTTP client requests (using `axios`, `fetch`, `http`, etc.)
- Request/response details and timing
- Error handling and retry attempts

### File System Operations

- File reads and writes (can be disabled for performance)
- Directory operations
- File system monitoring

## Advanced Configuration

### Custom Service Metadata

Enhance your service metadata by updating the resource attributes:

```typescript
const providerConfig: TracerConfig = {
  resource: resourceFromAttributes({
    "service.name": process.env.OTEL_SERVICE_NAME,
    "service.version": "2.1.0",
    "deployment.environment": process.env.NODE_ENV,
    "service.instance.id": "api-server-1",
    team: "backend",
  }),
  // ... rest of configuration
};
```

### Custom Spans and Attributes

Add custom spans and attributes to track business-specific operations:

```typescript
import { trace } from "@opentelemetry/api";

const tracer = trace.getTracer("express-app");

app.get("/users/:userId", async (req, res) => {
  const span = tracer.startSpan("get_user_operation");

  try {
    span.setAttributes({
      "user.id": req.params.userId,
      "operation.type": "user_lookup",
      "request.ip": req.ip,
    });

    // Your business logic here
    const user = await getUserById(req.params.userId);

    span.setStatus({ code: trace.SpanStatusCode.OK });
    res.json(user);
  } catch (error) {
    span.setStatus({
      code: trace.SpanStatusCode.ERROR,
      message: error.message,
    });
    span.recordException(error);
    res.status(500).json({ error: "Internal server error" });
  } finally {
    span.end();
  }
});
```

### Error Handling Integration

Integrate OpenTelemetry with Express error handling:

```typescript
import { trace } from "@opentelemetry/api";

// Error handling middleware
app.use(
  (
    error: Error,
    req: express.Request,
    res: express.Response,
    next: express.NextFunction,
  ) => {
    const span = trace.getActiveSpan();

    if (span) {
      span.setStatus({
        code: trace.SpanStatusCode.ERROR,
        message: error.message,
      });
      span.recordException(error);
    }

    console.error("Unhandled error:", error);
    res.status(500).json({ error: "Internal server error" });
  },
);
```

## Verification

1. **Check Application Startup**

   When you start your Express.js application, you should see OpenTelemetry initialization messages in the console.

2. **Generate Test Traffic**

   Make some requests to your Express.js application to generate telemetry data:

   ```bash
   curl http://localhost:3000/
   curl http://localhost:3000/health
   curl http://localhost:3000/api/users/123
   ```

3. **Verify Data in Last9**

   Log into your Last9 account and check that traces and metrics are being received in [Grafana](https://app.last9.io/grafana).

   Look for:

   - HTTP request traces with timing information
   - Middleware execution spans
   - Database query spans
   - Error traces for failed requests

## Troubleshooting

### Common Issues

**Instrumentation Not Working**
Ensure the instrumentation file is imported before any Express imports:

```javascript
// ✅ Correct order
require("./instrumentation");
const express = require("express");

// ❌ Incorrect order
const express = require("express");
require("./instrumentation");
```

**Missing Traces**
Enable debug logging to troubleshoot:

```javascript
const { diag, DiagConsoleLogger, DiagLogLevel } = require("@opentelemetry/api");
diag.setLogger(new DiagConsoleLogger(), DiagLogLevel.DEBUG);
```

**Environment Variables Not Set**
Verify your environment variables are properly configured:

```bash
env | grep OTEL_
```

**Network Connection Issues**
Test connectivity to Last9:

```bash
curl -v -H "Authorization: <your-auth-header>" <your-endpoint>
```

### Performance Considerations

- Use sampling in production to control trace volume
- Disable file system instrumentation to reduce noise
- Monitor the instrumentation overhead on your application
- Use batch processing for better performance
- Consider using the OpenTelemetry Collector for high-traffic applications

### Best Practices

- Import instrumentation before any other imports
- Set meaningful service names and versions
- Use consistent environment naming across services
- Add custom spans for critical business operations
- Implement proper error handling with span status updates
- Use structured logging alongside distributed tracing

## Supported Libraries

OpenTelemetry automatically instruments many popular Node.js libraries commonly used with Express.js:

- **HTTP Libraries**: `http`, `https`, `axios`, `node-fetch`
- **Databases**: `mysql`, `mysql2`, `pg`, `mongodb`, `ioredis`
- **Message Queues**: `amqplib` (RabbitMQ), `kafkajs`
- **Cloud Services**: `aws-sdk`, `@google-cloud/*`
- **Web Frameworks**: `express`, `koa`, `fastify`

For a complete list of supported libraries, visit the [OpenTelemetry Node.js documentation](https://opentelemetry.io/docs/instrumentation/js/auto/).

## Need Help?

If you encounter any issues or have questions:

- Join our [Discord community](https://discord.com/channels/652153247672729619/652153247672729621) for real-time support
- Contact our support team at [support@last9.io](mailto:support@last9.io)