Express.js

Use OpenTelemetry to instrument your JavaScript Express application and send telemetry data to Last9. This integration provides automatic instrumentation for HTTP requests, middleware operations, and other Express.js functionality.

You can either run OpenTelemetry Collector as a separate service or send telemetry directly from your application to Last9.

Prerequisites

Before setting up Express.js monitoring, ensure you have:

• Node.js 16.0 or higher installed
• Express.js application running
• Last9 account with integration credentials
• npm or yarn package manager available

1. Install OpenTelemetry Packages

   Install the required OpenTelemetry packages for Express.js instrumentation:

   npm install \
     @opentelemetry/api@1.9.0 \
     @opentelemetry/auto-instrumentations-node@0.59.0 \
     @opentelemetry/exporter-trace-otlp-grpc@0.201.1 \
     @opentelemetry/exporter-trace-otlp-http@0.201.1 \
     @opentelemetry/instrumentation@0.201.1 \
     @opentelemetry/resources@2.0.1 \
     @opentelemetry/sdk-node@0.201.1 \
     @opentelemetry/sdk-trace-base@2.0.1 \
     @opentelemetry/sdk-trace-node@2.0.1 \
     @opentelemetry/semantic-conventions@1.34.0

   These packages provide comprehensive instrumentation for Express.js applications and integrate with the Last9 platform.

2. Set Environment Variables

   Configure OpenTelemetry environment variables for your Express.js application. Replace the placeholder values with your actual Last9 credentials:

   export OTEL_SERVICE_NAME="<your_service_name>"
   export OTEL_EXPORTER_OTLP_ENDPOINT="{{ .Logs.WriteURL }}"
   export OTEL_EXPORTER_OTLP_HEADERS="Authorization={{ .Logs.AuthValue }}"
   export OTEL_TRACES_SAMPLER="always_on"
   export OTEL_RESOURCE_ATTRIBUTES="deployment.environment=local"
   export OTEL_LOG_LEVEL=error

   Environment Variables Explained:

   • OTEL_SERVICE_NAME: Name of your Express.js service
   • OTEL_EXPORTER_OTLP_ENDPOINT: Last9 endpoint URL for telemetry data
   • OTEL_EXPORTER_OTLP_HEADERS: Authentication header for Last9
   • OTEL_RESOURCE_ATTRIBUTES: Service metadata including environment

3. Create Instrumentation File

   Create an instrumentation file that sets up OpenTelemetry for your Express.js application. Choose the appropriate version based on your project setup:

   • TypeScript
   • JavaScript

   Create a file named instrumentation.ts in your project root:

   import {
     NodeTracerProvider,
     TracerConfig,
   } from "@opentelemetry/sdk-trace-node";
   import { BatchSpanProcessor } from "@opentelemetry/sdk-trace-base";
   import { OTLPTraceExporter } from "@opentelemetry/exporter-trace-otlp-http";
   import { registerInstrumentations } from "@opentelemetry/instrumentation";
   import { getNodeAutoInstrumentations } from "@opentelemetry/auto-instrumentations-node";
   import { resourceFromAttributes } from "@opentelemetry/resources";
   // Uncomment for troubleshooting:
   // import { diag, DiagConsoleLogger, DiagLogLevel } from "@opentelemetry/api";
   
   // For troubleshooting, set the log level to DiagLogLevel.DEBUG
   // diag.setLogger(new DiagConsoleLogger(), DiagLogLevel.DEBUG);
   
   const providerConfig: TracerConfig = {
     resource: resourceFromAttributes({
       ["service.name"]: process.env.OTEL_SERVICE_NAME,
       ["deployment.environment"]: process.env.NODE_ENV,
     }),
     spanProcessors: [new BatchSpanProcessor(new OTLPTraceExporter())],
   };
   
   // Initialize and register the tracer provider
   const provider = new NodeTracerProvider(providerConfig);
   provider.register();
   
   // Automatically instrument HTTP and Express (additional instrumentations can be added similarly)
   registerInstrumentations({
     instrumentations: [
       getNodeAutoInstrumentations({
         // instrumentation-fs is disabled to reduce the noise of spans related to file operations
         "@opentelemetry/instrumentation-fs": {
           enabled: false,
         },
       }),
     ],
   });

4. Import Instrumentation in Your Application

   Import the instrumentation file at the very top of your main application file, before any other imports. This is crucial for proper instrumentation setup.

   • TypeScript
   • JavaScript

   If your main file is server.ts, import the instrumentation at the top:

   import "./instrumentation";
   import express from "express";
   // ... other imports
   
   const app = express();
   // ... rest of your application code

   Important: Make sure to import the instrumentation script before any express package is imported.

5. Run Your Express Application

   Start your Express.js application normally. The OpenTelemetry instrumentation will automatically begin collecting and sending telemetry data to Last9:

   # For development
   npm start
   
   # Or with nodemon for development
   nodemon server.js
   
   # For production
   NODE_ENV=production npm start

Understanding the Instrumentation

The instrumentation code performs the following steps:

1. Sets up Trace Provider with your application’s name as the service name
2. Configures OTLP Exporter with Last9 endpoint and authentication
3. Registers Auto-Instrumentation for Express.js and common Node.js libraries
4. Optimizes Performance by using batch processing and disabling noisy file system instrumentation

What Gets Instrumented

When you use automatic instrumentation with Express.js, OpenTelemetry automatically captures:

HTTP Requests

• Request method, URL, and headers
• Response status codes and timing
• Route parameters and query strings
• Request and response body sizes

Express Middleware

• Middleware execution order and timing
• Custom middleware operations
• Route handler performance
• Error handling middleware

Database Operations

• SQL queries and execution times (MySQL, PostgreSQL, etc.)
• MongoDB operations
• Redis commands
• Database connection pooling

External API Calls

• HTTP client requests (using axios, fetch, http, etc.)
• Request/response details and timing
• Error handling and retry attempts

File System Operations

• File reads and writes (can be disabled for performance)
• Directory operations
• File system monitoring

Advanced Configuration

Custom Service Metadata

Enhance your service metadata by updating the resource attributes:

const providerConfig: TracerConfig = {
  resource: resourceFromAttributes({
    "service.name": process.env.OTEL_SERVICE_NAME,
    "service.version": "2.1.0",
    "deployment.environment": process.env.NODE_ENV,
    "service.instance.id": "api-server-1",
    team: "backend",
  }),
  // ... rest of configuration
};

Custom Spans and Attributes

Add custom spans and attributes to track business-specific operations:

import { trace } from "@opentelemetry/api";

const tracer = trace.getTracer("express-app");

app.get("/users/:userId", async (req, res) => {
  const span = tracer.startSpan("get_user_operation");

  try {
    span.setAttributes({
      "user.id": req.params.userId,
      "operation.type": "user_lookup",
      "request.ip": req.ip,
    });

    // Your business logic here
    const user = await getUserById(req.params.userId);

    span.setStatus({ code: trace.SpanStatusCode.OK });
    res.json(user);
  } catch (error) {
    span.setStatus({
      code: trace.SpanStatusCode.ERROR,
      message: error.message,
    });
    span.recordException(error);
    res.status(500).json({ error: "Internal server error" });
  } finally {
    span.end();
  }
});

Error Handling Integration

Integrate OpenTelemetry with Express error handling:

import { trace } from "@opentelemetry/api";

// Error handling middleware
app.use(
  (
    error: Error,
    req: express.Request,
    res: express.Response,
    next: express.NextFunction,
  ) => {
    const span = trace.getActiveSpan();

    if (span) {
      span.setStatus({
        code: trace.SpanStatusCode.ERROR,
        message: error.message,
      });
      span.recordException(error);
    }

    console.error("Unhandled error:", error);
    res.status(500).json({ error: "Internal server error" });
  },
);

Verification

1. Check Application Startup

   When you start your Express.js application, you should see OpenTelemetry initialization messages in the console.

2. Generate Test Traffic

   Make some requests to your Express.js application to generate telemetry data:

   curl http://localhost:3000/
   curl http://localhost:3000/health
   curl http://localhost:3000/api/users/123

3. Verify Data in Last9

   Log into your Last9 account and check that traces and metrics are being received in Grafana.

   Look for:

   • HTTP request traces with timing information
   • Middleware execution spans
   • Database query spans
   • Error traces for failed requests

Troubleshooting

Common Issues

Instrumentation Not Working Ensure the instrumentation file is imported before any Express imports:

// ✅ Correct order
require("./instrumentation");
const express = require("express");

// ❌ Incorrect order
const express = require("express");
require("./instrumentation");

Missing Traces Enable debug logging to troubleshoot:

const { diag, DiagConsoleLogger, DiagLogLevel } = require("@opentelemetry/api");
diag.setLogger(new DiagConsoleLogger(), DiagLogLevel.DEBUG);

Environment Variables Not Set Verify your environment variables are properly configured:

env | grep OTEL_

Network Connection Issues Test connectivity to Last9:

curl -v -H "Authorization: <your-auth-header>" <your-endpoint>

Performance Considerations

• Use sampling in production to control trace volume
• Disable file system instrumentation to reduce noise
• Monitor the instrumentation overhead on your application
• Use batch processing for better performance
• Consider using the OpenTelemetry Collector for high-traffic applications

Best Practices

• Import instrumentation before any other imports
• Set meaningful service names and versions
• Use consistent environment naming across services
• Add custom spans for critical business operations
• Implement proper error handling with span status updates
• Use structured logging alongside distributed tracing

Supported Libraries

OpenTelemetry automatically instruments many popular Node.js libraries commonly used with Express.js:

• HTTP Libraries: http, https, axios, node-fetch
• Databases: mysql, mysql2, pg, mongodb, ioredis
• Message Queues: amqplib (RabbitMQ), kafkajs
• Cloud Services: aws-sdk, @google-cloud/*
• Web Frameworks: express, koa, fastify

For a complete list of supported libraries, visit the OpenTelemetry Node.js documentation.

Need Help?

If you encounter any issues or have questions:

• Join our Discord community for real-time support
• Contact our support team at support@last9.io