# Node.js

> Instrument Node.js applications with OpenTelemetry for comprehensive tracing and observability

Source: https://last9.io/docs/integrations/languages/nodejs/

Use OpenTelemetry to instrument your Node.js application and send telemetry data to Last9. This integration provides automatic instrumentation for HTTP requests, database operations, file system access, and external API calls in your Node.js applications.

You can either run OpenTelemetry Collector as a separate service or send telemetry directly from your application to Last9.

## Prerequisites

Before setting up Node.js monitoring, ensure you have:

- Node.js 16.0 or higher installed
- Node.js application to instrument
- Last9 account with integration credentials
- npm or yarn package manager available

1.  **Install OpenTelemetry Packages**

    Install the required OpenTelemetry packages for comprehensive Node.js instrumentation:

    ```bash
    npm install \
      @opentelemetry/api@1.9.0 \
      @opentelemetry/auto-instrumentations-node@0.59.0 \
      @opentelemetry/exporter-trace-otlp-grpc@0.201.1 \
      @opentelemetry/exporter-trace-otlp-http@0.201.1 \
      @opentelemetry/instrumentation@0.201.1 \
      @opentelemetry/resources@2.0.1 \
      @opentelemetry/sdk-node@0.201.1 \
      @opentelemetry/sdk-trace-base@2.0.1 \
      @opentelemetry/sdk-trace-node@2.0.1 \
      @opentelemetry/semantic-conventions@1.34.0
    ```

    These packages provide:

    - **Core APIs**: Base OpenTelemetry functionality
    - **Auto-instrumentation**: Automatic tracing for popular libraries
    - **OTLP Exporters**: Direct export to Last9
    - **Node.js SDK**: Optimized for Node.js runtime

2.  **Set Environment Variables**

    Configure OpenTelemetry environment variables for your Node.js application:

    ```bash
    export OTEL_SERVICE_NAME="<your_service_name>"
    export OTEL_EXPORTER_OTLP_ENDPOINT=$last9_otlp_endpoint
    export OTEL_EXPORTER_OTLP_HEADERS="Authorization=$last9_otlp_auth_header"
    export OTEL_TRACES_SAMPLER="always_on"
    export OTEL_RESOURCE_ATTRIBUTES="deployment.environment=production"
    export OTEL_LOG_LEVEL=error
    ```

    Replace `<your_service_name>` with a descriptive name for your Node.js service (e.g., `api-server`, `user-service`, `payment-processor`).

    Use the same `OTEL_SERVICE_NAME` and `deployment.environment` values that your infrastructure telemetry uses. For example, if ECS logs use `service.name=checkout-service` and `deployment.environment=production`, configure Node.js traces with `OTEL_SERVICE_NAME="checkout-service"` and `OTEL_RESOURCE_ATTRIBUTES="deployment.environment=production"`. Consistent names let Last9 group logs, metrics, traces, dashboards, and alerts under the same service and environment.

3.  **Create Instrumentation File**

    Create an instrumentation file that sets up OpenTelemetry for your Node.js application. Choose the appropriate version based on your project setup:

**TypeScript**

    Create a file named `instrumentation.ts` in your project root:

    ```typescript
    import {
      NodeTracerProvider,
      TracerConfig,
    } from "@opentelemetry/sdk-trace-node";
    import {
      BatchSpanProcessor,
      SpanProcessor,
      Span,
    } from "@opentelemetry/sdk-trace-base";
    import { OTLPTraceExporter } from "@opentelemetry/exporter-trace-otlp-http";
    import { registerInstrumentations } from "@opentelemetry/instrumentation";
    import { getNodeAutoInstrumentations } from "@opentelemetry/auto-instrumentations-node";
    import { ATTR_SERVICE_NAME } from "@opentelemetry/semantic-conventions";
    import { resourceFromAttributes } from "@opentelemetry/resources";
    import * as api from "@opentelemetry/api";
    import {
      W3CTraceContextPropagator,
      W3CBaggagePropagator,
      CompositePropagator,
    } from "@opentelemetry/core";

    // For troubleshooting, uncomment the following lines to enable OpenTelemetry debug logging:
    // import { diag, DiagConsoleLogger, DiagLogLevel } from "@opentelemetry/api";
    // diag.setLogger(new DiagConsoleLogger(), DiagLogLevel.DEBUG);

    // BaggageSpanProcessor - propagates W3C Baggage entries as span attributes for RUM correlation
    class BaggageSpanProcessor implements SpanProcessor {
      onStart(span: Span, parentContext: api.Context): void {
        const baggage = api.propagation.getBaggage(
          parentContext || api.context.active(),
        );
        if (!baggage) return;
        for (const [key, entry] of baggage.getAllEntries()) {
          span.setAttribute(key, entry.value);
        }
      }
      onEnd(): void {}
      forceFlush(): Promise<void> {
        return Promise.resolve();
      }
      shutdown(): Promise<void> {
        return Promise.resolve();
      }
    }

    const providerConfig: TracerConfig = {
      resource: resourceFromAttributes({
        [ATTR_SERVICE_NAME]: process.env.OTEL_SERVICE_NAME,
        "deployment.environment": process.env.NODE_ENV,
      }),
      // SDK auto-reads OTEL_EXPORTER_OTLP_ENDPOINT and OTEL_EXPORTER_OTLP_HEADERS from env
      spanProcessors: [
        new BaggageSpanProcessor(),
        new BatchSpanProcessor(new OTLPTraceExporter()),
      ],
    };

    // Initialize and register the tracer provider
    const provider = new NodeTracerProvider(providerConfig);
    provider.register({
      propagator: new CompositePropagator({
        propagators: [
          new W3CTraceContextPropagator(),
          new W3CBaggagePropagator(),
        ],
      }),
    });

    // Automatically instrument the Node.js application
    registerInstrumentations({
      instrumentations: [
        getNodeAutoInstrumentations({
          // Disable noisy file system instrumentation
          "@opentelemetry/instrumentation-fs": {
            enabled: false,
          },
        }),
      ],
    });
    ```

**JavaScript**

    Create a file named `instrumentation.js` in your project root:

    ```javascript
    // instrumentation.js
    const { NodeSDK } = require("@opentelemetry/sdk-node");
    const {
      getNodeAutoInstrumentations,
    } = require("@opentelemetry/auto-instrumentations-node");
    const {
      OTLPTraceExporter,
    } = require("@opentelemetry/exporter-trace-otlp-http");
    const { resourceFromAttributes } = require("@opentelemetry/resources");
    const {
      ATTR_SERVICE_NAME,
    } = require("@opentelemetry/semantic-conventions");
    const { BatchSpanProcessor } = require("@opentelemetry/sdk-trace-base");
    const api = require("@opentelemetry/api");
    const {
      W3CTraceContextPropagator,
      W3CBaggagePropagator,
      CompositePropagator,
    } = require("@opentelemetry/core");

    // For troubleshooting, uncomment the following lines to enable OpenTelemetry debug logging:
    // const { diag, DiagConsoleLogger, DiagLogLevel } = require('@opentelemetry/api');
    // diag.setLogger(new DiagConsoleLogger(), DiagLogLevel.DEBUG);

    // BaggageSpanProcessor - propagates W3C Baggage entries as span attributes for RUM correlation
    class BaggageSpanProcessor {
      onStart(span, parentContext) {
        const baggage = api.propagation.getBaggage(
          parentContext || api.context.active(),
        );
        if (!baggage) return;
        for (const [key, entry] of baggage.getAllEntries()) {
          span.setAttribute(key, entry.value);
        }
      }
      onEnd() {}
      forceFlush() {
        return Promise.resolve();
      }
      shutdown() {
        return Promise.resolve();
      }
    }

    // Create and configure SDK
    // SDK auto-reads OTEL_EXPORTER_OTLP_ENDPOINT and OTEL_EXPORTER_OTLP_HEADERS from env
    const sdk = new NodeSDK({
      resource: resourceFromAttributes({
        [ATTR_SERVICE_NAME]: process.env.OTEL_SERVICE_NAME,
        "deployment.environment": process.env.NODE_ENV,
      }),
      textMapPropagator: new CompositePropagator({
        propagators: [
          new W3CTraceContextPropagator(),
          new W3CBaggagePropagator(),
        ],
      }),
      spanProcessors: [
        new BaggageSpanProcessor(),
        new BatchSpanProcessor(new OTLPTraceExporter()),
      ],
      instrumentations: [
        getNodeAutoInstrumentations({
          "@opentelemetry/instrumentation-fs": { enabled: false },
        }),
      ],
    });

    // sdk.start() is synchronous in sdk-node 0.201.x+
    try {
      sdk.start();
      console.log("[OpenTelemetry] Instrumentation initialized");
    } catch (error) {
      console.error("[OpenTelemetry] Failed to initialize tracing", error);
    }

    process.on("SIGTERM", () => sdk.shutdown().finally(() => process.exit(0)));
    process.on("SIGINT", () => sdk.shutdown().finally(() => process.exit(0)));
    ```

4.  **Import Instrumentation in Your Application**

    Import the instrumentation file at the very top of your main application file, before any other imports. This is crucial for proper instrumentation setup.

**TypeScript**

    If your main file is `server.ts` or `app.ts`, import the instrumentation at the top:

    ```typescript
    import "./instrumentation";
    import express from "express";
    import { createServer } from "http";
    // ... other imports

    const app = express();
    // ... rest of your application code
    ```

**JavaScript**

    If your main file is `server.js` or `app.js`, require the instrumentation at the top:

    ```javascript
    require("./instrumentation");
    const express = require("express");
    const http = require("http");
    // ... other requires

    const app = express();
    // ... rest of your application code
    ```

    **Important**: Make sure to import/require the instrumentation script before any other modules to ensure proper auto-instrumentation.

5.  **Run Your Node.js Application**

    Start your Node.js application normally. The OpenTelemetry instrumentation will automatically begin collecting and sending telemetry data to Last9:

    ```bash
    # For development
    npm start

    # Or with nodemon for development
    nodemon server.js

    # For production with PM2
    pm2 start server.js --name "my-api"

    # Direct node execution
    node server.js
    ```

## Understanding Node.js Instrumentation

The instrumentation setup performs the following steps:

1. **Service Configuration**: Sets up trace provider with your application's name as service name
2. **OTLP Export**: Configures direct export to Last9 endpoint with authentication
3. **Auto-Instrumentation**: Registers automatic tracing for popular Node.js libraries
4. **Performance Optimization**: Uses batch processing and efficient span management

## What Gets Instrumented

When you use automatic instrumentation with Node.js, OpenTelemetry automatically captures:

### HTTP Operations

- **Incoming Requests**: HTTP server requests with method, URL, headers, and response codes
- **Outgoing Requests**: HTTP client requests using `http`, `https`, `axios`, `fetch`, etc.
- **Request/Response Bodies**: Configurable body capture for debugging
- **Request Timing**: Complete request lifecycle timing

### Database Operations

- **SQL Databases**: MySQL, PostgreSQL, SQLite queries and connection pooling
- **NoSQL Databases**: MongoDB, Redis operations and commands
- **ORMs**: Sequelize, TypeORM, Prisma database operations
- **Connection Management**: Database connection lifecycle and health

### File System Operations

- **File I/O**: File read/write operations (disabled by default to reduce noise)
- **Directory Operations**: Directory listing and manipulation
- **Stream Operations**: File streaming and processing

### External Services

- **API Calls**: External HTTP API interactions
- **Message Queues**: RabbitMQ, Kafka, Redis pub/sub operations
- **Cache Operations**: Redis, Memcached interactions
- **Cloud Services**: AWS SDK, Google Cloud, Azure SDK operations

## Advanced Configuration

### Custom Service Metadata

Enhance your service metadata with additional resource attributes:

```typescript
const providerConfig: TracerConfig = {
  resource: resourceFromAttributes({
    [ATTR_SERVICE_NAME]: process.env.OTEL_SERVICE_NAME,
    "service.version": "2.1.0",
    "deployment.environment": process.env.NODE_ENV,
    "service.instance.id": "api-server-1",
    team: "backend",
  }),
};
```

### Custom Spans and Attributes

Add custom spans and attributes to track business-specific operations:

```typescript
import { trace } from "@opentelemetry/api";

const tracer = trace.getTracer("my-app");

async function processUserData(userId: string) {
  const span = tracer.startSpan("process_user_data");

  try {
    span.setAttributes({
      "user.id": userId,
      "operation.type": "user_processing",
      "operation.priority": "high",
    });

    // Your business logic here
    const userData = await fetchUserData(userId);
    const processedData = await processData(userData);

    span.setStatus({ code: trace.SpanStatusCode.OK });
    return processedData;
  } catch (error) {
    span.setStatus({
      code: trace.SpanStatusCode.ERROR,
      message: error.message,
    });
    span.recordException(error);
    throw error;
  } finally {
    span.end();
  }
}
```

### Performance Monitoring

Monitor Node.js specific metrics:

```typescript
import { metrics } from "@opentelemetry/api";

const meter = metrics.getMeter("my-app");

// Create custom metrics
const requestCounter = meter.createCounter("http_requests_total", {
  description: "Total number of HTTP requests",
});

const memoryGauge = meter.createObservableGauge("nodejs_memory_usage_bytes", {
  description: "Node.js memory usage",
});

// Record memory usage
memoryGauge.addCallback((result) => {
  const memUsage = process.memoryUsage();
  result.observe(memUsage.heapUsed, { type: "heap_used" });
  result.observe(memUsage.heapTotal, { type: "heap_total" });
  result.observe(memUsage.rss, { type: "rss" });
});
```

## Verification

1. **Check Application Startup**

   When you start your Node.js application, you should see OpenTelemetry initialization messages in the console.

2. **Generate Test Traffic**

   Make some requests to your Node.js application to generate telemetry data:

   ```bash
   curl http://localhost:3000/
   curl http://localhost:3000/health
   curl http://localhost:3000/api/users
   ```

3. **Test Database Operations**

   If your app uses databases, perform some database operations to generate database spans.

4. **Verify Traces in Last9**

   Log into your Last9 account and check that traces are being received in the [Traces dashboard](https://app.last9.io/traces).

   Look for:

   - HTTP request traces with timing information
   - Database operation spans
   - External API call traces
   - Error traces for failed operations

## Troubleshooting

### Common Issues

**Instrumentation Not Working**

Ensure the instrumentation file is imported before any other modules:

```javascript
// ✅ Correct order
require("./instrumentation");
const express = require("express");

// ❌ Incorrect order
const express = require("express");
require("./instrumentation");
```

**Missing Traces for Specific Libraries**

Check if the library has specific instrumentation available:

```bash
npm list | grep @opentelemetry/instrumentation
```

**Environment Variables Not Set**

Verify your environment variables are properly configured:

```bash
env | grep OTEL_
```

**Network Connection Issues**

Test connectivity to Last9:

```bash
curl -v -H "Authorization: $last9_otlp_auth_header" $last9_otlp_endpoint
```

### Performance Considerations

- **Sampling**: Use sampling in production to control trace volume
- **Batch Processing**: BatchSpanProcessor is enabled by default for performance
- **Memory Usage**: Monitor memory usage impact of instrumentation
- **File System**: Keep file system instrumentation disabled in production to reduce noise

### Best Practices

- **Import Order**: Always import instrumentation first
- **Service Naming**: Use descriptive, consistent service names
- **Environment Configuration**: Use different service names per environment
- **Custom Spans**: Add spans for critical business operations
- **Error Handling**: Implement proper error handling with span status updates
- **Graceful Shutdown**: Handle application shutdown to flush remaining traces

## Supported Libraries

OpenTelemetry automatically instruments many popular Node.js libraries:

- **HTTP Libraries**: `http`, `https`, `axios`, `node-fetch`, `request`
- **Web Frameworks**: `express`, `koa`, `fastify`, `hapi`
- **Databases**: `mysql`, `mysql2`, `pg`, `mongodb`, `ioredis`
- **ORMs**: `sequelize`, `typeorm`, `prisma`
- **Message Queues**: `amqplib` (RabbitMQ), `kafkajs`
- **Cloud Services**: `aws-sdk`, `@google-cloud/*`, `@azure/*`

For a complete list of supported libraries, visit the [OpenTelemetry Node.js documentation](https://opentelemetry.io/docs/instrumentation/js/auto/).

Please get in touch with us on [Discord](https://discord.com/invite/Q3p2EEucx9) or [Email](mailto:support@last9.io) if you have any questions.