Skip to content
Last9 named a Gartner Cool Vendor in AI for SRE Observability for 2025! Read more →
Last9
Talk to an Expert

NGINX

Monitor NGINX web server with comprehensive metrics, access logs, and error logs using OpenTelemetry Collector

Monitor your NGINX web server and reverse proxy with comprehensive metrics and log collection using OpenTelemetry Collector. This integration provides visibility into server performance, request patterns, and error conditions.

Prerequisites

  • NGINX web server installed and running
  • OpenTelemetry Collector
  • Access to NGINX configuration files
  • Last9 account with OTLP endpoint configured

Features

  • Status Metrics: Active connections, request rates, server status
  • Performance Metrics: Request duration, response codes, upstream health
  • Access Logs: Detailed request information and patterns
  • Error Logs: Error conditions and debugging information
  • SSL/TLS Metrics: Certificate status and SSL performance
  • Upstream Monitoring: Backend server health and load balancing

Installation

  1. Install NGINX (if not already installed)

    sudo apt-get update
    sudo apt-get install -y nginx
    

    # Start and enable NGINX
    sudo systemctl start nginx
    sudo systemctl enable nginx

  2. Install OpenTelemetry Collector

    # Download and install collector
    wget https://github.com/open-telemetry/opentelemetry-collector-releases/releases/download/v0.118.0/otelcol-contrib_0.118.0_linux_amd64.deb
    sudo dpkg -i otelcol-contrib_0.118.0_linux_amd64.deb

  3. Enable NGINX Status Module

    Create /etc/nginx/conf.d/status.conf:

    server {
        listen 8080;
        server_name localhost;
    

        # Status endpoint for metrics
        location /nginx_status {
            stub_status on;
            access_log off;
            allow 127.0.0.1;
            allow ::1;
            # Add your collector's IP if running remotely
            # allow 10.0.0.100;
            deny all;
        }
    

        # Health check endpoint
        location /health {
            access_log off;
            return 200 "healthy\n";
            add_header Content-Type text/plain;
        }
    }

    Verify stub_status module is available:

    nginx -V 2>&1 | grep -o with-http_stub_status_module

  4. Configure NGINX Logging

    Update your main NGINX configuration (/etc/nginx/nginx.conf):

    http {
        # Enhanced logging format for better observability
        log_format json_combined escape=json '{'
            '"timestamp": "$time_iso8601", '
            '"remote_addr": "$remote_addr", '
            '"remote_user": "$remote_user", '
            '"request": "$request", '
            '"status": $status, '
            '"body_bytes_sent": $body_bytes_sent, '
            '"request_time": $request_time, '
            '"upstream_response_time": "$upstream_response_time", '
            '"upstream_addr": "$upstream_addr", '
            '"http_referrer": "$http_referer", '
            '"http_user_agent": "$http_user_agent", '
            '"http_x_forwarded_for": "$http_x_forwarded_for", '
            '"host": "$host", '
            '"request_method": "$request_method", '
            '"request_uri": "$request_uri", '
            '"server_protocol": "$server_protocol", '
            '"ssl_protocol": "$ssl_protocol", '
            '"ssl_cipher": "$ssl_cipher"'
        '}';
    

        # Access logs with JSON format
        access_log /var/log/nginx/access.log json_combined;
    

        # Error logs
        error_log /var/log/nginx/error.log warn;
    

        # Your server configurations...
        include /etc/nginx/conf.d/*.conf;
        include /etc/nginx/sites-enabled/*;
    }

  5. Test and Reload NGINX

    # Test configuration
    sudo nginx -t
    

    # Reload NGINX
    sudo systemctl reload nginx
    

    # Verify status endpoint
    curl http://localhost:8080/nginx_status

OpenTelemetry Collector Configuration

Create /etc/otelcol-contrib/config.yaml:

receivers:
  # NGINX metrics via stub_status
  nginx:
    endpoint: "http://localhost:8080/nginx_status"
    collection_interval: 30s


  # NGINX access logs
  filelog/nginx_access:
    include:
      - /var/log/nginx/access.log
    operators:
      - type: json_parser
        timestamp:
          parse_from: attributes.timestamp
          layout: "2006-01-02T15:04:05-07:00"
      - type: move
        from: attributes.remote_addr
        to: attributes["client.ip"]
      - type: move
        from: attributes.request_method
        to: attributes["http.method"]
      - type: move
        from: attributes.status
        to: attributes["http.status_code"]
      - type: move
        from: attributes.request_time
        to: attributes["http.request_duration"]
      - type: move
        from: attributes.body_bytes_sent
        to: attributes["http.response_size"]
      - type: add
        field: attributes["log.source"]
        value: "nginx.access"
      - type: add
        field: attributes["service.name"]
        value: "nginx"


  # NGINX error logs
  filelog/nginx_error:
    include:
      - /var/log/nginx/error.log
    operators:
      - type: regex_parser
        regex: '^(?P<timestamp>\d{4}/\d{2}/\d{2} \d{2}:\d{2}:\d{2}) \[(?P<level>\w+)\] (?P<pid>\d+)#(?P<tid>\d+): \*(?P<connection_id>\d+) (?P<message>.*)'
        timestamp:
          parse_from: attributes.timestamp
          layout: "2006/01/02 15:04:05"
      - type: severity_parser
        parse_from: attributes.level
        mapping:
          debug: debug
          info: info
          notice: info2
          warn: warn
          error: error
          crit: fatal
          alert: fatal
          emerg: fatal
      - type: add
        field: attributes["log.source"]
        value: "nginx.error"
      - type: add
        field: attributes["service.name"]
        value: "nginx"


processors:
  # Batch processing for efficiency
  batch:
    timeout: 10s
    send_batch_size: 1024
    send_batch_max_size: 2048


  # Memory limiter
  memory_limiter:
    limit_mib: 256
    spike_limit_mib: 64


  # Resource detection
  resourcedetection/system:
    detectors: ["system", "env"]
    timeout: 5s
    system:
      hostname_sources: ["os"]


  # Add resource attributes
  resource:
    attributes:
      - key: service.name
        value: nginx
        action: upsert
      - key: service.version
        from_attribute: nginx.version
        action: insert
      - key: deployment.environment
        value: production
        action: insert


  # Transform metrics to add useful attributes
  transform/nginx:
    metric_statements:
      - context: datapoint
        statements:
          - set(attributes["server.type"], "nginx")
          - set(attributes["monitoring.tool"], "opentelemetry")


  # Filter out health checks from access logs
  filter/health_checks:
    logs:
      exclude:
        match_type: regexp
        bodies:
          - '.*"request_uri": "/health".*'
          - '.*"request_uri": "/nginx_status".*'


exporters:
  # Last9 OTLP exporter
  otlp/last9:
    endpoint: $last9_otlp_endpoint
    headers:
      Authorization: $last9_otlp_auth_header
    compression: gzip
    retry_on_failure:
      enabled: true
      initial_interval: 1s
      max_interval: 30s
      max_elapsed_time: 300s


  # Debug exporter for troubleshooting
  debug:
    verbosity: basic
    sampling_initial: 5
    sampling_thereafter: 200


extensions:
  health_check:
    endpoint: 0.0.0.0:13133


service:
  extensions: [health_check]
  pipelines:
    # Metrics pipeline
    metrics:
      receivers: [nginx]
      processors:
        [
          memory_limiter,
          resourcedetection/system,
          resource,
          transform/nginx,
          batch,
        ]
      exporters: [otlp/last9]


    # Logs pipeline
    logs:
      receivers: [filelog/nginx_access, filelog/nginx_error]
      processors:
        [
          memory_limiter,
          resourcedetection/system,
          resource,
          filter/health_checks,
          batch,
        ]
      exporters: [otlp/last9]

Docker Deployment

For containerized deployments:

# docker-compose.yml
version: "3.8"


services:
  nginx:
    image: nginx:latest
    ports:
      - "80:80"
      - "443:443"
      - "8080:8080" # Status endpoint
    volumes:
      - ./nginx.conf:/etc/nginx/nginx.conf:ro
      - ./conf.d:/etc/nginx/conf.d:ro
      - ./logs:/var/log/nginx
      - ./ssl:/etc/nginx/ssl:ro
    networks:
      - web
    labels:
      - "monitoring.service=nginx"


  otel-collector:
    image: otel/opentelemetry-collector-contrib:0.118.0
    command: ["--config=/etc/otel-collector-config.yaml"]
    volumes:
      - ./otel-config.yaml:/etc/otel-collector-config.yaml:ro
      - ./logs:/var/log/nginx:ro
    environment:
      - LAST9_OTLP_ENDPOINT=$last9_otlp_endpoint
      - LAST9_OTLP_AUTH_HEADER=$last9_otlp_auth_header
    networks:
      - web
    depends_on:
      - nginx


networks:
  web:
    name: web

Advanced Monitoring

SSL/TLS Monitoring

Add SSL monitoring to your NGINX configuration:

server {
    listen 443 ssl http2;
    server_name your-domain.com;


    ssl_certificate /etc/nginx/ssl/certificate.crt;
    ssl_certificate_key /etc/nginx/ssl/private.key;


    # SSL session information in logs
    log_format ssl_extended '$remote_addr - $remote_user [$time_local] '
                          '"$request" $status $body_bytes_sent '
                          '"$http_referer" "$http_user_agent" '
                          'ssl_protocol="$ssl_protocol" ssl_cipher="$ssl_cipher"';


    access_log /var/log/nginx/ssl_access.log ssl_extended;


    location / {
        proxy_pass http://backend;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
    }
}

Upstream Health Monitoring

Configure upstream monitoring:

upstream backend {
    least_conn;
    server backend1.example.com:8080 max_fails=3 fail_timeout=30s;
    server backend2.example.com:8080 max_fails=3 fail_timeout=30s;
    server backend3.example.com:8080 max_fails=3 fail_timeout=30s backup;
}


server {
    listen 80;


    location / {
        proxy_pass http://backend;
        proxy_next_upstream error timeout http_502 http_503 http_504;
        proxy_set_header Host $host;
        proxy_set_header X-Upstream-Response-Time $upstream_response_time;
    }
}

Rate Limiting Monitoring

http {
    # Rate limiting zones
    limit_req_zone $binary_remote_addr zone=login:10m rate=5r/m;
    limit_req_zone $binary_remote_addr zone=api:10m rate=100r/m;


    server {
        location /api/ {
            limit_req zone=api burst=20 nodelay;
            limit_req_status 429;


            # Log rate limit events
            access_log /var/log/nginx/rate_limit.log json_combined;


            proxy_pass http://api_backend;
        }
    }
}

Custom Metrics

Add custom metric collection using NGINX Plus or third-party modules:

# Add to otel-config.yaml
receivers:
  prometheus/nginx:
    config:
      scrape_configs:
        - job_name: "nginx-vts"
          static_configs:
            - targets: ["localhost:9113"]
          scrape_interval: 30s


  httpcheck/nginx:
    targets:
      - endpoint: http://localhost/health
        method: GET
    collection_interval: 60s

Alerting and Dashboards

Example metrics to monitor:

  • nginx_connections_active: Active client connections
  • nginx_connections_reading: Connections reading request headers
  • nginx_connections_writing: Connections writing responses
  • nginx_connections_waiting: Idle connections waiting for requests
  • nginx_requests_total: Total client requests
  • HTTP status codes: 2xx, 3xx, 4xx, 5xx response patterns
  • Request duration: Response time percentiles
  • Upstream response time: Backend performance

Troubleshooting

Common Issues

  1. No metrics appearing:

    • Verify stub_status module is enabled: nginx -V 2>&1 | grep stub_status
    • Check status endpoint accessibility: curl localhost:8080/nginx_status
    • Ensure collector has network access to NGINX

  2. Missing logs:

    • Verify log file permissions: ls -la /var/log/nginx/
    • Check log format is valid JSON
    • Ensure collector can read log files

  3. High resource usage:

    • Reduce collection frequency
    • Filter out high-volume endpoints
    • Optimize log format

Debug Commands

# Check NGINX configuration
nginx -t


# Test status endpoint
curl -s http://localhost:8080/nginx_status


# Monitor logs in real-time
tail -f /var/log/nginx/access.log


# Check collector health
curl http://localhost:13133

Performance Optimization

# Optimize NGINX for monitoring
http {
    # Reduce log I/O
    access_log /var/log/nginx/access.log json_combined buffer=32k flush=5s;


    # Optimize worker processes
    worker_processes auto;
    worker_connections 1024;


    # Enable gzip compression
    gzip on;
    gzip_types text/plain application/json;
}

Your NGINX infrastructure will now provide comprehensive observability data to Last9, enabling detailed monitoring of web server performance, request patterns, and error conditions.