Skip to content
Last9 named a Gartner Cool Vendor in AI for SRE Observability for 2025! Read more →
Last9
Talk to an Expert

OpenTelemetry Operator

Deploy OpenTelemetry Operator for automatic instrumentation and observability in Kubernetes clusters

Deploy OpenTelemetry Operator, Collector, and Cluster Monitoring in your Kubernetes environment using our automated setup script. This integration provides automatic instrumentation for applications and comprehensive observability for your cluster.

Prerequisites

Before setting up the OpenTelemetry Operator, ensure you have:

  • ✅ Kubernetes Cluster: A running Kubernetes cluster (v1.19+)
  • ✅ kubectl: Configured and connected to your cluster
  • ✅ helm: Installed (v3.9 or higher)
  • ✅ git: Installed for downloading setup scripts
  • ✅ Cluster Admin Access: Required for creating cluster-wide resources
  • ✅ Last9 Account: With integration credentials for traces, logs, and metrics

  1. Download the Setup Script

    First, download the automated setup script from the Last9 GitHub repository:

    # Download shell script directly from GitHub
    curl -O https://raw.githubusercontent.com/last9/opentelemetry-examples/main/otel-collector/otel-operator/last9-otel-setup.sh
    chmod +x last9-otel-setup.sh

  2. Choose Your Installation Option

    The setup script provides multiple installation options based on your observability needs:

    Install Everything: OpenTelemetry Operator + Collector + Cluster Monitoring

    ./last9-otel-setup.sh \
      endpoint="{{ .Logs.WriteURL }}" \
      token="{{ .Logs.AuthValue }}" \
      monitoring-endpoint="{{ .Metrics.WriteURL }}" \
      username="{{ .Metrics.Username }}" \
      password="{{ .Metrics.WriteToken }}"

    This option provides:

    • 🔍 Traces: Automatic application instrumentation via OpenTelemetry Operator
    • 📊 Metrics: Complete cluster and application metrics monitoring
    • 📝 Logs: Log collection and forwarding capabilities

    Replace the placeholder values with your actual Last9 credentials from the Last9 Integrations page.

  3. Verify Installation

    Check that all components are deployed and running correctly:

    # Check all pods in last9 namespace should be up and running
    kubectl get pods -n last9

    Depending on your installation option, you should see pods like:

    NAME                                                   READY   STATUS    RESTARTS   AGE
    opentelemetry-operator-controller-manager-xxx-xxx     2/2     Running   0          2m
    last9-otel-collector-xxx-xxx                          1/1     Running   0          2m
    prometheus-last9-k8s-monitoring-prometheus-0          2/2     Running   0          2m (if monitoring included)

  4. Enable Application Auto-Instrumentation

    To enable automatic OpenTelemetry instrumentation for your applications, add the appropriate annotation to your deployment:

    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: my-java-app
    spec:
      template:
        metadata:
          annotations:
            instrumentation.opentelemetry.io/inject-java: "last9/l9-instrumentation"
        spec:
          containers:
            - name: my-app
              image: my-java-app:latest

    After adding the annotation, restart your deployment:

    kubectl rollout restart deployment/my-app -n <your-namespace>

Advanced Configuration

Custom Environment Settings

Override the deployment.environment attribute to match your environment:

./last9-otel-setup.sh \
  endpoint="{{ .Logs.WriteURL }}" \
  token="{{ .Logs.AuthValue }}" \
  monitoring-endpoint="{{ .Metrics.WriteURL }}" \
  username="{{ .Metrics.Username }}" \
  password="{{ .Metrics.WriteToken }}" \
  env=production

Running on Nodes with Taints

If your Kubernetes nodes have taints (e.g., dedicated monitoring nodes), you need to provide tolerations:

  1. Create Tolerations Configuration

    Create a file named tolerations.yaml:

    # tolerations.yaml
    tolerations:
      - key: "monitoring"
        operator: "Equal"
        value: "true"
        effect: "NoSchedule"
    

    nodeSelector:
      monitoring: "true"
    

    nodeExporterTolerations:
      - operator: "Exists"

  2. Run Installation with Tolerations

    ./last9-otel-setup.sh \
      endpoint="{{ .Logs.WriteURL }}" \
      token="{{ .Logs.AuthValue }}" \
      monitoring-endpoint="{{ .Metrics.WriteURL }}" \
      username="{{ .Metrics.Username }}" \
      password="{{ .Metrics.WriteToken }}" \
      tolerations-file=tolerations.yaml

Understanding the Components

OpenTelemetry Operator

The operator provides:

  • Automatic Instrumentation: Injects OpenTelemetry libraries into applications
  • Collector Management: Manages OpenTelemetry Collector deployments
  • Custom Resources: Provides CRDs for OpenTelemetry configuration
  • Language Support: Supports Java, Node.js, Python, .NET, and Go applications

OpenTelemetry Collector

The collector handles:

  • Data Reception: Receives traces, metrics, and logs from applications
  • Data Processing: Filters, transforms, and enriches telemetry data
  • Data Export: Forwards processed data to Last9 endpoints
  • Service Discovery: Automatically discovers services in Kubernetes

Cluster Monitoring (Optional)

When enabled, provides:

  • Kubernetes Metrics: Cluster state, resource usage, and performance metrics
  • Node Metrics: Hardware and OS metrics from cluster nodes
  • Application Metrics: Custom application metrics and performance indicators

Verification

  1. Check Operator Status

    Verify the OpenTelemetry Operator is running:

    kubectl get pods -n opentelemetry-operator-system
    kubectl logs -n opentelemetry-operator-system deployment/opentelemetry-operator-controller-manager

  2. Verify Instrumentation Configuration

    Check the instrumentation resource:

    kubectl get instrumentation -n last9
    kubectl describe instrumentation l9-instrumentation -n last9

  3. Test Application Instrumentation

    Deploy a test application and verify traces are being generated:

    kubectl logs <your-app-pod> | grep -i opentelemetry

  4. Verify Data in Last9

    Log into your Last9 account and check that data is being received:

Uninstallation

To remove all OpenTelemetry and monitoring components:

./last9-otel-setup.sh uninstall-all

This command will:

  • Remove all OpenTelemetry components
  • Delete cluster monitoring stack
  • Clean up secrets and configurations
  • Remove the last9 namespace

Troubleshooting

Operator Not Starting

Check operator logs and resource requirements:

kubectl logs -n opentelemetry-operator-system deployment/opentelemetry-operator-controller-manager
kubectl describe pod -n opentelemetry-operator-system <operator-pod-name>

Application Not Instrumented

Verify annotation syntax and instrumentation resource:

kubectl describe pod <your-app-pod>
kubectl get instrumentation -n last9 -o yaml

Common issues:

  • Incorrect annotation name or value
  • Missing instrumentation resource
  • Unsupported application runtime

Collector Issues

Check collector configuration and logs:

kubectl logs -n last9 <collector-pod-name>
kubectl describe configmap -n last9 <collector-config>

Data Not Appearing in Last9

Verify credentials and network connectivity:

kubectl get secrets -n last9
kubectl logs -n last9 <collector-pod-name> | grep -i error

Best Practices

  • Resource Limits: Set appropriate resource limits for operator and collector components
  • Environment Labeling: Use consistent environment labels across all services
  • Instrumentation Testing: Test auto-instrumentation in development before production
  • Monitoring Stack Health: Monitor the health of OpenTelemetry components themselves
  • Regular Updates: Keep the operator and collectors updated to latest versions

Additional Resources

Need Help?

If you encounter any issues or have questions: