Google Compute Engine (GCE)

Use Last9’s OpenTelemetry endpoint to ingest logs and host metrics from GCE instances using Otel Collector.

Prerequisites

Before setting up Google Compute Engine monitoring, ensure you have:

• Google Compute Engine instances running
• Administrative access to your GCE instances
• OpenTelemetry Collector installation access
• Last9 account with integration credentials

1. Install OpenTelemetry Collector

   There are multiple ways to install the Otel Collector. Every Collector release includes APK, DEB and RPM packaging for Linux amd64/arm64/i386 systems.

   Note: systemd is required for automatic service configuration.

   • RPM Package
   • DEB Package

   Installing rpm package:

   wget https://github.com/open-telemetry/opentelemetry-collector-releases/releases/download/v0.118.0/otelcol-contrib_0.118.0_linux_amd64.rpm
   sudo rpm -ivh otelcol-contrib_0.118.0_linux_amd64.rpm

   More installation options can be found here.

   Note: We recommend installing otel-collector-contrib version 0.118.0.

2. Configure OpenTelemetry Collector

   The default path for otel config is /etc/otelcol-contrib/config.yaml.

   You can edit it and update it with below configuration. The configuration is annotated with comments which should be addressed before applying the configuration.

   The configuration provides sample examples of both JSON and regex parsers.

   receivers:
     hostmetrics:
       collection_interval: 60s
       scrapers:
         cpu:
           metrics:
             system.cpu.logical.count:
               enabled: true
         memory:
           metrics:
             system.memory.utilization:
               enabled: true
             system.memory.limit:
               enabled: true
         load:
         disk:
         filesystem:
           metrics:
             system.filesystem.utilization:
               enabled: true
         network:
         paging:
         processes:
         process:
           mute_process_user_error: true
           mute_process_io_error: true
           mute_process_exe_error: true
           metrics:
             process.cpu.utilization:
               enabled: true
             process.memory.utilization:
               enabled: true
             process.threads:
               enabled: true
             process.paging.faults:
               enabled: true
     otlp:
       protocols:
         grpc:
           endpoint: 0.0.0.0:4317
         http:
           endpoint: 0.0.0.0:4318
   
     # Detailed configuration options can be found at https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/filelogreceiver
     filelog:
       # File path pattern to read logs from. Update this to the destination from where you want to read logs.
       include: [/tmp/*.log]
       include_file_path: true
       # attributes:
       # A map of key: value pairs to add to the entry's attributes.
       # resource:
       # A map of key: value pairs to add to the entry's resource.
       retry_on_failure:
         enabled: true
   
   processors:
     batch:
       timeout: 20s
       send_batch_size: 100000
       send_batch_max_size: 100000
     resourcedetection/gcp:
       detectors: ["gcp"]
     transform/logs:
       error_mode: ignore
       log_statements:
         - context: resource
           statements:
             - set(attributes["service.name"], "my-service") # Change this to your service name
     resourcedetection/system:
       detectors: ["system"]
       system:
         hostname_sources: ["os"]
   
   exporters:
     debug:
       verbosity: detailed
     otlp/last9:
       endpoint: "{{ .Logs.WriteURL }}"
       headers:
         "Authorization": "{{ .Logs.AuthValue }}"
   
   service:
     pipelines:
       logs:
         receivers: [filelog]
         processors: [resourcedetection/gcp, transform/logs, batch]
         exporters: [otlp/last9]
       metrics:
         receivers: [hostmetrics]
         processors: [resourcedetection/system, batch]
         exporters: [otlp/last9]

3. Start OpenTelemetry Collector

   Run the otel collector using systemctl command:

   sudo systemctl start otelcol-contrib
   sudo systemctl status otelcol-contrib
   sudo systemctl restart otelcol-contrib

4. Check Collector Logs

   Monitor the collector logs to ensure it’s working correctly:

   sudo journalctl -u otelcol-contrib -f

Verification

To verify the setup is working:

1. Check that the collector service is running
2. Review the collector logs for any errors
3. Log into your Last9 account to confirm GCE metrics and logs are being received

Need Help?

If you encounter any issues or have questions:

• Join our Discord community for real-time support
• Contact our support team at support@last9.io