# CAST AI Audit Logs

> Ship CAST AI Kubernetes cost-optimization audit logs to Last9 via OpenTelemetry

Source: https://last9.io/docs/integrations/others/castai-audit-logs/

Stream CAST AI audit events — cluster connections, autoscaling decisions, configuration changes, user actions — into Last9 using CAST AI's custom OpenTelemetry Collector.

## Prerequisites

- Kubernetes cluster with `kubectl` and [Helm](https://helm.sh/) v3.14+
- CAST AI API key with **Audit Log read** scope ([CAST AI docs](https://docs.cast.ai/docs/authentication#obtaining-api-access-key))
- CAST AI cluster ID (optional; restricts events to a single cluster)
- Last9 OTLP **endpoint** and **Basic auth header** (region-specific) from [Integrations → OpenTelemetry](https://app.last9.io/integrations?integration=OpenTelemetry)

## Installation

Add the CAST AI Helm repository:

```bash
helm repo add castai-helm https://castai.github.io/helm-charts
helm repo update castai-helm
```

Save the following as `values.yaml`:

```yaml
castai:
  apiKey: ""
  clusterID: ""
  apiURL: "https://api.cast.ai"

config:
  receivers:
    castai_audit_logs:
      api:
        url: ${env:CASTAI_API_URL}
        key: ${env:CASTAI_API_KEY}
      poll_interval_sec: 30
      page_limit: 100
      storage:
        type: "persistent"
        filename: "/var/lib/otelcol/file_storage/audit_logs_poll_data.json"
      filters:
        cluster_id: ${env:CASTAI_CLUSTER_ID}

  processors:
    transform:
      error_mode: ignore
      log_statements:
        - context: log
          statements:
            - set(body, String(attributes["event"])) where IsMap(attributes["event"])
            - delete_key(attributes, "event") where IsMap(attributes)
            - set(time, observed_time)
    resource:
      attributes:
        - key: service.name
          value: castai-audit-logs
          action: upsert
    batch:
      timeout: 5s
      send_batch_size: 100

  exporters:
    otlphttp/last9:
      endpoint: "<your-last9-otlp-endpoint>"
      encoding: json
      headers:
        Authorization: "<your-last9-otlp-basic-auth>"

  extensions:
    health_check:
      endpoint: "0.0.0.0:13133"

  service:
    extensions: [health_check]
    telemetry:
      logs:
        level: info
    pipelines:
      logs:
        receivers: [castai_audit_logs]
        processors: [transform, resource, batch]
        exporters: [otlphttp/last9]
```

Install the chart:

```bash
helm install audit-logs castai-helm/castai-audit-logs-receiver \
  --namespace castai-logs --create-namespace \
  --set castai.apiKey="$CASTAI_API_KEY" \
  --set castai.clusterID="$CASTAI_CLUSTER_ID" \
  --set-string "config.exporters.otlphttp/last9.endpoint=$LAST9_OTLP_ENDPOINT" \
  --set-string "config.exporters.otlphttp/last9.headers.Authorization=$LAST9_OTLP_AUTH" \
  --values values.yaml
```

## Why the transform processor is required

The CAST AI receiver emits each audit event as an OTel `LogRecord` where the full event payload sits under `Attributes["event"]`. The `Body`, `service.name`, and `Timestamp` fields are not populated by default. Without the transform and resource processors above, records reach Last9 but are not indexed in a way that surfaces them via standard queries.

| Statement                                     | Purpose                                                                                                                                                                                                            |
| --------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `set(body, String(attributes["event"]))`      | Serialize the event payload into `LogRecord.Body` so Last9 indexes the content.                                                                                                                                    |
| `delete_key(attributes, "event")`             | Avoid duplicating the full event payload in both `Body` and `Attributes`.                                                                                                                                          |
| `set(time, observed_time)`                    | The receiver emits `LogRecord.Timestamp` as `0` (Unix epoch) for audit events, so records land outside every recent-time query window in Last9 and stay invisible. Override to the observed (ingestion) timestamp. |
| `resource` processor upsert of `service.name` | The receiver does not set `service.name`. Without it, records cannot be filtered by service.                                                                                                                       |

The `encoding: json` exporter option ensures the OTLP/HTTP payload uses the JSON variant.

## Verification

```bash
kubectl get pods -n castai-logs
kubectl logs -n castai-logs audit-logs-castai-audit-logs-receiver-0 --tail=50
```

Look for `processing new audit log` entries. Then query the [Logs Explorer](https://app.last9.io/logs) with:

```
service.name = "castai-audit-logs"
```

Trigger a new audit event with any action in the CAST AI console, such as a new node template, an autoscaler policy change, or a cluster reconnection. Confirm that the event appears in Last9 within one poll cycle.

## Uninstall

```bash
helm uninstall audit-logs -n castai-logs
kubectl delete namespace castai-logs
```

## References

- [CAST AI audit-logs-receiver on GitHub](https://github.com/castai/audit-logs-receiver)
- [CAST AI audit log documentation](https://docs.cast.ai/docs/audit-log-exporter)
- [Last9 example repository](https://github.com/last9/opentelemetry-examples/tree/main/kubernetes/castai-audit-logs)

---

## Troubleshooting

- **`invalid api access key, response code: 403`**

  The CAST AI API key is missing the Audit Log read scope. Create a new key in the CAST AI console with that scope, then run `helm upgrade --reuse-values --set castai.apiKey=<new-key>`.

- **No audit events visible in Last9**

  Verify that all four fixes are present in the config: `body` set, `time` overridden, `service.name` upserted, and exporter `encoding: json`. If one of them is missing, Last9 accepts the events (`HTTP 200 {"partialSuccess":{}}`), but the Logs Explorer cannot retrieve them.

- **Pod restarts with config unmarshal errors**

  The CAST AI audit-logs-receiver image ships a restricted set of OpenTelemetry Collector components. The supported exporters are `otlp_http` (alias `otlphttp`), `coralogix`, `datadog`, `splunk_hec`, `debug`, and `file`. The gRPC `otlp` exporter is not available. Use `otlphttp` for Last9.

Please get in touch with us on [Discord](https://discord.com/invite/Q3p2EEucx9) or [Email](mailto:support@last9.io) if you have any questions.