# Kubernetes profiling

> Install the Last9 eBPF profiler Helm chart and explore continuous profiles from Kubernetes workloads

Source: https://last9.io/docs/integrations/observability/kubernetes-profiling/

Last9 continuous profiling shows where CPU time is spent in workloads running
on Linux Kubernetes nodes. The customer-installable Helm chart runs the
upstream OpenTelemetry eBPF profiler as a DaemonSet and sends selected profiles
to Last9 over OTLP/HTTP.

## Prerequisites

- Kubernetes 1.27 or later with Linux worker nodes.
- Helm 3 and cluster-admin access to install the DaemonSet and its read-only
  workload-discovery permissions.
- Supported eBPF/BTF kernel features, `/proc`, `/sys`, and writable bpffs on
  the worker nodes.
- The profile ingestion endpoint and authorization value provided by Last9.

The profiler uses host PID visibility and explicit eBPF-related capabilities.
Review those permissions with your Kubernetes administrator before installing.

## Install the chart

Add the public chart repository and pin the supported chart version:

```bash
helm repo add last9 https://last9.github.io/helm-charts
helm repo update
mkdir -p charts
helm pull last9/last9-profiler --version 0.1.1 --destination ./charts
```

### Get the Last9 endpoint and credentials

Open [Integrations → OpenTelemetry](https://app.last9.io/integrations?integration=OpenTelemetry)
in Last9 and copy the server OTLP endpoint and Basic authorization value shown
for your organization. Use the profiles endpoint supplied for this chart; it
must end in `/v1/profiles`. If the credentials are not visible, ask an
organization admin or your Last9 contact for server ingestion credentials. Do
not use a query token or paste a credential into a tracked values file.

Create a Secret in the release namespace. The `authorization` value is passed
to the Last9 OTLP endpoint as the HTTP `Authorization` header; keep it in your
secret manager or a protected file.

```bash
kubectl create namespace last9-profiler
kubectl create secret generic last9-profiler-credentials \
  --namespace last9-profiler \
  --from-file=authorization=/secure/path/profiling-authorization
```

Create a values file and replace the endpoint with the customer-specific
HTTPS profiles endpoint:

```yaml
last9:
  ingestEndpoint: "https://REPLACE_WITH_LAST9_HOSTNAME/v1/profiles"
  authorization:
    existingSecret: last9-profiler-credentials
    existingSecretKey: authorization
deploymentEnvironment: "production"
clusterName: "REPLACE_WITH_CLUSTER_NAME"
security:
  acceptHostAccess: true
```

`deploymentEnvironment` and `clusterName` are attached to every profile as
resource attributes. Set them explicitly so the Profiles UI can filter data by
environment and cluster.

Install the pinned archive:

```bash
helm install last9-profiler ./charts/last9-profiler-0.1.1.tgz \
  --namespace last9-profiler \
  --values ./last9-profiler-values.yaml
kubectl rollout status daemonset/last9-profiler \
  --namespace last9-profiler --timeout=180s
```

The chart requires an HTTPS DNS endpoint ending in `/v1/profiles`. It does not
create an inbound Service or persist profile payloads on the node.

## Select workloads

Add the opt-in label to the pod template of each workload to profile:

```yaml
metadata:
  labels:
    last9.profiling/enabled: "true"
```

For a Deployment, the label belongs under `spec.template.metadata.labels`.
Roll out the workload after adding the label, then wait for samples to arrive.

## View profiles

Open [Profiles](https://app.last9.io/v2/organizations/last9/profiles) and
filter by cluster, namespace, service, environment, and profile type. Select a
profile to inspect the flamegraph and call-stack details.

## Flamegraph names

The chart transports profile data; it does not add executable symbols. Readable
function names depend on the profiled runtime and symbols available to the
upstream eBPF profiler. Stripped or minimal images, including shell-only
BusyBox workloads, can show `[unknown]` frames even when collection and
ingestion are healthy.

To validate readable flamegraphs, use a symbolizable Go, Java, or similar
application rather than a shell CPU-burn fixture. Unknown historical frames
are not re-symbolized after ingestion.

The OpenTelemetry Demo includes workloads built with different runtimes and
image layouts. Some processes may therefore appear as `[unknown]` even when
the profiler is collecting and profiles are arriving. Confirm ingestion with
a known symbolizable workload before treating unknown names as an installation
failure.

## Upgrade and uninstall

Keep the chart version pinned and upgrade only to a newer supported release:

```bash
helm upgrade last9-profiler ./charts/last9-profiler-0.1.1.tgz \
  --namespace last9-profiler \
  --values ./last9-profiler-values.yaml
```

To remove the profiler, uninstall the release. The customer-owned credentials
Secret is not chart-managed and should be deleted separately if no longer
needed:

```bash
helm uninstall last9-profiler --namespace last9-profiler
kubectl delete secret last9-profiler-credentials --namespace last9-profiler
```