Fluent Bit Helm Chart: Simplify Log Collection in Kubernetes

Collecting logs in Kubernetes often starts as a simple goal, and quickly turns into a game of “where did that log line go?” Between sidecars, DaemonSets, and countless config options, it’s easy to get lost.

Fluent Bit helps cut through the noise. It's fast, lightweight, and plays well with Kubernetes. And when you deploy it using Helm charts? The setup becomes way more manageable.

This guide covers the how and the why, without overcomplicating the what.

Why Fluent Bit Works So Well in Kubernetes

Fluent Bit is purpose-built for cloud-native environments. It’s not just another log shipper—it’s lean, fast, and built to handle the noisy, high-throughput world of Kubernetes.

Compared to its bulkier sibling Fluentd, Fluent Bit has a smaller memory footprint and faster startup time. That matters when you're running it as a DaemonSet across every node in your cluster.

Here’s why it’s a great choice:

• Resource-efficient: Low CPU and memory usage means it won’t fight your workloads for resources.
• Built for scale: Handles large volumes of logs without becoming a bottleneck.
• DaemonSet-friendly: Each node gets its Fluent Bit instance, so logs stay local until they’re shipped, reducing network load and increasing resilience.

How to Install and Configure the Fluent Bit Helm Chart

Pre-Installation Requirements

Before you hit install, double-check the following prerequisites to avoid errors later:

• Kubernetes version 1.20 or newer
  Make sure Beta APIs are enabled—Fluent Bit relies on them for certain features.
• Helm 3.0 or higher
  Helm 2 is outdated and no longer supported for most modern charts.
• RBAC permissions
  You’ll need permissions to create DaemonSets, service accounts, and other cluster-level resources. If you’re not a cluster admin, this is worth checking.
• Node capacity
  Fluent Bit is lightweight, but it still needs a bit of headroom to run. Ensure your nodes aren’t already maxed out.

Installing Fluent Bit Using the Helm Chart

Getting Fluent Bit up and running with Helm is relatively smooth, even if you’re new to Helm or logging pipelines. This setup uses Fluent Bit as a DaemonSet to collect logs from every node in your Kubernetes cluster.

Here’s how to get started.

Step 1: Add the Fluent Bit Helm Repo

First, add the official Fluent Bit Helm chart repository and update your local cache:

helm repo add fluent https://fluent.github.io/helm-charts
helm repo update

This gives you access to the latest chart versions, maintained by the Fluent Bit team.

Step 2: Install the Chart (Basic Setup)

To test things out, you can deploy Fluent Bit with its default configuration. It’ll collect logs and forward them to stdout, which is useful for validation before setting up a real log destination:

helm install fluent-bit fluent/fluent-bit

Once installed, you’ll see a Fluent Bit pod running on each node in your cluster—quietly collecting logs and piping them to your configured output (stdout by default).

Special Note for Kubernetes 1.25+

Kubernetes deprecated PodSecurityPolicies in version 1.25. If you're running on 1.25+ and see PSP-related errors during install, disable this feature in the chart’s values.yaml:

podSecurityPolicy:
  enabled: false

You can also pass it as a CLI override:

helm install fluent-bit fluent/fluent-bit --set podSecurityPolicy.enabled=false

Advanced Configuration with the Fluent Bit Helm Chart

Once Fluent Bit is up and running, the default config will only get you so far. Real workloads need routing, filtering, and parsing, tailored to how your teams debug or monitor things.

This is where the Helm chart shines. It’s flexible enough to support complex setups while staying relatively straightforward.

Step 1: Override Default Settings

To customize Fluent Bit, create a values.yaml file. This is where you define things like log destinations, filters, and parsers.

Here’s a simple example that sends all logs to Elasticsearch:

# values.yaml
config:
  outputs: |
    [OUTPUT]
        Name es
        Match *
        Host elasticsearch.logging.svc.cluster.local
        Port 9200
        Index fluent-bit
        Type _doc

Then install Fluent Bit using your config:

helm install fluent-bit fluent/fluent-bit -f values.yaml

Step 2: Send Logs to Multiple Destinations

You’re not limited to a single output. It’s common to route logs to different backends based on log type.

Say you want to:

• Ship app logs to Elasticsearch
• Forward system logs to Fluentd

Here’s how your values.yaml might look:

config:
  outputs: |
    [OUTPUT]
        Name es
        Match app.*
        Host elasticsearch.logging.svc.cluster.local
        Port 9200
        Index application-logs

    [OUTPUT]
        Name forward
        Match system.*
        Host fluentd.logging.svc.cluster.local
        Port 24224

The Match directive helps route logs based on tag patterns—usually tied to container names or namespaces.

Step 3: Add Metadata to Your Logs

If you’re trying to group logs by service, app, or team, you’ll want to enrich them with metadata. Fluent Bit makes this easy.

Option 1: Dynamic Metadata (Recommended)

Pull metadata straight from Kubernetes. This keeps things flexible and works well in multi-tenant clusters:

config:
  filters: |
    [FILTER]
        Name record_modifier
        Match *
        Record app_name ${kubernetes.namespace_name}
        Record sub_system ${kubernetes.container_name}

Option 2: Static Metadata

For smaller setups or simpler use cases, hardcoding works too:

config:
  filters: |
    [FILTER]
        Name record_modifier
        Match *
        Record app_name MyApplication
        Record sub_system MySubsystem

You can also combine both static values will override dynamic ones when specified. This gives you control without locking you into one style.

Log Filtering and Parsing with Fluent Bit

Logs don’t always show up in a neat, structured format, especially in Kubernetes. They can be messy, noisy, and hard to correlate without context. Fluent Bit gives you a full toolkit to clean them up, enrich them, and route them where they need to go.

Let’s walk through how to parse structured logs, add metadata, and handle tricky edge cases.

Parsing JSON Logs

If your app is already outputting JSON logs (good call), you can configure Fluent Bit to parse them automatically. Here's how to set it up in your values.yaml:

config:
  filters: |
    [FILTER]
        Name parser
        Match app.*
        Key_Name log
        Parser json
        Reserve_Data true
        
  parsers: |
    [PARSER]
        Name json
        Format json
        Time_Key timestamp
        Time_Format %Y-%m-%dT%H:%M:%S.%L

• Match app.*: Targets logs from containers with the app.* tag.
• Key_Name log: Tells Fluent Bit which field to parse as JSON.
• Reserve_Data true: Keeps the original log alongside the parsed fields.

This is especially useful when you want to extract structured fields for search or dashboards.

Enriching Logs with Kubernetes Metadata

Fluent Bit’s Kubernetes filter pulls in pod-level metadata—namespaces, pod names, labels, and more. This makes your logs usable when debugging.

Here’s how you wire it in:

config:
  filters: |
    [FILTER]
        Name kubernetes
        Match kube.*
        Kube_URL https://kubernetes.default.svc:443
        Kube_CA_File /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
        Kube_Token_File /var/run/secrets/kubernetes.io/serviceaccount/token
        Merge_Log On
        K8S-Logging.Parser On
        K8S-Logging.Exclude Off

• Merge_Log On: Merges container log with Kubernetes metadata.
• K8S-Logging.Parser On: Triggers the parser on logs marked as Kubernetes-formatted JSON.
• Match kube.*: Adjust the tag pattern to match your environment.

This makes it way easier to query logs by pod, namespace, or container name down the line.

Handling Kubernetes Labels with Special Characters

Kubernetes labels sometimes include characters like dots or dashes, which require special handling when accessed in Fluent Bit.

Here’s how to do it safely using bracket notation:

config:
  filters: |
    [FILTER]
        Name record_modifier
        Match *
        Record app_name ${kubernetes.labels["k8s-app"]}
        Record version ${kubernetes.labels["app.kubernetes.io/version"]}

Bracketed access ensures Fluent Bit parses labels with characters like . or / correctly—so you don't end up with null in your logs or a broken pipeline.

Managing Fluent Bit Resources and Tuning for Performance

Fluent Bit is built to be lightweight, but like any good tool, it needs the right resources to run smoothly, especially under load.

The Helm chart makes it easy to define resource limits and requests, so Fluent Bit doesn’t starve or overrun your nodes.

Setting Resource Limits

Here’s a baseline configuration that works well for most moderate workloads:

resources:
  limits:
    cpu: 200m
    memory: 200Mi
  requests:
    cpu: 100m
    memory: 128Mi

• requests ensure the scheduler allocates enough CPU and memory.
• limits cap Fluent Bit’s usage, preventing noisy neighbors from hogging everything.

If you’re running in a high-traffic environment—think log-heavy apps or dozens of pods per node—these defaults might be too conservative.

Tuning Fluent Bit for Higher Throughput

When things scale up, you’ll want to look beyond just CPU and memory. Buffering, flush intervals, and worker threads all impact how Fluent Bit performs under load.

Here’s an example of a more performance-tuned setup in your values.yaml:

config:
  service: |
    [SERVICE]
        Flush        5
        Daemon       off
        Log_Level    info
        HTTP_Server  On
        HTTP_Listen  0.0.0.0
        HTTP_Port    2020
        Workers      2

• Flush: Controls how often data is pushed to the output (in seconds).
• Workers: Spawns additional threads to process log streams in parallel.
• HTTP_Server: Enables a built-in endpoint for monitoring Fluent Bit’s health and metrics (accessible on port 2020).

This combo gives you more throughput, better responsiveness, and observability into how Fluent Bit is behaving under the hood.

Troubleshooting Fluent Bit in Kubernetes

Even with a stable Helm setup, Fluent Bit can occasionally run into hiccups—logs don’t show up, memory spikes, or certain nodes behave oddly. Here's how to diagnose and resolve the most common issues you’re likely to encounter.

Logs Are Missing or Not Reaching the Destination

If logs aren’t appearing where you expect them to, start with the basics.

Check Pod Deployment Across Nodes

Ensure that Fluent Bit DaemonSet pods are running on all nodes:

kubectl get pods -n fluent-bit -o wide

Fluent Bit should have a pod per node. If any are missing or stuck in CrashLoopBackOff, check node taints, tolerations, or pod scheduling constraints.

Inspect Fluent Bit Pod Logs

Next, look inside the pod logs for errors around input parsing, output connections, or general plugin failures:

kubectl logs -n fluent-bit fluent-bit-xxxxx

Look out for:

• Tag mismatches (e.g. Match value not matching your log source)
• Output plugin errors (e.g. failed to reach Elasticsearch, Loki, etc.)
• Parser errors that silently drop logs

Memory Usage Is Too High

If Fluent Bit starts hogging memory, it usually points to buffer configurations holding too much data in memory, or flush intervals that are too aggressive.

Tweak Buffer Size and Flush Settings

Try reducing how much log data is buffered before being sent, or increase the flush interval:

config:
  service: |
    [SERVICE]
        Flush 30
        
  outputs: |
    [OUTPUT]
        Name es
        Match *
        Buffer_Size 32KB

This helps Fluent Bit process logs in smaller batches, keeping memory use predictable even under load.

Metadata Isn’t Appearing Due to Permission Issues

When Kubernetes metadata doesn’t show up in logs, it’s often due to RBAC misconfiguration.

Confirm RBAC Role Bindings

The Helm chart typically sets this up for you, but it's worth double-checking if you're using strict security policies:

kubectl describe clusterrolebinding fluent-bit

Look for errors or missing roles tied to the Fluent Bit service account, especially around access to pods, namespaces, and logs.

Issues on Windows Nodes

Running Fluent Bit on Windows nodes introduces quirks, especially with DNS. Windows pods often don’t have DNS fully initialized when Fluent Bit starts, causing failures in resolving Kubernetes service endpoints.

Configure DNS Retry Behavior

Fluent Bit provides settings to deal with slow DNS initialization:

config:
  service: |
    [SERVICE]
        DNS_Mode         UDP
        DNS_Resolver     LEGACY
        DNS_Prefer_IPv4  On
        DNS_Retries      6
        DNS_Wait_Time    30

These retries help Fluent Bit wait out the initial DNS startup lag, especially on hybrid clusters with mixed OS nodes.

Handling Log Retention and Fluent Bit’s Position Database

By default, Fluent Bit tracks read positions in a local database file, preventing reprocessing logs from the beginning on pod restarts. But over time, this DB can grow or lead to stale data if not cleaned up.

Tune the Position Tracking Configuration

Here’s what to look for in your input configuration:

config:
  inputs: |
    [INPUT]
        Name              tail
        Path              /var/log/containers/*.log
        DB                /var/lib/fluent-bit/flb_kube.db
        DB.Sync           Normal
        Refresh_Interval  10

If needed, rotate or clean the DB periodically, especially in stateful clusters or when running Fluent Bit for long durations.

Securing and Maintaining Fluent Bit in Production

Once Fluent Bit is live in a production cluster, it’s time to think beyond just functionality. Securing your logging pipeline and keeping it updated are critical to preventing accidental data leaks or unexpected failures.

Enforcing Network Access Controls

Fluent Bit often handles sensitive log data, so it’s important to control who can talk to it, especially if the HTTP server is exposed for metrics or health checks.

You can enforce network access using Kubernetes NetworkPolicy:

networkPolicy:
  enabled: true
  ingress:
    - from:
        - podSelector:
            matchLabels:
              app: monitoring
      ports:
        - port: 2020
          protocol: TCP

This restricts traffic to only monitoring tools (like Prometheus or Grafana agents) accessing port 2020. Adjust labels and port ranges based on your environment.

Managing Secrets Safely

Avoid hardcoding credentials like API keys, Elasticsearch passwords, or tokens. Instead, mount them securely using Kubernetes Secrets:

env:
  - name: ES_PASSWORD
    valueFrom:
      secretKeyRef:
        name: elasticsearch-credentials
        key: password

This allows you to manage sensitive values externally, rotate them as needed, and ensure they’re not exposed in logs or configuration files.

Keeping Fluent Bit Updated

Like any critical component, keeping Fluent Bit up to date reduces the risk of known vulnerabilities and gives you access to performance improvements and bug fixes.

Updating with Helm is straightforward:

helm repo update
helm upgrade fluent-bit fluent/fluent-bit -f values.yaml

However, always test in staging first. New versions can introduce:

• Changes in configuration syntax
• Plugin behavior adjustments
• Breaking defaults for service or parser configs

Release notes are your friend—check them before any version bump, especially in production clusters.

Performance Optimization for Fluent Bit in Kubernetes

Fluent Bit is fast by default, but that doesn’t mean it's optimized for your workload out of the box. Whether you're processing verbose app logs or minimal system logs, tuning a few settings can make a real difference, especially as log volume grows.

Tune Buffer Sizes to Match Log Volume

Buffers control how much data Fluent Bit holds in memory before forwarding. If your services are noisy (lots of log lines per second), increasing the buffer size can reduce disk or network churn.

[OUTPUT]
  Buffer_Size 64KB  # Default is often 32KB

Tradeoff: Larger buffers improve I/O efficiency, but they increase memory usage. Start small, then adjust upward based on observed memory pressure.

Balance Flush Intervals for Latency vs Throughput

The Flush setting defines how often Fluent Bit sends logs to your output.

[SERVICE]
  Flush 5  # Flush logs every 5 seconds

• Shorter intervals (e.g. 1–2 seconds) reduce latency but increase CPU and network usage.
• Longer intervals (e.g. 10+ seconds) are better for batch efficiency, but can delay logs.

A 5–10 second interval is a reasonable starting point for most environments.

Use Multiple Workers to Parallelize Processing

On multi-core nodes, you can run multiple Fluent Bit workers to improve throughput, especially useful when the log volume is high or plugins are CPU-intensive.

[SERVICE]
  Workers 2

Monitor your node’s CPU and memory to avoid starving other workloads. Workers multiply throughput and memory usage.

Batch Output Efficiently

Most Fluent Bit output plugins (like Elasticsearch, Loki, or HTTP) support batching logs before sending.

Check plugin-specific settings like:

[OUTPUT]
  Name es
  Match *
  Buffer_Size 64KB
  Retry_Limit False

Or:

[OUTPUT]
  Name http
  Format json
  Batch_Size 200

Tuning these can reduce backpressure and improve network utilization—just make sure the destination system can handle the batch size without choking.

Integrating Fluent Bit with Service Meshes Like Istio

Running a service mesh like Istio? You’re not just dealing with application logs anymore—you’ve also got Envoy sidecar logs, which require different handling. Fluent Bit can collect and route with minimal setup.

To keep things organized, define separate input sources using tags:

config:
  inputs: |
    [INPUT]
        Name tail
        Path /var/log/containers/*istio-proxy*.log
        Tag istio.*
        Parser cri

    [INPUT]
        Name tail
        Path /var/log/containers/*app*.log
        Tag app.*
        Parser cri

This tagging strategy lets you apply separate filters, parsers, and outputs depending on the log type—great for directing traffic-heavy sidecar logs to one system, and app logs to another.

Best Practices for Configuration Management

If you’re running multiple environments—dev, staging, prod—you’ll want to avoid copy-pasting values files or manually tweaking config maps. Helm's templating engine makes this cleaner and more maintainable.

Example values-prod.yaml:

config:
  outputs: |
    [OUTPUT]
        Name es
        Match *
        Host {{ .Values.elasticsearch.host }}
        Index logs-{{ .Values.environment }}

elasticsearch:
  host: prod-elasticsearch.company.com
environment: production

This lets you standardize the structure of your Fluent Bit config while customizing based on the deployment environment.

Monitor Fluent Bit with Prometheus and Beyond

Once Fluent Bit is up and running, the next step is keeping tabs on how it's performing. You don’t want your log shipper becoming a black box—or worse, a bottleneck.

Built-in Metrics Endpoint

Fluent Bit includes an internal HTTP server that exposes runtime metrics, which you can scrape at:

/api/v1/metrics

This gives you visibility into input/output throughput, dropped records, retries, and memory usage—essential signals for production monitoring.

Enabling Prometheus Metrics Scraping

To hook into your existing Prometheus setup, enable the Prometheus-compatible metrics input and expose the HTTP server:

config:
  service: |
    [SERVICE]
        HTTP_Server On
        HTTP_Listen 0.0.0.0
        HTTP_Port 2020

  inputs: |
    [INPUT]
        Name prometheus_scrape
        Host 127.0.0.1
        Port 2020
        Metrics_Path /api/v1/metrics/prometheus

Once this is set up, point your Prometheus scrape configs to Fluent Bit's port (2020 by default), and you’ll start collecting exporter-style metrics right away.

Using Last9 for End-to-End Observability

While Prometheus gives you raw metrics, you’ll often need a more comprehensive view, especially when high-cardinality data (like dynamic labels, container names, or HTTP paths) is involved.

That’s where Last9 helps you. Last9 is a developer-first, budget-friendly observability platform designed to work with your Fluent Bit setup, not around it. It handles high-cardinality metrics without slow queries or budget burn, and supports native OpenTelemetry and Prometheus integrations out of the box.

With Fluent Bit forwarding logs, and Prometheus (or OTel) streaming metrics and traces, Last9 helps you:

• Correlate spikes in error logs with specific trace or metric anomalies
• Query high-cardinality datasets without query slowdowns
• Avoid paying per dashboard, query, or label

Fluent Bit gets your data out. Last9 helps you make sense of it, at scale, and under budget.

FAQs

Q: Can I run Fluent Bit as a Deployment instead of a DaemonSet?

A: Yes, but it's not recommended for log collection. DaemonSets ensure every node has a Fluent Bit instance, which is necessary for complete log coverage. Deployments work better for centralized processing scenarios.

Q: How do I handle multi-line logs like Java stack traces?

A: Use the multiline parser with appropriate rules. Configure the parser in your values file and reference it in your input configuration. The key is identifying the start pattern for multi-line records.

Q: What's the difference between Fluent Bit and Fluentd?

A: Fluent Bit is lightweight and designed for edge computing and containerized environments. Fluentd is more feature-rich but uses more resources. For Kubernetes log collection, Fluent Bit is usually the better choice.

Q: How do I limit which logs Fluent Bit collects?

A: Use input filtering with path patterns or namespace selectors. You can exclude specific pods or namespaces using the Kubernetes filter's exclude settings.

Q: Can Fluent Bit send logs to multiple destinations simultaneously?

A: Absolutely. Configure multiple OUTPUT sections with different match patterns. This allows you to route different log types to different destinations based on tags or content.

Q: How do I handle log retention when pods restart?

A: Fluent Bit uses a database file to track its position in log files. This ensures it picks up where it left off after restarts. Configure the DB parameter in your input section to control this behavior.

Q: What's the best way to manage secrets like API keys?

A: Store sensitive configuration in Kubernetes secrets and reference them using environment variables in your values file. This keeps credentials secure and out of your configuration files.

Q: How do I exclude certain namespaces from log collection?

A: Use the Kubernetes filter's exclusion settings or configure input paths to skip specific directories. You can also use the grep filter to exclude logs matching certain patterns.