# How to Run Elasticsearch on Kubernetes

> Understand how to deploy, scale, and manage Elasticsearch on Kubernetes with the right configs for storage, availability, and performance.

Source: https://last9.io/blog/elasticsearch-kubernetes-deployment/

[Elasticsearch](https://www.elastic.co/elasticsearch) stands as one of the most robust open-source search engines available today. Built on Apache Lucene, it handles complex search operations, real-time analytics, and large-scale data processing with impressive speed and accuracy.

[Kubernetes](https://kubernetes.io/) has transformed how we deploy and manage containerized applications. This orchestration platform automates deployment, scaling, and operations of application containers across clusters of hosts.

When you combine Elasticsearch with Kubernetes, you get a powerful foundation for scalable search infrastructure.

This blog covers how to deploy Elasticsearch on Kubernetes using native Kubernetes constructs and Helm.

## Should You Run Elasticsearch on Kubernetes?

Running Elasticsearch on Kubernetes can simplify operations if you're already managing stateful services in the same environment. But it comes with trade-offs that need careful consideration.

### When Kubernetes Is a Good Fit

Kubernetes works well when your Elasticsearch workloads are dynamic, multi-tenant, or need to scale automatically. Some common patterns include:

- **Log aggregation**  
  Ingest logs from multiple services using agents like Fluent Bit or Fluentd. Autoscaling helps handle spikes in log volume.
- **Search applications**  
  Serve user-facing search queries—product catalogs, documentation, or structured content. Kubernetes allows you to scale read nodes based on demand.
- **Metrics and observability**  
  Use Elasticsearch as a backend for storing metrics or trace data. Integrate with Kibana or Grafana for dashboards and alerts.
- **Analytics and aggregations**  
  Run large-scale aggregation queries over time-series or semi-structured data. Horizontal scaling improves query performance under load.

### Features That Support Elasticsearch

Several Kubernetes primitives help improve Elasticsearch resilience and uptime:

- **Pod anti-affinity**  
  Spread Elasticsearch pods across nodes to avoid single points of failure.
- **Rolling updates**  
  Apply configuration changes or upgrade Elasticsearch without taking the cluster offline.
- **Liveness and readiness probes**  
  Detect and recover failed pods automatically using health checks.
- **Load balancing**  
  Kubernetes Services route traffic evenly across healthy pods to improve request distribution.

### Operational Trade-Offs

- **Increased complexity**  
  You'll need working knowledge of both Elasticsearch internals and Kubernetes resource management.
- **Storage configuration**  
  Elasticsearch requires stable, high-performance storage. Use StatefulSets with persistent volume claims and test your StorageClass for IOPS and latency.
- **Network performance**  
  Kubernetes networking adds some latency compared to bare metal. Monitor tail latency if your workload is sensitive to response times.

If you're running Elasticsearch on Kubernetes, [this guide on Elasticsearch metrics](https://last9.io/blog/elasticsearch-metrics/) helps you figure out which metrics to monitor to catch scaling issues, disk pressure, or cluster instability early.

## Elasticsearch Internals on Kubernetes

When you deploy Elasticsearch on Kubernetes, each node becomes a pod. These pods are typically managed by a StatefulSet, which gives them stable DNS identities and persistent volumes, both required for reliable cluster formation and data durability.

**Cluster Formation and Node Communication**

Elasticsearch uses internal discovery to form a cluster. With StatefulSets, each pod gets a predictable hostname (e.g., `elasticsearch-0.elasticsearch`) and its persistent disk. This allows Elasticsearch to:

- Elect master nodes
- Distribute shards across nodes
- Maintain replicas for fault tolerance

Kubernetes manages pod restarts using liveness and readiness probes. If a pod crashes or becomes unhealthy, it’s automatically rescheduled. Elasticsearch rebalances data as needed based on shard allocation rules.

**Shards, Replicas, and Node Awareness**

Each Elasticsearch index is split into shards, with replicas for redundancy. In Kubernetes, these shards are distributed across pods, which may land on different worker nodes.

To avoid having both a primary and its replica on the same physical node (and risking data loss), use shard allocation awareness:

```yaml
cluster.routing.allocation.awareness.attributes: k8s_node_name
node.attr.k8s_node_name: ${HOSTNAME}
```

This ensures Kubernetes schedules replicas across distinct nodes.

**Kibana Integration**

Kibana provides a UI to query, visualize, and debug your Elasticsearch data. You can deploy it in the same namespace using a standard `Deployment`:

```yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: kibana
  namespace: elastic-system
spec:
  replicas: 1
  selector:
    matchLabels:
      app: kibana
  template:
    metadata:
      labels:
        app: kibana
    spec:
      containers:
        - name: kibana
          image: docker.elastic.co/kibana/kibana:8.11.0
          ports:
            - containerPort: 5601
          env:
            - name: ELASTICSEARCH_HOSTS
              value: "http://elasticsearch:9200"
```

Once deployed, Kibana connects to the cluster and gives you dashboards, log views, and access to query tools.

Running Elasticsearch in Kubernetes often means dealing with version bumps or index schema updates, [this guide on the Reindex API](https://last9.io/blog/elasticsearch-reindex-api/) covers how to handle them without data loss.

## Kubernetes Requirements for Running Elasticsearch

A stable and performant Elasticsearch deployment on Kubernetes depends on the correct use of StatefulSets, storage, namespaces, and resource constraints.

### Use StatefulSets and Persistent Volumes

Elasticsearch pods must retain their identity and storage across restarts. Use a `StatefulSet` to ensure each pod gets:

- A stable network name (`elasticsearch-0`, `elasticsearch-1`, etc.)
- A dedicated volume that persists when the pod is rescheduled

Provision volumes using a `StorageClass` with high IOPS and support for dynamic provisioning:

```yaml
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: fast-ssd
provisioner: kubernetes.io/aws-ebs
parameters:
  type: gp3
  iops: "3000"
  throughput: "125"
volumeBindingMode: WaitForFirstConsumer
allowVolumeExpansion: true
```

Each Elasticsearch pod will claim its own `PersistentVolumeClaim` based on this class.

### Isolate Resources with Namespaces and RBAC

Use a dedicated namespace (e.g., `elastic-system`) to group all Elasticsearch resources. This makes it easier to manage access, apply limits, and avoid naming collisions.

Example namespace and minimal RBAC:

```yaml
apiVersion: v1
kind: Namespace
metadata:
  name: elastic-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: elasticsearch
rules:
  - apiGroups: [""]
    resources: ["pods", "services", "endpoints"]
    verbs: ["get", "list", "watch"]
```

Bind the role to a service account as needed based on your deployment model (Helm, ECK, or custom controllers).

### Resource Sizing and Performance Tuning

Elasticsearch is sensitive to CPU, memory, and disk throughput. Start with conservative resource limits and tune based on actual workload characteristics.

#### CPU

- Start with 1–2 cores per data node.
- Increase if you see slow query execution or heavy indexing load.

#### Memory

- Allocate 50% of the pod memory to the JVM heap.
- Leave the rest for the file system cache (used by Lucene).

A typical small node starts with:

```yaml
resources:
  requests:
    memory: "4Gi"
    cpu: "1"
  limits:
    memory: "4Gi"
    cpu: "2"
```

#### Storage

- Use SSD-backed volumes for all data nodes.
- Plan for 1.5–2× your expected data size to account for indexing overhead, replicas, and future growth.

#### Ongoing Monitoring

Track these metrics continuously:

- JVM heap usage
- Disk I/O saturation
- Pod restarts or evictions
- Indexing and query latency

Use these signals to tune vertical limits or scale out horizontally.

Understanding how different pod types work can help you make better decisions about running stateful workloads, [this guide on types of pods in Kubernetes](https://last9.io/blog/types-of-pods-in-kubernetes/) breaks it down clearly.

## How to Deploy Elasticsearch on Kubernetes

There are several ways to deploy Elasticsearch on Kubernetes, depending on your operational model, control requirements, and experience level.

1.  Raw YAML using StatefulSets
2.  Helm charts (Elastic’s official)
3.  ECK (Elastic Cloud on Kubernetes)

### Method 1: Deploy Using Raw YAML (StatefulSet)

This method gives you the most control. It’s ideal for teams that want to tune everything from node layout to volume provisioning, without relying on a package manager.

Here’s a basic 3-node Elasticsearch cluster using a `StatefulSet`:

```yaml
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: elasticsearch
  namespace: elastic-system
spec:
  serviceName: elasticsearch
  replicas: 3
  selector:
    matchLabels:
      app: elasticsearch
  template:
    metadata:
      labels:
        app: elasticsearch
    spec:
      containers:
        - name: elasticsearch
          image: docker.elastic.co/elasticsearch/elasticsearch:8.11.0
          ports:
            - containerPort: 9200
              name: rest
            - containerPort: 9300
              name: inter-node
          env:
            - name: cluster.name
              value: k8s-logs
            - name: node.name
              valueFrom:
                fieldRef:
                  fieldPath: metadata.name
            - name: discovery.seed_hosts
              value: "elasticsearch-0.elasticsearch,elasticsearch-1.elasticsearch,elasticsearch-2.elasticsearch"
            - name: cluster.initial_master_nodes
              value: "elasticsearch-0,elasticsearch-1,elasticsearch-2"
            - name: ES_JAVA_OPTS
              value: "-Xms1g -Xmx1g"
          volumeMounts:
            - name: data
              mountPath: /usr/share/elasticsearch/data
  volumeClaimTemplates:
    - metadata:
        name: data
      spec:
        accessModes: ["ReadWriteOnce"]
        storageClassName: "fast-ssd"
        resources:
          requests:
            storage: 100Gi
```

This configuration:

- Creates 3 Elasticsearch pods with stable hostnames
- Persists data using dynamically provisioned volumes (`StorageClass` = `fast-ssd`)
- Sets up inter-node communication on port `9300`

Apply it with:

```bash
kubectl apply -f elasticsearch-statefulset.yaml
```

Expose the cluster with a headless service:

```yaml
apiVersion: v1
kind: Service
metadata:
  name: elasticsearch
  namespace: elastic-system
spec:
  selector:
    app: elasticsearch
  ports:
    - port: 9200
      targetPort: 9200
      name: rest
    - port: 9300
      targetPort: 9300
      name: inter-node
  clusterIP: None
```

This setup provides stable DNS for each pod (e.g., `elasticsearch-0.elasticsearch`), which is required for cluster discovery.

### Method 2: Deploy Using Helm

Helm simplifies the deployment process using prebuilt charts and sane defaults. Use this method if you want to get started quickly or integrate into an existing Helm-based workflow.

1.  Add the official Elastic Helm repo:

```bash
helm repo add elastic https://helm.elastic.co
helm repo update
```

2.  Install Elasticsearch:

```bash
helm install elasticsearch elastic/elasticsearch \
  --namespace elastic-system \
  --create-namespace \
  --set replicas=3 \
  --set volumeClaimTemplate.resources.requests.storage=100Gi \
  --set resources.requests.memory=2Gi \
  --set resources.limits.memory=2Gi
```

This installs a 3-node cluster with PVCs and memory limits set. You can override more values using a `values.yaml` file or with `--set` flags.

The Helm chart handles:

- StatefulSet creation
- Headless services for internal communication
- Volume provisioning
- Basic resource configs

Use `helm upgrade` for config changes and `helm uninstall` to tear it down cleanly.

### Method 3: Deploy Using ECK (Elastic Cloud on Kubernetes)

Elastic Cloud on Kubernetes (ECK) is the most automated option. It runs an operator that watches for custom Elasticsearch resources and manages the cluster lifecycle.

Install the ECK operator:

```bash
kubectl apply -f https://download.elastic.co/downloads/eck/2.10.0/crds.yaml
kubectl apply -f https://download.elastic.co/downloads/eck/2.10.0/operator.yaml
```

Then create an Elasticsearch cluster using the custom resource:

```yaml
apiVersion: elasticsearch.k8s.elastic.co/v1
kind: Elasticsearch
metadata:
  name: quickstart
  namespace: elastic-system
spec:
  version: 8.11.0
  nodeSets:
    - name: default
      count: 3
      config:
        node.store.allow_mmap: false
      podTemplate:
        spec:
          containers:
            - name: elasticsearch
              resources:
                limits:
                  memory: 2Gi
                  cpu: 1
                requests:
                  memory: 2Gi
                  cpu: 1
      volumeClaimTemplates:
        - metadata:
            name: elasticsearch-data
          spec:
            accessModes:
              - ReadWriteOnce
            storageClassName: fast-ssd
            resources:
              requests:
                storage: 100Gi
```

ECK automates:

- Cluster discovery
- TLS and cert management
- Upgrades
- Node scaling
- Monitoring setup (if using the rest of the Elastic Stack)

This is the best choice if you're standardizing around the Elastic Stack or want full lifecycle automation.

### Choosing the Right Deployment Method

| Method   | Best For                             | Control Level | Complexity | Notes                                       |
| -------- | ------------------------------------ | ------------- | ---------- | ------------------------------------------- |
| Raw YAML | Custom setups, full control          | High          | Medium     | Best for fine-tuned ops teams               |
| Helm     | Quick deployments, CI/CD integration | Medium        | Low        | Easy to install, flexible via `values.yaml` |
| ECK      | Full Elastic Stack automation        | Low           | Low        | Operator-managed, but opinionated           |

Elasticsearch performance can tank if CPU limits are misconfigured. [this guide on Kubernetes CPU throttling](https://last9.io/blog/kubernetes-cpu-throttling/) explains what to watch out for.

## Set Up Elasticsearch on Kubernetes with AWS

Running Elasticsearch on Kubernetes is production-ready on AWS, provided the infrastructure is tuned for stateful workloads. Amazon EKS simplifies Kubernetes operations, while services like EBS and NLB offer the underlying performance and availability Elasticsearch needs.

### Use Amazon EKS for Kubernetes Management

[EKS (Elastic Kubernetes Service)](https://aws.amazon.com/eks/) provides a managed control plane, reducing operational overhead for Kubernetes clusters. This is particularly helpful when running stateful workloads like Elasticsearch that require high availability and predictable node behavior.

Key advantages for Elasticsearch:

- **Managed control plane**  
  AWS handles upgrades, failover, and scaling for Kubernetes masters. You only manage worker nodes.
- **Native integrations**  
  Out-of-the-box support for IAM roles, ALB/NLB provisioning, EBS volumes, and CloudWatch metrics.
- **Infrastructure as code support**  
  Use `eksctl` to automate EKS provisioning with consistent configurations:

```bash
eksctl create cluster \
  --name elasticsearch-cluster \
  --version 1.28 \
  --region us-west-2 \
  --nodegroup-name elasticsearch-nodes \
  --node-type m5.xlarge \
  --nodes 3 \
  --nodes-min 1 \
  --nodes-max 10 \
  --managed
```

Use instance types that support enhanced networking (e.g., `m5n`, `c5n`) for low-latency inter-node communication.

### Configure Load Balancers for External Access

Elasticsearch is often queried externally, either by apps, dashboards, or observability systems. Use a Kubernetes `Service` backed by an AWS Network Load Balancer (NLB) for external traffic routing:

```yaml
apiVersion: v1
kind: Service
metadata:
  name: elasticsearch-external
  namespace: elastic-system
  annotations:
    service.beta.kubernetes.io/aws-load-balancer-type: "nlb"
    service.beta.kubernetes.io/aws-load-balancer-cross-zone-load-balancing-enabled: "true"
spec:
  type: LoadBalancer
  selector:
    app: elasticsearch
  ports:
    - port: 9200
      targetPort: 9200
      protocol: TCP
```

Restrict access to known IP ranges using AWS Security Groups:

```bash
aws ec2 authorize-security-group-ingress \
  --group-id sg-xxxxxxx \
  --protocol tcp \
  --port 9200 \
  --cidr 203.0.113.0/24
```

Use IAM roles and access policies if exposing via an API Gateway or other secured proxy.

### Storage Options for Elasticsearch Pods

Elasticsearch requires fast, consistent storage with high IOPS and throughput. AWS provides several volume types suited to different workload intensities:

- **EBS gp3**  
  Good default option. Set custom baseline IOPS and throughput for predictable performance.
- **EBS io2**  
  Best for large-scale indexing or search clusters with heavy disk activity. Offers higher durability and guaranteed performance.
- **Instance store (ephemeral)**  
  Very fast but non-persistent. Use only when you can rebuild lost data from upstream systems (e.g., log ingestion pipelines).

Provision volumes using a dedicated `StorageClass`:

```yaml
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: fast-ssd
provisioner: kubernetes.io/aws-ebs
parameters:
  type: gp3
  iops: "3000"
  throughput: "125"
volumeBindingMode: WaitForFirstConsumer
allowVolumeExpansion: true
```

### Network and Node Considerations

Elasticsearch is a chatty service. Inter-node communication (for replication, cluster state updates, and shard allocation) is frequent. Use instance types that support:

- **Enhanced networking** (ENA) for lower latency
- **High network throughput** (e.g., 10 Gbps or more on `m5n`, `r5n`, `c5n` instances)

Pin critical Elasticsearch pods using `nodeSelectors` or `affinity rules` to ensure they land on appropriate nodes.

## Final Thoughts

Elasticsearch runs well on Kubernetes if storage, networking, and resource configs are tuned correctly. Start with a small StatefulSet or Helm deployment. Use managed services like EKS to reduce setup overhead.

If you’re using Elasticsearch to handle observability data — logs, metrics, traces, you're likely dealing with high cardinality, ingestion spikes, and noisy alerts. Kubernetes adds even more variability: autoscaling, ephemeral pods, and dynamic labels can quickly overwhelm your Elasticsearch cluster.

This is where [Last9](https://last9.io/) helps: it gives you a [Control Plane](https://last9.io/control-plane/) to decide which metrics to drop, pre-aggregate, or forward, reducing telemetry load before it reaches Elasticsearch. That means fewer noisy alerts, better control over ingestion, and visibility that reflects actual system pressure, not just surface-level spikes.

[Get started with us](https://app.last9.io/) today or [book a time with us](https://last9.io/schedule-demo/) to know more!

## FAQs

**1\. Should You Run Elasticsearch on Kubernetes?**  
Yes, if you're already using Kubernetes and comfortable managing stateful workloads. Elasticsearch benefits from Kubernetes features like rolling updates, horizontal scaling, and consistent environments, but it also requires careful tuning around storage, networking, and resource limits.

**2\. How Do I Deploy Elasticsearch on Kubernetes?**  
You can deploy Elasticsearch using:

- StatefulSets with custom YAML configs
- Helm charts from the Elastic Helm repo
- Elastic Cloud on Kubernetes (ECK), an operator-based approach by Elastic

Each method has tradeoffs based on control, automation, and operational complexity.

**3\. How Do I Set Up Elasticsearch for High Availability in Kubernetes?**  
Key configs for HA include:

- At least three master-eligible nodes
- Use of Persistent Volumes via StatefulSets
- Pod anti-affinity rules to avoid co-locating critical pods
- Liveness/readiness probes for automatic recovery
- Enabling replicas and proper shard allocation awareness

**4\. How Do You Scale an Elasticsearch Cluster on Kubernetes?**  
Use `kubectl edit` or Helm to increase the replica count of your data or ingest nodes. Ensure the underlying infrastructure (CPU, memory, disk IOPS) can support the additional load. Horizontal scaling works well if your workloads are balanced across shards.

**5\. What Is Elasticsearch in Kubernetes?**  
It’s a deployment of Elasticsearch where each node runs as a pod, typically managed by a StatefulSet. Kubernetes handles orchestration, scheduling, and restarts, while Elasticsearch handles clustering and data distribution.

**6\. What Are the Requirements for Running Elasticsearch on Kubernetes?**  
You’ll need:

- StatefulSets for pod identity and storage
- SSD-backed Persistent Volumes (like AWS gp3 or io2)
- Proper CPU/memory tuning (e.g., 4Gi RAM per node, 50% heap)
- A dedicated namespace and RBAC configuration
- StorageClasses and resource requests tuned for Elasticsearch

**7\. What Is the Elastic Stack?**  
The Elastic Stack—formerly ELK Stack—includes:

- Elasticsearch (search engine)
- Logstash (data pipeline)
- Kibana (visualization layer)
- Beats (lightweight data shippers)

It's commonly used for centralized logging, metrics, and observability.

**8\. What Is the ELK Stack?**  
The ELK Stack refers to Elasticsearch, Logstash, and Kibana—the original trio before Beats and other tools were added. It's used to collect, store, and visualize log data.

**9\. What Is Kubernetes on AWS?**  
It’s typically run via Amazon EKS (Elastic Kubernetes Service), which provides a managed Kubernetes control plane. You’re responsible for node groups and workloads, but AWS handles upgrades and availability of the master components.

**10\. Is the Discovery Config Still Valid in Elasticsearch?**  
Yes. When deploying in Kubernetes, `discovery.seed_hosts` and `cluster.initial_master_nodes` are still required in your Elasticsearch pod environment variables or config. For StatefulSets, this usually includes static pod names like `elasticsearch-0`, `elasticsearch-1`, etc.

**11\. What Does the curl Command Do?**

```bash
curl http://10.102.46.55:9200/twitter/_search
```

This sends a query to the Elasticsearch node at `10.102.46.55` on the `twitter` index. If the index exists and the node is reachable, it returns documents. Useful to verify if data is still queryable.