Comparing Popular Service Mesh Offerings

When applications are developed in microservices or other distributed architectures, it becomes hard to manage, trace, and monitor them, as well as configure how the applications react in case of failure. When a service fails, it affects all the caller services, forcing you to implement a circuit breaker at the code level.

In addition, to figure out what’s going on between your services or to inspect a failure, you may need to add a tracing capability to your system by adding code in your application services or microservices. In scenarios like these, you don’t have much control over the network.

For instance, if you want to add mutual TLS (mTLS) to improve your microservices’ security, you must apply network-level configurations and application-level code changes to enable it. This is costly in terms of time and engineering.

However, there are easier ways to manage your microservices’ network traffic using a service mesh framework to secure it, trace what services are doing, and monitor any application-level actions.

Currently, many service mesh solutions are licensed and supported by well-known companies in the cloud-native sector. In this article, you’ll review these service mesh offerings based on their ease of use, features offered, and pricing.

What Is a Service Mesh

A service mesh is an infrastructure layer that provides a way to control how different parts of an application share data with one another by adding features to a network between those parts. Depending on the service mesh framework used, observability, traffic shifting, resiliency features such as circuit breaking, and mTLS can be configured once and enforced decentralized.

You can implement most of these features without a service mesh, but you must update your code. Moreover, a service mesh doesn’t require code changes; instead, it adds layers of additional components (or containers, depending on what the service mesh framework runs on) that implement the features framework or programming language independently.

In this article, we’ll be going over the following service meshes:

Istio, possibly the most popular service mesh framework, was announced in May 2017 by Google, IBM, and Lyft, and it is known for its extensive online community.

Linkerd is a Cloud Native Computing Foundation (CNCF) graduated open-source project. It's another popular service mesh framework known for its production-ready lightweight service mesh.

Amazon Web Services (AWS) also has its service mesh offering called AWS App Mesh, which is a fully managed service mesh intended for Amazon-specific services, like [Amazon Elastic Compute Cloud (EC2), Amazon Elastic Container Service (ECS), and Amazon Elastic Kubernetes Service (EKS)

Kuma is a more recent service mesh offering created by Kong and open-sourced in 2020. It was built in response to the previous service mesh frameworks’ issue of being heavy and difficult to manage or operate.

The last service mesh offering you’ll look into in this article is Consul, developed by Hashicorp. It provides a single binary that acts both as a client and a server, which enables it to be easily installed on any platform.

Comparing Service Mesh Offerings

Now take a more in-depth look at each tool and review them based on the overall architecture, ease of installation and use, supported protocols, ingress controller, monitoring and tracing, load balancing, resilience, security (mTLS and authorization), and licensing and pricing.

Istio

Istio is an open-source service mesh platform that layers transparently onto distributed applications to provide a more efficient way to secure, connect, and monitor microservices.

Architecture
Istio logically consists of a data plane and a control plane. The data plane is a set of Envoy-based proxies deployed as sidecars, while the control plane configures and manages the proxies to route the mesh traffic.

Installation and User Experience
Istio runs on a Kubernetes cluster or a virtual machine (VM). You can install it in many ways, including using the Kubernetes operator, using the istioctl command line interface (CLI), or using Helm.

While Istio provides managing its resources via its istioctl CLI, it doesn’t have an integrated UI. You have to install Kiali, a UI for managing or visually creating Istio Service Mesh objects.

Kiali can be integrated with Jaeger to add tracing features to the UI, which means you can do distributed tracing on the services in the Istio network.

Features
Istio supports gRPC, HTTP/2, and HTTP/1.x, WebSockets, and all Transmission Control Protocol (TCP)–based protocols. To provide ingress capability, it has an Envoy-based Istio gateway.

Along with Jaeger, which can also be integrated with the Kiali UI, Zipkin can enable distributed tracing through the Istio system.

Istio applies various load-balancing algorithms, such as round-robin or random least connection, and it supports percentage-based, header-based, and path-based traffic splits.

Istio also has many features for resiliency as well as circuit breaking, retries and timeouts, and fault and delay injection. It has mTLS support for all protocols, supports authorization rules via Authorization Policy, and uses external certificate authority (CA) when possible.

Licensing and Pricing
Istio is free and is licensed under Apache 2.0. It has enterprise support from various vendors, including Aspen Mesh, Solo.io, and Tetrate.

Linkerd

As one of the CNCF graduated open source projects, Linkerd advertises itself as the world’s lightest, fastest service mesh. Compared to the other service mesh offerings, it has a more straightforward and minimalistic structure.

Architecture
Like Istio, Linkerd is also split into a control plane and a data plane. For Linkerd, the control plane is a set of services, which aggregates telemetry data, provides a user-facing API, and provides control data to the data plane proxies. The data plane consists of the transparent Linkerd2 proxy, which handles all traffic to and from the service.

Installation and User Experience
Linkerd is relatively easy to install compared to the other service mesh frameworks due to its out-of-the-box configuration. It can be installed on a Kubernetes cluster using its CLI, Helm, or the marketplace install from your Kubernetes provider.

Linkerd makes it easy to access and manage its resources through a CLI named linkerd.

It also has a dashboard to manage its control plane and monitor the metrics per namespace and can provide dashboards through Grafana.

Features
Linkerd supports protocols such as gRPC, HTTP/2, and HTTP/1.x, WebSockets, and all TCP traffic. Unlike Istio, Linkerd doesn’t provide its ingress controller for the simplicity of its architecture. Instead, you can choose any ingress controller to work alongside Linkerd.

Regarding tracing capabilities, Linkerd doesn’t offer an immediate solution, but it traces all backends that support OpenCensus. Linkerd also has a Jaeger extension that can be installed by using the linkerd CLI, which enables tracing via Jaeger explicitly.

For routing and load balancing, Linkerd uses the [exponential weighted moving average EWMA load balancing algorithm and supports percentage-based traffic split through Server Name Indication (SNI).

Unlike Istio, Linkerd doesn’t support retries, timeouts, fault injections, and delay injections.

Although circuit breaking is implemented in the 1.x version of Linkerd, it is not yet implemented in 2.x. However, it’s planned for version 2.12.0.

mTLS is supported for most of the TCP-based traffic except for supporting the use of an external CA.

Licensing and Pricing
Linkerd is licensed under the Apache 2.0 license. Buoyant, the company that created Linkerd, provides full support and training for the service mesh.

AWS App Mesh

AWS App Mesh is a service mesh that provides application-level networking and service communication for microservices in a managed way. It supports many AWS services, such as EC2, ECS, EKS, and Kubernetes, running on AWS.

Architecture
AWS App Mesh consists of the following components:

Service mesh: A virtual network system between the services inside the mesh.
Virtual nodes: A virtual representation of a service in App Mesh or a Kubernetes service.
Virtual routers: A traffic manager for the virtual services within the mesh.
Routes: It forwards the traffic using simple rules, such as matching a service name prefix to one or more virtual nodes.
Virtual services: A service abstraction for the essential application services.
App Mesh sidecar: A sidecar container that applies the traffic rules for the virtual routers and nodes.
Proxy: It is based on the Envoy proxy and does traffic redirection.

Installation and User Experience
No installation is needed for AWS App Mesh. However, you must configure it based on your existing infrastructure on AWS.

Compared to Istio, it’s easier to set up with EKS because of its AWS-native integration and support. AWS App Mesh can be managed using AWS dashboards or the AWS CLI’s appmesh command.

Features
Like the other service mesh offerings, AWS App Mesh supports gRPC, HTTP/2, and HTTP/1.x, WebSockets, and all TCP-based protocols. It’s also based on the Envoy proxy, which makes it compatible with multiple solution approaches.

Monitoring for tracing is possible by using AWS X-Ray; however, enabling tracing for the services is not as easy as Istio because, to enable a service for AWS App Mesh tracing, you need to update the application code.

AWS App Mesh supports virtual routes for handling the traffic within the application mesh. Routes provide weight-based traffic split between the application services.

AWS App Mesh has features like rate limiting, timeouts, and retries with circuit breaking. However, unlike Istio, it doesn’t allow using custom error codes for retry configuration by only applying its retry default policy.

AWS App Mesh supports mTLS but doesn’t support automatic certificate rollover like Istio. But it does have AWS Identity and Access Management (IAM) integration, which is an advantage for AWS workloads and setup.

Licensing and Pricing
Unlike Istio and Linkerd, AWS App Mesh is not an open-source project; it’s a product of AWS, and there is no additional charge for using the service. You only pay for the AWS resources (EC2 instances, EKS, ECS with Fargate, etc.) consumed by the lightweight proxy that is deployed alongside your containers.

Kuma

Kuma is a CNCF sandbox project created by Kong. Kuma was created in response to heavy and difficult-to-operate service meshes. It offers a very modular structure by integrating with the applications already running on Kubernetes or non-Kubernetes infrastructures.

Architecture
There are two primary components of Kuma: the control plane and the data plane proxy. Logically, these are very similar to what Istio and Linkerd offer.

With the control plane component, Kuma accepts user input to create and configure policies like service meshes and to add services and configure their behavior within the meshes created. The data plane proxy is a bundled proxy implementation on top of the Envoy proxy, which processes both incoming and outgoing requests for the service.

Installation and User Experience
You can install Kuma on Kubernetes, run it on Docker, or install it on any operating system using the universal mode.

It supports vanilla Kubernetes and OpenShift for the Kubernetes installation, which can also be handled using Helm. Kuma can be installed on several operating systems, including Debian, CentOS, Ubuntu, Amazon Linux, and macOS.

You can manage Kuma’s resources via its `kumactl` CLI, and it has a graphical user interface (GUI) to manage the meshes or proxies of the data planes.

Features
Kuma supports gRPC, HTTP/2, and HTTP/1.x, WebSocket, TCP, and most importantly, the Apache Kafka protocol, which the other service mesh technologies lack. Kuma is also built on top of Envoy, like the other service mesh technologies except for Linkerd.

Kuma supports Zipkin and Datadog backends for traffic tracing, and Jaeger can be used as the Zipkin collector.

Kuma applies load balancing algorithms, such as round robin, random least connection, ring hash, and random Maglev. It supports percentage-based, header, and path-based traffic splits with transformations.

Regarding resiliency, Kuma provides circuit breaking, retry, and timeouts. It only supports method-based retries.

mTLS is supported by Kuma, and for the CA certificates, it can be integrated with HashiCorp Vault, cert-manager, and Azure Key Vault.

Licensing and Pricing
Kuma has the open-source Apache 2.0 license, and Kong provides enterprise-level support for Kuma.

Consul

Consul is a multi-networking system created by HashiCorp designed to be a service mesh solution. Like the other service mesh offerings, it solves the challenges of networking and security of cloud infrastructures and is developed actively by HashiCorp members and open-source contributors.

Architecture
Consul is a single binary application. Unlike Istio, it doesn't have or need different components to be installed, and it's lightweight like Linkerd.

The single binary acts as a server and a client, depending on where it's installed. For example, Consul implements an agent model that it runs on an instance as a client on each node. Each client contains a local cache that is updated by the server instances. This forms a decentralized cluster and differs from other solutions like Istio, which has the mixer the component at the center.

Installation and User Experience
You can install Consul on any operating system, such as macOS, Windows, and Linux. It uses the Homebrew package manager for MacOS and Chocolatey for Windows. Linux, it supports Debian, RHEL, Fedora, and Amazon Linux installations via their particular package managers, and it supports Kubernetes installation.

Consul has a Consul UI that lets you view the server and clients, the registered services and their sidecars, and registered gateways, including the ingress. Compared to the other UIs, it has extra features such as viewing and updating the ACL tokens and key-value pairs for configuration.

To access and manage Consul via a terminal, you can use the consul CLI, which enables you to do many operations that you can do on the UI.

Features
Consul supports both gRPC and HTTP/1.x, HTTP/2, and TCP connections. You can configure Consul UI for fetching and observing the metrics, or you can enable the Prometheus metric provider. With this, it becomes possible to configure Grafana dashboards to show service-specific metrics.

Consul uses path-based traffic routing and provides layer seven networking for traffic shifting and load balancing. It can also be integrated with many load balancers systems such as NGNIX, HAProxy, and F5.

For resiliency, Consul provides circuit breaking, retry, and timeouts. To implement circuit breaking, you must configure the Envoy circuit-breaking feature and the outlier detection.

For mTLS, it provides an automatic TLS certificate management system so that both leaf and root certificates are rotated through the Consul cluster automatically with no disruption of connections. It also has Vault support that adds another level of security in terms of keeping and protecting sensitive data.

Licensing and Pricing
Consul is open source and is licensed under an MPL-2.0 license. In addition, HashiCorp provides an enterprise license and support.

Interested in more such comparisons? Then may we suggest our blog on comparing all the popular time series databases.

Conclusion

In this article, you’ve looked at several service mesh offerings and compared them based on their features, licensing and pricing, architecture, and user experience. You’ve also learned about their technical features, like supported protocols, built-in ingress controller availability, monitoring and tracing capabilities, load balancing, resilience, and security.