How to Use In-Cluster Communication to Secure Kubernetes

Introduction to Linkerd Service Mesh

Introduction

Developing with Kubernetes in mind means developing Cloud Native applications. CNCF defines cloud-native as:

“Cloud-native technologies empower organizations to build and run scalable applications in modern, dynamic environments such as public, private, and hybrid clouds. Containers, service meshes, microservices, immutable infrastructure, and declarative APIs exemplify this approach.”

These techniques enable loosely coupled systems that are resilient, manageable, and observable. Combined with robust automation, they allow engineers to make high-impact changes frequently and predictably with minimal toil.

Source: https://github.com/cncf/foundation/blob/master/charter.md

Cloud-native applications require a different architectural approach. Service mesh helps achieve the cloud-native architectural goals by moving complexity from the application to the underlying infrastructure

Visit Linkerd documentation if you need a refresher about service mesh.

What Problems do Service Meshes Solve?

Service meshes add observability, security, and reliability features to “cloud-native” applications by transparently inserting this functionality at the platform layer rather than the application layer.

Platform Level Metrics: without changing configuration or source code, track low-level metrics
Mutual TLS — mLTS: add encryption and certificates based identity to cluster workloads
Improved Resiliency: latency aware load balancing, retries, timeouts, and advanced deployment patterns
Authorization Policy: enforce traffic rules on services level

Service meshes are an advanced and complex topic, so instead of focusing on all functionalities at once, we will tackle one at a time. We will explore how a service mesh can help with securing inter-cluster communication.

Authenticate and Encrypt Infernal Traffic

The premise of a service mesh is to move accidental architectural complexity such as logging, monitoring, encryption, and authentication to the underlying platform rather than require individual applications to implement similar logic on a framework or programming level.

There are 2 popular service meshes, Istio and Linkerd. We are going to use Linkerd to see how to encrypt and authenticate traffic, but the same would work with Istio.

In 2021 Linkerd moved to graduated status of CNCF projects, joining projects like Kubernetes, etcd, rook or helm

Once installed on the cluster, the Linkerd control plane will inject sidecars to Kubernetes system pods. From there, we can inject sidecars into the pods and create a service mesh.

Demo Example

If you would like to try out how Linkerd secures and authenticates traffic in a Kubernetes cluster, head over to Katacoda and follow this self-paced lab.

Head over to Katacoda to practice!

The Lab Scenario Flow

Install Linkerd CLI linkerd version
Check if the cluster is ready for the control plane installationlinkerd check --pre
Install Linkerd control plane

linkerd install | kubectl apply -f -

Check Linkerd installationlinkerd check
Create sample deployment:

kubectl create deployment --image=gcr.io/kuar-demo/kuard-amd64:blue kuard

Wait for the pod to come up:

kubectl wait deployment kuard --for=condition=Available --timeout=1m

Forward traffic to the pod:

kubectl port-forward deploy/kuard 8080:8080 --address 0.0.0.0 &

Inject Linkerd sidecar

Before injecting the linkerd sidecar, let’s see how many containers run inside kuard pod. There should be only kuard container running.

kubectl get pods -l app=kuard -o jsonpath='{.items[*].spec.containers[*].name}{"\n"}'

Inject Linkerd sidecar into kuard pod:

kubectl get deploy kuard -o yaml \
  | linkerd inject - \
  | kubectl apply -f -

Make sure the Linkerd sidecar is applied correctly:

linkerd -n default check --proxy

We should see the Linkerd sidecar running in the pod:

kubectl get pods -l app=kuard -o jsonpath='{.items[*].spec.containers[*].name}{"\n"}'

Install observability extension:

helm repo add linkerd https://helm.linkerd.io/stable

helm install linkerd-viz \
  --set dashboard.enforcedHostRegexp=.* \
  linkerd/linkerd-viz

Make sure the installation succeededlinkerd check
Launch the dashboard in the background:

linkerd viz dashboard --address 0.0.0.0 &

Check if mTLS works between system pods and kuard pod:

Mutual TLS

All pods with the injected linkerd sidecar communicate by default with encrypted and authenticated traffic.

Restart pod to enable tap configuration:

kubectl rollout restart deployment kuard

Explore dashboard to see connectivity details.

Conclusion:

Before you jump on and install Linkerd or Istio on your cluster, make sure that you understand the operational and cognitive complexity that the service mesh brings on board.

For the majority of the workloads, you don’t need a service mesh. There are often alternatives that fulfill the requirement you need without adding complexity with features you don’t need.

Previously published here.