mTLS & Service Mesh Security

By default, pod-to-pod traffic in Kubernetes is unencrypted. Network Policies can restrict which pods talk to each other, but not the authenticity of those connections. Mutual TLS (mTLS) solves that: both client and server present certificates, so every connection is both encrypted and identity-verified. This guide covers mTLS from first principles, through Istio configuration, to zero-trust network architecture.

Default Network Posture

With no additional configuration, the Kubernetes network is flat:

• Any pod can connect to any other pod by IP.
• Traffic between pods is not encrypted — a compromised node can inspect east-west traffic.
• Traffic source is not authenticated — a pod can impersonate another service by spoofing headers.
• Network Policies (guide 19) restrict connectivity but do not encrypt or authenticate traffic.

In a zero-trust model, you assume the network is hostile even inside the cluster. mTLS addresses both encryption and identity in one mechanism.

What mTLS Does

Regular TLS: the client verifies the server's certificate. The server does not verify the client.

Mutual TLS: both sides present certificates. The server verifies the client's certificate before accepting the connection. In Kubernetes, each workload identity (typically tied to a ServiceAccount) gets a certificate issued by the cluster's internal CA.

Sidecar Proxy Architecture

Service meshes like Istio and Linkerd inject a sidecar proxy (Envoy or a lightweight equivalent) into every pod. The proxy transparently intercepts all inbound and outbound traffic using iptables rules set up by an init container. The application code requires no changes.

# Label the namespace for automatic sidecar injection
kubectl label namespace production istio-injection=enabled

# Verify sidecar is injected (pod should have 2 containers)
kubectl get pods -n production
# NAME                   READY   STATUS    RESTARTS
# frontend-7d9f6-xkw2p   2/2     Running   0   ← 2/2 = app + istio-proxy

# Check proxy version
kubectl exec -n production frontend-7d9f6-xkw2p \
  -c istio-proxy -- pilot-agent request GET server_info | jq .version

Istio mTLS Configuration

Istio runs in permissive mode by default — mTLS is preferred but plain text is accepted. Switch to strict mode to reject all non-mTLS connections.

apiVersion: security.istio.io/v1beta1
kind: PeerAuthentication
metadata:
  name: default
  namespace: production    # applies to all workloads in this namespace
spec:
  mtls:
    mode: STRICT           # reject plain-text connections

---
# Or cluster-wide in the istio-system namespace
apiVersion: security.istio.io/v1beta1
kind: PeerAuthentication
metadata:
  name: default
  namespace: istio-system   # applies globally
spec:
  mtls:
    mode: STRICT

apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
  name: backend-access
  namespace: production
spec:
  selector:
    matchLabels:
      app: backend
  action: ALLOW
  rules:
  - from:
    - source:
        principals:
        # Only allow requests from the frontend ServiceAccount
        - "cluster.local/ns/production/sa/frontend"
    to:
    - operation:
        methods: ["GET", "POST"]
        paths: ["/api/*"]

mTLS Without a Mesh

A full service mesh is significant operational overhead. For simpler scenarios, you can implement mTLS directly in the application or at the infrastructure level.

apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: backend-cert
  namespace: production
spec:
  secretName: backend-tls
  duration: 24h           # short-lived; rotated automatically by cert-manager
  renewBefore: 1h
  subject:
    organizations: ["my-cluster"]
  commonName: backend.production.svc.cluster.local
  dnsNames:
  - backend.production.svc.cluster.local
  - backend.production.svc
  issuerRef:
    name: cluster-ca
    kind: ClusterIssuer

The application then loads the certificate from the mounted Secret and configures its TLS listener/client to require and verify peer certificates. This approach works well for a small number of services that need mutual auth without managing a full mesh.

Zero-Trust Patterns

Zero-trust networking assumes that the network boundary is already breached. Core principles applied to Kubernetes:

Certificate Rotation

Istio's CA (Istiod) issues workload certificates with a 24-hour TTL by default and rotates them automatically. The sidecar proxy fetches new certificates before expiry via the xDS API — no pod restart required.

# Check certificate expiry in a running sidecar
kubectl exec -n production frontend-7d9f6-xkw2p \
  -c istio-proxy -- \
  openssl s_client -connect backend.production:8080 \
  -cert /etc/certs/cert-chain.pem \
  -key /etc/certs/key.pem 2>/dev/null | \
  openssl x509 -noout -dates

# Istio certificate status
istioctl proxy-config secret frontend-7d9f6-xkw2p.production

# Force cert rotation (mostly for testing)
kubectl delete secret istio.frontend -n production

When You Actually Need a Mesh

Service meshes add real operational complexity (more components, sidecar overhead, debugging layers). Use one when you need:

• Cluster-wide mTLS — enforcing encryption and identity for all pod-to-pod traffic without code changes.
• Fine-grained authorization policies — AuthorizationPolicy at the HTTP method/path level, tied to SPIFFE identity.
• Traffic management — canary deployments, fault injection, retries, circuit breaking via VirtualService/DestinationRule.
• Unified observability — automatic metrics (RED), distributed tracing, and access logs for every service without instrumentation.