Networking

Kubernetes DNS & Service Discovery

● Intermediate ⏱ 12 min read

When a pod needs to talk to a database, it shouldn't need to know the database pod's IP — pod IPs change every restart. Kubernetes solves this with a cluster-internal DNS server called CoreDNS. Every Service gets a stable DNS name, and pods use short hostnames that CoreDNS resolves to ClusterIP addresses. No service discovery client, no hardcoded IPs.

CoreDNS

CoreDNS runs as a Deployment in the kube-system namespace, fronted by a ClusterIP Service named kube-dns. The kubelet configures every new pod to use kube-dns's ClusterIP as the pod's DNS resolver — written into /etc/resolv.conf inside the container.

inside any pod

cat /etc/resolv.conf
# nameserver 10.96.0.10         ← kube-dns ClusterIP
# search default.svc.cluster.local svc.cluster.local cluster.local
# options ndots:5

CoreDNS watches the Kubernetes API for Service and Endpoint changes and keeps its DNS records up to date automatically. When you create a Service, its DNS record is live within seconds — no TTL wait, no manual registration.

DNS resolution flow inside a pod

pod (namespace: default)

app container

lookup: "db"

→ tries db.default.svc.cluster.local

→

kube-system / CoreDNS

kube-dns :53

Watches API for Services

db.default.svc.cluster.local

→ 10.96.42.100

→

ClusterIP Service

svc/db

10.96.42.100

The pod resolves the short name "db" using search domains — no hardcoded IPs needed

CoreDNS resolves Service short names to ClusterIP addresses. The kubelet writes CoreDNS's IP into every pod's /etc/resolv.conf.

Service DNS Names

Every Service gets a DNS A record (or AAAA for IPv6) in the cluster DNS. The fully qualified domain name (FQDN) follows this pattern:

<service-name>.<namespace>.svc.<cluster-domain>

# Examples (default cluster domain is cluster.local):
my-service.default.svc.cluster.local
postgres.db.svc.cluster.local
redis.cache.svc.cluster.local

From within the same namespace, you can use just the Service name as a hostname. From a different namespace, use <service>.<namespace> — the search domains fill in the rest.

From namespace	Targeting	Short name works?	Use
`default`	`default/my-svc`	Yes	`my-svc`
`default`	`db/postgres`	Partial	`postgres.db`
any	any	Always	`postgres.db.svc.cluster.local`

Services also get a DNS SRV record for each named port: _<port-name>._<proto>.<service>.<namespace>.svc.cluster.local — useful for clients that need to discover port numbers dynamically.

Search Domains & ndots

The search and ndots lines in /etc/resolv.conf control how short names get resolved. With ndots:5, any name with fewer than 5 dots is tried with each search domain appended before trying it as a bare FQDN.

resolv.conf search path for "db" (0 dots)

# 0 dots < ndots:5, so search domains are tried first:
1. db.default.svc.cluster.local   ← ✓ found (if Service "db" exists in default ns)
2. db.svc.cluster.local
3. db.cluster.local
4. db                             ← bare lookup (falls through to upstream DNS)

⚠️

ndots:5 causes extra DNS queries for external names

Resolving api.github.com (2 dots, less than 5) triggers 3 failed cluster lookups before the real query. For latency-sensitive external calls, use the full FQDN with a trailing dot — api.github.com. — to skip search expansion, or lower ndots in the pod's DNS config.

Pod DNS Names

Pods also get DNS records, but they're less commonly used because pod IPs change on every restart. The FQDN uses the pod IP with dashes instead of dots:

# Pod IP: 10.244.1.5  →  10-244-1-5.<namespace>.pod.cluster.local

# Example:
10-244-1-5.default.pod.cluster.local

# Pods can also get a hostname if spec.hostname is set:
spec:
  hostname: my-pod
  subdomain: my-service   # must match a headless Service name
# → my-pod.my-service.default.svc.cluster.local

StatefulSets use this mechanism automatically — each pod gets a stable DNS name based on its ordinal index and the headless Service name. See the StatefulSets guide for details.

Headless Services

A headless Service has clusterIP: None. CoreDNS does not assign a ClusterIP — instead, it returns an A record for each pod matching the Service selector. Clients receive multiple IPs and do their own load balancing or pick a specific pod by name.

headless-service.yaml

apiVersion: v1
kind: Service
metadata:
  name: mysql
  namespace: db
spec:
  clusterIP: None    # headless — no VIP, no kube-proxy rules
  selector:
    app: mysql
  ports:
  - port: 3306

ClusterIP Service vs Headless Service DNS

ClusterIP Service

DNS lookup

mysql.db → 10.96.1.5

↓

ClusterIP VIP

10.96.1.5

↓ kube-proxy DNAT

pod-0

pod-1

Headless Service

DNS lookup

mysql.db → [10.244.1.2, 10.244.2.3]

↓ direct pod IPs

mysql-0.mysql.db

10.244.1.2

mysql-1.mysql.db

10.244.2.3

Headless Services return individual pod IPs — clients choose; StatefulSet pods get stable per-pod hostnames

A headless Service bypasses the ClusterIP VIP. DNS returns per-pod A records — essential for StatefulSets where pods have distinct identities.

DNS Policies

The spec.dnsPolicy field on a Pod controls how /etc/resolv.conf is configured:

Policy	Behaviour	When to use
`ClusterFirst`	Default. Uses CoreDNS for cluster names, forwards unknown queries upstream	Almost always
`Default`	Inherits the node's DNS config — no cluster DNS	Host-network pods that need node-level resolution
`None`	Ignores all cluster DNS; you provide a custom `dnsConfig`	Custom stub resolvers, advanced tuning
`ClusterFirstWithHostNet`	Like `ClusterFirst` but for pods with `hostNetwork: true`	hostNetwork pods that still need cluster DNS

Custom DNS Config

Use spec.dnsConfig to tune DNS behaviour per pod — lower ndots, add custom search domains, or point to a custom nameserver alongside CoreDNS:

pod-custom-dns.yaml

apiVersion: v1
kind: Pod
metadata:
  name: my-app
spec:
  dnsPolicy: ClusterFirst
  dnsConfig:
    options:
    - name: ndots
      value: "2"    # reduce extra lookups for external FQDNs
    searches:
    - corp.internal   # extra search domain added to the list
  containers:
  - name: app
    image: my-app:latest

You can also patch the CoreDNS ConfigMap in kube-system to add stub zones — forwarding queries for a specific domain (e.g. corp.example.com) to an internal resolver while all other queries go to the upstream DNS.

CoreDNS Corefile — stub zone

.:53 {
    errors
    health
    kubernetes cluster.local in-addr.arpa ip6.arpa {
        pods insecure
        fallthrough in-addr.arpa ip6.arpa
    }
    # Forward corp.example.com to the internal resolver
    forward corp.example.com 192.168.1.53
    forward . /etc/resolv.conf
    cache 30
    loop
    reload
    loadbalance
}

Debugging DNS

Start with a debug pod that has DNS tools installed:

# Spin up a debug pod with nslookup / dig available
kubectl run dns-debug --image=busybox:1.28 --rm -it -- sh

# Inside the pod:
nslookup kubernetes.default        # should return 10.96.0.1
nslookup my-service.other-ns       # cross-namespace lookup
nslookup my-service.other-ns.svc.cluster.local   # explicit FQDN
cat /etc/resolv.conf               # check nameserver + search domains

# Check CoreDNS pods are running
kubectl get pods -n kube-system -l k8s-app=kube-dns

# Tail CoreDNS logs (enable "log" plugin in Corefile to see all queries)
kubectl logs -n kube-system -l k8s-app=kube-dns --tail=50 -f

# Verify the kube-dns Service has endpoints
kubectl get endpoints kube-dns -n kube-system

# Check a Service has DNS-resolvable endpoints
kubectl get endpoints my-service -n default

💡

DNS not resolving? Check the Service selector first

The most common DNS failure isn't CoreDNS — it's a Service with no matching pods. kubectl get endpoints <service> shows <none> if the selector doesn't match any pod labels. The DNS record exists but points nowhere, so connections time out instead of failing fast.

kubectl Commands

# List all Services in a namespace (see ClusterIPs)
kubectl get svc -n <namespace>

# Check endpoints behind a Service
kubectl get endpoints <service> -n <namespace>

# Run a one-shot DNS lookup from inside the cluster
kubectl run -it --rm dns-test --image=busybox:1.28 -- nslookup <service>.<namespace>

# View CoreDNS config
kubectl get configmap coredns -n kube-system -o yaml

# Patch CoreDNS config (edit in place)
kubectl edit configmap coredns -n kube-system

# Force CoreDNS to reload the Corefile
kubectl rollout restart deployment coredns -n kube-system

# Verify kube-dns Service address
kubectl get svc kube-dns -n kube-system