Security

Container Image Security

● Advanced ⏱ 12 min read

A container image is a filesystem snapshot and a set of instructions. Everything your application depends on at runtime — OS libraries, language runtimes, third-party packages — lives in that image. An unvetted image is an unvetted binary running inside your cluster. This guide covers how to scan, sign, verify, and restrict images so your cluster only runs what you explicitly trust.

Image Attack Surface

Common image-related risks:

Unpatched OS libraries — base images like ubuntu:latest are rebuilt infrequently; they accumulate CVEs over time.
Supply chain compromise — a dependency in your application code or a base image layer is hijacked upstream.
Mutable tags — myapp:latest can point to a different image digest on every pull. No traceability, no reproducibility.
Secrets baked into images — credentials, SSH keys, or API tokens added during build and left in a layer.
Unnecessary packages — tools like curl, bash, apt installed in production images expand the attack surface if a container is compromised.

Vulnerability Scanning

Scanning compares image layer contents against CVE databases (NVD, GHSA, vendor advisories). Integrate scanning into the build pipeline so vulnerabilities block the push, not the deploy.

Trivy — scan an image

# Scan by tag
trivy image myapp:1.2.3

# Fail CI if CRITICAL vulnerabilities found
trivy image --exit-code 1 --severity CRITICAL myapp:1.2.3

# Scan and output SARIF for GitHub Security tab
trivy image --format sarif --output trivy-results.sarif myapp:1.2.3

# Scan a local tarball (useful for air-gapped environments)
docker save myapp:1.2.3 | trivy image --input -

GitHub Actions — scan on push

- name: Scan image
  uses: aquasecurity/trivy-action@master
  with:
    image-ref: ${{ env.IMAGE }}:${{ github.sha }}
    format: sarif
    output: trivy-results.sarif
    severity: CRITICAL,HIGH
    exit-code: 1

- name: Upload SARIF
  uses: github/codeql-action/upload-sarif@v3
  with:
    sarif_file: trivy-results.sarif

Image Signing — Cosign

Cosign (part of Sigstore) signs container images and stores the signature in the same OCI registry alongside the image. Verification happens at deploy time — no separate signature store needed.

Image build → scan → sign → verify → deploy pipeline

docker build

→

trivy scan

→

docker push

→

cosign sign

→

K8s admission

verify sig

→

pod scheduled

Step 5: Kyverno / Connaisseur / Policy Controller checks that the image digest has a valid Cosign signature from your CI key before allowing the pod to run.

Images are scanned at build time, signed after push, and the signature is verified by an admission controller before the pod is scheduled.

sign an image with Cosign

# Generate a key pair (store private key in a secret manager, not in the repo)
cosign generate-key-pair

# Sign an image by digest (tags are mutable; always sign by digest)
IMAGE=registry.example.com/myapp@sha256:abc123...
cosign sign --key cosign.key "$IMAGE"

# Keyless signing with OIDC (Sigstore public Rekor log — no key management)
COSIGN_EXPERIMENTAL=1 cosign sign "$IMAGE"

# Verify a signature
cosign verify --key cosign.pub "$IMAGE"

Admission Verification

Signing is useless without verification at deploy time. Use a policy engine as an admission webhook to check signatures before allowing pods to run.

Kyverno ClusterPolicy — verify Cosign signature

apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: verify-image-signatures
spec:
  validationFailureAction: Enforce
  rules:
  - name: check-image-signature
    match:
      any:
      - resources:
          kinds: ["Pod"]
    verifyImages:
    - imageReferences:
      - "registry.example.com/myapp:*"
      attestors:
      - count: 1
        entries:
        - keys:
            publicKeys: |-
              -----BEGIN PUBLIC KEY-----
              MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAE...
              -----END PUBLIC KEY-----

Distroless & Minimal Images

Distroless images contain only the application and its runtime dependencies — no shell, no package manager, no debugging tools. They dramatically reduce the CVE surface and make it harder for an attacker to move laterally after compromising a container.

multi-stage build with distroless

# Build stage — full toolchain
FROM golang:1.22 AS builder
WORKDIR /app
COPY . .
RUN CGO_ENABLED=0 go build -o server .

# Final stage — distroless, no shell, no apt
FROM gcr.io/distroless/static-debian12:nonroot
COPY --from=builder /app/server /server
USER nonroot:nonroot
ENTRYPOINT ["/server"]

Image	Size	Shell	When to use
`ubuntu:24.04`	~78 MB	bash	Development / debugging only
`debian:12-slim`	~74 MB	sh	Apps that need apt packages
`alpine:3.19`	~7 MB	sh	Small footprint, musl libc
`gcr.io/distroless/static`	~2 MB	none	Static binaries (Go, Rust)
`gcr.io/distroless/base`	~20 MB	none	glibc dynamic binaries
`scratch`	0 MB	none	Fully static single-binary apps

Private Registries

For private registries, create an image pull Secret and reference it in the pod spec or the namespace's default ServiceAccount.

# Create a registry credential Secret
kubectl create secret docker-registry regcred \
  --docker-server=registry.example.com \
  --docker-username=myuser \
  --docker-password=mytoken \
  -n production

# Reference in pod spec
spec:
  imagePullSecrets:
  - name: regcred

# Or attach to the default ServiceAccount so all pods in the namespace use it
kubectl patch serviceaccount default -n production \
  -p '{"imagePullSecrets":[{"name":"regcred"}]}'

Image Pull Policy

Policy	Behavior	When to use
Always	Pull on every pod start. Registry must be reachable.	Mutable tags (`:latest`) or CI environments.
IfNotPresent	Pull only if the image isn't cached on the node.	Immutable tags (digest or semver). Default for non-`latest` tags.
Never	Never pull. Fail if not cached.	Air-gapped environments; images pre-loaded via node provisioning.

⚠️

Pin by digest, not tag

Use image: myapp@sha256:abc123 in production. Tags are mutable — myapp:1.0.0 can be overwritten. A digest is immutable and cryptographically identifies exactly what runs. Combined with image signing, this closes the substitution attack surface.

kubectl Commands

# List all unique images running across the cluster
kubectl get pods -A -o jsonpath='{range .items[*]}{range .spec.containers[*]}{.image}{"\n"}{end}{end}' | sort -u

# Check which pods use a specific image
kubectl get pods -A --field-selector=status.phase=Running \
  -o json | jq '.items[] | select(.spec.containers[].image | contains("myapp")) | .metadata.name'

# Inspect image pull secrets on a ServiceAccount
kubectl get serviceaccount default -n production -o yaml

# Force a pull of the latest image by restarting a deployment
kubectl rollout restart deployment/myapp -n production

# Check if an image is cached on a node (requires node access)
crictl images | grep myapp