Production Operations

GitOps with Flux & ArgoCD

● Advanced ⏱ 20 min read

GitOps is a deployment model where git is the single source of truth for cluster state. A controller running inside the cluster continuously reconciles live state against the git-declared state. No more kubectl apply from CI pipelines; no manual changes that drift away from what's tracked. Every change goes through a PR, every deployment is auditable, and rollback is a git revert.

What Is GitOps

Four principles (from the OpenGitOps spec):

Declarative — the desired state is expressed declaratively (YAML manifests, Helm values, Kustomize overlays).
Versioned and immutable — desired state is stored in git. History is the audit log.
Pulled automatically — an in-cluster agent pulls changes from git; CI never pushes to the cluster.
Continuously reconciled — the agent detects drift and corrects it without human intervention.

GitOps reconcile loop — git is the source of truth

Developer

opens PR

→

Git Repo

PR merged

→

Flux / ArgoCD

detects change
reconciles

→

Cluster

state matches
git

↺ controller also polls git every 30–60s — catches drift even without a push event

GitOps: developer merges a PR → in-cluster agent detects the change → reconciles cluster state to match git. CI never touches the cluster directly.

Flux vs ArgoCD

	Flux	ArgoCD
Architecture	Set of controllers (source-controller, kustomize-controller, helm-controller…). No UI by default.	Single application server + UI. API-first.
UI	Optional (Weave GitOps). CLI-primary.	Built-in web UI — visual app tree, sync status, diff view.
Multi-tenancy	Tenant isolation via namespace-scoped resources. No built-in RBAC UI.	AppProject CRD scopes what each team can deploy. RBAC built in.
Secret management	SOPS native integration. Mozilla SOPS encrypts secrets in git.	Plugin-based. Works with Sealed Secrets, ESO, Vault.
Progressive delivery	Flagger (companion project) — canary, A/B, blue/green.	Argo Rollouts — same feature set, tighter ArgoCD integration.
Best for	Operator-focused teams who prefer Kubernetes-native CRDs and CLI.	Teams who want a UI, multi-team RBAC, and applicationset fleet deploys.

Flux — Core Concepts

Flux is composed of independent controllers. Each CRD has one responsibility:

Flux — GitRepository + Kustomization

# GitRepository: tells source-controller where to pull from
apiVersion: source.toolkit.fluxcd.io/v1
kind: GitRepository
metadata:
  name: gitops-repo
  namespace: flux-system
spec:
  interval: 1m                  # poll interval
  url: https://github.com/myorg/gitops
  ref:
    branch: main
  secretRef:
    name: github-token          # SSH key or HTTPS token secret

---
# Kustomization: tells kustomize-controller what to apply
apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
  name: myapp-production
  namespace: flux-system
spec:
  interval: 5m
  path: ./apps/myapp/overlays/production
  prune: true                   # delete resources removed from git
  sourceRef:
    kind: GitRepository
    name: gitops-repo
  targetNamespace: production
  healthChecks:
  - apiVersion: apps/v1
    kind: Deployment
    name: myapp
    namespace: production

Flux — HelmRelease

apiVersion: helm.toolkit.fluxcd.io/v2
kind: HelmRelease
metadata:
  name: ingress-nginx
  namespace: ingress-nginx
spec:
  interval: 10m
  chart:
    spec:
      chart: ingress-nginx
      version: ">=4.0.0 <5.0.0"
      sourceRef:
        kind: HelmRepository
        name: ingress-nginx
        namespace: flux-system
  values:
    controller:
      replicaCount: 2
      resources:
        requests: {cpu: "100m", memory: "128Mi"}

ArgoCD — Core Concepts

ArgoCD Application — deploy from git

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: myapp
  namespace: argocd
spec:
  project: default
  source:
    repoURL: https://github.com/myorg/gitops
    targetRevision: main
    path: apps/myapp/overlays/production
  destination:
    server: https://kubernetes.default.svc
    namespace: production
  syncPolicy:
    automated:
      prune: true             # delete removed resources
      selfHeal: true          # revert manual kubectl changes
      allowEmpty: false
    syncOptions:
    - CreateNamespace=true
    - PrunePropagationPolicy=foreground
    retry:
      limit: 3
      backoff:
        duration: 5s
        maxDuration: 3m
        factor: 2

Repo Layout

monorepo GitOps layout

gitops/
├── apps/                         ← application manifests
│   ├── myapp/
│   │   ├── base/                 ← shared Deployment, Service, HPA
│   │   └── overlays/
│   │       ├── staging/          ← patches: 1 replica, lower resources
│   │       └── production/       ← patches: 3 replicas, prod resources
│   └── other-app/
├── infrastructure/               ← cluster add-ons (not app-specific)
│   ├── cert-manager/
│   ├── ingress-nginx/
│   └── monitoring/
└── clusters/                     ← per-cluster Flux/ArgoCD config
    ├── prod-us/
    │   └── flux-system/
    │       ├── gotk-components.yaml
    │       └── gotk-sync.yaml    ← points to apps/ and infrastructure/
    └── staging/

Secrets in GitOps

Secrets can't go into git plaintext. Two common patterns:

SOPS with Flux — encrypt secrets in git

# Encrypt a secret with SOPS (age key)
sops --age=age1... --encrypt --in-place secret.yaml

# Flux decrypts at apply time using the cluster's age key
# Store the age private key as a K8s secret in flux-system:
kubectl create secret generic sops-age \
  --namespace=flux-system \
  --from-file=age.agekey=/path/to/age.key

# Add decryption to the Kustomization:
spec:
  decryption:
    provider: sops
    secretRef:
      name: sops-age

External Secrets Operator with ArgoCD

# Don't store secrets in git at all.
# Commit an ExternalSecret CRD that references Vault/AWS SM.
# ArgoCD syncs the ExternalSecret; ESO controller fetches the value.
apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
  name: db-password
spec:
  secretStoreRef:
    name: aws-secrets
    kind: ClusterSecretStore
  target:
    name: db-password
  data:
  - secretKey: password
    remoteRef:
      key: prod/myapp/db
      property: password

Progressive Delivery

Progressive delivery extends GitOps: instead of replacing all replicas at once, route a small percentage of traffic to the new version, measure error rate and latency, then promote or rollback automatically.

Argo Rollouts — canary with analysis

apiVersion: argoproj.io/v1alpha1
kind: Rollout
metadata:
  name: myapp
spec:
  replicas: 10
  strategy:
    canary:
      steps:
      - setWeight: 10          # 10% to canary
      - pause: {duration: 5m}  # wait 5 min
      - analysis:              # check metrics before proceeding
          templates:
          - templateName: error-rate
      - setWeight: 50
      - pause: {duration: 10m}
      - setWeight: 100
  selector:
    matchLabels:
      app: myapp
  template: ...               # same as Deployment pod template

Drift Detection & Reconciliation

Both Flux and ArgoCD detect when live cluster state diverges from git state (someone ran kubectl edit, an operator mutated a resource). With selfHeal: true (ArgoCD) or automatic reconciliation (Flux), the controller reverts the drift within the next sync interval.

# Flux — check sync status
flux get kustomizations -A
flux get helmreleases -A
flux logs --all-namespaces --level=error

# Force immediate reconciliation
flux reconcile kustomization myapp-production --with-source

# ArgoCD — check app health and sync status
argocd app list
argocd app get myapp
argocd app diff myapp         # show diff between git and live

# Force sync
argocd app sync myapp