Introduction

GitOps extends DevOps practices by using Git as the single source of truth for declarative infrastructure and applications.

Core principles:

1. Declarative description: All infrastructure and applications defined declaratively
2. Git as source of truth: Desired state stored in Git, not in running clusters
3. Automated synchronization: Software automatically syncs cluster state to Git state
4. Pull-based updates: Operators pull changes from Git, not pushed by CI

Benefits:

• Auditable: Every change recorded in git history
• Reproducible: Environment recreation from Git is deterministic
• Fast rollback: Revert to previous commit for instant rollback
• Self-healing: Drift between Git and cluster automatically corrected
• Developer-friendly: Standard git workflows for deployments

ArgoCD Architecture and Installation

ArgoCD runs as a Kubernetes controller and continuously monitors Git repositories, comparing desired state with actual cluster state.

Architecture components:

Git Repository → ArgoCD → Kubernetes Cluster
     ↑              ↓
     ←←←←diff/sync←←←←

Installation:

# Namespace install
kubectl create namespace argocd
kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml

# Or with Helm
helm repo add argo https://argoproj.github.io/argo-helm
helm install argocd argo/argo-cd -n argocd --create-namespace

ArgoCD CLI:

# Install CLI
brew install argocd

# Login (get initial password)
argocd login --insecure --username admin --password $(kubectl get pods -n argocd -l app.kubernetes.io/name=argocd-server -o jsonpath='{.items[0].spec.containers[0].env[?(@.name=="ARGOCD_AUTH_SECRET")].value}') localhost:8080

# Add repo
argocd repo add https://github.com/myorg/manifests --username myuser --password mytoken

# Sync application
argocd app sync myapp

ArgoCD server access:

# ingress.yaml for external access
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: argocd-server-ingress
  namespace: argocd
  annotations:
    nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"
    nginx.ingress.kubernetes.io/ssl-passthrough: "true"
spec:
  ingressClassName: nginx
  rules:
    - host: argocd.mycorp.example.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: argocd-server
                port:
                  number: 443

Application and ApplicationSet Resources

ArgoCD defines applications that point to Git repositories containing Kubernetes manifests.

Basic Application:

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: myapp
  namespace: argocd
spec:
  project: default
  source:
    repoURL: https://github.com/myorg/manifests.git
    targetRevision: HEAD
    path: apps/myapp/overlays/production
  destination:
    server: https://kubernetes.default.svc
    namespace: myapp
  syncPolicy:
    automated:
      prune: true
      selfHeal: true

Helm-based Application:

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: myapp
spec:
  source:
    repoURL: https://github.com/myorg/charts.git
    chart: myapp
    targetRevision: 1.2.0
    helm:
      valueFiles:
        - values-production.yaml
      parameters:
        - name: image.tag
          value: v2.1.0
      releaseName: myapp
  destination:
    server: https://kubernetes.default.svc
    namespace: myapp

ApplicationSet for GitOps automation across clusters:

apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: myapp-multicluster
spec:
  generators:
    - git:
        repoURL: https://github.com/myorg/manifests.git
        revision: HEAD
        directories:
          - path: clusters/production/*
    - clusters:
        selector:
          matchLabels:
            environment: production

  template:
    metadata:
      name: myapp-{{name}}
    spec:
      project: default
      source:
        repoURL: https://github.com/myorg/manifests.git
        path: apps/myapp
        targetRevision: HEAD
      destination:
        server: "{{server}}"
        namespace: myapp
      syncPolicy:
        automated:
          prune: true
          selfHeal: true

Flux Installation and Configuration

Flux uses a different operator model with GitRepository and Kustomization resources.

Installation with Flux CLI:

# Install Flux CLI
brew install fluxcd/tap/flux

# Bootstrap to cluster
flux bootstrap github \
  --owner=myorg \
  --repository=flux-infra \
  --branch=main \
  --path=./clusters/production \
  --personal

Core Flux resources:

# GitRepository defines the source
apiVersion: source.toolkit.fluxcd.io/v1
kind: GitRepository
metadata:
  name: myapp
  namespace: flux-system
spec:
  interval: 1m
  url: https://github.com/myorg/manifests.git
  ref:
    branch: main
  secretRef:
    name: git-credentials
  ignore: |
    # Ignore documentation in subfolders
    /**/*.md

# Kustomization defines what to apply
apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
  name: myapp
  namespace: flux-system
spec:
  interval: 5m
  path: ./apps/myapp/overlays/production
  prune: true
  wait: true
  sourceRef:
    kind: GitRepository
    name: myapp
  targetNamespace: myapp
  healthChecks:
    - apiVersion: apps/v1
      kind: Deployment
      name: myapp
      namespace: myapp

Multi-environment with Flux:

# Production kustomization depends on staging
apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
  name: myapp-staging
spec:
  dependsOn:
    - name: myapp-base
  # ...

---
apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
  name: myapp-production
spec:
  dependsOn:
    - name: myapp-staging # Wait for staging to be healthy
  # ...

Drift Detection and Correction

GitOps operators continuously monitor and correct drift.

ArgoCD drift detection:

# Check application health and sync status
argocd app get myapp

# Show diff between Git and cluster
argocd app diff myapp

# Manually sync after resolving drift
argocd app sync myapp

ArgoCD health assessment:

# Custom health check for custom resources
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: myapp
spec:
  ignoreDifferences:
    - group: apps
      kind: Deployment
      jsonPointers:
        - /spec/replicas # Ignore replica count differences

Flux drift handling:

# Suspend reconciliation
flux suspend kustomization myapp

# Resume reconciliation
flux resume kustomization myapp

# Force reconciliation
flux reconcile kustomization myapp --with-source

GitOps with Helm and Kustomize

Both tools work well with GitOps operators.

HelmRelease with ArgoCD:

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: myapp
spec:
  source:
    chart: nginx
    repoURL: https://charts.bitnami.com/bitnami
    targetRevision: 15.0.0
    helm:
      releaseName: myapp
      values:
        replicaCount: 3
        service:
          type: LoadBalancer

HelmRelease with Flux:

apiVersion: helm.toolkit.fluxcd.io/v2
kind: HelmRelease
metadata:
  name: myapp
spec:
  interval: 1h
  chart:
    spec:
      chart: myapp
      version: 1.0.0
      sourceRef:
        kind: HelmRepository
        name: myorg-charts
  values:
    replicaCount: 3
  install:
    crds: Create
  upgrade:
    crds: CreateReplace

Kustomize with ArgoCD:

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: myapp
spec:
  source:
    repoURL: https://github.com/myorg/manifests.git
    path: apps/myapp/overlays/production
    kustomize:
      commonLabels:
        app.kubernetes.io/managed-by: argocd
      images:
        - name: myapp
          newTag: v2.1.0

When to Use / When Not to Use

When GitOps makes sense

GitOps earns its keep when you need audit trails for infrastructure changes. If your team has ever spent hours tracking down which change caused a production incident, Git history solves that by default. Every deployment is a commit, every rollback is a git revert.

Use GitOps when you manage multiple clusters. Manually keeping staging, production, and disaster-recovery clusters in sync is error-prone. GitOps operators ensure all clusters converge to the same desired state without human intervention.

GitOps also helps when you need compliance documentation. Regulated industries require evidence that production matches approved configurations. Git provides that automatically — the commit hash is the audit log entry.

When to stick with CI-driven deployments

If your team is small and your infrastructure changes infrequently, GitOps adds operational overhead without much benefit. The operator needs maintaining, credentials need rotating, and drift detection needs monitoring. For a single cluster with two engineers, a well-written CI pipeline can do the job without the extra components.

GitOps also does not play well with stateful workloads that modify their own state. A database that accepts writes directly cannot be fully GitOps-controlled because the operator cannot distinguish intentional changes from drift.

GitOps Tool Selection Flow

flowchart TD
    A[Team needs GitOps?] --> B{Multiple clusters?}
    B -->|Yes| C{ArgoCD vs Flux?}
    B -->|No| D[Stick with CI-driven deploys]
    C -->|Need UI and ApplicationSets| E[ArgoCD]
    C -->|Need fine-grained Flux CRDs| F[Flux]
    A --> G{Need audit trail?}
    G -->|Yes| B
    G -->|No| D

ArgoCD vs Flux Comparison

Both operators implement GitOps, but they differ in philosophy and capability.

Aspect	ArgoCD	Flux
UI	Built-in web UI	CLI and external dashboards only
Multi-cluster	ApplicationSets for scale	Cluster API bootstrap
Learning curve	Simpler concepts	Flux CD v2 operators are more granular
Extension model	Plugins and config management	CRD-based operators
Helm support	Native with value overrides	HelmRelease CRD
GitHub integration	Tight with Applications	Reconciler pattern
Maturity	CNCF graduated	CNCF graduated

Choose ArgoCD when you want a UI for non-Kubernetes engineers to view deployment status. Choose Flux when you need deep integration with Kubernetes primitives or when you prefer everything as a custom resource.

Production Failure Scenarios

Common GitOps Failures

Failure	Impact	Mitigation
Git credentials expire	Operator stops syncing, drift accumulates	Use service accounts with token rotation
PR merged with bad YAML	Broken manifests deployed to production	Enable diff-before-sync, require reviews
Large manifest causes timeout	Application stuck in progressing state	Split into smaller Applications
Cluster unreachable	Sync fails, ArgoCD/Flux marks app out-of-sync	Configure retry intervals appropriately
drift detection too sensitive	Constant re-syncing, wasting resources	Configure ignoreDifferences in ArgoCD

Sync Failure Recovery Flow

flowchart TD
    A[Sync Triggered] --> B{Manifest Valid?}
    B -->|Invalid YAML| C[Sync Blocked]
    B -->|Valid| D{Resources Healthy?}
    D -->|No| E[Mark Degraded]
    D -->|Yes| F[Sync Complete]
    C --> G[Check Git commit]
    E --> H[Alert Team]
    F --> I[Monitor for Drift]
    G --> H
    H --> J[Fix and Force Sync]

Secret Rotation Without Disruption

GitOps and secrets require care. Never commit plain-text secrets to Git.

# Use sealed-secrets or external secrets operator
# Encrypt secrets before committing
kubectl create secret generic db-creds \
  --from-literal=password=supersecret \
  --dry-run=client \
  -o yaml | kubeseal --cert pub-cert.pem

# Flux external secrets example
apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
  name: db-creds
spec:
  refreshInterval: 1h
  secretStoreRef:
    name: vault-backend
    kind: ClusterSecretStore
  target:
    name: db-creds
  data:
    - secretKey: password
      remoteRef:
        key: prod/db
        property: password

Observability Hooks

Track GitOps health through sync status, drift metrics, and reconciliation times.

What to monitor:

• Sync status per application (OutOfSync, Synced, Degraded)
• Time since last successful sync
• Reconciliation duration (Flux) or sync duration (ArgoCD)
• Drift count across all managed resources
• Failed sync attempts and error messages

# ArgoCD - check sync status
argocd app get myapp --watch

# ArgoCD - list out-of-sync apps
argocd app list -o wide | grep OutOfSync

# Flux - check reconciliation status
flux get kustomizations

# Flux - check reconciliation logs
flux logs --kind=Kustomization --name=myapp

# Prometheus metrics from ArgoCD
argocd_app_sync_total{app_name="myapp", phase="success"}
argocd_app_sync_total{app_name="myapp", phase="failure"}
argocd_app_metadata{sync_token="abcd123"} # tracks commit SHA

Common Pitfalls / Anti-Patterns

Committing sensitive data to Git

This is the most common GitOps mistake. Sealed secrets, external secrets, or vault integration are mandatory. There is no acceptable reason to put passwords in Git, even in private repositories.

Not using automated sync

Running ArgoCD or Flux in manual sync mode defeats the purpose. The value of GitOps is the automatic convergence to desired state. Manual sync means you have all the operational overhead of GitOps with none of the self-healing benefits.

Overly broad prune policies

With prune: true, the operator deletes resources removed from Git. On a shared cluster with multiple teams, this can cause incidents if your path patterns are too broad. Test prune behavior in staging before enabling in production.

Ignoring sync wave ordering

When deploying multiple applications that depend on each other, the order matters. ArgoCD and Flux both support dependency ordering, but it is not enabled by default. Without it, your database might try to start before the PVC is created.

Using the same repository for everything

Monorepos with thousands of applications create performance problems. The operator scans the entire repo on every change. Split by team or by deployment boundary to keep sync times reasonable.

Trade-off Analysis

Aspect	ArgoCD	Flux	Jenkins X
UI	Full UI dashboard	CLI / WebUI only	Web UI
Multi-cluster	ApplicationSets (native)	Workload partitioning	Limited
Learning curve	Moderate	Steeper	Steep
GitOps model	Pull-based	Pull-based	Pull-based
Helm support	Yes	Yes	Yes
Kustomize support	Yes	Yes (Kustomization)	Yes
Extensibility	Plugins	Go modules	Plugins
Enterprise features	Argo CD Enterprise	Weave GitOps (Enterprise)	Limited

Interview Questions

Further Reading

• ArgoCD Documentation - Official ArgoCD docs
• Flux Documentation - Official Flux CD docs
• GitOps Principles - OpenGitOps principles
• Weave GitOps - Enterprise GitOps (Flux)
• ArgoCD ApplicationSets - Multi-cluster automation
• Flux Multi-Tenant - Tenant isolation patterns

Conclusion

Key Takeaways

• GitOps makes Git the source of truth for both infrastructure and applications
• ArgoCD provides a UI and scales well with ApplicationSets
• Flux uses CRD-based operators for fine-grained control
• Always use external secrets or sealed secrets, never plain-text credentials
• Automated sync with self-heal is what separates GitOps from CI-driven deployments
• Configure ignoreDifferences to prevent alert fatigue from managed fields

GitOps Health Checklist

# Verify ArgoCD sync status
argocd app list

# Check Flux reconciliation
flux get all --namespace flux-system

# View sync history
argocd app history myapp

# Force a clean sync
argocd app sync myapp --force

# Flux: reconcile with source refresh
flux reconcile kustomization myapp --with-source

# Check for drift
argocd app diff myapp

# Verify secrets are not in Git
git log --all --full-history -S "password" -- "*.yaml"