GitOps: Managing Infrastructure Through Pull Requests
GitOps workflows that manage Kubernetes infrastructure through pull requests with ArgoCD and Flux: repo structure, sync policies, health checks, rollbacks.
Traditional deployment pipelines push changes into production—a CI server builds artifacts and deploys them to targets. GitOps inverts this: the desired state lives in Git, and a controller running inside the cluster continuously reconciles reality with the declared state. No manual kubectl commands, no imperative scripts, no drift between what you think is deployed and what actually runs.
The pull request becomes the deployment mechanism. Review a manifest change, merge it, and the cluster converges. Roll back a deployment by reverting a commit.
The GitOps Model
GitOps has four principles: declarative configuration, version-controlled desired state, automated application, and continuous reconciliation.
# ❌ Imperative deployment — fragile, no audit trail
# kubectl set image deployment/api api=myapp:v2.3.1
# kubectl scale deployment/api --replicas=5
# kubectl apply -f hotfix-configmap.yaml
# Who ran this? When? Was it reviewed?
# What happens when the cluster drifts?# ✅ Declarative GitOps — Git IS the source of truth
# environments/production/api/deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: api
namespace: production
labels:
app: api
version: v2.3.1
spec:
replicas: 5
selector:
matchLabels:
app: api
template:
metadata:
labels:
app: api
version: v2.3.1
spec:
containers:
- name: api
image: registry.example.com/api:v2.3.1
ports:
- containerPort: 3000
resources:
requests:
cpu: 250m
memory: 256Mi
limits:
cpu: 500m
memory: 512Mi
livenessProbe:
httpGet:
path: /health
port: 3000
initialDelaySeconds: 10
periodSeconds: 15
readinessProbe:
httpGet:
path: /ready
port: 3000
initialDelaySeconds: 5
periodSeconds: 10Every change is a commit. Every deployment is a merge. Every rollback is a revert.
Repository Structure
Separate application code from deployment configuration. Application repos contain source code and CI pipelines. Config repos contain Kubernetes manifests organized by environment.
# infrastructure-config/ (GitOps config repo)
├── base/ # Shared manifests
│ ├── api/
│ │ ├── deployment.yaml
│ │ ├── service.yaml
│ │ ├── hpa.yaml
│ │ └── kustomization.yaml
│ └── worker/
│ ├── deployment.yaml
│ ├── service.yaml
│ └── kustomization.yaml
├── environments/
│ ├── staging/
│ │ ├── api/
│ │ │ ├── kustomization.yaml # Patches for staging
│ │ │ └── replica-patch.yaml
│ │ └── kustomization.yaml
│ └── production/
│ ├── api/
│ │ ├── kustomization.yaml # Patches for prod
│ │ ├── replica-patch.yaml
│ │ └── resource-patch.yaml
│ └── kustomization.yaml
└── argocd/ # ArgoCD application definitions
├── staging.yaml
└── production.yaml
Using Kustomize, base manifests define the common configuration. Environment overlays patch only what differs—replica counts, resource limits, environment variables.
# environments/production/api/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- ../../../base/api
namespace: production
patches:
- path: replica-patch.yaml
- path: resource-patch.yaml
images:
- name: registry.example.com/api
newTag: v2.3.1# environments/production/api/replica-patch.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: api
spec:
replicas: 5ArgoCD Application Configuration
ArgoCD watches the Git repository and syncs the cluster state to match. The Application resource defines what to watch and how to sync.
# argocd/production.yaml
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: production-api
namespace: argocd
finalizers:
- resources-finalizer.argocd.argoproj.io
spec:
project: production
source:
repoURL: https://github.com/myorg/infrastructure-config
targetRevision: main
path: environments/production
destination:
server: https://kubernetes.default.svc
namespace: production
syncPolicy:
automated:
prune: true # Remove resources deleted from Git
selfHeal: true # Revert manual cluster changes
allowEmpty: false # Prevent accidental deletion of all resources
syncOptions:
- CreateNamespace=true
- PrunePropagationPolicy=foreground
- PruneLast=true
retry:
limit: 3
backoff:
duration: 5s
factor: 2
maxDuration: 1m
ignoreDifferences:
- group: apps
kind: Deployment
jsonPointers:
- /spec/replicas # Ignore HPA-managed replicasThe selfHeal: true setting is critical—it means manual kubectl changes get reverted automatically. The source of truth is Git, and the cluster continuously converges toward it.
Automated Image Updates
When a CI pipeline builds a new image, it needs to update the GitOps config repo. This creates a commit that ArgoCD picks up.
# .github/workflows/build-and-update.yml
name: Build and Update Config
on:
push:
branches: [main]
jobs:
build:
runs-on: ubuntu-latest
outputs:
image-tag: ${{ steps.meta.outputs.version }}
steps:
- uses: actions/checkout@v4
- name: Build and push image
id: meta
run: |
TAG="v$(date +%Y%m%d)-${GITHUB_SHA::8}"
echo "version=$TAG" >> $GITHUB_OUTPUT
docker build -t registry.example.com/api:$TAG .
docker push registry.example.com/api:$TAG
update-config:
needs: build
runs-on: ubuntu-latest
steps:
- name: Checkout config repo
uses: actions/checkout@v4
with:
repository: myorg/infrastructure-config
token: ${{ secrets.CONFIG_REPO_TOKEN }}
- name: Update image tag
run: |
cd environments/staging/api
kustomize edit set image \
registry.example.com/api:${{ needs.build.outputs.image-tag }}
- name: Commit and push
run: |
git config user.name "CI Bot"
git config user.email "ci@example.com"
git add .
git commit -m "chore: update api image to ${{ needs.build.outputs.image-tag }}"
git pushHealth Checks and Rollback
ArgoCD monitors resource health after sync. If a deployment fails health checks, you can configure automatic rollback or catch it in the sync status.
# Custom health check for the API
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: production-api
spec:
# ... source and destination config
syncPolicy:
automated:
prune: true
selfHeal: true
# Resource health assessment
info:
- name: deployment-strategy
value: rolling-update# Check sync status
argocd app get production-api
# View sync history
argocd app history production-api
# Rollback to previous sync
argocd app rollback production-api 2
# Or simply revert in Git (preferred — maintains Git as source of truth)
git revert HEAD
git push origin main
# ArgoCD detects the change and syncs backThe Git-based rollback is the correct GitOps approach. Rather than using ArgoCD's rollback command, revert the commit in Git. This keeps the Git repository as the single source of truth and generates a clear audit trail with the revert commit message explaining why the rollback happened.
Drift Detection and Alerts
Even with selfHeal enabled, you should monitor for drift attempts—they often indicate operational issues or team members bypassing the GitOps workflow.
# Prometheus alert for ArgoCD sync failures
apiVersion: monitoring.coreos.com/v1
kind: PrometheusRule
metadata:
name: argocd-alerts
namespace: monitoring
spec:
groups:
- name: argocd
rules:
- alert: ArgoCDSyncFailed
expr: |
argocd_app_info{sync_status="OutOfSync"}
* on(name) group_left()
(time() - argocd_app_info{sync_status="OutOfSync"})
> 300
for: 5m
labels:
severity: warning
annotations:
summary: >-
Application {{ $labels.name }}
has been out of sync for more than 5 minutes
description: >-
Check ArgoCD for sync errors.
Run: argocd app get {{ $labels.name }}
- alert: ArgoCDAppDegraded
expr: |
argocd_app_info{health_status="Degraded"} == 1
for: 3m
labels:
severity: critical
annotations:
summary: >-
Application {{ $labels.name }}
health is degradedKey Takeaways
GitOps makes Git the single source of truth for infrastructure state—every deployment is a merge, every rollback is a revert, and the cluster continuously reconciles toward the declared state with no imperative commands needed. Separate application code repositories from infrastructure configuration repositories so that image builds trigger config updates through automated commits, maintaining a clear boundary between what the application is and how it runs. Enable selfHeal and prune in ArgoCD sync policies to ensure manual cluster changes get automatically reverted and deleted resources get cleaned up, preventing configuration drift that accumulates when teams bypass the GitOps workflow. Monitor for out-of-sync states and degraded health with alerts because drift attempts often indicate process gaps or operational issues—catching them quickly preserves the integrity that makes GitOps valuable.
