Kubernetes Resource Management and Autoscaling Strategies
Kubernetes resource requests, limits, QoS classes, Horizontal and Vertical Pod Autoscalers and cluster autoscaling, balanced for cost and reliability.
Kubernetes resource management is the difference between an efficient, responsive cluster and one that either wastes money on unused capacity or falls over during traffic spikes. Getting it right requires understanding how requests and limits interact with the scheduler, how Quality of Service classes determine eviction order, and how the three autoscaler types complement each other.
Most teams either set resources too conservatively—wasting 60% of their cluster capacity—or skip them entirely and discover OOM kills in production.
Requests, Limits, and QoS Classes
Requests tell the scheduler how much capacity a pod needs and are guaranteed. Limits cap the maximum a pod can consume. The combination determines the pod's Quality of Service class.
# ❌ No resource specifications — worst reliability
apiVersion: apps/v1
kind: Deployment
spec:
template:
spec:
containers:
- name: api
image: api:latest
# BestEffort QoS — first to be evicted under pressure
# Scheduler doesn't know how much capacity is needed# ✅ Properly configured resources
apiVersion: apps/v1
kind: Deployment
metadata:
name: api-server
spec:
replicas: 3
template:
spec:
containers:
- name: api
image: api:v2.1.0
resources:
requests:
cpu: "250m" # 0.25 CPU cores guaranteed
memory: "256Mi" # 256 MiB guaranteed
limits:
cpu: "1000m" # Can burst to 1 core
memory: "512Mi" # Hard cap — OOMKilled if exceeded
# Burstable QoS — requests != limits
# Evicted after BestEffort pods under memory pressure
- name: sidecar-proxy
image: envoy:v1.28
resources:
requests:
cpu: "100m"
memory: "64Mi"
limits:
cpu: "100m"
memory: "64Mi"
# Guaranteed QoS — requests == limits
# Last to be evicted, highest priority// Decision framework for setting resources
interface ResourceStrategy {
workloadType: string;
requestGuidance: string;
limitGuidance: string;
qosTarget: "Guaranteed" | "Burstable" | "BestEffort";
}
const strategies: ResourceStrategy[] = [
{
workloadType: "Latency-sensitive API",
requestGuidance:
"Set to p95 usage from production metrics",
limitGuidance:
"CPU: 2-4x request (allow burst). " +
"Memory: 1.5-2x request (prevent OOM)",
qosTarget: "Burstable",
},
{
workloadType: "Background worker",
requestGuidance:
"Set to average usage — can tolerate scheduling delays",
limitGuidance:
"CPU: no limit (let it use idle capacity). " +
"Memory: 2x request",
qosTarget: "Burstable",
},
{
workloadType: "Database / Stateful",
requestGuidance:
"Set requests == limits for predictable performance",
limitGuidance:
"Same as requests — no bursting, no throttling",
qosTarget: "Guaranteed",
},
{
workloadType: "Batch job",
requestGuidance:
"Minimal requests — tolerates preemption",
limitGuidance:
"Memory limit only — let it use available CPU",
qosTarget: "Burstable",
},
];Horizontal Pod Autoscaler (HPA)
HPA adjusts replica count based on metrics. The default CPU-based scaling works for compute-bound workloads, but custom metrics handle the rest.
# CPU-based HPA
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
name: api-server-hpa
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: api-server
minReplicas: 3
maxReplicas: 20
metrics:
- type: Resource
resource:
name: cpu
target:
type: Utilization
averageUtilization: 70
# Custom metric: requests per second from Prometheus
- type: Pods
pods:
metric:
name: http_requests_per_second
target:
type: AverageValue
averageValue: "100"
behavior:
scaleUp:
stabilizationWindowSeconds: 30
policies:
- type: Percent
value: 50
periodSeconds: 60
- type: Pods
value: 4
periodSeconds: 60
selectPolicy: Max # Use the policy that allows more pods
scaleDown:
stabilizationWindowSeconds: 300 # Wait 5 min before scaling down
policies:
- type: Percent
value: 10
periodSeconds: 60
selectPolicy: Min # Use the policy that removes fewer podsThe behavior section is critical. Without it, HPA scales down aggressively during brief traffic dips, causing capacity problems when traffic returns. The stabilization window and conservative scale-down policy prevent thrashing.
Vertical Pod Autoscaler (VPA)
VPA adjusts resource requests based on actual usage. It's most useful for workloads where the right resource values aren't obvious—it observes and recommends.
apiVersion: autoscaling.k8s.io/v1
kind: VerticalPodAutoscaler
metadata:
name: api-server-vpa
spec:
targetRef:
apiVersion: apps/v1
kind: Deployment
name: api-server
updatePolicy:
updateMode: "Off" # Recommendation-only mode
# "Auto" would restart pods to apply new resources
# Start with "Off" to review recommendations first
resourcePolicy:
containerPolicies:
- containerName: api
minAllowed:
cpu: "100m"
memory: "128Mi"
maxAllowed:
cpu: "2000m"
memory: "2Gi"
controlledResources: ["cpu", "memory"]# Check VPA recommendations
kubectl describe vpa api-server-vpa
# Output includes:
# Recommendation:
# Container Recommendations:
# Container Name: api
# Lower Bound: Cpu: 150m, Memory: 200Mi
# Target: Cpu: 350m, Memory: 384Mi
# Uncapped Target: Cpu: 350m, Memory: 384Mi
# Upper Bound: Cpu: 800m, Memory: 700MiVPA and HPA shouldn't both control CPU for the same deployment. The typical pattern: VPA manages memory requests, HPA manages replica count based on CPU or custom metrics.
Cluster Autoscaler Configuration
The cluster autoscaler adds or removes nodes when pods can't be scheduled or nodes are underutilized.
# Cluster autoscaler configuration (Helm values)
autoDiscovery:
clusterName: production
tags:
- k8s.io/cluster-autoscaler/enabled
- k8s.io/cluster-autoscaler/production
extraArgs:
# Don't scale down nodes with running pods from
# kube-system (except DaemonSets)
skip-nodes-with-system-pods: "true"
# Wait 10 minutes before considering a node for removal
scale-down-unneeded-time: "10m"
# Node must be below 50% utilization to be removed
scale-down-utilization-threshold: "0.5"
# Maximum time to wait for a node to be ready
max-node-provision-time: "15m"
# Don't scale down more than 1 node at a time
max-graceful-termination-sec: "600"
# Balance similar node groups
balance-similar-node-groups: "true"
# Handle GPU node groups separately
expander: "priority"// Node pool strategy for mixed workloads
interface NodePoolConfig {
name: string;
instanceType: string;
minNodes: number;
maxNodes: number;
taints: string[];
labels: Record<string, string>;
useCase: string;
}
const nodePools: NodePoolConfig[] = [
{
name: "general",
instanceType: "m5.xlarge",
minNodes: 3,
maxNodes: 20,
taints: [],
labels: { "workload-type": "general" },
useCase: "API servers, web frontends, background workers",
},
{
name: "memory-optimized",
instanceType: "r5.2xlarge",
minNodes: 0,
maxNodes: 10,
taints: ["workload=memory:NoSchedule"],
labels: { "workload-type": "memory" },
useCase: "In-memory caches, data processing",
},
{
name: "spot-batch",
instanceType: "c5.2xlarge",
minNodes: 0,
maxNodes: 50,
taints: ["lifecycle=spot:NoSchedule"],
labels: { "workload-type": "batch", lifecycle: "spot" },
useCase:
"Batch jobs, CI runners — tolerant of interruption",
},
];Putting It All Together: The Autoscaling Stack
The three autoscalers work together in a layered system.
# Complete autoscaling setup for a production API
# 1. Deployment with right-sized resources (from VPA recommendations)
apiVersion: apps/v1
kind: Deployment
metadata:
name: payment-api
spec:
replicas: 3
template:
spec:
containers:
- name: payment-api
resources:
requests:
cpu: "350m"
memory: "384Mi"
limits:
cpu: "1000m"
memory: "768Mi"
topologySpreadConstraints:
- maxSkew: 1
topologyKey: topology.kubernetes.io/zone
whenUnsatisfiable: DoNotSchedule
labelSelector:
matchLabels:
app: payment-api
---
# 2. HPA scales pods based on request rate
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
name: payment-api-hpa
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: payment-api
minReplicas: 3
maxReplicas: 30
metrics:
- type: Pods
pods:
metric:
name: http_requests_per_second
target:
type: AverageValue
averageValue: "80"
behavior:
scaleDown:
stabilizationWindowSeconds: 300
---
# 3. VPA tunes memory requests (recommendation mode)
apiVersion: autoscaling.k8s.io/v1
kind: VerticalPodAutoscaler
metadata:
name: payment-api-vpa
spec:
targetRef:
apiVersion: apps/v1
kind: Deployment
name: payment-api
updatePolicy:
updateMode: "Off"
resourcePolicy:
containerPolicies:
- containerName: payment-api
controlledResources: ["memory"]
# 4. Cluster autoscaler adds nodes when HPA creates
# pods that can't be scheduledKey Takeaways
Resource requests are guaranteed allocations that the scheduler uses for placement decisions—setting them too low causes scheduling on overcommitted nodes, while too high wastes cluster capacity. Quality of Service classes (Guaranteed, Burstable, BestEffort) determine eviction order under memory pressure, so latency-sensitive workloads should have requests close to limits for stable performance. HPA behavior policies prevent scaling thrashing—a 5-minute stabilization window before scale-down and conservative removal rates (10% per minute) avoid capacity problems during brief traffic dips. VPA belongs in recommendation mode initially, observing actual usage to suggest right-sized resource values before enabling automatic updates that restart pods. The three autoscalers layer naturally: VPA right-sizes individual pod resources, HPA adjusts replica count based on load, and cluster autoscaler provisions nodes when pending pods can't be scheduled. Multiple node pools with taints and tolerations let you match workload types to instance types—spot instances for batch jobs, memory-optimized nodes for caches, and general-purpose nodes for APIs.
