The Cost of Getting Resource Requests Wrong
Resource requests and limits are the most consequential configuration in a Kubernetes cluster, and most teams get them wrong. Over-request, and you're paying for capacity that sits idle. Under-request, and the scheduler packs too many pods onto nodes, leading to CPU throttling and OOM kills during traffic spikes. I've seen both failure modes cost teams significant money and reliability.
The tricky part isn't setting initial values. It's keeping them accurate as your application's resource profile changes with every deployment.
Requests vs Limits: What the Scheduler Actually Does
The scheduler uses requests — not limits — to decide where to place a pod. If your pod requests 500m CPU and 512Mi memory, the scheduler finds a node with at least that much allocatable capacity remaining. The node might have 2 CPU and 4Gi of memory available, and the scheduler will happily place three more pods requesting 500m each on that node.
Limits cap the maximum a container can consume. CPU limits are enforced via CFS (Completely Fair Scheduler) throttling — when a container exceeds its CPU limit, it gets throttled, meaning it waits for its next time slice. Memory limits are enforced with an OOM kill — exceed the limit, and the kernel kills the process.
This distinction matters because the failure modes are completely different. CPU throttling increases latency but the pod keeps running. An OOM kill terminates the pod and potentially loses in-flight work.
The Case Against CPU Limits
There's a growing consensus that setting CPU limits on most workloads does more harm than good. Here's why: CPU is a compressible resource. If a node has spare CPU capacity and your pod is limited to 1 CPU, your pod can't use that spare capacity even though it's sitting there idle. Your pod gets throttled at 1 CPU while the node has 3 CPU doing nothing.
The counterargument is that without CPU limits, a runaway process can starve other pods on the node. That's true, but resource requests already guarantee a minimum allocation. If pod A requests 500m and pod B requests 500m, they'll each get at least 500m even under contention. Limits only matter when there's spare capacity to fight over.
Our team removed CPU limits from most workloads about a year ago. The result: p99 latency dropped 15% because pods weren't being throttled during brief CPU bursts, and our cloud spend didn't increase because the total node capacity stayed the same.
resources:
requests:
cpu: 500m
memory: 512Mi
limits:
# no cpu limit — let pods burst when capacity is available
memory: 1GiRight-Sizing with Actual Metrics
Setting resource requests based on developer estimates is how you end up with every pod requesting 2 CPU and 4Gi when it actually uses 200m and 300Mi. You need actual usage data.
The Vertical Pod Autoscaler (VPA) in recommendation mode is the best tool for this. Install it, create a VPA resource for each workload, and it'll analyze actual CPU and memory usage to suggest appropriate requests:
apiVersion: autoscaling.k8s.io/v1
kind: VerticalPodAutoscaler
metadata:
name: api-server-vpa
spec:
targetRef:
apiVersion: apps/v1
kind: Deployment
name: api-server
updatePolicy:
updateMode: "Off" # recommend only, don't auto-apply
resourcePolicy:
containerPolicies:
- containerName: api-server
minAllowed:
cpu: 100m
memory: 128Mi
maxAllowed:
cpu: "4"
memory: 4GiAfter a week of production traffic, check the recommendations:
$ kubectl describe vpa api-server-vpa
Recommendation:
Container Recommendations:
Container Name: api-server
Lower Bound:
Cpu: 234m
Memory: 287Mi
Target:
Cpu: 412m
Memory: 398Mi
Upper Bound:
Cpu: 893m
Memory: 712MiThe "target" is what VPA thinks you should set as your request. The "upper bound" is a reasonable limit. I typically set requests at the target and memory limits at the upper bound with 20% headroom.
Prometheus Queries for Manual Right-Sizing
If you're not running VPA, you can query Prometheus directly. This query shows the 95th percentile CPU usage over the past 7 days for each container in a namespace:
quantile_over_time(0.95,
rate(container_cpu_usage_seconds_total{
namespace="production",
container!=""
}[5m])[7d:]
) by (pod, container)Compare that against what's requested:
kube_pod_container_resource_requests{
namespace="production",
resource="cpu"
} by (pod, container)The ratio between actual p95 usage and the request tells you how over-provisioned you are. In my experience, most teams are requesting 3-5x what they actually use at peak. That's 3-5x the infrastructure cost for that workload.
Memory: The Resource You Can't Get Wrong
Unlike CPU, memory is incompressible. You can't throttle memory usage — the kernel either grants the allocation or kills the process. This means your memory requests and limits need to account for the worst case, not the average case.
Set memory requests to the p99 working set size — not the RSS, which includes shared memory and cached pages that the kernel can reclaim. The metric you want is container_memory_working_set_bytes:
max_over_time(
container_memory_working_set_bytes{
namespace="production",
container="api-server"
}[7d]
)Set the memory limit 20-30% above the request. This gives your application room for occasional spikes without getting OOM killed, while still protecting the node from a true memory leak consuming everything.
QoS Classes and Eviction Priority
Kubernetes assigns a Quality of Service class to each pod based on its resource configuration. This determines eviction priority when a node runs out of resources:
Guaranteed — requests equal limits for both CPU and memory. These pods are evicted last. Use for critical workloads like databases.
BestEffort — no requests or limits set at all. These are evicted first. Fine for batch jobs that can be rescheduled, terrible for anything user-facing.
Burstable — everything else. Requests are set but don't equal limits. This is where most workloads should land.
The eviction order within the Burstable class depends on which pods are using the most memory relative to their request. A pod using 900Mi with a 500Mi request gets evicted before a pod using 400Mi with a 300Mi request. This is another reason accurate requests matter — under-requesting makes your pod a prime eviction candidate.
Namespace-Level Guardrails
LimitRanges and ResourceQuotas are your safety net. A LimitRange sets default requests and limits for pods that don't specify them, and caps the maximum any single container can request:
apiVersion: v1
kind: LimitRange
metadata:
name: default-resources
namespace: production
spec:
limits:
- default:
memory: 512Mi
defaultRequest:
cpu: 100m
memory: 256Mi
max:
cpu: "4"
memory: 8Gi
type: ContainerA ResourceQuota caps total consumption for the entire namespace. Without one, a team can accidentally scale their deployment to 100 replicas and consume the entire cluster's capacity:
apiVersion: v1
kind: ResourceQuota
metadata:
name: production-quota
namespace: production
spec:
hard:
requests.cpu: "40"
requests.memory: 80Gi
limits.memory: 160Gi
pods: "200"We set these on every namespace in production. The quota numbers should be large enough that normal operations don't hit them, but small enough to prevent one team's incident from affecting everyone else.
Continuous Right-Sizing as a Practice
Right-sizing isn't a one-time project. Application resource profiles change with every feature deployment. A new caching layer reduces CPU usage. A richer response format increases memory allocation. The requests you set three months ago are probably wrong today.
Build a monthly review into your operations cadence. Pull VPA recommendations or Prometheus queries for all production workloads, compare against current requests, and adjust. Automate what you can — some teams run VPA in auto mode for stateless workloads — but always review changes to stateful workloads manually. An automated memory reduction on a database pod that's actually using that memory for its buffer cache will ruin your day.
Handling Java and JVM Applications
JVM applications deserve special mention because their memory behavior confuses Kubernetes. The JVM allocates a heap at startup and manages memory internally. The container's memory usage (as seen by cgroup) includes the heap, metaspace, thread stacks, native memory, and memory-mapped files.
A common mistake: setting Xmx512m and a container memory limit of 512Mi. The JVM heap is 512MB, but total JVM memory usage will be 700-800MB. The container gets OOM killed because the limit doesn't account for non-heap memory.
The fix is to set the JVM heap to roughly 70% of the container memory limit, and configure the JVM to respect container memory limits using UseContainerSupport and MaxRAMPercentage flags. Setting MaxRAMPercentage to 70.0 leaves 30% for non-heap allocations. This has eliminated OOM kills in our Java services.
For Go applications, the memory picture is simpler but GOMAXPROCS needs attention. By default, Go sets GOMAXPROCS to the number of CPUs visible to the process - which in a container is the host's CPU count, not the container's limit. A Go process on a 64-core node will spawn 64 OS threads even if it's limited to 1 CPU. Use the automaxprocs library from Uber to detect container CPU limits correctly.
Pod Priority and Preemption
Priority classes work alongside resource requests to determine scheduling order. When the cluster is full, a high-priority pod can preempt lower-priority pods. This matters for resource sizing because it means your critical services can always find capacity, even during burst periods.
Create priority classes that match your service tiers. System components get the highest priority. Customer-facing services get high priority. Background workers get normal priority. Batch jobs and development workloads get low priority.
The interaction between priority and the Cluster Autoscaler is worth understanding: preempted pods become unschedulable, which triggers the autoscaler to add nodes. The high-priority pod runs immediately on the space freed by preemption, and the evicted pods wait for new nodes. This gives you fast scaling for critical services without maintaining excess capacity.