Kubernetes Resource Limits: CPU, Memory, and Quality of Service

Kubernetes clusters have finite resources. Nodes have CPU and memory that pods consume. Without resource constraints, one pod can starve others, degrade cluster stability, and make scheduling unpredictable. Kubernetes provides mechanisms to declare resource requirements, set hard limits, and classify pods by their importance.

This post explains requests vs limits, Quality of Service classes, and the namespace-level constraints that keep clusters healthy.

If you need Kubernetes fundamentals first, see the Kubernetes fundamentals post. For advanced scheduling patterns, see the Advanced Kubernetes post.

Introduction

Nodes have finite CPU and memory. Pods consume both. Without explicit resource declarations, the scheduler picks nodes based purely on capacity, and running containers can grab as much as they want. The result is noisy neighbors starving other pods, or node-level OOM events taking down services you did not even know were sharing the machine.

Resource requests and limits fix this. Requests tell the scheduler what a container needs to run. Limits cap what it can consume before Kubernetes steps in. Those declarations also assign your pods a Quality of Service class that determines eviction order when resources run short. Getting these values right is one of the most direct things you can do for cluster stability.

This post walks through requests and limits, the three QoS classes, and the namespace constraints (LimitRange, ResourceQuota) that keep one team from eating the entire cluster. You will also learn how to read actual resource usage with kubectl top and Vertical Pod Autoscaler so you are not guessing in the dark.

Requests vs Limits Explained

Every container in a pod can specify resource requests and resource limits:

apiVersion: v1
kind: Pod
metadata:
  name: web-app
  namespace: production
spec:
  containers:
    - name: web-app
      image: nginx:1.25
      resources:
        requests:
          memory: "256Mi"
          cpu: "250m"
        limits:
          memory: "512Mi"
          cpu: "500m"

Requests define what a container needs. The scheduler uses requests to decide which node to place the pod on. A node must have at least as much allocatable resources as the pod’s requests.

Limits define the maximum resources a container can use. When a container hits its memory limit, Kubernetes terminates and restarts it. When it hits its CPU limit, Kubernetes throttles the container.

CPU representation

CPU is measured in cores. You can express it as a whole number (1 CPU = 1 core) or millicores (1000m = 1 CPU). Common values:

• 100m = 0.1 CPU (one-tenth of a core)
• 250m = 0.25 CPU
• 500m = 0.5 CPU
• 1000m = 1 CPU

CPU is compressible. If a container exceeds its CPU limit, Kubernetes throttles it. The container does not get killed for CPU alone.

Memory representation

Memory is measured in bytes. You can use suffixes: Ki (kibibytes), Mi (mebibytes), Gi (gibibytes).

• 128Mi = 128 mebibytes (~134 MB)
• 256Mi = 256 mebibytes (~268 MB)
• 1Gi = 1 gibibyte (~1.07 GB)

Memory is not compressible. If a container exceeds its memory limit, Kubernetes terminates it with an OOMKilled status.

What happens when limits are exceeded

State: Terminated
Reason: OOMKilled
Exit Code: 137

Exit code 137 indicates the container was killed by the OOM (Out of Memory) killer. Frequent OOMKilled pods indicate you need to increase memory limits or optimize the application’s memory usage.

QoS Classes

Kubernetes assigns pods to Quality of Service classes based on their resource requests and limits:

QoS Class	Criteria	Behavior
Guaranteed	requests == limits for all containers	Last to be evicted
Burstable	requests < limits (or some limits not set)	Evicted after Guaranteed
BestEffort	No requests or limits set	First to be evicted

When to Use Each QoS Class

The table above lays out the criteria, but the decision comes down to three factors: how critical the workload is, whether it needs burst capacity, and whether you can afford to reserve full resources.

Use Guaranteed for anything that cannot go down. Databases with persistent connections, licensed software that checks hardware fingerprints, and control plane components that other pods depend on all fit here. The cost is reservation: a Guaranteed pod with a 2Gi memory request holds that 2Gi even at 2% utilization. If you have 50 Guaranteed pods and most are idle, you are paying for idle capacity across the cluster.

Use Burstable for the majority of application workloads. A web API that sits at 5% CPU most of the day but spikes to 60% during traffic surges belongs in Burstable. It gets enough resources to stay scheduled, can use extra capacity when available, and does not force you to reserve peak-level resources for every hour of the day. Most Kubernetes workloads fall here by default when you set requests and limits based on observed usage.

Use BestEffort only for workloads you can afford to lose at any moment. A batch job that picks up work from a queue, processes it, and reports results can tolerate eviction if the work is idempotent or gets re-queued. CI agents that run ephemeral builds, dev environment pods that developers restart manually, and any workload where the cost of eviction is near zero are candidates. The moment someone cares whether the pod stays running, it should not be BestEffort.

Rule of thumb: Most workloads should be Burstable. Set Guaranteed for workloads that must not be evicted under any circumstances. Never run production workloads as BestEffort.

QoS Decision Flow

flowchart TD
    A[Pod submitted] --> B{Requests == Limits\nfor all containers?}
    B -->|Yes| G[Guaranteed QoS]
    B -->|No| C{Any requests\nor limits set?}
    C -->|Yes| Bu[Burstable QoS]
    C -->|No| BE[BestEffort QoS]
    Bu --> D{Node under\nmemory pressure?}
    BE --> D
    G --> D
    D -->|Guaranteed| L1[Last to evict]
    D -->|Burstable| L2[Middle eviction]
    D -->|BestEffort| L3[First to evict]

Guaranteed pods

containers:
  - name: database
    image: postgres:15
    resources:
      limits:
        memory: "2Gi"
        cpu: "2000m"
      requests:
        memory: "2Gi"
        cpu: "2000m"

Pods with identical requests and limits get the highest QoS. Use this for critical workloads that should not be evicted.

BestEffort pods

containers:
  - name: batch-job
    image: my-batch-job
    resources: {}

No resource specifications means BestEffort. These pods are first in line for eviction when the node runs low on resources.

Burstable pods

containers:
  - name: web-app
    image: nginx:1.25
    resources:
      requests:
        memory: "128Mi"
        cpu: "100m"
      limits:
        memory: "512Mi"
        cpu: "500m"

Most pods fall into Burstable. They have some guaranteed resources but can burst above their requests when available.

LimitRange for Namespace Quotas

A LimitRange sets default, minimum, and maximum resource limits for pods and containers in a namespace. Without it, pods without resource specs become BestEffort.

apiVersion: v1
kind: LimitRange
metadata:
  name: default-limits
  namespace: production
spec:
  limits:
    - max:
        memory: "4Gi"
        cpu: "2000m"
      min:
        memory: "64Mi"
        cpu: "50m"
      default:
        memory: "256Mi"
        cpu: "250m"
      defaultRequest:
        memory: "128Mi"
        cpu: "100m"
      type: Container

This LimitRange:

• Sets maximum memory and CPU per container
• Sets minimum memory and CPU per container
• Applies default limits when containers specify no limits
• Applies default requests when containers specify no requests

Without this LimitRange, a container with no resource specs gets no guaranteed resources and can be evicted first.

Applying LimitRange

kubectl apply -f limitrange.yaml
kubectl describe limitrange default-limits -n production

The output shows the actual limits applied:

Type        Resource  Min   Max   Default Request  Default Limit
Container    cpu       50m   2     100m             250m
Container    memory    64Mi  4Gi   128Mi            256Mi

ResourceQuota for Cluster-Wide Limits

A ResourceQuota limits total resource consumption in a namespace. Use it to prevent any single namespace from consuming all cluster resources.

apiVersion: v1
kind: ResourceQuota
metadata:
  name: production-quota
  namespace: production
spec:
  hard:
    requests.cpu: "10"
    requests.memory: "20Gi"
    limits.cpu: "20"
    limits.memory: "40Gi"
    pods: "50"
    persistentvolumeclaims: "10"

This quota limits the entire production namespace to 10 CPU requests, 20Gi memory requests, 20 CPU limits, 40Gi memory limits, 50 pods, and 10 persistent volume claims.

Viewing quota usage

kubectl describe resourcequota production-quota -n production

The output shows current usage against the hard limits. When a quota is exhausted, Kubernetes rejects new resource creation in that namespace.

The kubectl describe output breaks down usage by resource type, so you can see how much of your 20Gi memory request limit is currently consumed across all pods in the namespace. It also shows pods count, persistentvolumeclaims, and any other objects the quota tracks.

If usage is approaching a hard limit, you have a few options: scale down pods, increase the quota (if cluster headroom exists), or clean up unused resources. Unlike LimitRange (which enforces per-container constraints at pod creation), ResourceQuota enforces aggregate limits across the entire namespace. Both constraints work together — a pod that passes the LimitRange checks can still get rejected by ResourceQuota if the namespace has already consumed its total allocation.

For programmatic monitoring, query the quota status directly:

kubectl get resourcequota production-quota -n production -o jsonpath='{.status}' | jq

This returns the current usage and hard limits as JSON, which you can feed into alerting tools. Set up an alert when usage exceeds 80% of any hard limit so you catch quota exhaustion before it blocks new deployments.

Pod Resource Testing and Tuning

Finding the right requests and limits takes measurement. Kubernetes lets you profile pod behavior before setting values in production.

kubectl run with resource specs

kubectl run -it --rm load-generator \
  --image=busybox \
  --restart=Never \
  -- requests.cpu=500m \
  -- requests.memory=128Mi \
  --limits.cpu=1000m \
  --limits.memory=256Mi \
  -- sh

Use this to run temporary pods and observe resource consumption with monitoring tools.

Vertical Pod Autoscaler

The Vertical Pod Autoscaler (VPA) analyzes historical resource usage and recommends or automatically applies better resource requests:

apiVersion: autoscaling.k8s.io/v1
kind: VerticalPodAutoscaler
metadata:
  name: web-app-vpa
  namespace: production
spec:
  targetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: web-app
  updatePolicy:
    updateMode: "Auto"

In Auto mode, VPA evicts and reschedules pods with updated resource specs. In Off mode, it only provides recommendations.

VPA helps you find baseline resource requirements without manual profiling.

Horizontal Pod Autoscaler

The Horizontal Pod Autoscaler (HPA) scales pod replicas based on CPU, memory, or custom metrics:

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: web-app-hpa
  namespace: production
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: web-app
  minReplicas: 3
  maxReplicas: 10
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 70

The HPA scales between 3 and 10 replicas to maintain 70% average CPU utilization. For memory-based scaling:

metrics:
  - type: Resource
    resource:
      name: memory
      target:
        type: Utilization
        averageUtilization: 80

Trade-off Analysis

Requests vs Limits Trade-offs

Aspect	Requests Only	Requests + Limits
Scheduling	Predictable node placement	Predictable scheduling + resource capping
QoS class	Burstable (best available)	Guaranteed if equal, otherwise Burstable
Memory protection	None	Prevents OOMKill from affecting node
CPU behavior	No throttle, unlimited burst	CPU throttled at limit
Production readiness	Insufficient	Required for production

QoS Class Trade-offs

QoS Class	Scheduling guarantee	Eviction priority	Resource efficiency	Use when
Guaranteed	Highest	Last evicted	Low (reserved full)	Critical infrastructure, licensed software
Burstable	Medium	Middle	High (flexible)	Most application workloads
BestEffort	None	First evicted	Highest	Batch jobs only

Trade-off reality: Guaranteed pods reserve their full resource request even when idle. BestEffort pods get whatever is left, making them efficient but risky for production.

LimitRange vs ResourceQuota

Aspect	LimitRange	ResourceQuota
Scope	Per namespace, per container	Per namespace, total namespace
What it controls	Min/max/default requests/limits	Total CPU, memory, pod count
Enforcement	Applied when pod is created	Checked when resources are created
Use case	Prevent BestEffort pods, set defaults	Prevent namespace from monopolizing cluster

Vertical Pod Autoscaler vs Manual Resource Tuning

Aspect	VPA (Automated)	Manual Tuning
Effort	Low	High (profiling required)
Precision	Based on historical data	Based on engineered judgment
Disruption	Pod evictions in Auto mode	None (no changes unless you apply)
Best for	Initial baseline discovery	Production fine-tuning

CPU Throttling Trade-offs

Aspect	With CPU Limits	Without CPU Limits (Guaranteed)
Latency	p99 spikes from throttling	Consistent latency
Resource cost	More efficient node utilization	May over-provision requests
Fairness	Prevents one pod dominating CPU	Pod can use all allocated CPU
Recommendation	Avoid for latency-sensitive svcs	Use for critical services

For latency-sensitive services where consistent p99 latency matters, set CPU limits equal to requests for Guaranteed QoS, or remove CPU limits entirely and rely on requests for scheduling.

Memory Limit Trade-offs

Aspect	Low memory limit	High memory limit
OOM risk	Frequent OOMKilled pods	Rare OOMKilled pods
Node efficiency	Higher (more pods per node)	Lower (reserved memory)
Debugging	Easier to spot (frequent crashes)	May hide memory leaks
Recommendation	Set ~20-30% above expected peak	Set for expected peak + headroom

Production Failure Scenarios

BestEffort Pods Evicted Under Memory Pressure

BestEffort pods are first in line for eviction when a node runs low on memory. If your cluster has a lot of them, eviction events pile up fast.

The kubelet’s eviction manager handles this. It periodically scans resource usage on the node and evicts pods when available memory drops below a threshold. It ranks pods by QoS class first, then by actual memory usage relative to requests. BestEffort pods go first — a pod using 10Mi gets evicted before a Burstable pod using 1Gi, purely because of its QoS classification. BestEffort pods are unsuitable for any workload where you actually care about uptime.

Eviction order: BestEffort pods first, then Burstable pods whose usage exceeds their requests, then Guaranteed pods as a last resort.

Symptoms to watch for:

• Frequent evictions in kubectl get events --all-namespaces | grep Evicted
• Pods restarting with OOMKilled as the last termination reason
• One namespace with constant churn while the rest of the cluster looks fine

Mitigation:

1. Set resource requests for all production pods. Even minimal requests promote a pod to Burstable QoS.
2. Use LimitRange to enforce minimum resource requests per namespace so new containers cannot become BestEffort.
3. Monitor kube_pod_info grouped by QoS class to track how many BestEffort pods exist in each namespace.
4. For batch workloads that genuinely have no resource requirements, isolate them in a dedicated namespace with their own ResourceQuota.

# Check eviction history in a namespace
kubectl get events --namespace production --field-selector reason=Evicted

# Count pods by QoS class
kubectl get pods -o jsonpath='{range .items[*]}{.metadata.name}{"\t"}{.status.qosClass}{"\n"}{end}' | sort | uniq -c

OOMKilled Pods from Memory Limit Too Low

When a container exceeds its memory limit, Kubernetes kills it with OOMKilled status. This is one of the most common production issues I see. The fix is almost always raising the memory limit.

Check actual memory usage first:

kubectl describe pod <pod-name>  # Look for OOMKilled in state
kubectl top pod <pod-name>  # Check actual memory usage

Set limits about 20-30% above what you see in staging to account for traffic spikes.

CPU Throttling Impacting Latency

CPU limits throttle containers even when the node has free CPU. For latency-sensitive services, this is where things go sideways.

Kubernetes enforces CPU limits through the Linux CFS (Completely Fair Scheduler) bandwidth control. When a container hits its CPU limit within a scheduling period, the kernel withholds CPU time until the next period. Kubernetes does not kill the container — it just makes it wait. The container sits there unable to use the CPU sitting idle on the node because its bandwidth limit for this period is already consumed.

You see this in latency metrics. A service that normally responds in 5ms p99 might show 50ms or 200ms p99 under CPU throttling even though the node has idle cores available. Average CPU masks this because the container runs fine most of the time and only slows down during throttled periods.

Diagnosing CPU throttling:

# Check CPU throttling metrics for a container
kubectl exec -it <pod-name> -- cat /sys/fs/cgroup/cpu/cpu.stat

# Look for throttling in container metrics
kubectl top pods --containers --show-labels | grep throttle

The metric to watch is nr_throttled. If it is high relative to the total period count, the container is regularly hitting its CPU limit.

High p99 latency with low average CPU usage is the telltale sign. If your latency charts show spikes but average CPU stays below 50%, CPU throttling is the cause.

Workloads where this matters most:

• API servers with strict SLA latency targets (p99 < 50ms)
• Real-time data pipelines processing events with deadlines
• gRPC services where a single slow response blocks a client thread
• Any service where request queuing is not an option

Remediation:

1. Remove CPU limits entirely. The container still respects its request for scheduling, but once scheduled it can burst above the request when free CPU exists.
2. Set CPU limits equal to requests. This promotes the pod to Guaranteed QoS and eliminates throttling at the cost of reserved capacity. If the service needs 250m of CPU to handle normal load, set both request and limit to 250m.
3. Right-size the CPU limit based on actual peak measurement. Use VPA in recommendation mode first to understand realistic burst peaks, then set limits to cover those peaks without overprovisioning.

For batch workloads where latency is irrelevant, CPU limits are useful to prevent one job from monopolizing available CPU. For latency-sensitive services, treat CPU limits as a knob to use carefully or not at all.

Anti-Patterns

Setting Identical Requests and Limits for All Pods

Treating every pod the same wastes resources. A web server handling 1000 req/s has different needs than a batch job processing queues.

Profile each application type separately and set appropriate resource specs.

Workload profiles fall into a few distinct categories:

Workload Type	CPU Profile	Memory Profile	Example
Request-driven web servers	Moderate, stable	Moderate, bounded	API gateways, frontend services
Background workers	Low CPU, variable memory	Low CPU, higher memory	Queue processors, batch transformers
Data-intensive services	High CPU, high memory	High memory, possible growth	ML inference, image processing
Cached services	Low CPU	High memory, no growth	Redis, in-memory aggregators

A web server handling 1000 requests per second might need 250m CPU and 256Mi memory under normal load. Under peak load with 5000 req/s, it might need 1000m CPU and 512Mi memory. A batch job that runs every hour might need 500m CPU but only 128Mi memory because it processes one message at a time and releases memory immediately after.

If you set identical values for both, the web server gets throttled during traffic spikes while the batch job wastes resources it never uses. Run kubectl top pod over a representative period to establish baselines before setting production values.

Not Setting Memory Limits

Memory limits prevent runaway processes from consuming all node memory and causing node-level OOM events. Always set memory limits, especially for applications that can experience memory leaks.

Without memory limits, a single container can consume all available memory on a node. When that happens, the Linux kernel triggers an OOM killer at the node level. Unlike container-level OOMKilled events (which kill only the offending container), node-level OOM kills can take down every pod running on that node.

The distinction matters:

Scenario	What Gets Killed	Who Is Affected
Container hits memory limit	Only that container	Just your pod
Node runs out of memory	All containers on the node	Every workload sharing the node

Memory leaks are a frequent culprit. A Java app with a growing heap, a Python process that keeps objects around, a Go service whose goroutine stack keeps expanding — all of these can quietly fill gigabytes of node memory over hours or days. Without a container-level memory limit, nothing stops them.

Even well-behaved applications that do not leak memory can cause issues. A service that uses 128Mi during quiet hours and 1Gi at peak traffic absorbs whatever is available, but that memory is not returned to the node until the container stops. Other pods on the same node get starved during the quiet hours because the first pod is still holding memory it is not actively using.

Set memory limits to the expected peak plus 20-30% headroom. If staging shows a service using up to 800Mi under load, set the limit to 1024Mi (1Gi). This gives the container room to handle bursts without being killed, while preventing a single runaway process from consuming the entire node.

Over-Provisioning CPU Limits

Setting CPU limits very high (like 4 cores for a simple web server) defeats the purpose of limits. The scheduler uses requests for node allocation decisions, not limits.

Set CPU limits based on actual expected peak load, not theoretical maximum.

The scheduler decides pod placement using requests, not limits. Set requests to 100m and limits to 4000m, and the scheduler only guarantees 100m of CPU when picking a node. The4000m limit does not reserve 4000m on any node — it just caps what the container can use if it tries to burst.

This creates a false sense of capacity. Imagine Node A has 10 pods with 100m requests each (1000m total), placed on a node with 4 cores. Each pod has a 4000m limit, but those limits are invisible to the scheduler. The node is oversubscribed: it has 4000m of CPU but the pods collectively want up to 40000m if they all hit their limits at once. Kubernetes allows this because only requests matter for placement.

When traffic spikes hit all 10 pods simultaneously, each tries to use close to its limit. The node has only 4 cores to share. Kubernetes throttles them all, and p99 latency spikes because none of them can get the CPU they are trying to use.

Base CPU limits on realistic peak load, not theoretical maximums. A web server that processes requests in 10ms of CPU time per request and handles 100 concurrent requests might need 1000m of CPU for brief bursts. Setting the limit to 4000m “just in case” provides no scheduling benefit and hides the actual resource picture.

For latency-sensitive services, set CPU limits equal to requests for Guaranteed QoS, or drop CPU limits entirely and let the service use whatever CPU is available. For batch workloads that benefit from bursting, set limits high enough to capture realistic burst peaks but not so high that they hide an undersized request.

Interview Questions

Further Reading

• Kubernetes Official Documentation - Managing Resources
• QoS Classes
• LimitRange
• ResourceQuota
• Vertical Pod Autoscaler
• Horizontal Pod Autoscaler

Conclusion

Use this checklist when configuring Kubernetes resource limits:

• Set resource requests for all production containers (scheduler uses these for placement)
• Set memory limits to prevent runaway processes from impacting node stability
• Set CPU limits based on actual application needs, not maximum theoretical values
• Used LimitRange to enforce default requests/limits per namespace
• Used ResourceQuota to cap total namespace resource consumption
• Profile applications with VPA or monitoring tools before setting production values
• Monitor OOMKilled events and adjust memory limits accordingly
• Monitor CPU throttling metrics and adjust CPU limits if latency issues appear
• Set Guaranteed QoS (requests == limits) for critical infrastructure pods
• Avoid BestEffort for any production workload