,

Kubernetes Overprovisioning: Why Requests Drift and How to Fix It

Learn why Kubernetes resource requests drift above workload needs, how to measure CPU and memory gaps, and how to validate safer rightsizing changes.

Laurent Gil Avatar
Featured image for Kubernetes Overprovisioning: Why It Happens

Kubernetes request overprovisioning means assigning pod CPU or memory requests above the capacity a workload needs to meet its performance and reliability goals. It commonly develops through defensive estimates, copied manifests, and incident increases that nobody revisits. The scheduler reserves room using requests, so persistent padding can reduce node packing and increase infrastructure cost. Detect it by comparing requests with representative usage and application performance, then validate gradual adjustments. Requests and limits serve different purposes: exceeding a memory request does not, by itself, trigger an OOM kill.

This guide focuses on pod request drift: why CPU and memory settings become stale and how to review them safely. Deliberate node capacity overprovisioning uses low-priority placeholder pods to keep spare capacity available. That is a separate latency-versus-cost decision, not necessarily waste.

Key takeaways

  • Request overprovisioning is a gap between workload reservations and demonstrated needs; deliberate spare node capacity is a separate decision.
  • Requests influence scheduling and contention. Limits impose runtime ceilings, so lowering a request is not the same as lowering a limit.
  • Measure provisioned, requested, and used capacity separately. The reportโ€™s 69% figure describes provisioned versus requested CPU, not requests versus usage.
  • Use historical usage and application performance to identify candidates; no percentile or headroom multiplier guarantees safe sizing.
  • Review HPA interactions, pilot changes gradually, and pair rightsizing with node consolidation to turn released capacity into potential savings.

Why do Kubernetes requests only ever go up?

Every request value in your cluster is a small decision made under pressure. Nobody sets a request too high on purpose. They set it high enough to stop the pager. Then the value becomes permanent, because nothing in the default workflow pushes it back down.

The incentive problem: punished for OOMKills, never for waste

Reliability incidents get immediate attention; unused reservations often remain invisible to the team deploying a service. Kubernetes scheduling and memory-pressure behavior add real trade-offs, but a larger request is not blanket protection.

Under memory pressure, kubelet ranks pods by whether usage exceeds requests, pod priority, and usage relative to requests. A pod using 600Mi against a 500Mi request is above its request; one using 600Mi against 700Mi is not. Both 700Mi and 2Gi requests exceed that usage, so those values alone do not establish which pod will be evicted first. See the Kubernetes eviction rules.

For node-level OOM selection, kubelet assigns an oom_score_adj influenced by QoS and, for Burstable containers, memory requests. Actual memory consumption also affects the final score; a larger request does not guarantee survival. Container-limit OOM kills are a separate constraint. CPU requests affect scheduling and CPU shares under contention, while CPU limits impose a runtime ceiling. Keep these mechanisms separate when investigating a failure.

The waste side has no owner. Nobody gets paged for a node packed at 12% utilization. Nobody opens a postmortem because requests sat far above usage, because there is no incident. The cost lands on a cloud bill in a different team’s spreadsheet, months later. So the rational move is to request more, especially for an engineer optimizing for a quiet night. That is a scheduler design working exactly as documented.

Drift accelerates because nothing ever pulls requests back

The 2026 State of Kubernetes Optimization Report reports CPU overprovisioning rising from 40% to 69% year over year. Its accompanying text describes the gap between provisioned capacity and what workloads request. This is a different measurement from the request-to-usage ratio used below. The report comparison does not establish that an identical set of workloads was followed without configuration changes.

Two mechanisms can widen the gap. Workload templates and incident fixes introduce conservative resource values, while requests often remain unchanged after releases alter demand. Node autoscalers use scheduling requirements to provision capacity; workload autoscalers can use observed resource metrics to adjust requests. Saying that all autoscalers ignore usage would conflate these distinct controllers.

Average utilization across the fleet tells the same story from the other end. You can provision enormous amounts of compute and still be technically “healthy.” Health checks look at error rates, not at the distance between what you asked for and what you got.

The three drivers of request drift

If you want to fix this inside your own cluster, the useful move is to find where the padding enters. In practice it comes from three places. Each one leaves a fingerprint you can see in git.

DriverHow it looks in git historyThe tellThe fix
Defensive padding at deploy timeInitial commit sets requests several times the observed usage, often with a comment like # conservative or a copied value from an example chartRequest-to-P95 ratio above 2x on a workload that has never been scaled or tunedUse representative CPU and memory measurements, startup peaks, and service objectives; validate changes gradually
Manifests copied with their requests attachedNew service added by cloning an existing Deployment, with resources: carried over unchanged while image and args differThe same request values appear across services with very different traffic shapesParameterize the template and require reviewed, workload-specific resource values
Incident bumps that never get revertedA one-line commit raising CPU or memory mid-incident, usually with a hotfix branch and no follow-up commitRequest values that rose in the same hour as a rollback or a deploy of a hotfixPair every incident bump with a ticket that has a date, an owner, and a re-measurement window

Defensive padding at deploy time

The first commit is where most of the gap is born. A team ships a new service, has no production data yet, and picks a number that feels safe. Usually that number is calibrated against a staging environment with a fraction of the traffic. Sometimes it is copied from a chart default written for a different workload entirely. Because the service then runs fine, the value gets promoted from a guess to an assumption.

Compare each workloadโ€™s requests with representative usage over an observation window that includes its traffic cycle. A request-to-P95 CPU ratio above 2x is a review signal, not proof of unnecessary capacity. Check latency, throttling, startup demand, and missed peaks before lowering a value.

Manifests copied with their requests attached

Copy-paste is the quietest driver, because the resulting manifests look intentional. A new API service gets created by cloning the frontend’s Deployment, editing the image and the args, and leaving resources: untouched. Now a batch job with one request per second carries the same 2 CPU request as a latency-sensitive gateway. Nobody chose that.

You can see the signature at the cluster level. When the same request values recur across services with vastly different traffic shapes, you are looking at inherited numbers, not measured ones. Templates and Kustomize bases make sense for everything except the resources block, which has to be workload-specific by definition.

Incident bumps that never get reverted

The third driver is the most defensible in the moment. It is also the most expensive over time. Something is spiking, the on-call raises CPU during the incident, the spike passes, everyone moves on. The commit stays. Do that a handful of times across a year and requests climb monotonically, even as the workload’s real profile flattens.

These bumps are also the easiest to find. Correlate resource changes against deploy and rollback events, and the incident-driven ones cluster tightly around hotfix commits. If your team runs a GitOps loop, a manual kubectl edit gets reverted by the reconciler. A committed bump persists instead, which is why committed bumps dominate long-term drift. This process, where configuration diverges from intent over time, is the subject of our piece on Kubernetes configuration drift.

How drift compounds: releases change profiles, requests stay

A request set once is a snapshot of a workload that no longer exists. The container image changes, the dependency versions move, the traffic mix shifts, and the request stays exactly where it was. Multiply that across a few dozen releases a year per service. The number drifts far from the workload it is supposed to describe.

Runtime upgrades make this vivid. A newer JVM or Node runtime can change baseline memory by hundreds of megabytes. A GC change can shift CPU in ways no one predicted. The Go or Python upgrade on your roadmap will do something to the profile. Your manifest will not notice.

Request changes also have second-order effects that hide the problem. Requests drive scheduling, as covered in our guide to Kubernetes requests and limits. An inflated request therefore reduces how many pods fit on a node. Fewer pods per node means more nodes, which means the autoscaler buys capacity that becomes provisioned compute nobody asked for. The gap between provisioned, requested, and used capacity is worth measuring separately. Our piece on Kubernetes capacity planning walks through that arithmetic.

For an illustrative calculation, suppose a cluster has 44.87 vCPU provisioned, 24.9 vCPU requested, and average usage of 3.94 vCPU over the same observation period. These are CPU-capacity and usage measurements, not โ€œvCPU per hourโ€; this example is not presented as a verified customer result.

Requests are about 6.3 times usage (24.9 รท 3.94). Provisioned capacity is about 11.4 times usage (44.87 รท 3.94), and utilization is approximately 8.8% (3.94 รท 44.87). The request-to-usage gap is 20.96 vCPU; the provisioned-to-requested gap is 19.97 vCPU. Measure them separately rather than treating one percentage as both.

These illustrative gaps identify different investigations. Review workload reservations for the request-to-usage gap, and node packing, constraints, and availability requirements for the provisioned-to-requested gap. Neither gap translates directly into guaranteed bill savings.

There is one more compounding loop. When you over-request and the cluster autoscaler adds nodes, the fleet gets more headroom than it needs. That headroom hides the next incident, which quietly rewards the team that padded. Each cycle reinforces the last, and the cluster gets more expensive to run while getting no more reliable.

How do I detect overprovisioned requests?

Detection beats guessing, and the method matters more than the tooling. If a service is genuinely unused rather than merely over-requested, follow the idle Kubernetes resources audit before tuning its requests. Two common measurement shortcuts fail here.

The first shortcut is a snapshot. Sampling usage at one moment tells you what happened at that moment, not what the workload needs. Burstable traffic looks over-requested at 3 a.m. and under-provisioned at the peak.

Averages can hide peaks by smoothing them together with idle periods. Startup can also produce peaks, not just low utilization. Compare historical percentiles and maxima with application performance before deciding whether a request is excessive.

Use representative historical usage as a screening baseline. Fourteen days can cover weekly traffic, but seasonal peaks, infrequent jobs, and recent releases may need another window. P95 CPU and P99 memory are useful starting measurements, not universally safe request values. Check memory peaks and runtime behavior separately; percentile sampling can miss short spikes.

Choose headroom from measured peaks, startup behavior, latency objectives, and acceptable contention. Test candidate requests gradually instead of applying a fixed multiplier across the fleet. For Java services, use the separate Java heap and non-heap memory sizing workflow alongside container metrics.

The following illustrative PromQL query compares each containerโ€™s P95 CPU usage with its current positive CPU request. A ratio above 1 means the measured percentile exceeds the request; it does not mean the container has exceeded a CPU limit. A low ratio identifies a candidate for investigation. Scope the query to one cluster and deduplicate scrape replicas before using it.

# P95 CPU usage over 14 days as a ratio to the request
# > 1.0 = P95 usage exceeds the request
# < 0.5 = request is more than 2x P95 usage
sum by (namespace, pod, container) (
  quantile_over_time(0.95,
    rate(container_cpu_usage_seconds_total{container!="", container!="POD"}[5m])[2w:5m]
  )
)
/
(max by (namespace, pod, container) (
  kube_pod_container_resource_requests{resource="cpu"}
) > 0)

Keeping pod and container labels prevents a hot replica from being averaged across all replicas. Pod churn means a current pod may have much less than fourteen days of history; a time range cannot create missing samples. These queries compare historical usage with current requests, so annotate request changes and releases. Workload-level analysis needs stable owner mappings and an explicit aggregation strategy.

The [2w:5m] subquery needs at least 14 days of Prometheus retention. If your Prometheus keeps 15 days, you have margin. If it keeps 7, run the same query over [7d:5m] and accept a slightly noisier answer. Be warned that a [2w:5m] subquery over rate() is expensive at fleet scale. Run it against recording rules or a long-term store, not against a live Prometheus serving dashboards.

The memory example uses a gauge, so it can apply a range selector directly. Working-set bytes are one view of memory demand, not a complete predictor of OOM behavior. Compare peaks, container limits, application memory, and termination evidence as well. See the Prometheus function reference for rate and quantile semantics.

# P99 memory working set over 14 days as a ratio to the request
sum by (namespace, pod, container) (
  quantile_over_time(0.99,
    container_memory_working_set_bytes{container!="", container!="POD"}[2w]
  )
)
/
(max by (namespace, pod, container) (
  kube_pod_container_resource_requests{resource="memory"}
) > 0)

Read the output as a triage list for each workload:

  • A ratio above 1.0 means the measured usage percentile exceeds the request. Investigate performance and contention before deciding whether to raise it.
  • A ratio between 0.5 and 1.0 is not proof of healthy sizing; check peaks and application behavior.
  • A ratio below 0.5 identifies a possible reduction candidate, subject to representative measurements and safety checks.
  • A ratio below 0.1 may indicate a large relative gap. Prioritize absolute capacity and cost impact, and check for infrequent jobs or missing history.

Start with one namespace and a manageable set of workloads. Rank candidates by the absolute gap as well as the ratio: a large percentage on a tiny request may matter less than a modest percentage across many replicas. Review service importance and measurement quality before selecting a pilot.

If requests drop on paper but node count does not move, compare node allocatable against summed requests. A DaemonSet, a kube-system component, or a Pending pod with a large request can hold the ceiling where it was.

For an HPA using resource utilization targets, utilization is usage divided by request. Lowering a CPU request can raise reported utilization at unchanged load and cause additional replicas. Absolute-value and external-metric targets behave differently. Review the metric type and scaling behavior before resizing.

The fix ladder: manual audit, VPA recommend mode, continuous automated rightsizing

Choose an operating model based on scale, workload risk, and the quality of your measurements. A low usage-to-request ratio alone does not justify skipping review or enabling broad automation. Start with a representative workload and explicit acceptance criteria.

Manual review, recommendation mode, and controlled automation can complement each other. Choose an operating model based on workload risk, release frequency, and operational capacity.

Rung 1: manual audit

Rank candidates by absolute resource gap, then review historical peaks, startup behavior, HPA settings, and service objectives. Validate a small rollout against latency, errors, restarts, and node pressure, with a clear rollback path. This can also identify requests that need to increase.

Budget for the review load. A manual audit produces dozens of pull requests, each of which needs a reviewer who understands that service. At fifty workloads it works. At five hundred it becomes a backlog, and the drift keeps worsening while the queue drains.

Rung 2: VPA recommend mode

Vertical Pod Autoscaler with updateMode: "Off" produces recommendations without applying resource updates. Installing VPA still adds components and permissions to review. Inspect recommendation bounds and history rather than assuming a fixed percentile is suitable for every workload. The upstream VPA documentation describes the supported modes.

In recommendation-only mode, an owner must review and apply changes. VPA also offers modes that apply recommendations; it is not limited to advisory operation. Choose one resource-management owner per workload and verify how it interacts with HPA and GitOps.

Rung 3: continuous automated rightsizing

Continuous automated workload rightsizing can keep recommendations aligned with changing demand. Cast AI Workload Autoscaler adjusts requests based on runtime observations and configured policies. Request sizing and node bin-packing are distinct parts of the optimization workflow. The 2026 report describes a representative cluster with provisioned CPU reduced by roughly half after automated rightsizing; that is an observed example, not a guaranteed result for another cluster.

Review exactly which requests and limits a policy changes, how it applies them, and how to suspend it before rollback. Use gradual rollout and validate disruption controls for the chosen update path; PodDisruptionBudgets do not cover every restart or node-pressure event. Start with the Workload Autoscaler documentation and our rightsizing safety guide.

Lower requests can create room for node consolidation, but savings require actual changes to billable capacity. Placement rules, disruption budgets, DaemonSets, minimum node counts, and commitments can limit the outcome. Node autoscaling is one way to reclaim capacity; a planned manual node reduction is another. Track workload health and the bill separately.

Turn request review into a continuous process

Request drift is an operational problem: estimates and emergency increases outlive the conditions that produced them. Compare requests with representative usage, investigate the largest absolute gaps, and test changes against application reliability. Keep provisioned-to-requested capacity separate from requested-to-used capacity when measuring results.

Start with the Kubernetes requests and limits guide, then review the automated rightsizing safety model. To assess your own cluster, start a free trial with a read-only cluster assessment.

Frequently asked questions

Why do Kubernetes requests become overprovisioned?

Conservative estimates, copied manifests, and incident-driven increases can leave requests above current workload needs. Without a review loop, those values persist after traffic or application behavior changes. Higher requests can affect scheduling and memory-pressure behavior, but do not guarantee reliability.

How much headroom should Kubernetes requests include?

There is no universal safe multiplier. Use representative CPU and memory history, startup peaks, latency objectives, and expected bursts. P95 CPU and P99 memory can help screen candidates, but validate changes gradually and examine memory peaks separately.

Does lowering requests cause OOM kills?

Lowering a memory request alone does not lower the container memory limit or directly trigger a limit-based OOM kill. It can affect placement and behavior under node memory pressure. CPU requests affect scheduling and CPU shares; CPU limits govern quota throttling. Check limits, QoS, HPA behavior, and node pressure before applying changes.

How do I measure actual Kubernetes resource usage?

Use historical cAdvisor metrics in Prometheus: rate of container_cpu_usage_seconds_total for CPU and container_memory_working_set_bytes as one memory signal. Compare representative percentiles and peaks with current requests. Account for pod churn, missing samples, duplicate scrapes, release changes, and workload-specific traffic cycles.

What automates Kubernetes request tuning?

Vertical Pod Autoscaler can produce recommendations in Off mode and apply changes in other supported update modes. Cast AI workload autoscaling also adjusts resource settings according to configured policies. Review supported update mechanisms, disruption behavior, limits policies, and recovery procedures before enabling production automation.

Cast AI โ€บ Blog โ€บ Kubernetes Overprovisioning: Why Requests Drift and How to Fix It