Kubernetes Capacity Planning: How to Size a Cluster You Cannot Predict

Key takeaways

• CPU requests average 69% above actual usage across production Kubernetes clusters (Cast AI 2026 State of Kubernetes Optimization Report).
• Planning against requested resources means planning against numbers that are mostly padding.
• The three quantities to track are: used, requested, and provisioned. Plan against used.
• Stateful workloads represent 7% of deployments but require a separate capacity planning pass due to AZ-locked storage.
• Commit 60-70% of your stable, rightsized baseline to reserved capacity. Cover the rest with Spot.
• Review capacity plans every 30-90 days and immediately after major events.

Kubernetes autoscaling was supposed to eliminate the need for capacity planning. In practice, it has made capacity planning harder. The reason is structural. Autoscalers provision nodes in response to pending pods, and pods pend when requested CPU or memory is unavailable, not when actual usage is high. When requests are inflated, autoscalers treat the cluster as full and add nodes. The cluster grows. Costs rise. Utilization stays at 8%.

According to the Cast AI 2026 State of Kubernetes Optimization Report, which analyzed tens of thousands of clusters across AWS, GCP, and Azure, 69% of requested CPU goes entirely unused. That figure rose from 40% the prior year. The average production cluster runs at 8% CPU utilization.

This guide covers how to break that loop. It explains how to separate the three quantities that make Kubernetes capacity planning workable: what your workloads actually use, what they request, and what your nodes provision. It then covers headroom sizing, spike vs. growth planning, stateful workloads, commitment management, and review cadence.

Why Kubernetes capacity planning is different

Traditional capacity planning compares current usage against available capacity. When usage reaches roughly 70%, you add hardware. The logic is direct.

Kubernetes adds three layers of indirection that break this logic. Understanding each layer is essential before any sizing decision makes sense.

Layer one: scheduling happens against requests, not usage

Kubernetes schedules pods by requested resources, not by actual utilization. A pod requesting 4 CPU but consuming 0.4 CPU still occupies 4 CPU worth of scheduling space on a node. The node appears 80% allocated even if actual consumption is 8%. Other pods cannot schedule onto that node, not because it is full, but because its requests say it is.

Layer two: HPA responds to utilization relative to requests

HPA calculates utilization as the ratio of actual usage to requested CPU. At 8% actual use of reserved CPU, that ratio stays low. HPA does not scale out. The cluster fills up with reserved-but-idle capacity instead of adding replicas to serve real traffic.

Layer three: node autoscalers provision against pending pods

Cluster Autoscaler and Karpenter provision new nodes when pending pods cannot fit onto existing nodes. Those pods pend because they cannot find requested headroom, not because nodes are actually saturated by CPU or memory consumption. The autoscaler responds to a scheduling signal, not a utilization signal.

Together, these three layers create a self-reinforcing loop. Requests are set defensively (a rational choice — nobody wants OOM kills at 2am). Autoscalers add nodes to match inflated requests. Over time, the cluster grows to accommodate padding, not real load. Utilization stays at 8% while costs keep rising.

The solution is not to disable autoscaling. Autoscaling handles real traffic variation. The solution is to give autoscalers accurate signals by rightsizing the requests they respond to. Once requests reflect actual usage patterns, provisioned nodes shrink to fit real demand, and kubernetes capacity planning becomes something you can compute against. For a deeper look at how Kubernetes cost optimization fits within this framework, that pillar page covers the full picture.

The three numbers: used, requested, provisioned

Sound kubernetes cluster sizing starts with measuring three quantities per cluster, not one.

• Used: CPU and memory consumed at runtime by all running containers. This is what kubectl top pods reports.
• Requested: CPU and memory reserved for scheduling. Autoscalers respond to this number. Node allocatable capacity is measured against it.
• Provisioned: Total CPU and memory across all nodes in the cluster, regardless of whether any pod uses that capacity.

Consider a representative production cluster from the Cast AI 2026 State of Kubernetes Optimization Report:

That cluster provisions more than 11x what it consumes. The gap between requested (24.9) and used (3.94) is the overprovisioning gap — the padding baked into resource requests. The gap between provisioned (44.87) and requested (24.9) is scheduler headroom: capacity held above scheduled workloads for burst and node failure absorption.

Both gaps matter, but they require different fixes. Closing the request gap requires rightsizing. Closing the provisioned gap requires consolidation. Sound capacity planning starts by measuring all three numbers, then working backwards: from used to set requests, and from requests to size provisioned headroom. Automated workload rightsizing covers the mechanics of correcting requests at the container level.

Measuring your own gap

Before making any capacity changes, measure your current state. Three commands get you the data you need.

Step 1: get node-level allocatable and requested resources

kubectl get nodes -o custom-columns='NAME:.metadata.name,CPU:.status.allocatable.cpu,MEM:.status.allocatable.memory'

This shows allocatable CPU and memory per node in a format that is stable across Kubernetes versions. For per-node request detail, inspect individual nodes:

kubectl describe node <name> | grep -A10 "Allocated resources"

Compare across nodes to find fragmentation and underutilized nodes that are candidates for consolidation.

Step 2: get cluster-wide actual usage

kubectl top nodes
kubectl top pods --all-namespaces

Compare actual CPU and memory consumption per node and per pod against the Requests figure from Step 1. The ratio is your current utilization efficiency.

Note: kubectl top requires metrics-server, which is not installed by default on all clusters. If you use the Prometheus adapter instead, skip kubectl top and use the PromQL query below.

For clusters running Prometheus, this query measures actual CPU usage relative to requests at container granularity, far more scalable than kubectl top across large node counts:

max by(namespace, pod, container)(
  rate(container_cpu_usage_seconds_total[5m])
  /
  on(namespace, pod, container)
    (kube_pod_container_resource_requests{resource="cpu"} > 0)
)

A ratio below 1.0 indicates the container is using less CPU than it requests, overprovisioned. A ratio below 0.2 is a high-confidence rightsizing candidate: the request is more than 5x actual usage.

Step 3: calculate the gap

For each namespace, divide total used CPU by total requested CPU. If that ratio is below 0.35 (35%), your requests need rightsizing before any capacity plan you build will reflect reality. A ratio below 0.15 means you are planning against numbers that are more than 85% padding.

Track these three numbers weekly. Changes in the gap are early signals of workload behavior shifts — and the first indicator that your capacity plan needs revision.

How much headroom do you actually need?

Once requests reflect actual usage via rightsizing, the question becomes how much provisioned headroom to hold above total requested capacity. The standard guidance is 15-30% above peak requested. The right number depends on two variables: autoscaler provisioning speed and what happens to user traffic during that window.

Autoscaler reaction time changes the math

Karpenter provisions new nodes in 45-90 seconds (with pre-baked AMIs; custom bootstrap scripts extend this). Cluster Autoscaler takes 3-4 minutes. HPA requires at minimum 75 seconds to trigger after a metric breach (15-second check cycle, multiple observation periods required). By default, HPA v2 allows setting behavior.scaleUp.stabilizationWindowSeconds: 0 to reduce this window for latency-sensitive workloads. GKE’s native capacity buffer restores active headroom from standby in roughly 30 seconds.

Faster autoscalers reduce the headroom you must hold in reserve. If your autoscaler takes 90 seconds to provision and your peak spike takes 5 minutes to reach maximum load, you need enough pre-provisioned capacity to absorb 30% of the spike while nodes join. If your autoscaler takes 4 minutes, you need to cover more of the spike yourself.

A practical rule of thumb: add 20-30% headroom above your p95 rightsized request baseline. For HPA-managed workloads, also account for the autoscaler reaction window, if HPA takes 75-120 seconds to scale and your spike completes in 3 minutes, you need enough headroom to serve 40-65% of spike traffic before HPA catches up. Start at 20-25% headroom and increase only if you observe pending pods or error budget burn during traffic events.

Pause pods and capacity buffers

One proven approach is pause pods: lightweight pods that hold scheduling space on nodes. When real workloads need that space, Karpenter or Cluster Autoscaler evicts the pause pods to make room and provisions replacement capacity in the background. This decouples headroom management from application autoscaling.

GKE formalizes this pattern with its native capacity buffer, where standby capacity refills active buffers in approximately 30 seconds. For teams using Karpenter, the consolidationPolicy and consolidateAfter settings control how aggressively nodes are reclaimed and how much headroom is retained between consolidation cycles. For a deeper look at how bin-packing interacts with headroom, see the Kubernetes bin-packing guide.

Planning for spikes vs planning for growth

Spike planning and growth planning are fundamentally different problems. Conflating them is one of the most common kubernetes resource planning mistakes.

Spike planning

Spike planning is about reaction latency. The central question: how much pre-provisioned headroom prevents user-facing impact during a sudden traffic surge?

Relevant signals for spike planning include HPA and KEDA thresholds, historical peak-to-trough ratios per namespace, autoscaler provisioning speed, and error budget consumption during past spike events. Autoscaling tools handle spikes operationally. See the best Kubernetes autoscaling tools comparison for a full breakdown. For capacity planning purposes, what matters is the reaction time each autoscaler contributes, because that determines the headroom floor.

Growth planning

Growth planning is about infrastructure investment timelines. The central question: when will current provisioned capacity be insufficient for next quarter’s workload, and what commitments need to be made before that point?

Relevant signals include 60-90 day utilization trends per namespace, deployment velocity, new workload onboarding rate, and commitment term lengths. One-year or three-year Reserved Instances require earlier decision points than month-to-month Spot.

A critical discipline in growth planning: separate seasonal peaks from baseline growth. A cluster that spikes to 3x usage during a monthly batch run does not need 3x baseline committed capacity. Commit to the inter-seasonal floor instead — typically 40-50% of seasonal peak for high-seasonality workloads. Cover the rest with Spot Instances.

Stateful workloads anchor the plan

Stateful workloads represent 7% of deployments per Cast AI internal data from the 2026 report. Despite that small percentage, they have a disproportionate impact on capacity plans. They cannot be treated like stateless Deployment pods.

Why stateful workloads require separate planning

Standard Kubernetes scheduling assumes pods are fungible. StatefulSets are not. The core constraint is storage.

EBS ReadWriteOnce (RWO) PVCs bind to a single node and a single availability zone. If a StatefulSet pod migrates to a different AZ, the PVC cannot follow. The reattachment fails, and the pod stays pending. Even same-AZ reattachment takes 30-90 seconds. During that window, the pod is unavailable. For StatefulSets backing a database, queue, or stateful cache, that window is a partial outage.

Additionally, StatefulSet pods receive stable network identities and ordered deployment behavior. They resist the automated bin-packing techniques that work cleanly for stateless Deployments. As a result, stateful node pools tend to accumulate fragmentation over time — unused capacity that cannot be reclaimed without disruption.

How to plan stateful node capacity

Plan stateful and stateless node pools separately. For stateful workloads, apply these rules:

• Use dedicated NodePools or node groups per availability zone, aligned to your PVC-bound AZs.
• Apply AZ affinity rules that match PVC-bound AZ to pod scheduling AZ. This prevents silent reattachment failures.
• Set maxUnavailable: 0 in PodDisruptionBudgets or apply the karpenter.sh/do-not-disrupt: "true" annotation to prevent autoscaler-driven disruption.
• Hold 25-35% headroom in stateful pools — more than the standard 15-30% for stateless — because stateful pods cannot freely migrate to absorb fragmentation.

A minimal StatefulSet AZ affinity example:

# StorageClass for zone-pinned PVCs (use WaitForFirstConsumer)
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: regional-ssd
provisioner: kubernetes.io/gce-pd  # or ebs.csi.aws.com for EKS
volumeBindingMode: WaitForFirstConsumer  # PVC provisioned in pod's zone
reclaimPolicy: Retain

---
# StatefulSet with per-pod AZ pinning via node affinity
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: database
spec:
  replicas: 3
  template:
    spec:
      affinity:
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
            - matchExpressions:
              - key: topology.kubernetes.io/zone
                operator: In
                values:
                - us-east-1a  # Pin to specific AZ where PVC exists

Use WaitForFirstConsumer on the StorageClass so the PVC is provisioned in the same AZ as the first pod scheduled there. For multi-AZ StatefulSets, use a Topology-Aware Volumes controller or provision one StatefulSet replica per AZ with separate PVCs.

Storage capacity in the plan

PVC provisioned size frequently exceeds actual used size by a wide margin. Track actual storage utilization per PVC, not just provisioned gigabytes. EBS volume expansion is online for ext4 and xfs file systems, but it requires forward capacity accounting — you need to plan for the expansion before the volume fills, not after.

For workloads that require cross-AZ pod mobility, ReadWriteMany (RWX) storage (EFS, NFS-based solutions) eliminates the AZ constraint. The trade-off is cost and latency. Plan those separately from RWO-bound workloads.

For more on how node consolidation interacts with stateful workloads and EBS constraints, see the Kubernetes bin-packing and node consolidation guide.

Commitments: how much to reserve when the floor keeps moving

Reserved Instances and Savings Plans are where kubernetes forecasting meets cloud billing. They are also where most teams make their most expensive mistakes.

The core problem with instance-based reservations

Modern autoscalers change which instance types are running. Karpenter selects instance families dynamically, optimizing for cost and Spot availability. If you commit to m5.2xlarge Reserved Instances and Karpenter switches to m6i.2xlarge for cost reasons, those RIs may not apply to the running compute. You pay the commitment and the on-demand rate simultaneously.

The solution is spend-based commitment instruments:

• AWS Compute Savings Plans: apply to any EC2 compute, any instance family, any region.
• Azure Savings Plan for Compute: apply across VM sizes and regions.
• GCP Compute Flexible CUDs: apply across machine types within a region.

These instruments survive instance-type changes. They apply to whatever compute runs. For Kubernetes clusters using modern autoscalers, spend-based instruments are the correct tool.

How to size commitments correctly

Three steps in sequence:

1. Rightsize first. Committing at 8% utilization means buying discounts on compute you are not using. Fix the requests before sizing commitments. Otherwise, you lock in waste at a discounted rate.
2. Observe for 60-90 days post-rightsizing. This window establishes your stable floor. Use the p10 of daily CPU spend — not the average, not the peak — to find the true baseline. To find your p10 daily baseline across all pods: quantile_over_time(0.10, sum(kube_pod_container_resource_requests{resource="cpu"})[30d:1d]) Note: on large clusters, this 30-day range query may be expensive. Scope to a specific namespace with a namespace=~'prod-.*' label filter or use a shorter range (7d:1d) as a starting point.
3. Commit 60-70% of the stable floor to Compute Savings Plans or equivalent. Cover the remaining 30-40% with Spot Instances for variable and bursty workloads.

The commitment utilization gap

The industry average for commitment utilization sits at 60-70%. With automated management, teams can reach approximately 98% utilization, according to the Cast AI 2026 report. At $500,000 per year in committed spend, the difference between 65% and 98% utilization represents roughly $165,000 of reservations going unused every year.

The gap exists because manual commitment sizing is a one-time decision applied to a dynamic system. Autoscalers change cluster composition continuously. Without automation tracking and rebalancing commitments as the cluster changes, utilization drifts downward. For more on how Karpenter optimization interacts with commitment sizing, that guide covers the provisioning side of this equation.

Non-production in the plan

Non-production clusters are easy to exclude from capacity planning. They are also where some of the worst waste accumulates.

A typical dev cluster runs at most 50 active hours per week. It is idle for 118 of the 168 hours in a week — nights, weekends, non-working hours. If it is provisioned like production (copied manifests, same resource requests), it consumes full production-scale compute for all 118 idle hours. Research shows 10-16x overprovisioning is common in dev environments, because teams copy production manifests without adjusting requests for lighter workloads.

Scheduling the cluster down during idle hours alone saves approximately 65% of monthly dev environment cost. That single lever — without any rightsizing or Spot migration — produces the largest cost reduction of any dev environment optimization.

For capacity planning purposes, treat non-production as a separate domain:

• Track non-production costs separately from production. Combined reporting hides how much each environment costs.
• Do not apply production Reserved Instances or Savings Plans to dev workloads. Dev clusters are intermittent; commitments require sustained usage to yield value.
• Apply schedule-based shutdown during idle hours. Scale to zero during nights and weekends.
• Rightsize dev requests independently. Dev workloads commonly need 5-10x less CPU than the production manifest specifies.

For a detailed treatment of non-production cluster cost controls, see Kubernetes dev environment costs: what to cut when nobody is watching.

Reviewing the plan: cadence and triggers

A capacity plan written once and never revisited is not a plan. It is a guess that ages poorly. Kubernetes workloads change continuously: new services ship, traffic patterns shift, and the request gap widens as teams add padding without removing it.

Scheduled cadence

Weekly checks focus on early warning signals: pending pods, autoscalers at maximum, OOM kill rate increases, and CPU throttling spikes. These signals do not necessarily require changing the plan. They indicate the plan may soon need revision.

Monthly reviews compare requested vs. actual utilization per namespace and workload. They also review the node pool size distribution. A pool consistently running at 90%+ needs more headroom in the plan. A pool consistently running at 30% is overprovisioned or is a consolidation candidate.

Quarterly audits review commitment coverage against actual spend, adjust commitment sizes based on growth trends, and check whether the cluster workload mix has changed — new stateful services introduced, GPU workloads added, or major deployments retired.

Trigger-based reviews

Beyond the scheduled cadence, certain events should trigger an immediate plan review regardless of timing:

• Any deployment that increases total requested CPU by more than 30%.
• Autoscaler at maximum capacity for more than 48 consecutive hours.
• OOM kill rate increase of more than 20% week-over-week.
• Commitment utilization dropping below 75%.
• First stateful workload or first GPU workload added to the cluster.
• Major team growth or acquisition that adds new workloads at scale.

Automated platforms help here. Cast AI’s Workload Autoscaler refreshes rightsizing recommendations every 30 minutes and reacts immediately to OOM events or usage spikes exceeding 50% above the current recommendation. That refresh rate is effectively continuous review. Manual review cycles cannot match it, but they remain valuable for strategic decisions: commitment sizing, node pool architecture, and growth planning.

Conclusion

Kubernetes capacity planning is not about predicting demand with precision. It is about separating three quantities — used, requested, and provisioned — and making decisions at each layer based on accurate data rather than defensive padding.

The 69% overprovisioning gap documented in the Cast AI 2026 report is not a failure of autoscaling. Autoscaling is doing exactly what it was configured to do. The failure is in the requests that autoscaling responds to. Fix the requests, and every downstream decision — headroom sizing, commitment coverage, node pool planning — becomes computable against a real number.

For stateful workloads, apply a separate planning pass with AZ-level storage constraints and dedicated node pools. For non-production, apply schedule-based controls and rightsize independently from production. For commitments, use spend-based instruments and observe 60-90 days of post-rightsizing data before sizing.

How Cast AI closes the loop

Cast AI closes the capacity planning loop without manual intervention at each step. PrecisionPack adjusts pod CPU and memory requests every 30 minutes, reacting immediately to OOM events and usage spikes exceeding 50% above the current recommendation. Kubernetes also includes VPA (Vertical Pod Autoscaler) as a built-in rightsizing alternative, though it requires pod restarts to apply new requests and does not work well alongside HPA on CPU metrics, making it a poor fit for most stateless production workloads. Karpenter then provisions against accurate requests instead of inflated ones, reducing average provisioned CPU by approximately 50% across clusters analyzed in the 2026 report.

On the consolidation side, Cast AI’s Evictor runs continuous bin-packing across AWS, GCP, and Azure. In documented cases, it reduced a six-node cluster to three nodes at 80% CPU utilization. Combined with Spot Instances, that approach delivered 66% total cost reduction. For stateful workloads that traditional bin-packing tools cannot move, container live migration enables pod migration with minimal disruption, unlocking consolidation for previously excluded workloads.

On commitment management, Cast AI’s Commitments Utilization feature applies reserved spend automatically across clusters. Teams using automated commitment management in Cast AI’s 2026 fleet data achieved commitment utilization above 98%, compared to a typical 50-70% without automation.

If your cluster’s utilization is below 30%, start by measuring the three numbers today: used, requested, and provisioned. Close the request gap before building any plan on top of it. Then use that accurate baseline for every decision downstream.

Frequently Asked Questions