Most Kubernetes clusters run at 8% average CPU utilization. The nodes are running. The bills are arriving. The workloads are not filling the machines. The default scheduler is not to blame for that number, it was designed for availability, not for cost. Changing it requires understanding how placement decisions happen, and what to layer on top to reclaim the wasted capacity.
Key Takeaways
- Kubernetes bin-packing places pods densely on fewer nodes, reducing idle node cost.
- The default scheduler uses LeastAllocated scoring, which spreads pods and leaves nodes underutilized.
- Switching to MostAllocated scoring tightens pod placement, but it does not remove already-running idle nodes.
- Node consolidation, through tools like Karpenter or Cast AI Evictor, continuously removes underutilized nodes after the fact.
- Cast AI Evictor cut node count by 50% in one measured case, keeping remaining nodes at 80% CPU utilization.
What Bin-Packing Is and Why It Cuts Cost
Bin-packing is a classic computer science problem: fit the most items into the fewest containers. In Kubernetes, the “items” are pods and the “containers” are nodes. The goal is to maximize node utilization so that every node you pay for is doing real work.
The Cost Math
Cloud compute charges by node, not by pod. A cluster with 10 nodes at 20% CPU utilization costs the same as a cluster with 10 nodes at 80% utilization. The dense cluster accomplishes four times more work per dollar spent.
The numbers from production clusters are striking. According to the Cast AI 2026 State of Kubernetes report, 69% of clusters over-provision CPU, up from 40% just two years prior. Average CPU utilization sits at 8%. Datadog found that 83% of container costs go to idle resources. These are not edge cases. This is the default state of most Kubernetes clusters.
Consider a cluster running 6 nodes at 15% CPU each. A bin-packing pass might consolidate that workload onto 2 nodes at 45% CPU – eliminating four node bills instantly and creating headroom for autoscaling to respond faster.
Bin-Packing vs. Node Consolidation
These terms are related but distinct. Bin-packing refers to the scheduling decision at pod placement time – which node receives a new pod. Consolidation refers to the ongoing process of identifying underutilized nodes, migrating their pods elsewhere, and deleting the now-empty nodes.
First, you need scheduling that places pods densely. Then, you need consolidation to reclaim nodes that fragmented over time. Both layers are necessary for sustained cost reduction.
Why the Default Scheduler Does Not Pack for Cost
The Kubernetes scheduler prioritizes availability. Its default behavior spreads pods across nodes to reduce blast radius when a node fails. This is a reasonable availability decision – and it directly conflicts with cost efficiency.
LeastAllocated: An Availability Decision With a Cost
The default scoring plugin is NodeResourcesFit with a LeastAllocated strategy. This scores nodes higher when they have more free capacity. The scheduler always prefers the emptiest node for each new pod, spreading pods thin across many nodes each running at low utilization.
This behavior is not a bug – it serves its design goal. For teams trying to cut cloud spend, though, LeastAllocated works directly against them. Every pod scheduled onto a new node keeps that node alive on the cloud bill.
Resource Requests Break Bin-Packing Before It Starts
Resource requests compound the problem. The scheduler allocates based on requests, not actual usage. A pod requesting 2 CPU cores but using only 200m appears to the scheduler as a 2-core pod. In practice, a node might show as 80% allocated but only 8% utilized.
Oversized requests are extremely common. When developers set requests conservatively to avoid OOMKills, they inflate the scheduler’s view of demand. The scheduler cannot place pods that would easily fit on actual CPU headroom. Nodes fill up on paper while staying idle in practice.
Fixing resource requests is a prerequisite for effective bin-packing. Cast AI Autopilot right-sizes resource requests using actual usage data, giving the scheduler accurate inputs before bin-packing logic runs.
How to Pack Tighter (Karpenter and Automation)
There are two practical layers for improving Kubernetes node utilization: change how the scheduler places new pods, and continuously consolidate nodes that accumulated waste over time.
Layer 1: Switch the Scheduler to MostAllocated
The MostAllocated scoring strategy reverses the default. Instead of preferring the emptiest node, the scheduler scores nodes higher when they already carry more load. New pods land on the most utilized node first, filling it before the scheduler touches an emptier one.
Apply this configuration to your scheduler:
apiVersion: kubescheduler.config.k8s.io/v1
kind: KubeSchedulerConfiguration
profiles:
- pluginConfig:
- args:
scoringStrategy:
resources:
- name: cpu
weight: 1
- name: memory
weight: 1
type: MostAllocated
name: NodeResourcesFitThat said, MostAllocated only affects new pod placements – it does not move existing pods. At large scale (500+ nodes), filling nodes aggressively increases the blast radius of node failures. Plan your PodDisruptionBudgets accordingly.
Layer 2: Consolidation with Karpenter
Karpenter adds node lifecycle management on top of scheduling. Its consolidation feature identifies underutilized nodes and migrates their pods to other nodes, then removes the empty nodes. This handles waste that accumulated before you changed the scheduler, and it continues working over time.
A basic disruption configuration for consolidation:
# Karpenter v1.0+ NodePool disruption configuration
disruption:
consolidationPolicy: WhenUnderutilized
consolidateAfter: 30m
budgets:
- nodes: "10%"The value above uses 30 minutes as a conservative starting point. Reduce consolidateAfter gradually in lower-traffic environments. Never set below 30 seconds in production without extensive PDB and pod scheduling validation.
The budgets field limits how many nodes Karpenter disrupts simultaneously. For production clusters, start with 10% and tune from there.
To pause consolidation temporarily without changing your NodePool permanently:
kubectl patch nodepool default --type merge -p '{"spec":{"disruption":{"consolidationPolicy":"WhenEmpty"}}}'This switches to empty-node-only consolidation, which is much less aggressive. Revert with WhenUnderutilized when you are ready to resume full consolidation.
Availability Trade-Offs
Denser packing means fewer nodes absorbing a failure. Solid pod anti-affinity rules and PodDisruptionBudgets are essential before enabling aggressive consolidation in production. Stateful workloads require careful disruption budget configuration – a node running a sole database pod should not participate in consolidation without explicit safeguards.
Aggressive consolidation – short consolidateAfter values – causes pod churn during normal traffic fluctuations, not just during scale events. The recommendation for production: start at 30 minutes and tune down only after measuring the impact on pod restart rates and latency. Before enabling WhenUnderutilized, verify PDB coverage for all replicated workloads. If a Deployment has no PDB, consolidation can disrupt every replica simultaneously.
A PodDisruptionBudget sets the minimum number of replicas that must stay available during voluntary disruptions. Without one, Karpenter consolidation can evict all replicas of a Deployment simultaneously. Here is a minimal PDB for a two-replica stateless service:
apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
name: api-pdb
namespace: production
spec:
minAvailable: 1
selector:
matchLabels:
app: api-serviceSet minAvailable: 1 for stateless services. For stateful workloads or services with strict SLOs, set maxUnavailable: 0 to prevent any simultaneous evictions.
StatefulSet and Persistent Volume Considerations
StatefulSets with ReadWriteOnce PVCs – EBS on EKS, Persistent Disk on GKE – cannot be freely rescheduled across nodes. The PVC must detach from the source node and reattach on the destination. This takes 30–90 seconds and fails silently if the destination node is in a different availability zone.
Exclude StatefulSet pods from consolidation by adding the karpenter.sh/do-not-disrupt: "true" annotation, or configure separate NodePools for stateful and stateless workloads. Mixing them in a single pool leads to consolidation cycles that stall or roll back when PVC reattachment times out.
How Cast AI Automates Bin-Packing
Karpenter consolidation runs on AWS EKS. Cast AI Evictor provides continuous, multi-cloud bin-packing across AWS, GCP, and Azure without EKS-specific configuration.
Configuring MostAllocated and Karpenter consolidation requires ongoing tuning. Production clusters change. New workloads land with misconfigured requests. Node pools fragment over time as the cluster scales up and down. Cast AI addresses this with a continuous, autonomous loop.
Evictor’s Workflow
Cast AI Evictor runs a continuous consolidation cycle. It identifies the least-utilized node in the cluster, evicts pods gracefully while respecting PodDisruptionBudgets, reschedules those pods onto remaining nodes with verified capacity, then deletes the empty node and begins the cycle again.
Evictor only targets workloads with two or more replicas. Single-replica pods stay in place. Because Evictor only moves pods when a viable replacement node exists, evictions do not leave replicas unscheduled.
Evictor also integrates with Cast AI’s right-sizing. Before consolidating, it uses actual usage data to verify that remaining nodes can absorb the evicted pods without overcommitting real CPU and memory – closing the resource-request accuracy problem that breaks manual bin-packing.
To monitor consolidation activity, watch the karpenter_disruptions_actions_performed_total metric (Karpenter) or castai_evictor_evictions_total (Cast AI). Alert if the eviction rate exceeds 5 per 10 minutes during business hours – that rate indicates churn rather than planned consolidation.
Measured Results
In a documented case, Cast AI Evictor reduced a 6-node cluster to 3 nodes. The remaining 3 nodes ran at 80% CPU utilization. Note: 80% is used here as a measured outcome, not a target. For production clusters, plan for 60–70% sustained CPU utilization to leave headroom for traffic spikes and HPA scale-out delays.
For teams running EKS on Spot instances, combining Cast AI bin-packing with Spot node selection reduced total cluster cost by 66% – combining bin-packing density gains with Spot instance pricing on EKS (workload profile: stateless microservices, before-and-after measured over 30 days). The two levers compound: bin-packing reduces the number of nodes needed, and Spot reduces the per-node cost of those that remain.
At 500 nodes, manual tuning of scheduler configs and disruption budgets becomes a full-time job. Evictor runs autonomously and adapts as workload patterns shift, without requiring engineers to intervene each time the cluster changes state.
Conclusion
Most teams discover bin-packing after they have already optimized their application code and right-sized their database instances. By then, compute cost feels fixed. It is not. The cluster still carries the structural overhead built in by a scheduler that was never told to optimize for cost. Fixing that overhead takes two changes: one to the scheduler profile, one to the consolidation policy. Neither requires downtime. Both compound over time as workloads grow and shrink.
For teams looking to connect bin-packing with a broader autoscaling strategy, see the guide to Kubernetes autoscaling for cloud cost optimization.
If you want to skip the scheduler configuration and Karpenter setup, Cast AI Evictor handles both layers automatically across AWS, GCP, and Azure. You can connect your first cluster and see savings in under 15 minutes.
Frequently Asked Questions
Bin-packing in Kubernetes is the practice of scheduling pods as densely as possible onto a smaller set of nodes. The goal is to maximize node utilization so the cluster runs fewer nodes and pays less for idle compute. A dense cluster runs workloads at 70–80% CPU utilization instead of the 8% average seen across most production clusters today.
The default Kubernetes scheduler uses a LeastAllocated scoring strategy. This strategy prefers nodes with the most free capacity, so each new pod lands on the emptiest node available. Pods spread thin across many nodes, each running at low utilization. This approach improves availability by reducing blast radius, but it increases cost significantly. Oversized resource requests make the problem worse by making nodes appear full when they are actually idle.
There are three steps. First, switch the scheduler’s NodeResourcesFit plugin to MostAllocated scoring. This directs new pods to the most utilized nodes. Second, right-size your resource requests so the scheduler sees accurate demand instead of inflated estimates. Third, enable node consolidation using Karpenter’s WhenUnderutilized policy or Cast AI Evictor to continuously remove nodes that accumulated waste over time. All three steps together produce sustained utilization improvement.
