no IP addresses available in range set: The node's pod IP pool is fully allocated

This CNI error means every address in the node's pod CIDR is assigned. New pods on that node cannot get an IP, so they stay in ContainerCreating while the rest of the cluster works normally.

Applies to: Clusters using host-local or equivalent CNI IPAM

What it means

Most CNI plugins allocate each node a block of pod addresses and assign from it locally. When the block is exhausted, sandbox creation fails with a message like failed to allocate for range 0: no IP addresses available in range set. There are two ways to reach that point, and they need different fixes. The honest one is that the node is running as many pods as its block has addresses — a /24 gives roughly 254, and a node configured for 110 pods will never hit it, while a smaller block might. The other is leaked allocations: pods that were removed without their IPAM entry being released, so the plugin believes addresses are in use when nothing holds them. The second case is diagnosable by comparing running pods against allocated addresses.

Most common causes

• Genuinely more pods on the node than its pod CIDR can address.
• Leaked IPAM entries from pods that were removed abnormally, often after a runtime or kubelet crash.
• A per-node CIDR that is smaller than the node's configured maximum pod count.
• A high rate of pod churn, where allocations are not released as fast as they are made.
• A cluster-wide pod CIDR too small to give every node an adequate block.
• Multiple CNI plugins or configurations on the node disagreeing about allocations.

How to diagnose it

1. Read the CNI error in the pod's events: kubectl describe pod POD.
2. Count pods on the node against the block size: kubectl get pods --all-namespaces -o wide --field-selector spec.nodeName=NODE | wc -l.
3. Check the node's pod CIDR: kubectl get node NODE -o jsonpath='{.spec.podCIDR}'.
4. Inspect the plugin's IPAM state on the node. For host-local IPAM this is a directory of files under /var/lib/cni/networks, one per allocated address.
5. Compare the number of allocation entries against the number of running pods. A large gap means leaked allocations.
6. Check for more than one configuration file in /etc/cni/net.d, since the runtime uses the first alphabetically and a stray file causes confusion.

How to fix it

1. Remove leaked IPAM entries following the plugin's documented procedure, after confirming the corresponding pods no longer exist.
2. Enlarge the per-node CIDR, which usually requires reconfiguring the CNI and may require replacing nodes.
3. Lower the node's maximum pod count so it cannot exceed its address block.
4. Add nodes to spread the pods, as an immediate mitigation.
5. Remove stray CNI configuration files so only the intended plugin is used.

Notes

Because the block is per node, this presents as one node rejecting every new pod while the cluster looks healthy. Any diagnosis that starts cluster-wide will miss it; the node name in the failing pods' -o wide output is the fastest clue.

Related

• FailedCreatePodSandBox — The runtime could not create the pod's network sandbox
• ContainerCreating — The kubelet is still preparing the container

Sources

• Kubernetes documentation — Network Plugins
• Kubernetes documentation — Cluster Networking
• Kubernetes documentation — Nodes