no route to host: The network stack had nowhere to send the packet

EHOSTUNREACH means the kernel could not find a path to the destination address, or a router rejected it. In a cluster this normally points at the CNI rather than at the application.

Applies to: All Kubernetes versions

What it means

no route to host is a routing-layer failure: either the local routing table has no entry covering the destination, or an intermediate router returned an ICMP unreachable. This distinguishes it usefully from a timeout, where packets were sent and simply disappeared. In Kubernetes it typically means the pod network is not fully established — a CNI agent that has not programmed routes on a node, an overlay that is not up between nodes, or a pod trying to reach an address range that the cluster network does not cover. It also appears when an iptables REJECT rule with an ICMP response is in the path, which some firewall configurations use.

Most common causes

• The CNI agent on the source node has not programmed routes to other nodes' pod ranges.
• The overlay network between nodes is not established, so cross-node pod traffic has no path.
• The destination address is outside any range the cluster network knows about.
• A firewall rule rejecting rather than dropping, which returns ICMP unreachable.
• A node that has just joined and whose routes have not propagated.
• A pod IP that no longer exists, where the route to that range is present but the host is not.

How to diagnose it

1. Establish the scope: does it fail from every node, or only some? kubectl run a test pod with a node selector to compare.
2. Check the CNI agent's health on the source node: kubectl get pods -n kube-system -o wide | grep NODE.
3. Inspect routes on the node: ip route should show routes for other nodes' pod CIDRs.
4. Check the node's assigned pod CIDR: kubectl get node NODE -o jsonpath='{.spec.podCIDR}'.
5. Look for reject rules in the node's firewall configuration.

How to fix it

1. Restart or repair the CNI agent on the affected node so routes are reprogrammed.
2. Ensure the cloud network permits traffic between nodes on the ports and protocols the CNI requires, including any encapsulation protocol.
3. Confirm the pod CIDR allocation is consistent between the cluster configuration and the CNI's own configuration — a mismatch produces exactly this.
4. Change firewall rules from reject to drop, or better, permit the traffic the cluster needs.
5. Wait for route propagation on a newly joined node, and check the CNI agent has completed startup there.

Notes

A cluster where same-node pod traffic works and cross-node traffic fails is almost always a CNI or underlying-network problem rather than anything to do with Services or policies. Testing both cases first saves a lot of time.

Related

• i/o timeout — The connection attempt got no response
• FailedCreatePodSandBox — The runtime could not create the pod's network sandbox

Sources

• Kubernetes documentation — Network Plugins
• Kubernetes documentation — Cluster Networking
• Kubernetes documentation — Debug Services