CRI context deadline exceeded: A call from the kubelet to the container runtime timed out

rpc error: code = DeadlineExceeded means the kubelet asked the runtime to do something and got no answer in time. The runtime is overloaded, blocked, or wedged.

Applies to: All Kubernetes versions

What it means

The kubelet communicates with the container runtime over gRPC, with timeouts on each call. A DeadlineExceeded error means the runtime did not respond within the deadline — not that the operation failed, which is an important distinction, because the operation may still be in progress or may have completed after the timeout. The usual causes are I/O saturation making every runtime operation slow, a very large number of containers on the node making listing expensive, or a single stuck operation blocking others. Repeated CRI timeouts are also what eventually produces a PLEG health failure and a NotReady node, so this error often appears just before a node goes down.

Most common causes

• Disk I/O saturation on the node, slowing every runtime operation.
• A very high container count making list operations expensive.
• A single operation blocked on a hung filesystem or an unresponsive storage backend.
• The runtime process being starved of CPU because reservations are not configured.
• A stuck container that the runtime cannot stop, blocking other calls.
• A pull of a very large image saturating the runtime's capacity.

How to diagnose it

1. Time a runtime call by hand on the node: time crictl ps. A slow response confirms it.
2. Check I/O saturation: iostat -x 1 or the equivalent.
3. Count containers on the node: crictl ps -a | wc -l.
4. Read the runtime log for stuck operations: journalctl -u containerd -n 300.
5. Check whether the node has also reported PLEG problems or gone NotReady.
6. Look for a hung network filesystem mount, which blocks operations that touch it.

How to fix it

1. Address the I/O bottleneck — faster storage, or workloads that write less.
2. Reduce container density on the node.
3. Configure kubeReserved and systemReserved so the kubelet and runtime are not starved by pods.
4. Clear a hung mount or stuck container that is blocking the runtime.
5. Restart the runtime to recover from a wedged state, draining the node first where possible.
6. Replace nodes where this recurs, since it usually indicates a persistent hardware or configuration issue.

Notes

A timeout does not mean the operation failed. The kubelet may retry an operation that actually succeeded, producing duplicate work and errors such as a reserved container name — which is why these two failures often appear together.

Related

• PLEG is not healthy — The kubelet's container state loop is too slow
• failed to reserve container name — A container with that name already exists

Sources

• Kubernetes documentation — Container Runtime Interface
• Kubernetes documentation — Debugging Kubernetes nodes with crictl
• Kubernetes documentation — Reserve Compute Resources for System Daemons