Probe: connection refused: The probe reached the pod but nothing was listening on that port

A probe failing with connection refused means the network path to the pod is fine and the port is closed. Either the application has not started listening yet, it is listening on a different port, or it is bound to localhost only.

Applies to: All Kubernetes versions

What it means

connect: connection refused is a definite answer from the pod's network stack: the packet arrived and no process was listening on that port. This is genuinely useful, because it eliminates network policy, DNS, and routing as causes — those produce timeouts, not refusals. What remains is a small set: the application has not yet bound the port, it bound a different port than the probe targets, or it bound only the loopback address. That last case is the one that catches people, because the application works perfectly when tested with kubectl exec … curl localhost and fails every probe, since the kubelet probes the pod's routable IP rather than its loopback.

Most common causes

• The application has not finished starting and has not bound its port yet — normal during startup, a problem if it persists.
• The application is bound to 127.0.0.1 instead of 0.0.0.0, so only processes inside the same network namespace can reach it.
• The probe's port does not match the port the application actually listens on.
• A named port in the probe that does not match any containerPort name in the spec.
• The application crashed after starting, so the port that was open is now closed.
• The probe targets a sidecar's port rather than the application's, or vice versa.

How to diagnose it

1. Check what is listening inside the container: kubectl exec POD -- ss -ltnp, or netstat -ltn if the image has it. Look at the bind address, not just the port.
2. A bind shown as 127.0.0.1:8080 rather than 0.0.0.0:8080 or *:8080 is the answer.
3. Compare the probe's port against the container's: kubectl get pod POD -o jsonpath='{.spec.containers[*].ports}'.
4. Test from another pod using the pod's IP rather than localhost: kubectl run t --rm -it --image=curlimages/curl -- curl -v POD_IP:PORT.
5. If the refusal only happens at startup, compare the time to bind against the probe's initial delay.

How to fix it

1. Bind the application to all interfaces. Nearly every server has a host or bind-address setting, and the containerised default should be 0.0.0.0.
2. Correct the probe's port, or the named port reference, so it matches the listening port.
3. Add or extend a startupProbe if the application simply needs longer before it binds.
4. Make sure the probe targets the right container in a multi-container pod — probes are per-container.

Notes

Refused and timed out are worth treating as completely different diagnoses. A refusal proves the packet arrived, which rules out the entire network layer; a timeout does not. Reading the distinction correctly saves most of the investigation.

Related

• Probe: timeout — The probe got no answer within its timeout
• Readiness probe failed — The pod was removed from service endpoints

Sources

• Kubernetes documentation — Configure Liveness, Readiness and Startup Probes
• Kubernetes documentation — Debug Services
• Kubernetes documentation — Pod Lifecycle: container probes