connection refused: The destination was reached and nothing was listening on that port

A refused connection is a definite answer: the packet arrived and the port was closed. That rules out routing, DNS, and network policy, which all produce timeouts instead.

Applies to: All Kubernetes versions

What it means

connection refused is a TCP reset from the destination's network stack, meaning no process was listening on that port. In a cluster this is unusually informative because of what it eliminates: the name resolved, the route worked, and no policy dropped the packet — all of which would have produced a timeout. What remains is a small set of possibilities: the application is not listening yet, it is listening on a different port, it is bound to loopback only, or the Service is forwarding to the wrong target port. The loopback case is the one that produces the most confusion, because the application works when tested from inside its own container and refuses every connection from outside it.

Most common causes

• The application has not finished starting and has not bound its port.
• The application is bound to 127.0.0.1 rather than 0.0.0.0.
• The Service's targetPort does not match the port the container listens on.
• The container crashed, so the port that was open is now closed.
• The client is connecting to the right pod but the wrong port entirely.
• A sidecar proxy that has not started, in a mesh where it intercepts traffic.

How to diagnose it

1. Check what is listening and on which address: kubectl exec POD -- ss -ltn. A bind to 127.0.0.1 is the answer if present.
2. Test against the pod IP directly, bypassing the Service: kubectl run t --rm -it --image=curlimages/curl -- curl -v POD_IP:PORT.
3. Compare the Service's ports with the container's: kubectl get svc SERVICE -o yaml and kubectl get pod POD -o jsonpath='{.spec.containers[*].ports}'.
4. If the pod IP works and the Service does not, the problem is the Service's port mapping or its endpoints, not the application.
5. In a service mesh, check the sidecar is running and ready before concluding anything about the application.

How to fix it

1. Bind the application to 0.0.0.0.
2. Correct targetPort on the Service so it matches the container's listening port.
3. Fix whatever is crashing the container if the port was open and closed.
4. Wait for, or fix, the sidecar in a mesh deployment — its startup ordering relative to the application is a common source of early refusals.

Notes

Refused and timed out should be read as different diagnoses every time. A refusal proves end-to-end reachability, which is a large part of the investigation already done.

Related

• i/o timeout — The connection attempt got no response
• Service has no endpoints — No ready pods behind the Service

Sources

• Kubernetes documentation — Debug Services
• Kubernetes documentation — Service
• Kubernetes documentation — Connecting Applications with Services