Unhealthy: The event Kubernetes emits whenever any probe fails

Unhealthy is the event reason attached to every probe failure. The message that follows it says which probe failed and how, and that message is the actual diagnostic.

Applies to: All Kubernetes versions

What it means

Unhealthy is not a pod status — it is the reason field on an event the kubelet emits when a probe fails. The message names the probe type and the failure mode, for example Liveness probe failed: Get "http://10.1.2.3:8080/healthz": dial tcp 10.1.2.3:8080: connect: connection refused. Everything useful is in that string. Knowing how to read it saves most of the work: the probe type tells you what will happen next (a liveness failure restarts, a readiness failure removes from service), and the error tells you which layer failed — connection refused means no listener, a timeout means a slow listener, and an HTTP status means the application answered and declined.

Most common causes

• Any liveness, readiness, or startup probe failure. The reason is shared across all three.
• A connection refused, meaning nothing is listening on the probed port.
• A timeout, meaning something is listening but did not respond within timeoutSeconds.
• An unexpected HTTP status — anything outside 200–399 counts as a failure.
• An exec probe command exiting non-zero.
• A TCP probe failing to establish a connection.

How to diagnose it

1. Read the full message, not just the reason: kubectl describe pod POD, or kubectl get events --field-selector reason=Unhealthy for a namespace-wide view.
2. Note which probe type is named — that determines whether the pod is being restarted or merely removed from service.
3. Classify the error: connection refused (no listener), context deadline exceeded or Client.Timeout (too slow), an HTTP status code (application declined), or a non-zero exit (exec probe).
4. Check the frequency. Events are deduplicated with a count, so a count in the hundreds means a persistent problem rather than a blip.

How to fix it

1. Fix according to the classification: start the listener, widen the timeout, or fix what the application is reporting.
2. Correct the probe definition if the port, path, or scheme does not match what the container serves.
3. For exec probes, run the command manually inside the container to see its output and exit status.
4. Tune failureThreshold so that a single transient failure does not act, while a sustained one still does.

Notes

Because all three probe types share this event reason, alerting on Unhealthy alone conflates a pod being restarted with a pod being briefly taken out of rotation. Matching on the probe name in the message is what makes the signal useful.

Related

• Liveness probe failed — The container was restarted as unhealthy
• Probe: connection refused — Nothing is listening on the probed port

Sources

• Kubernetes documentation — Configure Liveness, Readiness and Startup Probes
• Kubernetes documentation — Pod Lifecycle: container probes
• Kubernetes documentation — Debug Running Pods