KubeErrors

Exit code 139: The process was terminated by SIGSEGV — a segmentation fault

139 is 128 + 11, meaning SIGSEGV. The process accessed memory it was not allowed to access. This is a crash in native code, not something Kubernetes did to the container.

Applies to: All Kubernetes versions, Linux nodes

What it means

Signal 11 is SIGSEGV, delivered by the kernel when a process references an invalid memory address — dereferencing a null or freed pointer, running off the end of a buffer, or exhausting the stack through unbounded recursion. Managed runtimes normally turn these into catchable errors, so a container exiting 139 is usually running native code: a C or C++ binary, a native extension loaded by Python or Node.js, or JNI code under a JVM. It is worth stressing what 139 is not — it is not an out-of-memory kill. Those report 137 with reason OOMKilled. Conflating the two sends people to raise memory limits for a pointer bug.

Most common causes

How to diagnose it

  1. Confirm the reason is not OOMKilled: kubectl describe pod POD under Last State. Reason and code must be read together.
  2. Read the previous container's logs for a stack trace: kubectl logs POD --previous.
  3. Determine whether the crash is input-dependent — if it follows a particular request, that is the fastest path to a reproduction.
  4. Check whether it is confined to one node, which would suggest hardware rather than software.
  5. Enable core dumps written to a mounted volume, and confirm the node's core_pattern allows them to be captured.

How to fix it

  1. Fix the defect in the native code. There is no cluster-level remedy for a segmentation fault.
  2. Ensure native extensions are built against the exact library versions present in the runtime image.
  3. Increase the stack size or remove unbounded recursion if the trace shows stack exhaustion.
  4. Roll back to the last image that did not crash while investigating, so the loop is stopped without pretending it is fixed.
  5. If the crash follows one node, drain it and have the hardware checked.

Notes

A container that segfaults during startup will restart, back off, and show CrashLoopBackOff with an empty current log. The evidence only exists in the previous container's output, so capture it before the pod is deleted and recreated.

Related

Sources

Pages on this site are written with AI assistance from the primary sources listed on each page, then checked against those sources before publishing.