DeadlineExceeded: The Job ran longer than activeDeadlineSeconds and was stopped

activeDeadlineSeconds is a wall-clock limit on a Job. When it expires the Job is terminated and marked Failed regardless of how the work was going — retries do not reset it.

Applies to: All Kubernetes versions

What it means

activeDeadlineSeconds bounds a Job's total lifetime from when it starts, and it takes precedence over backoffLimit: when it expires, running pods are terminated and the Job is marked Failed with reason DeadlineExceeded. The crucial detail is that the clock covers the whole Job, including time spent on failed attempts and backoff delays between them. A Job that retries a few times can therefore exhaust its deadline without any single attempt having taken long, which makes the failure look like a slow task when it is really an accumulation of retries.

Most common causes

• The work genuinely takes longer than the deadline allows.
• Retries and backoff consuming the deadline before a successful attempt finishes.
• The pod hanging on an external dependency that never answers.
• The pod spending its deadline waiting to be scheduled or waiting for a volume.
• A deadline copied from another Job without checking it fits this one.
• The workload having grown — the same job over more data crossing a deadline that used to be adequate.

How to diagnose it

1. Read the Job's status and timestamps: kubectl describe job JOB.
2. Check how the time was spent: were there several failed attempts, or one long one? kubectl get pods -l job-name=JOB with timestamps answers this.
3. Read the logs of the terminated pod to see how far it got.
4. Check whether time was lost before the work started, waiting for scheduling or volumes.
5. Compare against the job's normal duration if you have history.

How to fix it

1. Raise activeDeadlineSeconds if the work legitimately needs longer, allowing for retries.
2. Make the work faster or split it into smaller Jobs that each fit comfortably.
3. Add timeouts inside the application so a hanging dependency fails fast rather than consuming the whole deadline.
4. Lower backoffLimit so retries do not eat the budget when failures are deterministic.
5. Keep activeDeadlineSeconds set. Removing it means a hung Job runs forever, which is worse than a bounded failure.

Notes

activeDeadlineSeconds and backoffLimit bound different things: one caps total elapsed time, the other caps failure count. A Job with only one of them set is unbounded in the other dimension, and both failure modes are real.

Related

• BackoffLimitExceeded — The Job's pods failed too many times
• Exit code 143 — Terminated by SIGTERM

Sources

• Kubernetes documentation — Jobs
• Kubernetes documentation — CronJob