Celery time_limit on the gevent pool: a 1 s limit let a CPU task run 3 s

Update 2026-11-08: rebuilt every case on Celery 5.2.7 with gevent 22.10.2, and on Celery 5.7.0 with gevent 26.9.0.

I gave a Celery task time_limit=1 and ran it with -P gevent. The CPU-bound task kept going for 3 s, finished all its work, and was then reported as failed:

state=FAILURE result=TimeLimitExceeded(1) got_after=3.06s

The gevent pool enforces the limit with gevent.Timeout, which can only fire when the task yields to the event loop. CPU-bound code never does. Keep gevent for I/O and run CPU work on a prefork worker.

The workers guide says this in two sentences: the gevent pool “does not implement soft time limits” and “will not enforce the hard time limit if the task is blocking”. It leaves out what that looks like in a task, and how an exception handler makes it worse.

I found this by rereading old code and rebuilt each case on a real worker with a private Redis, on Python 3.11. Both version pairs behaved the same. I have no production logs from that time.

What a 1 s limit does on each pool #

Each task does 3 s of work:

The first row is the nasty one. The work completed, its side effects happened, and the task still reports failure. The timer was pending the whole time and fired at the first switch after the loop.

A CPU-bound task also stalls everything else in that worker process. With -c 8, a task sent during a 3 s CPU loop waited 2.8 s to start, against a few milliseconds on an idle worker. On prefork with -c 2, it still started within a few milliseconds.

Split fetch and parse into separate workers #

app.conf.task_routes = {
    "tasks.fetch": {"queue": "fetch"},
    "tasks.parse": {"queue": "parse"},
}

chain(fetch.s(url), parse.s()).apply_async()

celery -A app worker -n fetch@%h -P gevent -c 60 -Q fetch
celery -A app worker -n parse@%h -P prefork -c 2 -Q parse

With time_limit=1 on parse, a chain with a 3 s CPU parse ended in TimeLimitExceeded after 1.1 s, and a 0.1 s parse succeeded. Pass a file path or an object-store key between the two rather than the body.

Why a timeout can’t interrupt CPU work #

celery/concurrency/gevent.py in 5.7.0 wraps each task like this:

try:
    with Timeout(timeout):
        return apply_target(target, args, kwargs, callback,
                            accept_callback, getpid(),
                            propagate=(Timeout,), **rest)
except Timeout:
    return timeout_callback(False, timeout)

Timeout is a timer on the event loop, and everything runs in one thread. The loop gets control only when the current greenlet yields, typically at a monkey-patched socket call or sleep. Prefork stops a task from outside, and the hard limit replaces the child process.

The soft limit is simpler: on_apply takes soft_timeout through **_ and never uses it. The worker log says nothing about that.

A bare except: cancels the limit #

gevent.Timeout derives from BaseException , so except Exception lets it pass and a bare except: catches it. My code had a fallback like this several calls below the task:

try:
    return lookup(url)
except:
    log.exception("lookup failed, trying the fallback")
    return fallback(url)

The timer fires once. After the handler ate it, the task had no limit at all, and in the rebuild it finished three seconds past the deadline as SUCCESS.

My fix at the time, a commit called “Fail the task on timeout”, added except Timeout at the top of the task. A Timeout stops at the nearest handler that matches, so the nested bare except: on the fallback path kept swallowing it. Change the nested handler:

except Exception:
    log.exception("lookup failed, trying the fallback")
    return fallback(url)

If you need cleanup, catch Timeout and re-raise it. In the rebuild, the cleanup ran at 1.01 s, and Celery recorded TimeLimitExceeded, so there was no need to set the state by hand.

The soft limit goes the other way on prefork: SoftTimeLimitExceeded is an Exception, so a task that moves between pools needs its handlers checked for both.

For a Python loop you own, gevent.sleep(0) on each iteration gives the loop a turn, and the task then stopped at 1.01 s. That can’t help inside a library call that never yields.

The sibling trap on the same crawler: Retry is an Exception, and except Exception eats it .

Takeaway #

On the gevent pool, time_limit is only as good as the task’s willingness to yield, so treat it as a limit on I/O waits.

Run grep -rnE --include='*.py' 'except:[[:space:]]*$' . on anything that runs under -P gevent.