Unit 6.3 · about 6 h
37 / 324Task ownership, cancellation, timeouts and retries
A task that outlives its caller still consumes capacity. A retry that ignores ownership, deadlines or side effects can turn one transient failure into overload or duplicate work.
- implement
You can now
- Own related tasks with
TaskGroupand propagate cancellation after bounded cleanup. - Implement a monotonic end-to-end deadline with classified failures, capped backoff, jitter and a retry budget.
- Reject unsafe retries when a timeout leaves a side effect's outcome unknown.
Covers
- Task ownership, task groups, cancellation and cleanup
- Timeouts, deadlines, retry budgets, jitter and idempotency
Study, in this order
3 steps
- Async task lifecycle first passAlgoMaster Concurrency
Error Handling:
TaskGroupfor Structured Concurrency (Python 3.11+); Cancellation; Timeout withasyncio.timeout(Python 3.11+)Depth · Use the 3.14 reference below for exact failure and cancellation semantics, not older
gather-based ownership patterns. - CPython 3.14 task ownership and deadlinesPython concurrency documentation
Task cancellation; Task groups and version-change notes; Timeouts:
timeoutandtimeout_at; Running in threads:to_thread - Retries are additional loadTimeouts, retries, and backoff with jitter
Timeouts; Retries and backoff; Jitter
Depth · The five-layer example's 243 is \(3^5\) total attempts. Its phrase 'three retries' is inconsistent with that arithmetic; four attempts per layer would give \(4^5 = 1024\).
Focus on
TaskGroupawaits owned children on exit. A non-cancellation child failure cancels siblings and groups ordinary errors; one child'sCancelledErroralone does not fail the group. Clean up infinallyand normally re-raise cancellation.TaskGroupandtimeoutarrived in 3.11. In 3.13TaskGroupimproved simultaneous cancellation/count handling and closes coroutines submitted while inactive; in 3.14create_taskforwards allkwargs, includingeager_start. Do not depend on accidental task-start ordering.- Set one deadline with
loop.time()andtimeout_at. Queue waits, attempts and backoff spend the same budget. CatchTimeoutErroroutside thetimeoutcontext; cancellation is cooperative, not a hard real-time stop. - Retry only classified transient failures and idempotent operations. Pick one retry owner, cap total attempts and aggregate retry load, and inject
clock,randomandsleepfunctions for deterministic policy tests.
Reproduce closed-book
Build an event-gated
TaskGroup: one child waits indefinitely with a finally marker, another waits for it to start then raises ValueError. Assert the sibling is cancelled, cleanup ran and except* ValueError receives the failure. Separately assert a child raising CancelledError leaves a successful sibling intact, and an expired timeout around an await becomes TimeoutError outside the context. Add a fake dependency that fails twice then succeeds: three total attempts and full-jitter delays in [0, min(cap, base*2**k)]. Inject random and sleep to check the bounds without wall-clock assertions. Test permanent failure, exhausted budget, expired deadline and cancellation during backoff with no later attempt.Traps
- Catching
BaseExceptionin a retry loop and thereby retryingCancelledError. Cancellation is not a transient dependency failure. - Giving every attempt a fresh end-to-end timeout. This silently lets retries exceed the caller's deadline.
- Assuming cancellation of a
to_threadawaitstops an already-running thread or rolls back a remote side effect. Use cooperative worker stop signals and an explicit unknown-outcome policy.
Practice
The section mini-project covers this: TaskGroup-owned workers, request deadlines, classified retry budgets, jitter and cancellation cleanup against a fault-injected dependency.
Self-test
-
What differs between
TaskGroupandgatheron an ordinary child failure?TaskGroupcancels and awaits siblings before reporting grouped errors. Defaultgatherpropagates the first error without automatically cancelling its other awaitables. -
Where do you catch
asyncio.timeout'sTimeoutError?Outside itsasync withblock. Inside, expiry is delivered through cancellation of the current task. -
Three total attempts at each of five retrying layers yield what worst-case leaf load?
\(3^5 = 243\) leaf attempts for one original request. Three retries plus the initial attempt would instead yield \(4^5 = 1024\). -
Does a timed-out write prove the server did not commit it?
No. Its outcome may be unknown. Retry only with a valid idempotency contract or reconcile the result first.
Completion criterion
Stop when
Your tests show no child left running after group exit and no retry after cancellation, permanent failure, exhausted budget or deadline expiry; duplicate delivery has an explicit idempotency policy.
Mark passed, then open 6.4 Only mark this when you can do it without notes.Leads to
- 6.4 Backpressure, load shedding and graceful shutdown Make retries and cancellation obey the pipeline's admission and drain policy.
Resources for this unit
- AlgoMaster Concurrency Study Concurrency 101; the Python units on threads, the GIL, multiprocessing, asyncio and concurrent.futures; synchronization primitives; concurrency challenges; locking strategies; and concurrency patterns. Then solve Bounded Buffer, Thread-Safe Blocking Queue, Thread-Safe Rate Limiter and Task Scheduler with Dependencies.
- Python concurrency documentation Use asyncio task groups, cancellation, queues and timeouts; concurrent.futures; multiprocessing; threading; and signal documentation to verify lifecycle and failure behavior.
- Timeouts, retries, and backoff with jitter Read Timeouts, Retries and backoff, and Jitter for deadline scope, idempotency, one retry owner and bounded retry load. Interpret the five-layer example as three total attempts per layer, not three retries plus the initial attempt.
Also in this section
- High Performance Python, 3rd edition Read Chapter 2 Profiling to Find Bottlenecks, Chapter 9 Asynchronous I/O, Chapter 10 The multiprocessing Module and Chapter 12 Using Less RAM, and use the companion code (https://github.com/mynameisfiber/high_performance_python_3e) as experiment starting points. Change the workloads and measure the regimes where concurrency helps, loses or only moves the bottleneck.
- Operating Systems: Three Easy Pieces - concurrency Read Chapters 26-33, from threads through event-based concurrency, for locks, condition variables, semaphores, concurrency bugs and event loops. Do selected homework only for a weak mechanism.
- Docker guides Use the Python container guide and build best practices to run as non-root, pin dependencies, handle signals and test CPU and memory limits.
- OpenTelemetry Python instrumentation Manually instrument one request so its structured logs, one low-cardinality metric and one trace share a correlation identity. Use the telemetry to locate one seeded fault.
- Algorithmica - Getting Accurate Results Use Measuring the Right Thing and Reducing Noise for benchmark design: input distribution, latency versus throughput, warm-up, frequency scaling and pinning. The examples are C++; the checks transfer to Python.
- Site Reliability Engineering - overload and troubleshooting Read Chapter 21, The Pitfalls of Queries per Second, Client-Side Throttling, Utilization Signals, Handling Overload Errors and Conclusions. Read Chapter 12, Triage, Examine, Diagnose, Test and Treat, and Making Troubleshooting Easier for a service runbook. Skip datacenter load-balancing implementation details.
- Liveness, Readiness, and Startup Probes Read Liveness probe, Readiness probe and Startup probe for the different health decisions and failure actions. Apply the contracts to a local service; do not install Kubernetes or configure a cluster.