Unit 6.5 · about 6 h
39 / 324Profiling CPU and memory, and benchmarking honestly
Profiles locate expensive work; benchmarks test whether changing it helps. Mixing the two can turn profiler overhead, hidden queueing or a changed workload into an apparent improvement.
- measure
You can now
- Choose sampling, call instrumentation or allocation tracing to test a concrete bottleneck hypothesis.
- Distinguish CPU cost, transient allocation peaks and retained Python memory from waiting and process RSS.
- Build a correctness-checked benchmark with warm-up, repeated trials and raw results, then justify one optimization and a regime where it does not help.
Covers
- Sampling versus instrumentation and wall versus CPU time
- CPU profiles, allocation traces, RSS and memory retention
- Warmup, repeated trials, workload control and raw results
Study, in this order
4 steps
- High Performance Python, 3rd edition: targeted profilingHigh Performance Python, 3rd edition
Chapter 2: Profiling Efficiently (pp. 28-29), Using the
cProfileModule (pp. 41-45), Usingmemory_profilerto Diagnose Memory Usage (pp. 53-60), Introspecting an Existing Process with PySpy (pp. 63-64), Unit Testing During Optimization to Maintain Correctness (pp. 73-76), Strategies to Profile Your Code Successfully (p. 77)Depth · Learn tool selection and output interpretation. Do not install every profiler or assume the book's tool versions support both 3.14 builds; execute the stdlib reproduction below.
- Call profiles are not benchmarksPython concurrency documentation
Introduction to the profilers; Instant User's Manual:
tottimeandcumtime; What Is Deterministic Profiling? - Temporary peaks versus retained allocationsPython concurrency documentation
Examples: Compute differences; Record the current and peak size of all traced memory blocks; API:
get_traced_memory,reset_peakandSnapshot.compare_to - Measuring the right thing, with less noiseAlgorithmica - Getting Accurate Results
Getting Accurate Results: Measuring the Right Thing; Reducing Noise
Depth · The examples are C++. Carry over the checks, not the code: a fixed input distribution, latency versus throughput, warm-up, frequency scaling, pinning and treating small differences as an A/B test.
Focus on
- Sampling estimates where observed stacks spend time and can miss short events. Instrumentation records calls but perturbs execution. Choose a supported tool and timer for the hypothesis, not the prettiest report.
cProfiletottimeexcludes callees;cumtimeincludes them. Suspended async work and worker processes need separate interpretation; a request's wall time is not its CPU time.tracemallocsees traced Python allocations, not all native buffers or RSS. Compare snapshots after the same lifecycle stage for retention; reset the peak for transient allocations and record process memory separately.- Benchmark without profilers under matched input and offered load. Save correctness checks, hardware, Python build, worker count, start method, warm-up, all trials, throughput, latency distribution, rejection rate and queue depth. Include queue wait in end-to-end latency.
Reproduce closed-book
sum(list(range(n))) and sum(range(n)) for n = 100_000 and check both against n*(n-1)//2. Profile the materialized version with cProfile, then measure traced current and peak memory for each in fresh tracing sessions. The list version should have a larger temporary peak; do not require particular bytes or speedups. Retain 1,000 bytearray(1024) objects, compare snapshots before and after, clear the retaining list, collect garbage and compare again. Then benchmark both sum variants without tracing at small and larger n, with one warm-up and five trials each, saving CPU and full Python version/build with raw times. Explain an optimization's supported workload region rather than extrapolating to a service.Traps
- Timing task creation instead of awaiting completion, or hiding admission delay by starting the latency clock only after enqueue.
- Calling high RSS a Python-object leak. Allocator retention and native memory can keep RSS high after traced allocations fall.
- Reporting the fastest trial, changing total work, or benchmarking with
cProfileortracemallocenabled.
Practice
The section mini-project covers this: CPU and retention diagnosis plus a controlled before/after pipeline benchmark with one justified improvement and one rejected optimization.
Self-test
-
What does a high
cumtimebut lowtottimesuggest?The function's callees account for much of its observed cost. Inspect those callees before rewriting the wrapper. -
Can a snapshot after a function returns reveal its freed temporary list?
Not as a live allocation. Use peak tracing during the function for temporary memory and snapshots for allocations still retained. -
Why can low CPU usage coexist with high request latency?
The request may wait in a queue, on a lock, for I/O or for a downstream service. CPU profiles alone do not distinguish all those waits. -
Why keep failures and rejections in a concurrency benchmark?
A design can appear faster by dropping slow work. Compare completed throughput, end-to-end latency and failure rate at the same offered load.
Completion criterion
Stop when
You can identify the materialization peak and retention site from evidence, preserve the result oracle, and present raw unprofiled trials with a workload-limited conclusion rather than a single best time.
Mark passed, then open 6.6 Only mark this when you can do it without notes.Leads to
- 6.6 Structured logs, metrics and traces Combine profiles with request-level evidence to separate CPU work from dependency waits.
Comes after
- 3.9 Reference counts, cycle collection and weak referencesMemory growth in profiles often traces to cycles and caches.
- 6.7 Health checks, containers, supervision and runbooksRe-measure the service under its actual container limits after a recovery or optimization.
Resources for this unit
- Python concurrency documentation Use asyncio task groups, cancellation, queues and timeouts; concurrent.futures; multiprocessing; threading; and signal documentation to verify lifecycle and failure behavior.
- High Performance Python, 3rd edition Read Chapter 2 Profiling to Find Bottlenecks, Chapter 9 Asynchronous I/O, Chapter 10 The multiprocessing Module and Chapter 12 Using Less RAM, and use the companion code (https://github.com/mynameisfiber/high_performance_python_3e) as experiment starting points. Change the workloads and measure the regimes where concurrency helps, loses or only moves the bottleneck.
- Algorithmica - Getting Accurate Results Use Measuring the Right Thing and Reducing Noise for benchmark design: input distribution, latency versus throughput, warm-up, frequency scaling and pinning. The examples are C++; the checks transfer to Python.
Also in this section
- AlgoMaster Concurrency Study Concurrency 101; the Python units on threads, the GIL, multiprocessing, asyncio and concurrent.futures; synchronization primitives; concurrency challenges; locking strategies; and concurrency patterns. Then solve Bounded Buffer, Thread-Safe Blocking Queue, Thread-Safe Rate Limiter and Task Scheduler with Dependencies.
- Operating Systems: Three Easy Pieces - concurrency Read Chapters 26-33, from threads through event-based concurrency, for locks, condition variables, semaphores, concurrency bugs and event loops. Do selected homework only for a weak mechanism.
- Docker guides Use the Python container guide and build best practices to run as non-root, pin dependencies, handle signals and test CPU and memory limits.
- OpenTelemetry Python instrumentation Manually instrument one request so its structured logs, one low-cardinality metric and one trace share a correlation identity. Use the telemetry to locate one seeded fault.
- Timeouts, retries, and backoff with jitter Read Timeouts, Retries and backoff, and Jitter for deadline scope, idempotency, one retry owner and bounded retry load. Interpret the five-layer example as three total attempts per layer, not three retries plus the initial attempt.
- Site Reliability Engineering - overload and troubleshooting Read Chapter 21, The Pitfalls of Queries per Second, Client-Side Throttling, Utilization Signals, Handling Overload Errors and Conclusions. Read Chapter 12, Triage, Examine, Diagnose, Test and Treat, and Making Troubleshooting Easier for a service runbook. Skip datacenter load-balancing implementation details.
- Liveness, Readiness, and Startup Probes Read Liveness probe, Readiness probe and Startup probe for the different health decisions and failure actions. Apply the contracts to a local service; do not install Kubernetes or configure a cluster.