Unit 6.9 · about 2 h
282 / 324Unit cost, headroom and simpler designs
Reliability without cost is a wish. This unit turns the saturation and capacity numbers into a provisioning choice the team can afford, and keeps the simpler design on the table.
- derive
You can now
- Derive per-request unit cost in the constrained resource and the provisioned headroom for a stated workload.
- Defend the design against a 10x workload change with numbers: scale, shed, degrade or simplify.
- Compare against a simpler alternative and name what the complexity buys.
Covers
- Unit cost, headroom and simpler design alternatives
Study, in this order
3 steps
- Risk tolerance and the cost of reliabilityGoogle Site Reliability Engineering - selected core chapters
Risk Tolerance of Services (Target level of availability; Cost); Identifying the Risk Tolerance of Infrastructure Services
- Spare capacity and scaling limitsGoogle SRE Workbook - SLO, load, incident and canary exercises
Autoscaling (Configuring Conservatively: spare capacity for overload and redundancy; Setting Constraints: minimum and maximum bounds, quota)
- Simplicity defensePython Experimental Systems Kit - measurement slice
Design-defense questions (When should this remain a script instead of this package)
Depth · Use the one question as the simpler-alternative test for your design. The full kit returns in later stages.
Focus on
- Unit cost is per useful request in the constrained resource: 10 ms service time is 0.01 CPU-s per request, so 80 rps costs 0.8 CPU and 95 goodput rps costs 0.95 CPU.
- Headroom is provisioned over expected peak at a utilization target: 0.8 CPU at a 0.7 target provisions about 1.14 CPUs, with autoscale bounds and kill switches capping the bill when the model breaks.
- 10x (800 rps) is 8 CPU-seconds of work per second, so about 11.4 CPUs at the same 0.7 target, or shedding, degrading or removing work. Amdahl bounds splitting one request or job across CPUs: 90% parallel on 4 CPUs gives 3.08x, not 4x.
- The simpler alternative (one process, no cache, fewer moving parts) wins unless the complex design names the failure or cost it removes.
Reproduce closed-book
Traps
- Pricing peak capacity at 100% utilization. Queue math forbids it; buy the headroom or buy the queue.
- Counting hosts instead of the constrained resource. Cost follows CPU-s, bytes stored and bytes moved per request, not box count.
- Letting experiment capacity become permanent. Time-box canary and load-test capacity and remove it in the rollback plan.
Practice
AlgoMaster: Calculate Amdahl's Law Speedup
AmdahlSpeedupCalculator.speedup: 1/((1 - parallelFraction) + parallelFraction/processors), rounded to 5 places (examples: 0.9 on 4 gives 3.07692)
Attempt only after your own serial-fraction estimate. Compare the platform answer with the scale-out limit in your 10x defense.
Self-test
-
80 rps at 10 ms service with target \(\rho\) 0.7: provision?
\(0.8/0.7 = \text{about } 1.14\) CPUs. Round up to whole CPUs, set the autoscale minimum and bound the maximum. -
Amdahl with 0.9 parallel on 4 CPUs: speedup? What does it say about 10x load?
3.07692, and never above 10x with any CPU count. That caps speeding up one job. Serving 10x more independent requests needs 10x the CPU-seconds, or less work per request, shedding or degradation. -
A simpler single-process design handles 3x but not 10x. Keep the complex one?
Only with numbers: name the load where simplicity breaks, price both, and show what the complexity removes (failure domain, cost or toil).
Completion criterion
Stop when
You derive unit cost, headroom and the 10x table without notes and can defend the simpler alternative against your design with numbers.
Mark passed, then open 7.1 Only mark this when you can do it without notes.Leads to
- 6.3 SLIs, SLOs, error budgets and recovery objectives Revisit targets once cost bounds what reliability can be bought.
Comes after
- 6.4 Little's Law, capacity and queueing approximationsPrice the derived capacity and headroom.
- 6.8 Backups, runbooks, incident timelines and postmortemsPrice the prevention work the postmortem asks for.
Resources for this unit
- Google Site Reliability Engineering - selected core chapters Read Chapters 3, 4, 6, 10, 12, 17, 19-22: risk, SLOs, monitoring, alerting, troubleshooting, reliability tests, load balancing, overload and cascading failure. Apply every concept to the same bounded RPC and store rather than inventing separate examples.
- Google SRE Workbook - SLO, load, incident and canary exercises Use Chapters 2, 4, 9-11, 12, 14, 16 and 17. Define one SLO, estimate capacity, load through saturation, canary a changed version, roll it back, and write one evidence-based incident timeline.
- Python Experimental Systems Kit - measurement slice Use its artifact, bounded-pipeline, failure-injection, profiling and reproducibility requirements as the evidence checklist. Reuse the current service instead of building the full project again.
Also in this section
- OpenTelemetry Python - metrics, logs and traces Instrument request ID, queue wait, handler time, storage time, status and bounded labels. Export locally, retain raw load records and verify that instrumentation does not hide the bottleneck or create unbounded cardinality.
- AlgoMaster System Design - quantitative and failure drills Use Solve Little's Law, Calculate System Availability, Plan Reliability Recovery Targets, Calculate Amdahl's Law Speedup, Analyze Capacity Bottlenecks, Simulate Partition Quorums and Audit Failure Domains only after producing your own estimates. Compare the platform answer with measured service behavior.
- Gil Tene - How NOT to Measure Latency (talk) Use for coordinated-omission vocabulary only: closed-loop testers that wait for each reply, the missing samples under queueing, and the correction concept of recording intended send time. Do not adopt
wrk,heyorab; write the small Python generator in the saturation unit instead. - M/M/1 queue - model definition and stationary analysis Read Model definition (Poisson arrivals, exponential service, single server, infinite buffer, FIFO) and Stationary analysis (stability \(\lambda < \mu\), \(\rho = \lambda/\mu\), \(L = \rho/(1 - \rho)\), response time \(1/(\mu - \lambda)\) via Little's Law). Use the closed forms as approximations with assumptions stated, checked by your own simulation.