Unit 3.7 · about 4 h
255 / 324Connection shutdown, pools and latency decomposition
Setup cost decides reuse policy; shutdown order decides deploy safety. It assumes the Programming container health and SIGTERM-drain unit (programming/concurrency-profiling-operations/health-containers-runbooks); here the mechanism is sockets, accept queues, pools and load-balancer coordination.
- explain
- measure
You can now
- Explain half-close,
TIME-WAITandCLOSE-WAITownership, and the graceful shutdown order. - Size a connection pool from concurrency with Little's Law and key pools by security boundary.
- Decompose client latency into DNS, connect, TLS, queue, handler and dependency parts from client, server and trace evidence.
Covers
- Cancellation, half-close and graceful connection shutdown
- Connection pools, server queues and load-balancing vocabulary
- Client, network, queue, handler and dependency latency
Study, in this order
6 steps
- TCP termination and half-closeAlgoMaster Networks - pcap and concurrent server labs
The Normal Four-Segment Close; Understanding
TIME-WAIT; UnderstandingCLOSE-WAIT; Half-Close at the Socket API; Abrupt Termination withRST - listen backlog, accept, close and shutdownBeej's Guide to Network Programming - sockets to multiplexing
Section 5.5 (
listenand the backlog); Section 5.6 (accept); Section 5.9 (closeandshutdown) - Connection pooling and keep-aliveAlgoMaster Networks - pcap and concurrent server labs
How a Pool Handles a Request; Pool Keys Protect Routing and Security Boundaries; Size the Pool from Concurrency; Separate the Timeout Boundaries; Stale Connections Cannot Be Eliminated Completely
- Graceful shutdown and drainingAlgoMaster Networks - pcap and concurrent server labs
A Shutdown State Machine; The Order of Operations Matters; Budget the Grace Period; Testing Shutdown Under Load
- Where latency comes fromAlgoMaster Networks - pcap and concurrent server labs
The Four Per-Hop Delays; Protocol Round Trips Add Latency; Measuring an HTTP Latency Breakdown; Interpreting Common Network Measurements
- A latency decomposition worked exampleAlgoMaster Networks - pcap and concurrent server labs
Turn the Stream into a Timeline; Inspect the Edges of the Gap
Depth · Read as a decomposition method: phase table first, then edges of the dominant gap. You already studied its flow-control mechanism in the transport unit.
Focus on
- Each direction closes independently.
shutdownfor writing sendsFINwhile the read side stays open;closewith unread data still queued can reset the connection. TIME-WAITbelongs to the active closer and lasts twice the maximum segment lifetime; a lingeringCLOSE-WAITmeans the application never closed, which is a leak, not a TCP quirk.- Pool size follows offered concurrency: arrival rate times hold time, plus headroom. Pool keys must include every security boundary (host, TLS identity, credentials), never just the IP.
- One client number never decomposes itself. DNS, connect, TLS, queue, handler and dependency times come from different instruments; subtract only compatible timestamps.
Reproduce closed-book
curl -w, three times, and report the numbers with your curl and TLS library versions and warm or cold DNS.Traps
- Reusing a connection across hostnames that resolve to one IP. Certificates and authorization attach to names, not addresses.
- Treating starttransfer minus appconnect as server time. It still includes request travel, queueing and first-byte return.
- Draining by closing the listener only. In-flight requests, keep-alive connections and load-balancer caches need explicit propagation, a grace budget and a forced-stop step.
Practice
AlgoMaster: Design a Connection Pool Queue
acquire and release with FIFO waiter handoff; a release hands directly to the oldest waiter
The release-to-waiter handoff is the bounded-admission behavior your pool needs.
Self-test
-
ssshows many sockets inCLOSE-WAIT. Whose bug is it?The local application's. The peer closed its side; the local end never closed its socket, so the state persists. -
A pool holds 20 HTTP/1.1 connections at 600 requests/s with 50 ms holds. What happens?
Waiters accumulate without bound. The pool sustains about 400 requests/s; the excess queues, and latency and memory grow. -
Why must a pool discard a connection that received
Connection: close?The server will not accept another exchange on it. Lending it out guarantees the next borrower fails. -
A pcap shows a 5 s gap with zero-window advertisements and no retransmissions. Which latency component grew?
None of the network components. An endpoint stall at the receiver inflated total time; DNS, connect and TLS phases were already complete.
Completion criterion
Stop when
You size a pool from rate and hold time, place a CLOSE-WAIT leak and a zero-window stall from evidence, and write the shutdown order for a pooled service.
Leads to
- 6.2 Latency distributions, throughput and saturation Turn the decomposition into distributions and saturation analysis.
- 3.8 Deterministic network fault injection Test pool exhaustion and shutdown races under scripted faults.
Comes after
- 3.2 TCP and UDP transport behaviorTurn setup cost and window stalls into pool and latency policy.
- 3.3 TLS and HTTP versionsTurn handshake cost into reuse and pool policy.
Resources for this unit
- Beej's Guide to Network Programming - sockets to multiplexing Read What is a socket?, IP addresses and data structures, Client-server background, System calls or bust, Client-server examples, Slightly advanced techniques and sendall. Build the simple TCP pair first, then replace delimiter assumptions with a bounded length-prefixed protocol.
- AlgoMaster Networks - pcap and concurrent server labs Use Reading a Packet Capture: A Real Bug and Building a Concurrent TCP Echo Server after the Beej implementation. Reproduce one packet-level observation and one admission, deadline or shutdown behavior in your independently written service.
Also in this section
- Beej's Guide to Network Concepts - TCP and UDP path Read the OS and sockets, protocol layering, IP, UDP and TCP chapters alongside packet captures from your own client and server. Use it to explain transport behavior rather than memorize layer names.
- MIT 6.5840 - RPC, threads and fault vocabulary Read the introductory RPC and threads material plus the lab guidance. Use it to enumerate timeout ambiguity and duplicate effects. Stop before Raft and the full lab sequence.
- Replicated Log and Stream Processor - protocol slice Use only the protocol requirements, deterministic network fault matrix and delivery-contract questions. Build the single-node framed service now; keep replication and consensus for the later project.
- Designing Data-Intensive Applications, 2nd edition - Chapter 5 Read Formats for Encoding Data (printed pp. 163 to 177) for schema merits and compatibility, and Dataflow Through Services: REST and RPC (printed pp. 180 to 186) for service dataflow, version coexistence and the idempotence pointer.
- gRPC Deadlines guide Read Overview, Deadlines on the Client and Deadline Propagation for absolute deadlines, remaining-time forwarding and clock-skew shielding.
- gRPC Status Codes Read the full status list, including
DEADLINE_EXCEEDED,UNAVAILABLEand the codes never generated by the library, for the error taxonomy. - W3C Trace Context Read Section 3.2 for the
traceparentformat and Sections 3.4 to 3.5 for the legaltraceparentandtracestatemutations when forwarding.