Section 1 of 7 · Systems Foundations
Machine, Process, Memory & I/O Foundations
Systems stop feeling mysterious when CPU work, kernel-owned process state, translated memory and persistent I/O can be traced as one concrete path. This first section builds that path before adding concurrency or a network.
10 units 22 topics 0/10
Units
10 in this section
- 1.1 Instruction pipelines, branches and SIMD explain4 h
- 1.2 Caches, working sets, bandwidth and NUMA explain · measure4 h
- 1.3 Processes, system calls, fork, exec and scheduling explain · implement4 h
- 1.4 File descriptors, pipes, signals and process lifecycle implement4 h
- 1.5 Virtual memory, page tables, TLBs, page faults and mmap explain · measure6 h
- 1.6 Resident size, virtual size and allocator retention measure3 h
- 1.7 Files, page cache, fsync, atomic rename and durability explain · implement5 h
- 1.8 I/O access patterns and storage devices measure4 h
- 1.9 Namespaces, cgroups and container boundaries explain · implement5 h
- 1.10 Binary logic and machine language with Nand2Tetris implement8 h
Before moving on
Checkpoint
Trace a process from creation to exit and a checksummed file from application write to recovered state. Show one page-fault observation, one descriptor or signal transition, one container resource-limit failure and one measured locality change.
Deliberately out of scope
Leave for later
- Writing a complete kernel or filesystem
- Advanced CPU microarchitecture and kernel tracing
- GPU profiling, CUDA and AI performance engineering
Resources for this section
Shared by every unit above
- Operating Systems: Three Easy Pieces - selected foundations Read Chapters 4-7, 13-15, 18-21, 36, 39-40, 42 and 45. Work from process state to address translation, devices, files, journaling and integrity. For every chapter, draw the owned state and one failure boundary instead of collecting summaries.
- MIT 6.1810 - selected xv6 process and memory labs Use Fall 2025 Lectures 1, 3-6, 8 and 10-11. Complete the util lab, one syscall task, and the page-table inspection plus ugetpid task from pgtbl. Add cow only if copy-on-write remains unclear. Keep all work in your own course checkout and stop before the full xv6 sequence.
- OSTEP homework - process, paging, disk and integrity simulators Solve process-run.py, paging-linear-translate.py, paging-policy.py, disk.py, fsck.py and checksum.py for at least two seeds each before using the answer flag. Then reproduce one prediction using strace, /proc or a small local program.
- AlgoMaster Operating Systems - selected observation route After OSTEP, use the matching lessons as a second explanation and recall check: fork, exec, wait, and exit; Zombies and Orphans; Signals; Context Switching; The Process Address Space; Paging; The TLB: Hits, Misses, and Invalidation; Page Faults; Copy-on-Write; mmap and Memory-Mapped Files; How malloc Works; VSZ, RSS, PSS, and What They Mean; File Descriptors; The Page Cache and Write-Back; Durability and fsync; Crash Consistency; Linux Namespaces; Control Groups; PID 1 and Signal Handling; Container Resource Failures; and the Investigating lessons. Treat quizzes as recall checks and keep your own traces as evidence.
- Linux proc and strace references Use proc(5), getrusage(2), mmap(2), fsync(2), rename(2), wait(2) and signal(7) while tracing a parent-child pipe, page faults, file publication and a resource-limit failure. Record exact commands and raw observations.
- Nand2Tetris - Projects 1 and 4 only Use Project 1 for Boolean logic and Project 4 for machine-language execution only if the binary or instruction-level entry check fails. Do not take the full course as a second systems curriculum.
- Algorithmica - Algorithms for Modern Hardware Use the Instruction-Level Parallelism, SIMD Parallelism and RAM & CPU Caches chapters for measured pipeline, branch, vectorization, bandwidth, latency and cache-line effects. The figures are the author's Zen 2 measurements at a fixed 2 GHz: predict and measure your own.
- MIT 6.S894 Accelerated Computing - Throughput Processors Use Lecture 2 for the processor model behind this section: superscalar issue, out-of-order execution, speculation and SIMD's need for coherent control. The labs run on the course's remote machines and target AVX-512 and NVIDIA GPUs, so they belong to the GPU track.
- Auto-Vectorization in LLVM Use The Loop Vectorizer Usage and Diagnostics sections for the Clang remarks the pipelines unit asks for; the Features list is optional depth.
- AlgoMaster: hardware and container storage companions Use Memory Ordering, Barriers, and False Sharing (coherence and false-sharing headings only); Huge Pages, NUMA, and ASLR (NUMA headings only); Storage Hardware; RAID Fundamentals (comparison and backup boundaries); and Container File Systems. These fill hardware and image-layer gaps outside the pool's selected OS revision lessons.
Apply the section
Mini-project · Mechanisms laboratory: kernel, process, memory, file and container (12 to 16 h)
Build the repository-wide mechanisms laboratory that later sections reuse: small kernel, locality, process, memory, file and container experiments with correctness checks and retained raw output. Each experiment pairs a measurement with its environment so later claims stay reproducible.
Steps
- Create the repository with
lab/,kv/anddossier/directories, where each experiment is onemake checktarget that writes raw output beside its environment record of CPUs, kernel, compiler and interpreter versions and pinning. - Build scalar, vectorized, dependency and branch variants of one reduction kernel, verify each sum against a Python oracle, and compare enabled and disabled vectorization in saved
objdumpoutput with pinned repeated timings. - Extend the row-first and column-first traversals into working-set and stride sweeps, read the cache sizes from sysfs and predict where the knees fall, then check each sweep against a Python oracle and keep pinned repeated timings with a topology note.
- Write
lab/supervise.pyto run a pipe worker from a descriptor ownership diagram, keep a leaked-writer regression that fails before the fix, and showSIGTERMtermination with zombie observation and reaping and closure on every exit path understrace. - Trace first-touch faults and copy-on-write isolation, stage allocate, free, reuse and free-all cycles with snapshots of requested payload beside resident and virtual sizes, add one sparse mapping, and diagnose retained capacity apart from growth.
- Turn atomic publication into
lab/publish.pywith ordered write, file sync, rename and directory sync checked instrace, failure tests that leave the old file intact, and a cached read comparison labeled as page-cache reads. - Retain one inspected Docker container OOM with namespace, credential and mount diagrams, configured limits compared with effective limits, PID 1 duties and an orchestrator checklist, and add the Nand2Tetris gate and assembler suites as the logic oracle.
Deliverables
- The
lab/directory with one checked experiment per target, raw outputs, environment records and a passingmake check. - Kernel, locality and I/O tables with pinned repeats, saved assembly, cache sizes and filesystem notes.
lab/supervise.pyandlab/publish.pywith diagrams, regression tests and the memory and container reports.
Checks
- rubric · Self-grade with the rubric below
Rubric
| Criterion | Pass | Strong |
|---|---|---|
| Correctness before timing | Every experiment prints a check asserted by make check, and each table was verified against its Python oracle. | Each timing names its oracle and pinning, and the write-up explains assembly and cache evidence plus untested effects. |
| Processes and descriptors | The leaked-writer test fails before the fix and passes after, and signal handling reaps on every path shown in strace. | The diagram predicts the observed closes, and an injected parent failure still leaves no zombie and no open pipe. |
| Memory, files and containers | Staged snapshots separate retention from growth, publication failures keep the old file, and limits match effective values. | The diagnosis names the reusable capacity, the sparse case separates virtual from resident size, and the checklist ties each action to its trigger. |
Feeds into: Capstone mechanisms laboratory: every experiment here recurs there. The supervisor starts the service in fault tests, and publish_atomic publishes the store manifest.