Unit 5.5 · about 4 h
31 / 324Shell streams, exit status, signals, processes and permissions
Every experiment runs through a shell. An unquoted path or a swallowed exit code turns a clean failure into silent corruption.
- explain
- implement
You can now
- Write a fail-fast script that quotes paths, validates inputs, propagates exit status and cleans up on signals.
- Diagnose a failed process from exit status, streams, permissions, environment and signal behavior.
- Explain quoting, redirection, pipelines, job control and partial-output recovery without trial and error.
Covers
- Shell quoting, streams, exit status, signals and partial output
- Permissions, environment, processes, file descriptors and logs
Study, in this order
3 steps
- Course shell: language, streams and strict modeThe Missing Semester of Your CS Education
The shell language (bash): quoting, redirections, pipes, command substitution, shebang,
set -euo pipefailand the run-until-failure script; Exercises on streams, exit status, scripts and permissions - Command-line environment: streams, status and signalsThe Missing Semester of Your CS Education
Arguments; Streams; Environment variables; Return codes; Signals; Exercises under Return Codes and Signals and Job Control
- Bash execution control and background logsThe Art of Debugging
Bash: Controlling script execution (trace with
-x, abort on failures inside the script and in the pipeline, abort on undefined variables, temporarily turning offsetcommands);nohup
Focus on
- Quote every expansion and keep globs unquoted only when intended. Single quotes preserve literally; double quotes allow expansion.
- Use
set -euo pipefail,trapcleanup onEXIT, and makeINTandTERMexit (130 and 143) so theEXITtrap runs. Withoutpipefail,false | trueexits 0 and the failure vanishes. - Streams are separate contracts:
stdoutcarries data,stderrcarries diagnostics, the exit code carries the outcome. Redirect and check each one deliberately. - Background jobs report through
$!,waitandkill. Partial output stays recoverable when each stage writes its own log.
Reproduce closed-book
check-and-run.sh with a shebang, set -euo pipefail, a quoted $1 filename check with test -f, a temp file removed by trap on EXIT, traps that exit 130 on INT and 143 on TERM, a sleep run as sleep & wait $!, and separate stdout and stderr logs. Show that false | true exits 1 under pipefail and 0 without it on GNU bash 5.3.9, and that kill -TERM ends the script at once with exit code 143 and the temp file gone. Then drop the exit traps and watch TERM wait for the sleep and exit 0.Traps
set -edoes not fire insideifconditions,&&lists, or pipelines withoutpipefail. Test failure handling explicitly instead of assuming strictness.- Unquoted variables and command substitutions split on spaces. Filenames with spaces become several arguments, so quote or use arrays.
- A cleanup trap on
TERMthat does not exit. Bash runs it only after the foreground command finishes, then resumes the script, which can exit 0;trapEXITfor cleanup, exit from theINTandTERMtraps, andwaiton background work so the signal lands at once.
Practice
Missing Semester course-shell stream and status exercises
Exercises, items 5, 6, 8 and 9: split stdout and stderr, the $? one-liner, check.sh and chmod +x
Missing Semester command-line return-code exercise
Exercises under Return Codes: the run-until-failure script
Reuse your fail-fast header and trap in it.
Self-test
-
false | true; echo $?prints what with and withoutpipefail?1 withpipefail, 0 without. The pipeline status is the last command unlesspipefailpropagates the failure. -
Why must
cdbe a shell builtin?A child process cannot change its parent working directory, socdmust run in the shell itself. -
A script cleans up on Ctrl-C but not on
kill -9. Why?SIGKILLcannot be caught or ignored. Traps handleINTandTERMbut neverKILL, so temp files must tolerate it. -
Output still appears after
cmd 2>/dev/null. What happened?Usually the text is onstdout, notstderr. Otherwise another pipeline stage printed it, or the program writes to/dev/ttydirectly, as password prompts do.
Completion criterion
Stop when
Your script handles a filename with spaces, reports the right exit code for missing input, and leaves no temp files after SIGTERM.
Leads to
- 5.7 SSH trust, keys, forwarding and tmux sessions Run the same fail-fast scripts on remote hosts over SSH inside tmux.
- 5.4 Debuggers, logs, traces and competing hypotheses Use
straceand process state as debugger evidence for hangs and missing files.
Comes after
- 5.4 Debuggers, logs, traces and competing hypothesesDiagnose processes from the shell with streams, exit status, signals and
strace.
Resources for this unit
- The Missing Semester of Your CS Education Complete the 2026 lectures on the shell, command-line environment, development tools, debugging and profiling, Git, packaging and shipping code, and code quality. Apply every exercise to the same stage project.
- The Art of Debugging Use the Unix, Python and compiled-code sections during a real seeded defect. Follow the evidence and tiny-reproducer workflow rather than reading the unfinished book front to back.
Also in this section
- pytest documentation Use Get Started, fixtures, parametrization, monkeypatch, caplog and temporary paths to build behavior and failure-path tests without order dependence.
- Hypothesis documentation Follow the Quick start and property-based testing guides, then generate operation histories and retain the minimized example for one real defect.
- Pro Git Use Chapters 2-3 and the sections on debugging, internals and recovery while constructing a history for bisect and recovering a deliberately displaced commit.
- Stas' Python Cookbook - debugging and testing Use Part III and the packaging sections for debugger, traceback, pytest, logging, profiling and clean-environment recipes.
- AlgoMaster Python - mocking Use the Mocking lesson for
unittest.mock:MockversusMagicMock,patchtargets,autospecand the over-mocking trap. The pytest docs remain the reference for fixtures andmonkeypatch.