Unit 7.1 · about 3 h
283 / 324Security goals, trust boundaries and threat models
Every later unit assumes a written threat model. Without named assets and boundaries, least privilege and audit have nothing to attach to.
- explain
You can now
- State confidentiality, integrity, availability and accountability (every action traceable to an identity through audit) for one service, and say which one the design trades away under pressure.
- Draw assets, entry points, data flows and trust boundaries for a small RPC store.
- Write abuse cases for unauthorized mutation, cross-tenant reads and restore paths, and mark each as tested or residual risk.
Covers
- Confidentiality, integrity, availability and accountability
- Assets, adversaries, attack surfaces and trust boundaries
- Threat models, abuse cases and residual risk
Study, in this order
3 steps
- Confidentiality, integrity and availabilityBuilding Secure and Reliable Systems - selected design route
Chapter 1, Confidentiality, Integrity, Availability: the Confidentiality, Integrity and Availability sections, plus Reliability Versus Security design considerations
- Adversaries, profiles and methodsBuilding Secure and Reliable Systems - selected design route
Chapter 2, Understanding Adversaries: Attacker Motivations, Attacker Profiles from Hobbyists through Insiders, Attacker Methods (Threat Intelligence, Cyber Kill Chains, TTPs), Risk Assessment Considerations
- System model, threat identification and mitigationsOWASP Threat Modeling Cheat Sheet
System Modeling; Threat Identification with Choosing a Threat Modeling Technique; Response and Mitigations; Review and Validation
Focus on
- Security controls can block recovery: in the Chapter 1 password-manager story, the safe combination sat in the offline password manager it was meant to unlock. Model recovery and on-call paths with the same care as attacks.
- Adversary profiles set the budget: hobbyists through nation-states plus insiders. Defenses aim to raise attacker cost, not to stop every profile.
- The system model comes first: trust boundaries, data flows, data stores, processes and external entities, then technique choice.
- Each abuse case ends as a test or an explicit residual risk. Unowned risks are the failure, not the attack.
Reproduce closed-book
Closed-book, sketch the trust-boundary diagram of a single-tenant key-value RPC service, list its assets and two adversary profiles, and write three abuse cases (unauthorized write, cross-tenant read, restore from backup), each ending in a test or a residual-risk sentence.
Traps
- Treating uptime as the whole goal. Redundancy that serves stale or wrong-tenant data keeps the service up while breaking integrity and confidentiality.
- Modeling only external attackers. Chapter 2 treats insiders, who already hold access, as an adversary profile, and the password-manager outage needed no attacker at all.
- Collecting threats without mitigations or owners. The OWASP process continues into Response and Mitigations, then Review and Validation; a list of threats is not a threat model.
Practice
The section mini-project covers this: trust-boundary diagram, abuse cases and a residual-risk statement for the RPC store and its migration path.
Self-test
-
A service stays up during an incident but serves one tenant's records to another. Which goals failed and which held?
Confidentiality and integrity failed; availability held. Uptime alone says nothing about the other two. -
Why does Chapter 2 advise making yourself a more expensive target even against well-funded attackers?
Some attackers are economically motivated. Raising cost can remove the incentive even when perfect defense is impossible. -
What must the OWASP system model make clear before threat identification starts?
Trust boundaries, data flows, data stores, processes and the external entities that interact with the system. These are the likely attack points that STRIDE prompts are applied to. -
An abuse case has no test and no owner. What is it?
An unowned residual risk by default. Write it down as accepted risk or turn it into a test.
Completion criterion
Stop when
You can draw the boundary, name the adversaries, and close every abuse case as tested or accepted risk without reopening the books.
Mark passed, then open 7.2 Only mark this when you can do it without notes.Leads to
- 7.2 Identity, authorization, least privilege and secrets Turn the trust boundary into identities, scoped credentials and audit points.
- 6.7 Failure domains, correctness signals and safe rollout Place residual risks into failure domains and rollout guards.
Resources for this unit
- Building Secure and Reliable Systems - selected design route Read Chapters 1, 2, 4-9 and 12-14: adversaries, trade-offs, least privilege, understandability, changing landscapes, resilience, recovery, code, tests and deployment. Apply the chapter questions to the same RPC, store, tenant and migration boundary.
- OWASP Threat Modeling Cheat Sheet Model trust boundaries, data flows, data stores, processes and external entities, and list the assets they carry. Write misuse cases for unauthorized mutation, cross-tenant reads, stale credentials, sensitive logs and restore or migration paths, then turn each into a test or explicit residual risk.
Also in this section
- OpenDP theory - one bounded privacy mechanism Read A Framework to Understand DP and Differential Privacy with OpenDP (in the 0.16.0 user guide, Framework API), then Accuracy: Pitfalls and Edge Cases and the Differencing, Membership and Reconstruction attack notebooks. Define one privacy unit, clamp an aggregate, add calibrated noise, record \(\epsilon\) and \(\delta\) inputs, measure utility and list everything outside the guarantee.
- OSTEP - introductory security chapters Use Chapters 53-56 for introductory security, authentication, access control and cryptography vocabulary. Connect process and container isolation to the threat model and state where those boundaries do not protect the system.
- Systems Foundations Checkpoint - security and privacy slice Use only the tenant-access, privacy aggregate, abuse, migration, backup and residual-risk requirements as the final assessment. A toy privacy mechanism is evidence for its stated aggregate, not a claim that the service or model is private.
- Data Privacy and Security (MIT Data-Centric AI) Read the threat-model examples, shadow-training and metrics-based membership attacks, extraction by sampling plus perplexity ranking, empirical-defense evaluation, and the \((\epsilon, \delta)\) definition with DP-SGD.
- A Survey of Privacy Attacks in Machine Learning Use the attack taxonomy and threat model (adversarial knowledge, assets under attack) plus the causes of leaks and the defenses overview; skip the per-paper catalog.
- Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning Read the weak threat model (no model or training-set knowledge, about 50 poisoning samples) and the reported result of above 90 percent attack success; skip the optimization details.
- OWASP Input Validation Cheat Sheet Apply allowlist validation, parse-safely-then-validate, structured-data rules, and the common pitfalls (client-only checks, trusted internal sources, denylisting, logging rejected input).
- OWASP Deserialization Cheat Sheet Apply the Python clear-box rules (
pickle, unsafe YAML loaders,jsonpickle) and the language-agnostic methods (pure-data formats, signed data).