Section 4 of 7 · Programming & Data Foundations
Interfaces, Data & APIs
Research and production systems fail at boundaries. Typed contracts, explicit schemas and idempotent interfaces keep malformed data and retries from becoming silent corruption.
8 units 18 topics 0/8
Units
8 in this section
- 4.1 Typed interfaces, runtime validation and configuration explain · implement4 h
- 4.2 JSON, JSONL, CSV, columnar formats and bounded ingestion implement4 h
- 4.3 Schema versions, compatibility and migrations explain · implement4 h
- 4.4 Relational modeling, joins, aggregation and window functions implement6 h
- 4.5 Transactions, isolation anomalies and ORM boundaries explain · measure5 h
- 4.6 Indexes, selectivity and query plans explain · measure4 h
- 4.7 Layer boundaries and command-line contracts implement3 h
- 4.8 HTTP APIs, pagination, streaming and idempotency explain · implement5 h
Before moving on
Checkpoint
Expose one typed service through CLI and HTTP, ingest malformed and old-version records safely, and defend the schema, transaction and idempotency boundaries.
Deliberately out of scope
Leave for later
- Large web frameworks and frontend development
- Distributed databases and cross-service transactions
- ORM-specific architecture beyond one small comparison
Resources for this section
Shared by every unit above
- Official docs typed-service build path Build one record service in this order: structural protocols and generics; Pydantic models, validators, strict mode, serialization and settings; FastAPI request bodies, dependencies, errors and tests; then PostgreSQL transactions and EXPLAIN. Replay valid, malformed, duplicate and previous-schema requests through CLI and HTTP.
- AlgoMaster SQL - selected route Study SQL Foundations, Querying Data, Joins, Subqueries and CTEs, Window Functions, DDL, DML and Views, Performance Optimization, Transactions and Concurrency, and Data Modeling. Stop when the service queries and transaction tests pass.
- PostgreSQL transactions and EXPLAIN Use the transaction tutorial and EXPLAIN guide to demonstrate commit, rollback, one isolation anomaly or prevented anomaly, and a measured plan improvement on the database actually used.
- FastAPI tutorial Use Request Body, Dependencies, Handling Errors and Testing to expose the same service layer through HTTP. Keep framework models outside the domain core and preserve the same error semantics as the CLI.
- AlgoMaster Low-Level Design - one rehearsal Use one fitting design prompt to rehearse interface contracts, dependency direction and failure ownership. Treat the review as feedback, then verify behavior with your own tests.
- Stas' Python Cookbook - data and interfaces Use Serialization and Deserialization plus Environment and Arguments for focused implementation help. Compare JSON, JSONL, CSV and compressed data, and never use pickle for untrusted interchange.
- AlgoMaster Python - interfaces, generics and data files Use Interfaces and Protocols plus Generic Types for structural typing, and Working with JSON plus Working with CSV for JSONL, error handling,
newlineandDictReaderbehavior. Lesson quizzes check recall; implementation is verified with your own scripts. - SQLAlchemy 2.1 documentation - select, Session and eager loading Use the
select()tutorial for 2.x query style,Sessionbasics for the transaction boundary and identity map, and relationship loading for lazy versusselectinloadversusjoinedloadplusraiseload. Useselect()style andSession.executeorSession.scalarsonly, never the legacyQueryAPI. - Alembic tutorial - migration scripts and downgrading Use Create a Migration Script, Running our First and Second Migration, Getting Information and Downgrading to run a paired
upgradeanddowngraderound trip on a scratch database. - PyArrow Parquet read and write docs Use Reading and Writing Single Files plus
Tablefrom_pydictandfrom_pylistto convert normalized records to Parquet and test types, nulls, column projection and schema. - Stripe idempotent requests Use the
Idempotency-Keyheader contract, stored-response replay, parameter-mismatch errors andPOST-only scope to implement keyed replay for one endpoint.
Apply the section
Mini-project · One run service behind a CLI and an HTTP API, on PostgreSQL (12 to 16 h)
Turn
taskrunner into a service with one service layer behind a CLI and an HTTP API on PostgreSQL. Version the payload, migrate the schema both ways, and prove transaction, isolation and idempotency behavior with tests.Steps
- Write
RunServiceagainst aRunStoreprotocol with an in-memory fake for unit tests, validate the boundary model strictly, and test configuration precedence with the AlgoMaster Interfaces and Protocols exercise as reference. - Add a batch import that streams JSON Lines or CSV, quarantines malformed rows with line numbers, and writes accepted rows to Parquet with round-trip type and null checks.
- Migrate the schema with Alembic through an upgrade, downgrade and re-upgrade round trip on a scratch database, and test the versioned payload in both directions following the Alembic tutorial downgrade round trip.
- Debit credits and insert the run in one transaction with a rollback test, then reproduce a serialization anomaly under
REPEATABLE READwith a bounded retry, following the PostgreSQL transactions tutorial. - Predict each read query's row count before running it, then compare the listed filter's plan before and after adding its index and name one query the index does not help, using the AlgoMaster
LEFT JOINandGROUP BYexercises as reference. - Serve the same service through a CLI and a FastAPI app over the untouched service layer, testing the status-code contract, the capped page and store swapping with the FastAPI tutorial Testing extended example.
- Make submission idempotent with an
Idempotency-Keyledger claimed in the same transaction as the debit, and prove a replay stores one run while a mismatched body is rejected.
Deliverables
- The service package:
RunService, the fake and PostgreSQL stores, the CLI and the FastAPI app. - Alembic revisions with the round-trip transcript and both-directions payload tests.
- Transaction, isolation, idempotency and import tests with quarantine file, plans and predictions.
Checks
- rubric · Self-grade with the rubric below
Rubric
| Criterion | Pass | Strong |
|---|---|---|
| One service layer | The CLI and HTTP paths call the same service, the service module stays transport-free, and both stores pass the suite. | One parametrized suite runs every service test against the fake and the PostgreSQL store. |
| Boundaries and evolution | Malformed, extra-field and cross-version payloads are rejected with field-level errors, and the migration round trip succeeds. | The write-up classifies each schema change as additive or breaking and names which old clients keep working. |
| Transaction behavior | The failed debit leaves credits and run count unchanged, and the anomaly test retries to the correct balance. | The retry loop is also tested with an injected serialization failure and stops clearly at its limit. |
| HTTP contract and idempotency | Created, validation-error, not-found and capped-page behavior hold, and a keyed double submission stores one run. | A same-key different-body request is rejected, and a replay returns the stored response unchanged. |
| Data evidence | The import quarantines malformed rows with line numbers without loading the whole file, and each query matches its predicted count. | The plan comparison states table size and buffers, and explains the query the index does not help. |
Feeds into: The typed installable service through CLI and HTTP with schema, migration, transaction and idempotency tests; its database becomes the readiness check in operations.