Imports, virtual environments and src-layout packaging

The checkpoint builds and installs a package. Import evidence is also how every environment bug gets diagnosed.

explain
implement

You can now

Explain module caching, path resolution and package boundaries, including script versus -m execution.
Diagnose import shadowing and environment mismatch from sys.path and sys.modules evidence.
Build and install a typed src-layout package from a blank directory with a console entry point, verified in a clean environment.

Covers

Modules, packages, import resolution and module caching
Virtual environments, dependencies and editable installs
Src-layout packaging, pyproject metadata and console entry points

Study, in this order

8 steps

Import search path
Stas' Python Cookbook - selected path

Modules and Imports: sys.path order; computing paths from __file__; PYTHONPATH

Done
Module cache and dynamic import
Stas' Python Cookbook - selected path

Inspecting and reloading modules (the sys.modules cache; reload keeps old objects) and Dynamic import with importlib

Done
The module cache and packages
Python language reference

Section 5.3.1 The module cache (one execution per process); 5.2 Packages (regular versus namespace); 5.7 relative imports

Done
Searching and __main__
Python language reference

Section 5.3 Searching with 5.5 Path Based Finder; 5.8 __main__ (script versus -m path and the double-execution trap)

Done
Environment variables and argparse
Stas' Python Cookbook - selected path

Environment Variables and Program Arguments: environ handling; the argparse CLI shape; replaying the exact command line

Done
Versions and virtual environments
Stas' Python Cookbook - selected path

Versions and Dependencies: sys.version_info checks, importlib.metadata versions, venv creation and activation

Done
Creating the package files
Packaging Python Projects

Creating the package files (the src layout tree) through Configuring metadata; choose the setuptools backend

Done
Building and installing locally
Packaging Python Projects

Generating distribution archives (one build produces sdist and wheel) and Installing your newly uploaded package

Depth · Skip Uploading the distribution archives; do not publish to TestPyPI. Install from dist/ into a clean venv instead (verified with a setuptools-backend build and a clean-environment wheel install).

Done

Focus on

Importing executes the module once per process and caches it in sys.modules. A second import binds the same object; reload() re-executes but old objects keep old classes.
sys.path[0] differs by launch: a script prepends its own directory while -m prepends the current directory. The same file run as a script and imported as a module executes twice, as __main__ and under its real name.
Regular packages need __init__.py; namespace packages merge path portions without one. Relative imports need package context and fail in bare scripts.
Editable installs register the project in the environment; path hacks only fake it. The passing test is a clean-environment install with no sys.path edits.

Reproduce closed-book

From a blank directory, create a src-layout package named atlas_demo with an add() function and an atlas-demo entry point; build the sdist and wheel; install the wheel in a clean environment and run the entry point to print 5 with no sys.path edits.

Traps

Running tests from the package directory and importing the local tree by accident. The clean-environment install test exists to catch exactly this.
Two copies of one module: tests import pkg.mod while a script already ran it as __main__. State lives in two places and singletons split.
Using pip freeze for requirements. It captures the whole environment; record direct dependencies and verify with the clean install.

Practice

AlgoMaster Python: Exercise: Packages

Exercises 1 to 5, especially Exercise 3: Re-export a Flat Package API and Exercise 4: Use Relative Imports in a Subpackage

Do it after building the tutorial package; compare its layout with yours.

AlgoMaster Python: Exercise: Virtual Environments

Exercises 1 to 5, especially Exercise 2: Read pyvenv.cfg Settings and Exercise 5: Audit a Venv Against Requirements

Create the clean install-test venv from memory.

Self-test

import pkg.mod runs twice in one process. How many executions happen, and how do you prove it?
Why does python sub/show.py differ from python -m sub.show in sys.path[0]?
A script imports its sibling fine but fails after moving to a src layout. What changed?
What distinguishes an editable install from a PYTHONPATH hack?

Completion criterion

Stop when

Your entry point runs from a clean environment, and you can name sys.path[0] and the cached module for any launch.

Mark passed, then open 3.9 Only mark this when you can do it without notes.

Leads to

4.7 Layer boundaries and command-line contracts Layer boundaries and CLI contracts build on this package.
5.8 Dependency locks, run manifests and quality gates Lock the dependencies this package declares.

Comes after

3.6 Context managers, exceptions and deterministic cleanupConsole entry points consume the error-to-exit-code mapping.

Resources for this unit

Stas' Python Cookbook - selected path
Stas Bekmanprimarycourse
Run Part I on dictionaries, iteration, functions, closures, decorators, classes and context managers; Part II on modules, files, paths, environment and arguments; and Part IV on versions, dependencies, packaging and code quality. Finish by building one src-layout package from a blank directory.
Python language reference
Python Software Foundationrequiredreference
Check Data model, Execution model, Import system and Exceptions whenever a claim depends on Python semantics. These pages are the authority when the course and a runtime observation disagree.
Packaging Python Projects
Python Packaging Authorityrequiredimplementation
Follow the packaging tutorial once from a blank directory, using a src layout, pyproject.toml, an isolated build and a clean-environment installation test.

Also in this section

AlgoMaster Python - structured revision
AlgoMastersupportpractice
Use Functions, Modules and Packages, OOP, Advanced OOP, Exception Handling, Iterators and Generators, Type Hints and Annotations only after the cookbook pass. Open it for recall practice or a diagnosed weak mechanism, not as a second complete course.
dataclasses - data classes
Python Software Foundationreferencereference
Read the @dataclass parameters (eq, order, frozen, slots), field() with default_factory, frozen instances, post-init processing, replace, asdict and astuple. Use for the validated frozen dataclass in attributes-protocols-dataclasses.
typing - type hints and protocols
Python Software Foundationreferencereference
Read Protocols for structural typing and runtime_checkable limits, Generics for TypeVar generics, and assert_type for narrowing checks. Use for the Protocol and narrowing parts of type-hints-protocols; PEP 695 syntax itself is in the language reference.
gc - cycle garbage collector
Python Software Foundationreferencereference
Read the cycle collector interface: collect, get_count, garbage and disable/enable. Use for the cycle-collection and measurement parts of refcounting-gc-weakrefs.
weakref - weak references
Python Software Foundationreferencereference
Read weak-reference objects (ref callbacks, proxy objects) and the WeakValueDictionary example. Use for the weak-reference part of refcounting-gc-weakrefs.