`torch.compile`: capture, guards, graph breaks and dynamic shapes

Compile silently accepts partial graphs: a break just runs that part eagerly. Without break counts and equivalence checks, a speedup number can hide a semantics change.

measure

You can now

Compile a stable fragment and prove eager equivalence on values and gradients.
Trigger a graph break and a recompilation on purpose, then read each from the logs.
Choose fullgraph, dynamic and backend settings from the failure you want to forbid or allow.

Covers

torch.compile capture, guards and graph breaks
Dynamic shapes, recompilation and semantic comparison

Study, in this order

2 steps

`torch.compile` parameters
PyTorch - compiler troubleshooting

fullgraph, dynamic, backend and mode; guard failures with TORCH_LOGS=guards; recompile_limit of 8 before falling back to eager

Done
Breaks, guards, recompiles and logs
PyTorch - compiler troubleshooting

Terminology with the graph break, guards, recompilation and dynamic shapes definitions; Logging Tools with TORCH_LOGS graph_breaks, guards and recompiles; Simple Workarounds including data-dependent fixes; Reporting Issues, Ablation with eager and inductor backends

Depth · Read Terminology, Logging Tools and Simple Workarounds fully. Skim TorchDynamo internals and Reporting Issues.

Done

Focus on

Default compile is partial: each frame compiles what it can and runs the rest eagerly. fullgraph=True turns an unnoticed break into a loud error.
Guards are runtime checks on shapes, types and values. A failed guard recompiles, and eight recompiles exhaust the budget and fall back to eager.
Data-dependent control flow is the classic break: .item, tensor conditions and prints split the graph. Hoist, use torch.cond, or capture scalar outputs.
Equivalence comes before timing. Compare eager and compiled values and gradients with a tolerance, across shapes, before believing any speedup.

Reproduce closed-book

Compile a pure function and assert eager and compiled outputs match. Then compile a branch on flag.item() and read the TORCH_LOGS=graph_breaks line naming Tensor.item() as the break. Call torch._dynamo.reset(), compile the same function with fullgraph=True and show it raises. Finally count unique graphs with a counting backend: dynamic=False gives one per shape.

Traps

Reporting a speedup without a break count. A fully broken compile is just eager with overhead, and the timer will not tell you.
Feeding changing Python floats, such as a scheduled learning rate, into a compiled function. Under dynamic=False each new value recompiles until the limit of 8; the default recompiles once and then treats the float as dynamic. A tensor avoids both.
Reusing a function across compile experiments. Caches are per code object, so a fullgraph=True compile of a function already compiled without it can reuse the old entries and not raise. Call torch._dynamo.reset() between experiments.

Practice

The section mini-project covers this: triggered graph-break repair with eager comparison.

Self-test

Compiled code matches eager but the log names Tensor.item() as a graph break. What happened to the function?
Under dynamic=False, one shape reuses its graph while a new shape adds a second. What does dynamic=True change?
Timing collapses to eager speed after several shape changes. Which limit fired, and how do you see why?
Why compare values and gradients with tolerance before trusting a compile speedup?

Completion criterion

Stop when

You can trigger, name and fix one break and one recompile from the logs, with eager equivalence holding across your test shapes.

Mark passed, then open 8.9 Only mark this when you can do it without notes.

Leads to

8.9 Profiling training phases and verifying optimizations Measure compiled versus eager inside profiled phases.

Resources for this unit

PyTorch - compiler troubleshooting
PyTorchrequiredreference
Read graph breaks, guards, recompilations and logging. Compile one stable fragment, intentionally trigger a graph break or recompilation, inspect it, and compare eager and compiled outputs and gradients.

Also in this section

PyTorch - reliability and performance recipes
PyTorchprimarycourse
Use the Profiler, Automatic Mixed Precision, model loading and Benchmark recipes, then the Performance Tuning Guide. Apply each to one compact model from an earlier section and keep a CPU fallback when CUDA is unavailable.
PyTorch - checkpointing, numerical accuracy and CUDA semantics
PyTorchrequiredreference
Use the official user-guide entries for torch.utils.checkpoint, numerical accuracy, CUDA semantics, reproducibility and serialization. Predict memory categories before measuring and define resume equivalence before running.
Deep Learning Training Debugger
Roadmap-Finalreferenceassessment
Use the failure-injection, measurement and evidence rules as the deeper standard. The full Tier 2 project is a later route and is not added to this stage milestone.
TensorTonic - mixed-precision memory and transformer FLOPs
TensorTonicsupportpractice
Solve only Mixed-Precision Training Memory Accountant and Transformer Training FLOP Estimator. Reconcile both with the compact model's profiler and document every included and excluded term.
Learn PyTorch - profiler walkthrough
Daniel Bourkeoptionalimplementation
Use notebook 10 only if the official profiler route remains unclear. Transfer the relevant annotations into the compact model instead of keeping the tutorial code.
TrenTorch - precision, checkpointing and resume
TrenTorchsupportpractice
Solve Loss Scaling, FP16/BF16 Representable Range, Gradient Checkpointing and Resume-from-checkpoint in NumPy, then reproduce each behavior with the PyTorch API in the compact model.
Python os - `os.replace`
Python Software Foundationreferencereference
Read os.replace for the atomic-rename step of checkpoint writes: save to a temp file in the same directory, flush and fsync, then replace the final name.
Hands-On Machine Learning with Scikit-Learn and PyTorch
Aurelien Geron (O'Reilly, 2025)supporttheory
Read Appendix B, Reduced Precision Models and Mixed-Precision Training, for why float16 training underflows and what fp32 primary weights and loss scaling fix.