Section 4 of 8 · Deep Learning

Convolutional & Recurrent Architectures

Convolution and recurrence supply complementary inductive biases. Owning their shapes, costs, state and failure modes makes architecture choice explainable in interviews and experiments.

7 units 20 topics 0/7

Units

7 in this section

4.1 Convolution shapes: channels, padding, stride, dilation and groups
derive · implement
5 h
4.2 Pooling, transposed convolution, receptive fields and cost
derive
4 h
4.3 Equivariance, residual CNN blocks and resolution cost
explain · measure
5 h
4.4 Recurrent cells, unrolling and backpropagation through time
derive · implement
5 h
4.5 Recurrent gradients, GRU and LSTM gates
derive · implement
5 h
4.6 Hidden state, encoder-decoder models and teacher forcing
explain
4 h
4.7 Decoding, variable-length batches and leak tests
implement · measure
5 h

Before moving on

Checkpoint

Derive shape, receptive field and cost for a stated CNN, implement one convolution and one recurrent or gated cell, then prove padding or future input cannot alter a valid output.

Deliberately out of scope

Leave for later

Detection, segmentation, speech and video specialization
Large-scale architecture reproduction
Production sequence decoding systems

Resources for this section

Shared by every unit above

D2L - convolutional and recurrent architecture route
D2L authorscourse

primary

Use why convolution, convolution layer, padding and strides, pooling, channels, batch normalization and ResNet, then RNN, BPTT, scratch RNN, GRU, LSTM, encoder-decoder and seq2seq. Work through the scratch mechanisms before concise APIs.
Understanding Deep Learning - convolution and residual networks
Simon J. D. Princetheory

required

Use Chapters 10 and 11 to sharpen convolutional invariance, receptive-field reasoning, residual paths and normalization. D2L remains the recurrent implementation route.
Stanford CS231n - Assignments 1 and 2
Stanford Universitypractice

support

Use only the neural network, normalization, dropout, convolution and PyTorch components that match this section. Do not complete the course or unrelated vision tasks.
Deep-ML candidate - convolution and recurrence problems
Deep-MLpractice

optional

After the platform admission check, use at most one of 41 Conv2D, 62 RNN with BPTT, 287 GRU or 59 LSTM as a changed-prompt drill.
TrenTorch - convolution cost and recurrent cells
TrenTorchpractice

support

Solve FLOPs Estimation, GRU Forward and LSTM Forward after the D2L scratch models. Each reference cell matches torch.nn.GRUCell or LSTMCell.
PyTorch nn.Conv2d reference
PyTorchreference

reference

Read Signature and Shape for the exact floor formula with dilation, the groups divisibility rule with grouped and depthwise examples, and the weight layout \((C_{\text{out}}, C_{\text{in}}/\text{groups}, kH, kW)\). D2L and UDL teach the mechanism; this page is the shape contract to code against.
PyTorch sequence packing utilities
PyTorchreference

reference

Read pack_padded_sequence for lengths, enforce_sorted and batch-first at recognition depth: what packing skips and what contract it demands. Do not reimplement the packed format; pair it with the masked-loss implementation in D2L seq2seq.

Apply the section

Mini-project · Convolution, gated recurrence and leak tests (10 to 14 h)

Implement a convolution and a GRU or LSTM cell from tensor operations and check them against PyTorch. Derive a small CNN's shapes, receptive field and cost before measuring them, and prove with interventions that padding and future inputs cannot change valid outputs.

Steps

Pass the CS231n Assignment 2 checks for the naive convolution and max-pooling passes in ConvolutionalNetworks.ipynb and for the vanilla RNN step and sequence passes in RNN_Captioning_pytorch.ipynb.
Write conv2d with padding, stride, dilation and groups, using unfold or loops, and match torch.nn.functional.conv2d in float64 over a grid of settings that includes groups equal to channels.
For a stated small CNN, derive every output shape, the last receptive field, the parameter count and forward cost by hand. Check shapes and parameters in code, and check the receptive field by input gradients under a linearized setup that keeps every path alive.
Implement a GRU or LSTM cell in PyTorch's gate convention, load weights from torch.nn.GRUCell or LSTMCell, and match outputs and gradients over a 20-step unroll in float64.
Batch variable-length sequences with padding plus masks or packing, and test that changing padded values leaves every valid output and the final valid hidden state unchanged.
For a unidirectional model, test that changing the input at step \(t\) leaves outputs before \(t\) unchanged. Run the same test on a bidirectional model and show that it fails, which proves the test can fail.
Plot gradient norms through time for a vanilla RNN and your gated cell on a long sequence, and relate them to measured gate values.

Deliverables

Convolution and gated-cell modules with float64 differential tests against PyTorch.
The derivation sheet for the stated CNN, with code checks for shapes, parameters and receptive field.
The leak tests: padding invariance, future-input invariance and the bidirectional counter-case.

Checks

external-tests · CS231n Assignment 2: `ConvolutionalNetworks.ipynb` and `RNN_Captioning_pytorch.ipynb` checks
rubric · Self-grade with the rubric below

Rubric

Criterion	Pass	Strong
Convolution	Matches `F.conv2d` in `float64` for the tested padding, stride, dilation and group settings.	Gradients also match, and a property test samples random valid settings.
Derivation	Hand-derived shapes, parameters and multiply-adds match the code.	The gradient-based receptive field matches the derivation, including a dilated layer.
Gated cell	Outputs match the PyTorch cell with loaded weights.	Gradients through the 20-step unroll match too, and you show that a D2L-convention GRU does not match PyTorch even with the same weights, because the reset gate acts before rather than after the hidden-state matmul.
Leak tests	Padding and future-input interventions leave valid outputs unchanged within a stated tolerance.	The bidirectional counter-case fails the future-input test, and the padding test covers an all-padding row under a stated convention.
Gradients through time	Gradient norms by time step are plotted for both cells.	You tie the decay to measured gate values or to norms of the recurrent Jacobian.

Feeds into: Capstone artifact 2: this CNN or recurrent model is one option for the reproducible training run, and the leak tests can support a casebook postmortem.