Unit 1.2 · about 4 h
52 / 324Linear maps, span, independence and change of basis
Coordinates describe a vector; they are not the vector itself. Keeping that distinction prevents wrong transposes and explains why a better basis can simplify the same map.
- derive
You can now
- Construct a matrix representation from the images of an ordered basis and derive the matrix for a composition.
- Distinguish spanning, independence and being a basis, including uniqueness of coordinates.
- Derive the change-of-coordinates formula with separate input and output bases.
Covers
- Linear maps, matrix representations and composition
- Span, independence, basis and change of coordinates
Study, in this order
2 steps
- Independence and matrix representationsMIT 18.06 - Linear Algebra
Video Lecture 9, Independence, basis, and dimension; Lecture 30, Linear transformations and their matrices; Lecture 31, Change of basis; image compression
Depth · In Lecture 31, stop after change of basis; leave image compression for SVD. These are video numbers, not calendar session numbers.
- Basis change as a compositionMathematics for Machine Learning
Chapter 2, sections 2.5 to 2.6 (pp. 40 to 48) for definitions, then 2.7.1 to 2.7.2 (pp. 50 to 58), especially Theorem 2.20 and its proof on pp. 54 to 55
Depth · Read the basis-change proof line by line. Leave kernel/image in 2.7.3 for the next unit.
Focus on
- Spanning gives existence of coordinates; independence gives uniqueness. Both are needed for a basis.
- Column \(j\) is the output coordinate vector of the image of input basis vector \(j\). Composition acts right to left on column vectors.
- If \(x_{\text{old}}=Sx_{\text{new}}\) and \(y_{\text{old}}=Ty_{\text{new}}\), then the new representation is \(T^{-1}AS\). Only an endomorphism with the same basis change at both ends gives \(S^{-1}AS\).
Reproduce closed-book
dtype=np.float64, without random draws; compute C=np.linalg.solve(T,A@S) and np.linalg.norm(T@(C@z)-A@(S@z),2). The reference difference is 0.0; require it below 1e-12 rather than asserting floating-point equality.Traps
- \(S\) contains the new basis vectors in old coordinates, not the reverse conversion. The inverse belongs on the output conversion.
- A spanning list may contain redundant vectors. Its coefficients need not be unique.
- An affine map with a nonzero offset is not a linear map even when implemented with a matrix multiplication.
Practice
MML Exercise 2.20: bases on both sides of a linear map
Chapter 2, Exercise 2.20(a) to (f), printed pp. 68 to 69
Draw the two bases and solve the coordinate conversions before multiplying matrices. Part (f) checks the same output by two coordinate routes.
Self-test
-
Why does a basis give unique coordinates?
Spanning gives at least one expansion. Subtracting two expansions gives a zero linear combination. Independence forces every difference of coefficients to be zero. The two basis assumptions are used in different steps. -
For \(f(x)=Ax\) and \(g(y)=By\), which matrix represents \(g\) composed with \(f\)?
\(BA\): \(g(f(x))=B(Ax)\). If \(A\) is \(m\) by \(n\), \(B\) must have \(m\) columns; with \(B\) of shape \(p\) by \(m\) the result is \(p\) by \(n\). -
Why is using \(S^\top\) instead of \(S^{-1}\) wrong in general?
A basis matrix need only be invertible, not orthogonal. For \(S=\left[\begin{smallmatrix}1&1\\0&1\end{smallmatrix}\right]\), \(S^\top=\left[\begin{smallmatrix}1&0\\1&1\end{smallmatrix}\right]\), while \(S^{-1}=\left[\begin{smallmatrix}1&-1\\0&1\end{smallmatrix}\right]\). Only orthonormal basis columns imply \(S^\top=S^{-1}\).
Completion criterion
Stop when
You solve Exercise 2.20 through its two output routes and can derive the order \(T^{-1}AS\) without memorizing it.
Mark passed, then open 1.3 Only mark this when you can do it without notes.Leads to
- 1.3 The four subspaces, rank and solution sets Describe what the map reaches and what it loses.
- 2.2 Conditioning, basis expansions, ridge, lasso and elastic net Apply the distinction between the map and its coordinate representation.
Comes after
- 1.1 Proof habits and shape notationUse definitions to prove linearity and independence.
Resources for this unit
- MIT 18.06 - Linear Algebra Follow lectures 1-10, 14-17, 21-22, 25, 27, 29-31 and 33 with matching problems. This is the ordered first pass.
- Mathematics for Machine Learning Use Chapters 2-4 and §10.2 to check notation or re-derive a result. Do not treat it as a second course.
Also in this section
- MIT 6.042J - Proof strand Study textbook Chapters 1 and 3, §§4.1-4.4 and §§5.1-5.4 with matching Unit 1 problems. Leave the rest of the course.
- MIT 18.065 - Applied matrix bridge Use lectures 3-10 after 18.06 for SVD, PCA, low-rank approximation, matrix norms and solver difficulty.
- Introduction to Applied Linear Algebra Sample Chapters 1-13 and the extra exercises for computational practice in fitting, least squares and pseudoinverses.
- Essence of Linear Algebra Open only when a geometric picture is missing; it does not replace derivation or problem solving.
- NumPy broadcasting and matmul shape rules Read General broadcasting rules and Broadcastable arrays, then
matmulNotes for stacked matrices and one-dimensional promotion. These close the pool's array-semantics gap. - Fundamentals of Numerical Computation: conditioning and operation counts Python edition, section 2.8 through Residual and backward error for the fixed-matrix perturbation bound; section 2.5.3 for triangular-solve and LU flop counts. Use Exercise 2.8.7 for a condition-number proof, not the unseeded Hilbert experiment.