Matrices & Linear Transformations

Treat a matrix as a structured collection of numbers and as a transformation that maps input vectors to output vectors.

By the end of this lesson

You will be able to

Explain rows and columns in a feature matrix.
Determine whether two matrix shapes can be multiplied.
Explain matrix-vector multiplication as repeated weighted sums.
Interpret a matrix as a linear transformation.

Rows are observations, columns are features

A matrix is a rectangular array. In a typical tabular ML dataset, X has shape (n, p): n observations in rows and p features in columns. A weight vector w has p entries. Multiplying X @ w produces one score per row, so the result has shape (n,). Always check dimensions before multiplying.

X = np.array([[0.2, 0.4],
              [0.7, 0.1],
              [0.3, 0.8]])
w = np.array([2.0, -1.0])

print(X.shape)       # (3, 2)
print((X @ w).shape)  # (3,)

A matrix can transform a vector

A matrix defines a mapping from one vector space to another. Its columns show where the original basis directions are sent. Matrix multiplication composes transformations: applying one matrix and then another can be represented by multiplying the matrices in the corresponding order. For ML, the practical takeaway is that matrix multiplication performs many related linear calculations together.

A = np.array([[2.0, 0.0],
              [0.0, 0.5]])
x = np.array([3.0, 4.0])

print(A @ x)  # [6. 2.]

Apply what you learned

Exercise

Predict a batch of observations

Use matrix multiplication to produce one weighted score for each row in a small feature matrix.

Create a matrix X with four rows and three columns.
Create a weight vector w with three values.
Check and print both shapes before multiplying.
Calculate scores = X @ w and confirm that it contains four values.
Change the number of columns in X without changing w; observe and explain the shape error.

Done when

The multiplication succeeds when the inner dimensions match and produces one score per observation. Your explanation names the input and output shapes.

Free references

Study resources

3 links

01
Linear transformations and matrices | Chapter 3 Video
Watch the single chapter to see matrices map basis vectors and transform the plane.

Done
02
Matrices as transformations Article
Read the article’s matrix-column interpretation and complete its first transformation example.

Done
03
Matrix multiplication as composition | Chapter 4 Video
Watch this one chapter for why matrix multiplication composes transformations and why its order matters.

Done

Check your understanding

Self-test

2 questions

Question 1

If X has shape (100, 4) and w has shape (4,), what is the shape of X @ w?
Question 2

What do the columns of a matrix represent in a dataset feature matrix?

Completion check

Move on when

You can reason about matrix shapes and use X @ w to compute one weighted score per observation.

Mark passed and continue Back to module