Support Vector Machines

Understand maximum-margin classification and how kernels allow support vector machines to model nonlinear boundaries.

By the end of this lesson

You will be able to

Define a separating hyperplane and margin.
Explain which observations become support vectors.
Describe the soft-margin penalty C.
Explain the role of a kernel and why scaling matters.

The margin is the central idea

A support vector machine seeks a decision boundary with a large margin between classes. The closest training observations are support vectors; they determine the boundary. Real data is rarely perfectly separable, so a soft-margin formulation allows some violations and balances margin width against classification errors. In scikit-learn, C controls this trade-off: larger values penalize training errors more strongly and can lead to a less regularized fit.

from sklearn.svm import SVC

svm = Pipeline([
    ("prepare", preprocess),
    ("model", SVC(kernel="rbf", C=1.0, gamma="scale")),
])
svm.fit(X_train, y_train)

Kernels compare examples in a transformed space

A linear kernel uses the dot product directly. Polynomial or radial-basis-function (RBF) kernels allow nonlinear decision boundaries without explicitly constructing all transformed features. For RBF, gamma controls how local each training example's influence is: too small can underfit; too large can fit noise. SVMs are sensitive to feature scaling, and probability calibration is a separate concern when probabilities are needed. Large datasets can make kernel SVM training expensive.

Apply what you learned

Exercise

Compare linear and RBF SVMs

Fit linear and RBF support vector classifiers through scaled pipelines and compare their validation behavior.

Select a modest-size classification dataset for the experiment.
Build numeric scaling inside each pipeline.
Fit a linear SVM and an RBF SVM on the same training rows.
Vary C and, for RBF, gamma using validation or cross-validation.
Explain accuracy, training-time and interpretability trade-offs.

Done when

Each SVM receives scaled training features through a pipeline, parameters are selected without test data, and the comparison explains margin and kernel behavior.

Free references

Study resources

2 links

01
Support Vector Machines Part 1: Main Ideas Video
Watch this individual first SVM video. Focus on maximum margin, soft margins, support vectors and kernel intuition.

Done
02
Support Vector Machines Article
Read classification and kernel sections, including practical notes on scaling and parameter choice.

Done

Check your understanding

Self-test

2 questions

Question 1

What are support vectors?
Question 2

What does C control in a soft-margin SVM?

Completion check

Move on when

You can explain the margin and kernel idea and fit a scaled SVM without using test data to tune parameters.

Mark passed and continue Back to module