Logistic Regression & Probabilities

Turn a linear score into a class probability and understand how logistic regression separates two classes.

By the end of this lesson

You will be able to

Explain the sigmoid function and log-odds.
Distinguish a probability estimate from a thresholded class prediction.
Interpret coefficients and regularization.
Identify when a linear decision boundary may be too restrictive.

From a weighted score to a probability

Logistic regression first computes a linear score z = wᵀx + b, then maps it through the sigmoid function σ(z) = 1 / (1 + e⁻ᶻ). The result lies between zero and one and is interpreted as the model's estimated probability for the positive class. The logit, log(p / (1-p)), is linear in the features. It is still a linear model in its score; the sigmoid provides the probability mapping.

from sklearn.linear_model import LogisticRegression

classifier = Pipeline([
    ("prepare", preprocess),
    ("model", LogisticRegression(max_iter=1000)),
])
classifier.fit(X_train, y_train)
probabilities = classifier.predict_proba(X_valid)[:, 1]

A probability is not yet a decision

A predicted probability of 0.35 is not automatically a negative decision. A threshold converts probabilities into labels, and 0.5 is only a common default. A lower threshold may increase recall while reducing precision; a higher threshold often does the reverse. Coefficients describe conditional associations inside the model and depend on feature scaling, encoding, regularization and correlated inputs. They are not causal effects.

Apply what you learned

Exercise

Inspect logistic probabilities

Fit logistic regression to a binary dataset and inspect probabilities, coefficients and thresholded predictions.

Use a preprocessing Pipeline for numeric and categorical fields.
Fit logistic regression on training data only.
Inspect validation probabilities and confirm they lie between zero and one.
Compare predicted labels at thresholds 0.5 and 0.3.
Explain why the threshold changes decisions without changing fitted coefficients.

Done when

The model returns probability estimates and the learner can distinguish the fitted probability function from a later decision threshold.

Free references

Study resources

2 links

01
Logistic Regression Video
Watch the individual logistic regression video. Focus on the sigmoid mapping and the distinction between score and probability.

Done
02
Linear Models Article
Read Logistic Regression, including regularization and probability-related notes.

Done

Check your understanding

Self-test

2 questions

Question 1

What does the sigmoid function do in logistic regression?
Question 2

Does changing the classification threshold retrain logistic regression?

Completion check

Move on when

You can fit a logistic regression pipeline, inspect probabilities and demonstrate how a threshold changes labels.

Mark passed and continue Back to module