Unit 5.8 · about 3 h
167 / 324Permutation importance and partial dependence
Importance and dependence plots describe what the fitted model leans on, not what causes the outcome. Correlated features make both lie smoothly.
- measure
You can now
- Compute permutation importance on a held-out set with repeats and report importances with their spread.
- Distrust importance and PDP readings under correlated features and check with clustered or ICE views.
- State PDP and ICE results as model behavior on constructed points, never as causal effects.
Covers
- Permutation importance and correlated-feature limits
- Partial dependence and noncausal interpretation boundaries
Study, in this order
2 steps
- Permutation feature importancescikit-learn - model selection and evaluation route
The diabetes
Ridgeworked example (s50.204,bmi0.176,bp0.088,sex0.056 atn_repeats=30), Section 5.2.2 (impurity/MDI bias toward high-cardinality features) and Section 5.2.3 (correlated features share credit; cluster and keep one per group)Depth · Skip the multicollinear example figures. The breast-cancer warning paragraph carries the point.
- Partial dependence and ICE plotsscikit-learn - model selection and evaluation route
Section 5.1 (PDP marginalizes over complement features; correlated inputs create absurd constructed points), Section 5.1.2 (ICE lines with
kind='both'reveal hidden heterogeneity) and Section 5.1.4 (brute versus recursion can disagree under correlation)Depth · Stop before the multi-class dropdown. One binary or regression example is enough.
Focus on
- Permutation importance is model- and score-relative: shuffle one column, measure the score drop, repeat. Importance is the reference minus the corrupted mean.
- Compute it on held-out data with a decent model. On a bad model low importance means nothing; train versus held-out gaps flag overfit.
- Under correlation, permutation importance splits credit across partners and PDP averages over impossible combinations. Cluster correlated features first and overlay ICE.
- Impurity (MDI) importance favors high-cardinality and numeric features and rewards overfit. Never present it as interchangeable with permutation importance.
Reproduce closed-book
Ridge permutation snippet (n_repeats=30, random_state=0) and recover the top-4 order with s5 near 0.204. Then plot centered ICE lines for bmi: Ridge is linear, so every line is identical. Refit HistGradientBoostingRegressor(random_state=0) on the same split and its centered ICE lines end anywhere from about 36 to 152; describe where they spread.Traps
- Dropping unimportant correlated features one at a time. Partners cover for each other, so remove the cluster or nothing.
- Reading a PDP slope as raising \(X\) raises \(Y\). PDP shows model output on constructed rows with complement values borrowed from other rows; no intervention happened.
- Quoting MDI importances as feature truth. They are computed on training splits during fitting and biased toward fine-grained splitters.
Practice
scikit-learn permutation importance titanic check
Reproduce the RandomForest titanic result: random_cat and random_num sit near zero importance while real features stand out, with spread over repeats
Then repeat the diabetes snippet locally and confirm s5 at 0.204 +/- 0.050 before trusting your own run.
Self-test
-
Permutation importance of \(x_1\) and \(x_2\) (correlation 0.95) is near zero for both, yet dropping both tanks the score. Explain.
The fitted model reads the same signal through both columns, so shuffling one leaves the partner carrying most of it and each drop looks small. Removing both removes the signal. Cluster correlated features and judge the cluster. -
Why must permutation importance run on a held-out set with a decent model?
On training data it measures memorization leverage, and on a bad model the reference score is meaningless. Held-out drops measure reliance that generalizes. -
The temperature PDP falls above 35C but some ICE lines are flat. What do you conclude?
The average hides heterogeneity: heat hurts only a subgroup, an interaction. Report the ICE spread, not just the PD line. -
MDI ranks a 1000-level
idcolumn first while permutation ranks it near zero with large spread. Which do you trust?Permutation on held-out data. MDI is biased to high-cardinality splitters and computed in-sample: theidmemorizes but does not generalize.
Completion criterion
Stop when
Your permutation ranking matches the docs' diabetes order, and you can show one correlated-feature trap and one PDP-heterogeneity trap on your own model.
Mark passed, then open 5.9 Only mark this when you can do it without notes.Leads to
- 5.10 Ablations, negative controls, threats to validity and decision memos Test an important feature by removing it under controls.
- 5.9 Covariate, label and concept shift, drift and label delay Correlated structure is what shift breaks first.
Comes after
- 3.4 XGBoost, LightGBM and CatBoost configurationExplain the configured booster with model-agnostic tools.
- 3.8 Feature importance caveats and class weighting effectsCarry model-agnostic importance into evaluation.
- 5.7 Learning curves, error taxonomies and slicesExplain the worst slice with importance and dependence views.
Resources for this unit
- scikit-learn - model selection and evaluation route Follow Cross-validation, Hyperparameter tuning, Decision threshold tuning, Metrics and scoring, Probability calibration, Validation curves, Permutation importance and Common Pitfalls. Retain raw predictions and independently hand-check central metrics.
Also in this section
- ISLP - resampling and multiple testing Use Chapter 5 and the relevant selection-history parts of Chapter 13 with their Python labs. Connect resampling to uncertainty and multiplicity without repeating the full statistics stage.
- Classical ML Scientific Investigation Use Core requirements, architecture constraints, Milestones A-F and failure injection as the standard for end-to-end evidence. The full project remains a later capstone route.
- D2L - Environment and Distribution Shift Read 4.7, then construct one controlled covariate, label and concept shift. Record what changes, which signal detects it and what response the evidence supports.
- Deep-ML candidate - evaluation problems After the platform admission check, use at most one of 18 K-Fold CV, 260 Expected Calibration Error, 276 ROC, 277 AUC or 840 Stratified K-Fold. Hand-checked local metrics remain the evidence.
- SciPy - Resampling and Monte Carlo methods tutorials Work the notebooks linked from this page: Tutorial 3 (The Bootstrap) with
scipy.stats.bootstrap, includingpaired=True, and Tutorial 2b (Paired-Sample Permutation Tests) withpermutation_testandpermutation_type='samples'. Keep one resampling index stream across compared models. - El-Yaniv and Wiener - On the Foundations of Noise-free Selective Classification Read the Abstract and Section 2 (Preliminary Definitions: Definitions 1 and 2 for coverage and selective risk) for the risk-coverage tradeoff. Stop before the noise-free learning bounds.