Unit 4.1 · about 4 h
152 / 324Numeric transforms and categorical encoding
Every estimator downstream sees the transformed geometry, not the raw table. A scaler fit on the wrong rows or an encoder that errors on new categories breaks serving before any model runs.
- implement
You can now
- Build a leakage-safe numeric and categorical preprocessing path with fitted state, unknown-category behavior and schema tests.
- Choose scaling, clipping and imputation for skewed, heavy-tailed and missing features and state what each choice hides.
- Keep sparse categorical output sparse through the pipeline and densify only where the estimator requires it.
Covers
- Numeric transforms, scaling, clipping and missingness indicators
- Categorical encoding, unknown categories and sparse features
Study, in this order
3 steps
- End-to-end preprocessing: cleaning, encoding and scalingHands-On Machine Learning with Scikit-Learn and PyTorch - dimensionality reduction and unsupervised learning
Book Chapter 2: Clean the Data, Handling Text and Categorical Attributes, and Feature Scaling and Transformation (printed pp. 70-81). Notebook 02 cells: Prepare the Data for Machine Learning Algorithms: Data Cleaning with
SimpleImputermedianandmost_frequent; Handling Text and Categorical Attributes withOneHotEncoderand thesparse_outputnote; Feature Scaling withMinMaxScaler,StandardScaler, log transform and percentile flattening; outlier dropping withIsolationForestDepth · Stop before Custom Transformers. Ratio, RBF-similarity and cluster-similarity features belong to the next unit.
- Scaling, encoding and sparse behaviorscikit-learn - preprocessing data
8.3.1 standardization with the
with_mean=Falsesparse rule andMaxAbsScaler; 8.3.1.3RobustScalerfor outlier-heavy features; 8.3.4OneHotEncoderhandle_unknownignoreversuserror, infrequent handling andsparse_output - Missing-value strategies and indicatorsscikit-learn - imputation of missing values
8.4.2 univariate
medianandmost_frequentstrategies; 8.4.6 Marking imputed values:MissingIndicator, oradd_indicator=Trueon the imputer, to keep a missingness flag beside the imputed value; 8.4.7 which estimators acceptNaNdirectly
Focus on
- Fit every statistic on training rows only, then transform the rest. Refitting on evaluation rows leaks their distribution into the features.
OneHotEncoderwithhandle_unknown=ignorewrites a new category as all zeros; withhandle_unknown=errorit raises. Pick one before serving.StandardScalercenters, so it densifies sparse input unlesswith_mean=False.MaxAbsScalerpreserves sparsity.- A missingness indicator keeps the fact of missingness after imputation fills the value. Dropping the flag assumes missingness carries no signal.
Reproduce closed-book
handle_unknown=ignore. Transform two new rows, one with an unseen category and one with a missing number, and write the exact output rows including the all-zeros encoding.Traps
- Fitting the imputer or scaler on the full data before splitting. Validation scores then reuse evaluation statistics.
- Setting
sparse_output=Falseon a high-cardinality encoder inside a pipeline. Memory grows with rows times categories; keep sparse until an estimator needs dense. - Reading the percentile-flattened income as deleting outliers. It remaps tails to fixed bins; the
IsolationForestdrop in the same notebook is a separate commented-out choice.
Practice
Hands-On Machine Learning Ch2 Exercise 6: `StandardScalerClone`
Exercise solutions, Exercise 6: re-implement StandardScalerClone with inverse_transform, feature_names_in_ and get_feature_names_out, then check_estimator
Implement in your own repository. check_estimator must pass and inverse_transform must round-trip within floating-point tolerance.
Self-test
-
A scaler fit on all 10,000 rows gives mean 5.2; fit on the 8,000 training rows gives 5.0. Which one goes into the pipeline, and why?
The training-only 5.0. The full-data mean carries evaluation rows into every transformed training value. -
OneHotEncoder(handle_unknown=ignore)trained on[a, b]sees[c]. What row does it emit, and what doesinverse_transformreturn for it?It emits[0, 0].inverse_transformmaps all zeros back toNonewhen no category was dropped. Verified with scikit-learn 1.9. -
Why does
StandardScaleron a CSR matrix raise unlesswith_mean=False?Centering subtracts the mean from every entry including stored zeros, which would densify the matrix. Withwith_mean=Falseonly scaling applies and sparsity survives. -
When does an imputed column need a companion indicator column?
When missingness itself predicts the target, for example sensors that fail under load. The indicator keeps that signal after the value is filled.
Completion criterion
Stop when
Your clone passes check_estimator, round-trips five rows through fit_transform and inverse_transform, and your pipeline transforms unseen categories and missing values without refitting.
Leads to
- 4.2 Interactions, feature selection and fit-state serialization Carry the fitted encoders and scalers into pipelines with selection and saved state.
- 1.6 Training-only imputation, scaling, encoding and learned state Apply the training-only fitting rule to imputation, scaling and encoding.
Resources for this unit
- Hands-On Machine Learning with Scikit-Learn and PyTorch - dimensionality reduction and unsupervised learning Read Chapters 7 and 8 and run notebooks 07 and 08 for PCA, k-means, DBSCAN, agglomerative clustering, Gaussian mixtures and anomaly detection. Stop before extended semi-supervised and face-data projects.
- scikit-learn - preprocessing data Read 8.3.1 standardization with the sparse
with_mean=Falserule, 8.3.1.3 scaling data with outliers, and 8.3.4 categorical encoding withhandle_unknown, infrequent andsparse_outputbehavior. Apply after the HOML notebook build works. - scikit-learn - imputation of missing values Read 8.4.2 univariate median and
most_frequentstrategies, 8.4.6 Marking imputed values withMissingIndicatororadd_indicator=Truefor missingness flags, and 8.4.7 estimators that acceptNaNdirectly. Decide impute-only versus impute-plus-indicator per column.
Also in this section
- scikit-learn - decomposition, clustering and outlier detection Check decomposition orientation and solver behavior, cluster API semantics, and the distinction between novelty and outlier detection after the independent tests pass.
- ISLP - Unsupervised Learning Use Chapter 12 and its Python lab for the PCA derivations (12.2, including the low-rank approximation view and proportion of variance explained) and k-means (12.4.1), plus stability and interpretation limits, beside the notebook route.
- Deep-ML - unsupervised problems Use 17 K-Means Clustering, 19 PCA, 341 Gaussian Mixture Model with EM, 259 DBSCAN and 367 Isolation Forest as unit checks after the local implementation passes. Their tests were read in review; each card states what they pin, including the 367 path-length convention.
- scikit-learn - pipelines and composite estimators Read 8.1.1
Pipelinechaining with the no-leakage rule and transformer caching, and 8.1.4ColumnTransformerfor heterogeneous columns withremainderand fitted-state behavior. Use it to assemble and serialize the feature-selection pipeline. - UMAP - basic usage and parameters Work the Penguin scaling and
fit_transformsection and the Digitsfitandtransformsection, then read the parameters page for then_neighborslocal-global tradeoff andmin_distpacking. Stay in 2D exploratory views with fixed seeds.