Unit 4.8 · about 2 h
159 / 324t-SNE and UMAP as guarded exploratory views
Embeddings are maps for the eyes, not measurements. Unlabeled shapes become false findings unless every knob is recorded and every shape is re-tested.
- explain
You can now
- Explain what t-SNE preserves in local neighborhoods and what it distorts in global distances, densities and cluster sizes.
- Run t-SNE and UMAP with fixed seeds, scaled input and recorded
perplexity,n_neighborsandmin_dist. - Refuse semantic claims from embedding shapes without stability and downstream checks.
Covers
- t-SNE and UMAP for guarded exploratory visualization
Study, in this order
4 steps
- t-SNE runs and method comparisonHands-On Machine Learning with Scikit-Learn and PyTorch - dimensionality reduction and unsupervised learning
Book Chapter 7: LLE and Other Dimensionality Reduction Techniques, which introduces t-SNE (printed pp. 239-242), and Exercise 10 (p. 243). Notebook 07 cells: LLE section with the Swiss-roll
TSNEcomparison usinginitrandomandlearning_rateauto; Exercise 10: t-SNE on the first 5,000 MNIST digits with digit and class-image scatterplots, then the PCA, LLE and MDS comparisonDepth · Keep the exercise small: 5,000 points as written, not the full 70,000. The notebook warns that its MDS cells take 10 to 30 minutes; skip them or run on a subsample. Skip the Kernel PCA extra material.
- t-SNE behavior and practical tipsscikit-learn - decomposition, clustering and outlier detection
2.2.9 t-SNE: local-structure focus, KL non-convexity with seed dependence, global-structure warning, PCA
initmitigation, Barnes-Hut limits,perplexityand learning-rate guidance; 2.2.10 practical tips: scale features first and reconstruction error for dimension choice - UMAP usage and parametersUMAP - basic usage and parameters
Penguin data:
StandardScalerbeforefit_transform; Digits data:fitwithrandom_state=42, thentransformon the same data, which equalsembedding_Depth · Stay in 2D exploratory views. Skip the Bokeh hover plot.
- UMAP parametersUMAP - basic usage and parameters
n_neighbors: the local versus global tradeoff;min_dist: how tightly points packDepth · Skip the
n_componentsand metric sections.
Focus on
- t-SNE minimizes KL between neighborhood probabilities; the objective is non-convex, so seeds differ. Record the seed and try more than one.
- Perplexity sets the effective neighbor count;
n_neighborstrades local detail for global shape;min_distpacks or spreads the plot. None of them is a cluster count. - sklearn
TSNEhas notransformfor new points;UMAPhastransform. Refitting t-SNE on each new batch rebuilds the map. - Sizes, gaps and densities on the plot are layout artifacts until stability runs and labels say otherwise.
Reproduce closed-book
perplexity or n_neighbors with min_dist, run two seeds, and state two invalid readings: cluster size as population and gap width as distance.Traps
- Clustering the 2D embedding and reporting the groups as found structure. The map distorts distances; validate in the original space.
- Reading big gaps as big separations.
min_distandperplexityset the spread; change them and gaps move. - Fitting t-SNE once and shipping the picture. One seed is one local minimum; rerun and keep only stable shapes.
Practice
Hands-On Machine Learning Ch7 Exercise 10: t-SNE comparison
Exercise solutions, Exercise 10: t-SNE on the first 5,000 MNIST images with digit scatterplots, then the PCA, LLE and MDS comparison
Fix random_state, record perplexity and learning_rate, rerun with a second seed, and write which shapes persist before any interpretation. Skip the slow MDS cells or subsample first.
Self-test
-
Two t-SNE seeds show different island counts on the same data. Which map is right?
Neither yet. Rerun several seeds, keep shapes that persist, and check them with labels or clustering in the original space. -
Why scale before t-SNE or UMAP?
Both use neighbor distances. An unscaled large-unit feature decides every neighborhood;StandardScalerputs features on shared footing. -
UMAPtransformembeds new digits but sklearnTSNEcannot. Why?UMAPkeeps the fitted model and itstransformplaces new points into the existing embedding. scikit-learn'sTSNEhasfit_transformbut notransform, so new points need a refit that rebuilds the whole map. -
A UMAP plot shows one class as three islands. Claim: three subtypes. Allowed?
As a hypothesis only. Islands can come fromn_neighbors,min_distand seed choices; test with stability runs and independent evidence.
Completion criterion
Stop when
Two seeded runs agree on the shapes you describe, every knob is recorded, and you can name two invalid readings of the plot.
Mark passed, then open 5.1 Only mark this when you can do it without notes.Leads to
- 5.1 Holdout, cross-validation and resampling by the right unit Move from exploratory views to evaluation, starting with the sampling unit behind every estimate.
- 4.6 DBSCAN, agglomerative clustering and cluster validation Check density labels against the same guarded view.
Comes after
- 4.6 DBSCAN, agglomerative clustering and cluster validationOverlay cluster labels on an honest embedding to inspect shape.
Resources for this unit
- Hands-On Machine Learning with Scikit-Learn and PyTorch - dimensionality reduction and unsupervised learning Read Chapters 7 and 8 and run notebooks 07 and 08 for PCA, k-means, DBSCAN, agglomerative clustering, Gaussian mixtures and anomaly detection. Stop before extended semi-supervised and face-data projects.
- scikit-learn - decomposition, clustering and outlier detection Check decomposition orientation and solver behavior, cluster API semantics, and the distinction between novelty and outlier detection after the independent tests pass.
- UMAP - basic usage and parameters Work the Penguin scaling and
fit_transformsection and the Digitsfitandtransformsection, then read the parameters page for then_neighborslocal-global tradeoff andmin_distpacking. Stay in 2D exploratory views with fixed seeds.
Also in this section
- ISLP - Unsupervised Learning Use Chapter 12 and its Python lab for the PCA derivations (12.2, including the low-rank approximation view and proportion of variance explained) and k-means (12.4.1), plus stability and interpretation limits, beside the notebook route.
- Deep-ML - unsupervised problems Use 17 K-Means Clustering, 19 PCA, 341 Gaussian Mixture Model with EM, 259 DBSCAN and 367 Isolation Forest as unit checks after the local implementation passes. Their tests were read in review; each card states what they pin, including the 367 path-length convention.
- scikit-learn - preprocessing data Read 8.3.1 standardization with the sparse
with_mean=Falserule, 8.3.1.3 scaling data with outliers, and 8.3.4 categorical encoding withhandle_unknown, infrequent andsparse_outputbehavior. Apply after the HOML notebook build works. - scikit-learn - pipelines and composite estimators Read 8.1.1
Pipelinechaining with the no-leakage rule and transformer caching, and 8.1.4ColumnTransformerfor heterogeneous columns withremainderand fitted-state behavior. Use it to assemble and serialize the feature-selection pipeline. - scikit-learn - imputation of missing values Read 8.4.2 univariate median and
most_frequentstrategies, 8.4.6 Marking imputed values withMissingIndicatororadd_indicator=Truefor missingness flags, and 8.4.7 estimators that acceptNaNdirectly. Decide impute-only versus impute-plus-indicator per column.