Chapter 4 — Clustering

K-Means, hierarchical, dendrograms

Prof. Xuhu Wan

Chapter 4 · Introduction to Business Analytics

Clustering

Finding hidden groups in unlabelled data.

Prof. Xuhu Wan

ISOM, HKUST Business School · Wan Academy · 2026 Edition

The Clustering Workflow

Collect → Standardise → Choose K → Run → Interpret

Clustering is unsupervised learning: no target column, no “right answer”. The algorithm discovers structure in the features alone. Business uses include customer segmentation, anomaly detection, document grouping, and exploratory data analysis.

Important

The single biggest mistake is skipping standardisation. If you cluster Age (20–70) and Income ($15K–$140K) without standardising, the income axis dominates the Euclidean distance by a factor of ~1000.

Why Standardise — Visual Proof

K-Means in One Sentence

Repeatedly: (1) assign each point to its nearest centroid, then (2) move each centroid to its cluster’s mean. Stop when centroids stop moving.

Choosing K — The Elbow Plot

The elbow is a heuristic, not a proof. It tells you where adding clusters stops giving meaningful improvement.

Hierarchical Clustering — Bottom Up

Start with each point as its own cluster
Merge the closest pair
Repeat until one big cluster

The full tree (dendrogram) lets you read off any K after the fact by cutting horizontally.

Note

Ward linkage picks the merge that produces the smallest increase in within-cluster variance — the most common choice in practice. It tends to produce compact, balanced clusters.

Worked Example: NBA Players by Style

NBA front offices recover player archetypes — pure scorer, playmaker-big, defensive wing — from per-game stats alone, without coach labels.

Cross-tab shows which true archetypes each cluster recovered. The playmaker-big — uniquely high in rebounds and assists — usually maps cleanly; the two backcourt archetypes blur where a real scout would also hesitate.

Worked Example: Netflix Viewer Segmentation

Netflix runs 1300+ customer segments to decide which thumbnails and shows to recommend to whom.

Cluster 0 = action-heavy · Cluster 1 = K-drama-only · Cluster 2 = family-with-kids.

Working with an AI Copilot

When you paste cluster centroids into a chatbot and ask “what should I name these clusters?”, the AI will invent decorative names — “tech-savvy millennials”, “affluent urbanites” — with no evidence in the data.

Before asking the AI for a label, verify the cluster mean differences are both statistically significant and practically meaningful in business terms.
Ask the AI which features drove its proposed name — if it can’t point to specific feature gaps, the label is decorative, not analytic.
The AI cannot tell you whether K was right. That requires the elbow plot, a silhouette score, and a domain expert’s read.

Mistakes Library: Netflix Prize (2009)

Warning

In 2006 Netflix launched a $1 million prize for any team that could beat their internal recommender by 10%. Three years later, “BellKor’s Pragmatic Chaos” won with a 107-model ensemble blending matrix factorisation, restricted Boltzmann machines, and gradient-boosted residual models. Netflix paid the $1M — and never deployed the winning model.

Two reasons. First, the engineering cost of running 107 stitched-together models in production was prohibitive compared with the marginal accuracy gain. Second, data drift: during the three-year contest Netflix had pivoted from mailed DVDs to streaming, and the model’s gains on the original DVD-rating snapshot evaporated against streaming behaviour.

Lesson: an analysis that wins on a clean snapshot can fail in production. Always check whether your training data still describes the population at the moment the model runs.

Decision Memo — How Many Customer Segments Should Marketing Target?

You are a customer analytics intern at Netflix APAC. The current marketing model uses 8 viewer segments; your clustering analysis suggests 3 is enough. Draft the memo.

To: VP Marketing, Netflix APAC From: , customer analytics intern Subject: Re-target promotions to 3 — not 8 — viewer segments Date: 2026-05-15

Recommendation: Collapse the 8-segment marketing model to 3 high-leverage segments.

Evidence: - K-Means with K=3 captures 68% of variance. - Adding the 4th cluster only reduces WCSS by 4 additional points — elbow at K=3. - The 3 segments map cleanly onto distinct content preferences.

Caveats: - Features were standardised; without it, hours-per-week would dominate. - “Cluster identity” is a marketing label, not a customer attribute.

Next step: A/B test cluster-specific creative on the next promotion vs. the 8-segment baseline.

Chapter Summary

Method	Use when
K-Means	You know K · fast on large N · roughly spherical clusters
Hierarchical (Ward)	Small-to-mid N · don’t know K up front · want a tree
Always	Standardise features first
Choose K	Elbow + business knowledge

Full NBA player-archetype and mall-customer-segmentation case studies in the book — Chapter 4.

This concludes the course. Capstone projects use everything from Chapters 1–4 together.