Hierarchical Clustering

Hierarchical Clustering

Hierarchical Clustering is an unsupervised learning method that builds a tree of nested groups from similarities or distances between data points.

Unlike K-Means, which directly produces a flat partition for a chosen number of clusters, hierarchical clustering can build a nested sequence of merges or splits.

The hierarchy can help us inspect nested grouping structure, compare merge levels, and choose a useful clustering at different resolutions.

Hierarchical clustering is used in areas such as bioinformatics, gene-expression analysis, document grouping, customer analysis, image analysis, and exploratory scientific research. The resulting dendrogram shows similarity under the chosen method; it should not automatically be interpreted as a true evolutionary or causal tree.

Why Use Hierarchical Clustering?

K-Means asks us to choose K before fitting. Hierarchical clustering offers a different view: build a hierarchy first, inspect it, and then choose a useful cut when appropriate.

This is useful when we want to:

• Automatically reveal grouping structure
• Show relationships between clusters
• Allow flexible cluster selection later
• Preserve hierarchy information

It is therefore best viewed as an alternative clustering strategy, not as a method that historically appeared simply because K-Means required K.

Real-Life Analogy of Hierarchical Clustering

Imagine organizing books inside a giant library. Initially, all books are separate. Then gradually, books about similar topics are grouped together. For example: Science books together, History books together, Mathematics books together.

Notice something important: Groups form at multiple levels. This creates a hierarchy. Hierarchical Clustering works exactly like this. It forms Small clusters, Medium clusters, and Large clusters, creating a complete hierarchy of relationships.

Types of Hierarchical Clustering

Which type will we focus on? Most standard machine-learning libraries, including Scikit-learn's AgglomerativeClustering, focus on the bottom-up agglomerative approach. Divisive methods also exist, but their exact splitting strategy depends on the algorithm used.

Understanding Agglomerative Clustering Deeply

Agglomerative clustering starts with Every point as an independent cluster.

Then repeatedly: Find nearest clusters ➔ Merge them ➔ Recalculate distances ➔ Repeat until all points merge.

What Determines "Closest Clusters"? (Linkage Methods)

This is an important concept in Hierarchical Clustering. The algorithm needs a method to measure distance between clusters. This is called Linkage Criteria.

Distance Metrics & Mathematical Calculation

The algorithm needs a way to measure dissimilarity between points. A common choice is Euclidean distance. For A(2,3) and B(6,7):

Dendrogram — The Heart of Hierarchical Clustering

The most important visualization in Hierarchical Clustering is the Dendrogram. A dendrogram is a tree-like diagram showing Cluster merges, Similarity levels, and Hierarchical relationships.

Scalability: standard hierarchical clustering can become expensive as the number of samples grows because the hierarchy is built from many pairwise relationships. Memory and runtime are often roughly quadratic-scale concerns, although exact cost depends on the linkage and implementation. For very large datasets, K-Means, MiniBatchKMeans, sampling, or structured connectivity constraints may be more practical.

Implementing Hierarchical Clustering in Python

# Step 1 — Import Libraries
import numpy as np
import matplotlib.pyplot as plt
from scipy.cluster.hierarchy import dendrogram, linkage
from sklearn.datasets import make_blobs
from sklearn.cluster import AgglomerativeClustering
from sklearn.metrics import silhouette_score

# Step 2 — Create a reproducible teaching dataset
X, _ = make_blobs(
    n_samples=120,
    centers=3,
    cluster_std=0.70,
    random_state=42
)

# ==========================================
# APPROACH 1: Build and plot the hierarchy
# ==========================================
linked = linkage(X, method='ward')

plt.figure(figsize=(10, 6))
dendrogram(linked)
plt.title("Dendrogram using Ward Linkage")
plt.ylabel("Linkage distance")
plt.show()

# ==========================================
# APPROACH 2: Cut the hierarchy into 3 groups
# ==========================================
model = AgglomerativeClustering(
    n_clusters=3,
    linkage='ward'
)
labels = model.fit_predict(X)

print("Cluster sizes:", np.bincount(labels))
print("Silhouette score:", round(silhouette_score(X, labels), 3))

plt.scatter(X[:, 0], X[:, 1], c=labels, cmap='viridis')
plt.title("Agglomerative Clustering Output")
plt.show()

Real-Life Scenario — Gene Analysis

Suppose scientists analyze gene-expression measurements across samples. Genes with similar expression patterns may be grouped together for further investigation of possible shared biological behavior.

Hierarchical clustering can reveal nested similarity patterns in such measurements. The dendrogram is an exploratory clustering result; biological or evolutionary conclusions require separate domain evidence and analysis.

Hierarchical Clustering vs K-Means

Applications of Hierarchical Clustering

Common Mistakes

Quick FAQs

Final Summary

Hierarchical Clustering is an interpretable clustering algorithm in Machine Learning. It works by Measuring similarities, Merging or splitting clusters, Building hierarchical structures, and Visualizing relationships using dendrograms.

A full hierarchy can be built without choosing the final number of clusters first, but obtaining a flat clustering still requires a cut rule such as a cluster count or distance threshold. The method is especially useful when the nested grouping structure itself is worth inspecting.