Semi-Supervised Learning

Semi-Supervised Learning

Suppose you have 10,000 photos, but a human has labelled only 300 of them. Throwing away the other 9,700 photos feels wasteful, but paying people to label everything may be expensive. Semi-supervised learningasks whether the unlabelled examples can help us learn while we still use the labels we trust.

Supervised, Unsupervised, and Semi-Supervised

A Familiar Example: Classifying Emails

Imagine that you collected thousands of emails, but only a small number were manually labelled as Spam or Not Spam.

A semi-supervised method may use the labelled emails to anchor the classes and the unlabelled emails to learn additional structure. The exact mechanism depends on the algorithm.

Two Core Ways to Use Unlabelled Data

Self-Training: A Tiny Step-by-Step Example

Suppose a classifier predicts three unlabelled examples with these maximum class probabilities:

After adding A and B, the classifier can be fitted again. This process may continue for several rounds.

When Can Unlabelled Data Help?

Different semi-supervised algorithms rely on different assumptions. Three common intuitions are useful:

These are assumptions and modeling intuitions—not universal laws. For example, two classes can overlap heavily, or nearby points can have different labels. In such cases, unlabelled structure may not align with the prediction target.

Python Example: Label Spreading with Scikit-Learn

In Scikit-Learn's semi-supervised graph algorithms, unlabelled targets are represented by -1. The example below keeps only 4 labelled training examples per Iris class. The remaining training examples are supplied as unlabelled observations, while the test set stays completely separate.

import numpy as np
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.semi_supervised import LabelSpreading
from sklearn.metrics import accuracy_score

X, y = load_iris(return_X_y=True)

X_train, X_test, y_train, y_test = train_test_split(
    X,
    y,
    test_size=0.30,
    random_state=42,
    stratify=y
)

rng = np.random.default_rng(42)
y_semi = np.full_like(y_train, -1)

# Keep only 4 known labels from each class.
for class_id in np.unique(y_train):
    class_rows = np.where(y_train == class_id)[0]
    labeled_rows = rng.choice(class_rows, size=4, replace=False)
    y_semi[labeled_rows] = y_train[labeled_rows]

labeled_count = np.sum(y_semi != -1)
unlabeled_count = np.sum(y_semi == -1)

model = make_pipeline(
    StandardScaler(),
    LabelSpreading(
        kernel="knn",
        n_neighbors=7,
        alpha=0.2,
        max_iter=100
    )
)

model.fit(X_train, y_semi)
predictions = model.predict(X_test)

print("Labeled training samples:", labeled_count)
print("Unlabeled training samples:", unlabeled_count)
print("Held-out test samples:", len(y_test))
print("Test accuracy:", round(accuracy_score(y_test, predictions), 3))

Labeled training samples: 12
Unlabeled training samples: 93
Held-out test samples: 45
Test accuracy: 0.889

Label Propagation vs Label Spreading

With LabelSpreading, alpha controls how much an example adopts information from its neighbors versus retaining its initial label information. It is a model parameter to tune—not a universal constant.

Going Deeper: Other Semi-Supervised Approaches

Semi-Supervised vs Self-Supervised Learning

They are related because both can benefit from unlabelled data, but they are not synonyms. A self-supervised representation may later be fine-tuned with labelled data, while classical semi-supervised methods directly mix labelled and unlabelled examples during learning.

Real-World Situations Where It Can Be Useful

When to Consider It—and When to Be Careful

Evaluation: Keep the Test Labels Truly Hidden

It is legitimate for a semi-supervised algorithm to use unlabelled training features; that is the point. But final test labels must not leak into training, pseudo-label selection, threshold tuning, graph tuning, or model selection.

Scalability Note

Graph-based methods can become expensive as the number of examples grows. In Scikit-Learn's label-propagation family, the RBF kernel builds a fully connected dense similarity graph, whereas the KNN kernel can use a much sparser neighborhood graph. So “semi-supervised learning scales to millions of examples” is not a safe general claim—the algorithm and representation matter.

Common Mistakes

Quick Recap

Final Summary