Confusion Matrix

Confusion Matrix in Machine Learning

A confusion matrix is a table for understanding the mistakes made by a classification model. Instead of only asking "How many predictions were correct?", it asks a more useful question: "Which classes were confused with which other classes?"

Confusion matrices are useful in problems such as spam filtering, fraud screening, defect detection, credit-risk classification, security alerts, and medical research because different kinds of mistakes can have very different consequences.

What Is a Confusion Matrix?

A confusion matrix compares actual labels with predicted labels. In a binary classification problem, the four cells are True Positive, False Positive, False Negative, and True Negative.

Binary Classification Confusion Matrix

Suppose Positive is the class we especially want to detect, such as "Fraud" or "Spam". A common teaching layout is:

One Consistent Example for the Whole Lesson

Imagine 100 cases. There are 40 actual positives and 60 actual negatives. The model produces:

Understanding Each Component

Metrics Derived from the Same Matrix

We will use the same values throughout: TP = 35, TN = 50, FP = 10, FN = 5.

1. Accuracy — How many predictions were correct overall?

2. Precision — When the model says Positive, how often is it right?

3. Recall (Sensitivity) — How many real positives did we find?

4. Specificity — How many real negatives did we identify?

5. F1 Score — One number combining Precision and Recall

Multiclass Confusion Matrix

Confusion matrices are not limited to two classes. If a classifier predicts Cat, Dog, or Horse, the matrix can have three rows and three columns.

The diagonal cells are correct predictions. Off-diagonal cells tell us which classes are being confused. For multiclass precision/recall/F1, each class can be evaluated against the others using a one-vs-rest view and then summarized using averages.

Counts vs Normalized Confusion Matrices

Raw counts answer questions such as "How many cases were missed?" A normalized matrix can answer questions such as "What fraction of each true class was predicted as each class?"

Python Implementation

Scikit-learn can calculate the matrix, unpack TN/FP/FN/TP for a binary problem, create a display, and generate precision/recall/F1 summaries.

import numpy as np
import matplotlib.pyplot as plt
from sklearn.metrics import (
    confusion_matrix,
    classification_report,
    ConfusionMatrixDisplay,
)

# 1 = Dog (positive class), 0 = Not Dog (negative class)
y_true = np.array([1, 1, 1, 0, 1, 0, 1, 1, 0, 0])
y_pred = np.array([1, 0, 1, 0, 1, 1, 1, 1, 0, 0])

# Rows = true labels, columns = predicted labels
cm = confusion_matrix(y_true, y_pred, labels=[0, 1])
tn, fp, fn, tp = cm.ravel()

print("Confusion matrix:")
print(cm)
print(f"TN={tn}, FP={fp}, FN={fn}, TP={tp}")

print("\nClassification report:")
print(
    classification_report(
        y_true,
        y_pred,
        labels=[0, 1],
        target_names=["Not Dog", "Dog"],
        digits=2,
    )
)

# Current scikit-learn display API
ConfusionMatrixDisplay.from_predictions(
    y_true,
    y_pred,
    labels=[0, 1],
    display_labels=["Not Dog", "Dog"],
    cmap="Blues",
    values_format="d",
)
plt.title("Confusion Matrix")
plt.show()

Confusion matrix:
[[3 1]
 [1 5]]
TN=3, FP=1, FN=1, TP=5

Classification report:
              precision    recall  f1-score   support

     Not Dog       0.75      0.75      0.75         4
         Dog       0.83      0.83      0.83         6

    accuracy                           0.80        10
   macro avg       0.79      0.79      0.79        10
weighted avg       0.80      0.80      0.80        10

Common Mistakes