🧠 AI with Python – 📉🧩 Visualizing Decision Boundaries for SVM

Support Vector Machines (SVMs) are powerful classification models that work by finding the optimal hyperplane separating different classes.

But SVMs become far easier to understand when their decision boundaries are visualized — especially in 2D.

In this project, we train a simple SVM classifier and plot its decision regions, margins, and support vectors to clearly understand how the model draws boundaries between classes.

Understanding the Problem

Many learners struggle to intuitively understand how SVMs work.

Terms like margin, hyperplane, and support vectors can feel abstract without visual examples.

By restricting the dataset to two numerical features, we can:

• Visualize the classification space
• See how the SVM draws boundaries between classes
• Observe which points become support vectors
• Understand the effect of the kernel function

This visual approach makes SVM concepts much clearer.

1. Load and Prepare the Dataset

We use the classic Iris dataset, but restrict it to:

• two numerical features → 2D plotting
• two classes → clear binary boundary

from sklearn import datasets
import pandas as pd

iris = datasets.load_iris()
X = iris.data[:, :2]  # sepal length + sepal width
y = iris.target

# Keep only two classes for clean visualization
mask = y != 2
X = X[mask]
y = y[mask]

2. Train/Test Split

from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42, stratify=y
)

3. Train the SVM Classifier

We use a linear kernel for the first visualization.

from sklearn.svm import SVC

model = SVC(kernel="linear")
model.fit(X_train, y_train)

4. Plot the Decision Boundary

We generate a grid covering the entire plot area and classify every point to create filled decision regions.

import numpy as np
import matplotlib.pyplot as plt

h = 0.02
x_min, x_max = X[:,0].min() - 1, X[:,0].max() + 1
y_min, y_max = X[:,1].min() - 1, X[:,1].max() + 1

xx, yy = np.meshgrid(
    np.arange(x_min, x_max, h),
    np.arange(y_min, y_max, h)
)

Z = model.predict(np.c_[xx.ravel(), yy.ravel()])
Z = Z.reshape(xx.shape)

plt.figure(figsize=(8,6))
plt.contourf(xx, yy, Z, cmap="coolwarm", alpha=0.4)
plt.scatter(X[:,0], X[:,1], c=y, cmap="coolwarm", s=40, edgecolor="k")

5. Highlight Support Vectors

The points closest to the margin become support vectors — the core of SVM learning.

sv = model.support_vectors_
plt.scatter(
    sv[:,0], sv[:,1],
    s=120,
    facecolors='none',
    edgecolors='k',
    linewidths=1.5,
    label="Support Vectors"
)

plt.title("SVM Decision Boundary (Linear Kernel)")
plt.xlabel("Sepal Length")
plt.ylabel("Sepal Width")
plt.legend()
plt.show()

Key Takeaways

1. SVM decision boundaries are defined by support vectors, not all training points.
2. The margin is the region between the nearest points of each class — SVM maximizes this margin.
3. Changing kernels (linear → RBF → polynomial) drastically changes the shape of the boundary.
4. Decision boundary visualizations make SVM concepts intuitive and easy to grasp.
5. This approach is lightweight and perfect for exploring ML behavior interactively.

Conclusion

Visualizing SVM decision boundaries provides deep insight into how the model separates classes and makes predictions.

With just two features, the plot clearly shows margins, support vectors, and the classifier’s geometry — concepts that otherwise stay abstract.

This technique is widely used for teaching, debugging, and understanding ML behavior.

You can extend this further by trying different kernels (RBF, poly) or experimenting with noisy datasets.