🧠 AI with Python – 📈 Train a Support Vector Machine (SVM) Classifier

Introduction

Support Vector Machines (SVMs) are powerful supervised learning algorithms widely used for classification tasks.

They work by finding the optimal hyperplane that separates data points of different classes.

With kernels, SVMs can handle non-linear decision boundaries, making them flexible and effective.

In this post, we’ll train an SVM classifier using scikit-learn on the Iris dataset, evaluate its performance, and optionally visualize the decision boundaries in 2D.

Why SVM?

• Finds the maximum margin hyperplane, ensuring strong generalization.
• Works for linear and non-linear data (via kernels).
• Effective in high-dimensional spaces.
• Requires scaling of features for stable performance.

Loading the Dataset

We’ll use scikit-learn’s built-in Iris dataset, a classic classification dataset with four numeric features and three classes (setosa, versicolor, virginica).

from sklearn.datasets import load_iris

iris = load_iris()
X, y = iris.data, iris.target
feature_names = iris.feature_names
target_names = iris.target_names

Train/Test Split

We split the dataset into training and testing sets, stratifying by class to preserve balance.

from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.25, stratify=y, random_state=42
)

Building the SVM Pipeline

SVMs are sensitive to feature scales, so we combine StandardScaler and SVC in a Pipeline.

This ensures scaling happens inside cross-validation and prevents leakage.

from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.svm import SVC

clf = Pipeline([
    ("scaler", StandardScaler()),
    ("svm", SVC(kernel="rbf", C=1.0, gamma="scale", random_state=42))
])
clf.fit(X_train, y_train)

Evaluating the Model

We measure accuracy and use a classification report for detailed metrics.

from sklearn.metrics import accuracy_score, classification_report

y_pred = clf.predict(X_test)
print("Accuracy:", accuracy_score(y_test, y_pred))
print("Report:\n", classification_report(y_test, y_pred, target_names=target_names))

Confusion Matrix

A confusion matrix gives a visual summary of predictions vs. true labels.

from sklearn.metrics import ConfusionMatrixDisplay
import matplotlib.pyplot as plt

ConfusionMatrixDisplay.from_predictions(y_test, y_pred, display_labels=target_names, cmap="Blues")
plt.show()

Optional: Visualizing Decision Boundaries

For intuition, we can project the dataset onto two features (petal length and width) and plot the decision regions of an SVM.

import numpy as np

# Use only two features
X2 = X[:, [2, 3]]
X2_train, X2_test, y2_train, y2_test = train_test_split(
    X2, y, test_size=0.25, stratify=y, random_state=42
)

viz_clf = Pipeline([
    ("scaler", StandardScaler()),
    ("svm", SVC(kernel="rbf", C=1.0, gamma="scale", random_state=42))
]).fit(X2_train, y2_train)

# Create mesh grid
x_min, x_max = X2[:, 0].min() - 0.5, X2[:, 0].max() + 0.5
y_min, y_max = X2[:, 1].min() - 0.5, X2[:, 1].max() + 0.5
xx, yy = np.meshgrid(np.linspace(x_min, x_max, 300), np.linspace(y_min, y_max, 300))

Z = viz_clf.predict(np.c_[xx.ravel(), yy.ravel()]).reshape(xx.shape)

plt.contourf(xx, yy, Z, alpha=0.25, cmap="Paired")
plt.scatter(X2_train[:, 0], X2_train[:, 1], c=y2_train, edgecolor="k", cmap="Paired")
plt.xlabel(feature_names[2])
plt.ylabel(feature_names[3])
plt.title("SVM (RBF) Decision Boundaries – Petal Features")
plt.show()

Sample Output

• Accuracy: ~97% (varies slightly due to randomness)
• Classification Report: Strong precision and recall across all three classes.
• Confusion Matrix: Minimal misclassifications.
• Decision Boundary Plot: Clear separation of classes in 2D projection.

Key Takeaways

• SVMs are powerful classifiers that maximize the margin between classes.
• They handle non-linear boundaries via kernels like RBF.
• Always scale features before training an SVM.
• Best suited for small to medium datasets with well-defined boundaries.