🧠 AI with Python – Polynomial Regression with scikit-learn

Introduction

Linear regression is powerful, but it struggles when relationships are non-linear.

Polynomial Regression extends linear regression by first transforming the features into polynomial terms (e.g., x, x^2, x^3), then fitting a linear model on those expanded features.

In this post, we’ll build a Pipeline with PolynomialFeatures and LinearRegression, evaluate with standard metrics (MAE, MSE, R²), and visualize how well the curve fits a synthetic quadratic dataset.

When to Use Polynomial Regression

• Your residuals show curvature (systematic under/over-prediction with a line).
• You want a lightweight alternative to tree models for smooth trends.
• You need an interpretable curve with a small number of terms.

Minimal Implementation (Core Idea)

Build a Pipeline:

from sklearn.pipeline import Pipeline
from sklearn.preprocessing import PolynomialFeatures
from sklearn.linear_model import LinearRegression

degree = 2
model = Pipeline([
    ("poly", PolynomialFeatures(degree=degree, include_bias=False)),
    ("linreg", LinearRegression())
])

Fit & Evaluate (sketch):

model.fit(X_train, y_train)
y_pred = model.predict(X_test)

Use MAE, MSE, and R² to compare different degrees (e.g., 1–5).

Visualising the Fit

To see how well the model captures the pattern, sort X across the range, predict smoothly, and overlay the fitted curve on top of the scatter plot of noisy points and the true function.

Practical Tips

• Bias–Variance tradeoff: Higher degree reduces bias but can overfit.
• Try a small grid of degrees with cross-validation (GridSearchCV over poly__degree).
• For multi-feature data, polynomial terms explode combinatorially → consider regularization (Ridge/Lasso).
• Keep include_bias=False because LinearRegression already learns an intercept.

Key Takeaways

• Polynomial Regression = Linear Regression on engineered polynomial features.
• Use a Pipeline to keep transformations consistent and avoid leakage.
• Select the degree carefully; validate with proper metrics and CV.
• Add regularization for higher-degree polynomials or many features.