Polynomial Regression

Polynomial Regression

Polynomial Regression is a form of Regression used when the relationship between variables is nonlinear. Instead of fitting a straight line, it bends the line to fit the curves of the dataset.

Polynomial Regression in Simple Words

Linear Regression tries to describe a pattern with a straight line. Polynomial Regression is useful when the pattern clearly bends or curves. It gives the model extra features such as x² and x³ so a linear model can follow that curve.

What is Polynomial Regression?

Polynomial Regression is a supervised machine learning algorithm that models nonlinear relationships between the independent variable and dependent variable by using polynomial terms.

Although it is called "polynomial," it still belongs to the family of linear models because the coefficients are learned linearly.

Real-Life Intuition

Visual Understanding of Polynomial Regression

Why Polynomial Regression is Needed

Polynomial Regression can model some smooth nonlinear patterns by adding powers of the original features. Whether it predicts better must be checked on validation or unseen data.

Core Idea of Polynomial Regression

Polynomial Regression transforms features into higher-degree polynomial terms.

Where Polynomial Regression Fits

Understanding Polynomial Degrees

1. Quadratic Regression (Degree 2)

Produces a parabola-shaped curve.

2. Cubic Regression (Degree 3)

Captures more complex nonlinear relationships.

Feature Transformation

Polynomial Regression converts original features into polynomial features. By doing this, it treats the higher powers as completely new variables and trains a linear model on them.

Difference Between Linear and Polynomial Regression

Cost Function in Polynomial Regression

Polynomial Regression for a continuous target can use the same regression losses as Linear Regression. A common choice is Mean Squared Error (MSE), which measures the average squared prediction error.

Overfitting & Choosing the Degree

The polynomial degree controls how flexible the curve can become. Too low a degree may miss the pattern, while an unnecessarily high degree can start fitting noise and perform poorly on unseen data. Choose the degree using validation data or cross-validation rather than a fixed rule.

Assumptions & Practical Checks

Polynomial Regression is linear in its learned coefficients, so many Linear Regression assumptions still matter. Some checks are especially important when you want statistical inference, while others are practical modeling choices.

Industry Applications Tree

Implementation of Polynomial Regression in Python

Polynomial Regression is used when the relationship between the input variable and output variable is not perfectly linear. Instead of fitting a straight line, polynomial regression fits a curved line to better capture complex patterns in the data. In this example, we will build a polynomial regression model using Python and compare it with a simple linear regression model.

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt

from sklearn.linear_model import LinearRegression
from sklearn.preprocessing import PolynomialFeatures

data = {
    'Study_Hours': [1, 2, 3, 4, 5, 6, 7, 8, 9, 10],
    'Exam_Score': [45, 50, 54, 60, 68, 78, 85, 91, 96, 98]
}

df = pd.DataFrame(data)

print(df)

X = df[['Study_Hours']].values
y = df['Exam_Score'].values

linear_model = LinearRegression()
linear_model.fit(X, y)

poly_features = PolynomialFeatures(degree=3, include_bias=False)

X_poly = poly_features.fit_transform(X)

poly_model = LinearRegression()

poly_model.fit(X_poly, y)

plt.scatter(X, y, color='blue')
plt.plot(X, linear_model.predict(X), color='red')
plt.title('Linear Regression Model')
plt.xlabel('Study Hours')
plt.ylabel('Exam Score')
plt.show()

plt.scatter(X, y, color='blue')
plt.plot(X,
    poly_model.predict(poly_features.transform(X)),
    color='red')
plt.title('Polynomial Regression Model')
plt.xlabel('Study Hours')
plt.ylabel('Exam Score')
plt.show()

new_value = np.array([[7.5]])
linear_prediction = linear_model.predict(new_value)
print(linear_prediction)

poly_prediction = poly_model.predict(
    poly_features.transform(new_value)
)
print(poly_prediction)

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt

from sklearn.linear_model import LinearRegression
from sklearn.preprocessing import PolynomialFeatures

# 1. Create Dataset
data = {
    'Study_Hours': [1, 2, 3, 4, 5, 6, 7, 8, 9, 10],
    'Exam_Score': [45, 50, 54, 60, 68, 78, 85, 91, 96, 98]
}

df = pd.DataFrame(data)

# 2. Features and Target
X = df[['Study_Hours']].values
y = df['Exam_Score'].values

# 3. Linear Regression Model Training
linear_model = LinearRegression()
linear_model.fit(X, y)

# 4. Polynomial Feature Expansion & Model Training
poly_features = PolynomialFeatures(degree=3, include_bias=False)
X_poly = poly_features.fit_transform(X)

poly_model = LinearRegression()
poly_model.fit(X_poly, y)

# 5. Linear Regression Plot
plt.scatter(X, y, color='blue')
plt.plot(X, linear_model.predict(X), color='red')
plt.title('Linear Regression Model')
plt.xlabel('Study Hours')
plt.ylabel('Exam Score')
plt.show()

# 6. Polynomial Regression Plot
plt.scatter(X, y, color='blue')
plt.plot(X, poly_model.predict(poly_features.transform(X)), color='red')
plt.title('Polynomial Regression Model')
plt.xlabel('Study Hours')
plt.ylabel('Exam Score')
plt.show()

# 7. Make Predictions
new_value = np.array([[7.5]])
linear_prediction = linear_model.predict(new_value)
poly_prediction = poly_model.predict(poly_features.transform(new_value))

print("Linear Regression Prediction:", linear_prediction)
print("Polynomial Regression Prediction:", poly_prediction)

Evaluation Metrics

Polynomial Regression uses standard regression metrics to tell how well our curve fits the data:

Polynomial vs Other Algorithms

Common Questions About Polynomial Regression

Final Summary

Polynomial Regression is a powerful extension of Linear Regression designed to model nonlinear relationships. It expands features and fits a curve rather than a straight line.

Increasing the polynomial degree makes the model more flexible, but too much flexibility can fit noise and hurt generalization. A good workflow is to try modest degrees, monitor validation error, use cross-validation, and consider scaling or regularization when polynomial features become large or highly correlated.