Feature Engineering

Feature Engineering in Machine Learning

Feature engineering is the process of creating, transforming, and selecting useful input features from raw data so a Machine Learning model can learn patterns more effectively.

Feature Engineering in Simple Words

Common feature engineering techniques include handling missing values, encoding categorical variables, scaling numeric features, creating ratios and interactions, extracting date-time information, transforming skewed values, and representing text numerically.

In many structured-data Machine Learning projects, the model learns from the columns we give it. Feature Engineering means turning raw columns into more useful clues that make the pattern easier to learn.

What Is Feature Engineering in Machine Learning?

Feature Engineering is the process of creating new input variables, transforming existing data, extracting useful information, and improving feature quality so that Machine Learning models can learn patterns more effectively.

In simple words: Feature Engineering helps convert raw data into meaningful information that improves model performance.

Why is it Important?

Raw data is often incomplete, noisy, unstructured, inconsistent, and difficult for ML models to understand. Feature Engineering transforms messy real-world data into machine-friendly representations.

Real-Life Example: Predicting House Prices

Imagine we want to predict the selling price of a house. A raw row might look like this:

We can turn these raw values into features that are often easier for a model to use.

Common Input Types We Engineer

Feature engineering depends on the kind of raw data we have. The same idea is applied differently to numbers, categories, text, and dates.

Feature Engineering Workflow

1. Step 1: Understand the Data. Before engineering features, analyze distributions, identify missing values, understand business context, and detect anomalies.
2. Step 2: Clean the Data. Data cleaning includes removing duplicates, handling null values, fixing inconsistent formatting, and correcting errors.
3. Step 3: Create Useful Features. Transform raw information into valuable predictors.
4. Step 4: Select Important Features. Keep useful features and remove irrelevant or redundant ones.
5. Step 5: Validate the New Features. Compare model performance on validation data. A feature that sounds clever is useful only if it helps the real prediction task without causing leakage.

Common Feature Engineering Techniques

There are several widely used techniques for creating, transforming, and preparing features for different data and modeling needs.

1. Handling Missing Values

Real-world datasets often contain missing information. Methods to handle missing data include:

• Remove Missing Rows: May be reasonable when missingness is limited and deleting those rows is unlikely to bias the dataset.
• Mean Imputation: Replace missing values with the average.
• Median Imputation: Often a useful simple choice for skewed numerical data or data with extreme values.
• Mode Imputation: Useful for categorical variables.

For a fuller treatment, see Handling Missing Data.

import pandas as pd
import numpy as np

df = pd.DataFrame({'Age': [25, np.nan, 30, 22]})
print("Original:\n", df)

# Mean imputation
df['Age'] = df['Age'].fillna(df['Age'].mean())
print("\nAfter Imputation:\n", df)

Original:
     Age
0  25.0
1   NaN
2  30.0
3  22.0

After Imputation:
          Age
0  25.000000
1  25.666667
2  30.000000
3  22.000000

2. Encoding Categorical Variables

Many common Machine Learning estimators expect numerical input, so categories such as Red, Blue, and Green need an appropriate representation. Common approaches include Ordinal Encoding for ordered categories, One-Hot Encoding for nominal categories, and carefully designed target-based encoding for some high-cardinality problems.

See Encoding Categorical Data for the full comparison and leakage precautions.

import pandas as pd

df = pd.DataFrame({'Color': ['Red', 'Blue', 'Green']})
encoded = pd.get_dummies(df['Color'], dtype=int)
print(encoded)

   Blue  Green  Red
0     0      0    1
1     1      0    0
2     0      1    0

3. Feature Scaling

Different features may have very different ranges (for example, Age 18–60 vs Salary in lakhs). In distance-based and gradient-based methods, these scale differences can strongly affect the calculation even when the larger-number feature is not more important.

• Min-Max Scaling: Often maps values to a range such as [0,1]
• Standardization: Mean=0, Std=1
• Robust Scaling: Uses median and quantiles, making it less sensitive to extreme values

See Feature Scaling for worked StandardScaler, Min-Max, and RobustScaler examples.

from sklearn.preprocessing import StandardScaler
import pandas as pd

df = pd.DataFrame({'Salary': [50000, 60000, 100000]})
scaler = StandardScaler()

scaled = scaler.fit_transform(df)
print(scaled)

[[-0.9258201 ]
 [-0.46291005]
 [ 1.38873015]]

4. Creating Ratio and Interaction Features

Sometimes two existing columns become more useful when they are combined. This is one of the clearest examples of creating a new feature from raw data.

import pandas as pd

df = pd.DataFrame({
    'Area': [1200, 1500],
    'Bedrooms': [3, 5]
})

df['Area_per_Bedroom'] = df['Area'] / df['Bedrooms']
print(df)

   Area  Bedrooms  Area_per_Bedroom
0  1200         3             400.0
1  1500         5             300.0

Interaction features can also combine variables by multiplication or other domain-based rules. Create them only when the relationship makes sense and validate whether they help.

5. Binning (Discretization)

Binning groups continuous values into ranges such as Young, Adult, and Senior. It can make a feature easier to interpret and can reduce sensitivity to small numeric differences, but it also loses some information, so it should be used only when the grouping makes sense for the problem.

import pandas as pd

df = pd.DataFrame({'Age': [19, 28, 45, 60, 22]})

df['Group'] = pd.cut(df['Age'],
       bins=[0, 25, 40, 100],
       labels=['Young', 'Adult', 'Senior'])
       
print(df)

   Age   Group
0   19   Young
1   28   Adult
2   45  Senior
3   60  Senior
4   22   Young

6. Log Transformation

A log transformation is often explored for strongly right-skewed positive values such as income, website traffic, or sales. It compresses large values more than small values, which can make some relationships easier for a model to learn. It is not automatically helpful for every dataset.

import numpy as np
import pandas as pd

df = pd.DataFrame({'Income': [1000, 5000, 1000000]})
df['Log_Income'] = np.log1p(df['Income'])

print(df)

    Income  Log_Income
0     1000    6.908755
1     5000    8.517393
2  1000000   13.815512

7. Date-Time Feature Engineering

Dates contain several pieces of information. From one timestamp we can extract features such as day, month, year, weekday, quarter, or whether it falls on a weekend.

import pandas as pd

df = pd.DataFrame({'Date': pd.to_datetime(['2026-05-16', '2026-12-25'])})

df['Month'] = df['Date'].dt.month
df['Weekday'] = df['Date'].dt.weekday
df['Is_Weekend'] = df['Weekday'] >= 5

print(df)

        Date  Month  Weekday  Is_Weekend
0 2026-05-16      5        5        True
1 2026-12-25     12        4       False

8. Text Feature Engineering

Text data must be converted into numerical representation. Common Techniques include Bag of Words, TF-IDF, and Word Embeddings.

from sklearn.feature_extraction.text import TfidfVectorizer

texts = ["I love Machine Learning", "Machine Learning is powerful"]
vectorizer = TfidfVectorizer()

X = vectorizer.fit_transform(texts)
print(vectorizer.get_feature_names_out())
print(X.toarray())

['is' 'learning' 'love' 'machine' 'powerful']
[[0.         0.44943642 0.6316672  0.44943642 0.        ]
 [0.53404633 0.37997836 0.         0.37997836 0.53404633]]

Feature Engineering vs Feature Extraction vs Feature Selection

9. Feature Extraction

Feature Extraction creates a new representation from existing data, often through an algorithmic transformation. Feature Engineering is the broader process and can include domain-based creation, transformation, extraction, encoding, and other preprocessing choices. Examples of extraction include TF-IDF vectors, image descriptors, or PCA components.

10. Feature Selection

Not all features help prediction. Some features add noise, increase overfitting, or slow training. Selection methods include:

• Filter Methods: Statistical selection (e.g. Correlation analysis)
• Wrapper Methods: Model-based selection (e.g. Recursive Feature Elimination)
• Embedded Methods: Built into algorithms (e.g. Lasso Regression)

Continue with Feature Selection & Extraction for dedicated examples and selection methods.

Common Beginner Mistakes & Best Practices

Interview Knowledge Test

Common Questions About Feature Engineering