Whether you are just starting out or preparing for a data science interview, knowing these 10 algorithms deeply — not just their names — is what separates good ML practitioners from great ones. Here is a practical, code-backed guide to each.
Linear Regression
When to use
Predicting a continuous value with a roughly linear relationship between features and target.
Real-world example
House price prediction based on area, bedrooms and location.
from sklearn.linear_model import LinearRegression
model = LinearRegression()
model.fit(X_train, y_train)
print("R² score:", model.score(X_test, y_test))Interpretable coefficients. Fast. Requires feature scaling for gradient descent.
Logistic Regression
When to use
Binary or multiclass classification where you need probability outputs and interpretability.
Real-world example
Email spam detection, disease diagnosis (positive/negative).
from sklearn.linear_model import LogisticRegression model = LogisticRegression(max_iter=200) model.fit(X_train, y_train) proba = model.predict_proba(X_test)[:,1] # probabilities
Excellent baseline for any classification task. Fast, interpretable.
Decision Tree
When to use
When interpretability matters most and data has non-linear relationships.
Real-world example
Customer credit risk scoring — explainable to business stakeholders.
from sklearn.tree import DecisionTreeClassifier model = DecisionTreeClassifier(max_depth=5, random_state=42) model.fit(X_train, y_train)
No scaling needed. Prone to overfitting alone — use inside ensembles.
Random Forest
When to use
General-purpose go-to algorithm. Works on almost any structured dataset.
Real-world example
Fraud detection, medical diagnosis, customer churn prediction.
from sklearn.ensemble import RandomForestClassifier model = RandomForestClassifier(n_estimators=200, random_state=42) model.fit(X_train, y_train) importances = model.feature_importances_
Robust, handles missing data, gives feature importance. Slightly slower than single tree.
XGBoost
When to use
When you need maximum accuracy on structured/tabular data.
Real-world example
Kaggle competition winner for most structured data challenges.
import xgboost as xgb
model = xgb.XGBClassifier(n_estimators=300, learning_rate=0.05,
max_depth=6, random_state=42)
model.fit(X_train, y_train, eval_set=[(X_test, y_test)],
early_stopping_rounds=20, verbose=False)Best algorithm for tabular data. Requires careful hyperparameter tuning.
SVM
When to use
High-dimensional data (text, images) or when data is not linearly separable.
Real-world example
Text classification, face recognition with small datasets.
from sklearn.svm import SVC
from sklearn.preprocessing import StandardScaler
from sklearn.pipeline import Pipeline
model = Pipeline([('scaler', StandardScaler()), ('svm', SVC(kernel='rbf', C=1.0))])
model.fit(X_train, y_train)Scales poorly to large datasets (>100k rows). Always requires feature scaling.
K-Nearest Neighbors
When to use
Small datasets where prediction speed is not critical.
Real-world example
Recommendation systems, anomaly detection.
from sklearn.neighbors import KNeighborsClassifier model = KNeighborsClassifier(n_neighbors=5, metric='euclidean') model.fit(X_train, y_train)
No training phase. Very slow at prediction on large datasets.
Naive Bayes
When to use
Text classification with high-dimensional sparse features.
Real-world example
Spam filtering, sentiment analysis, document categorisation.
from sklearn.naive_bayes import MultinomialNB
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.pipeline import Pipeline
model = Pipeline([('tfidf', TfidfVectorizer()), ('nb', MultinomialNB())])
model.fit(texts_train, labels_train)Extremely fast. Works great with small datasets and text data.
K-Means Clustering
When to use
Grouping data without labels — customer segmentation, data exploration.
Real-world example
Customer segmentation by purchase behaviour.
from sklearn.cluster import KMeans model = KMeans(n_clusters=4, init='k-means++', n_init=10, random_state=42) model.fit(X) labels = model.labels_
Must specify K in advance. Assumes spherical, equal-sized clusters.
Neural Networks
When to use
Image, text, speech, large unstructured datasets.
Real-world example
Image classification, language translation, voice assistants.
from tensorflow import keras
model = keras.Sequential([
keras.layers.Dense(128, activation='relu', input_shape=(n_features,)),
keras.layers.Dropout(0.3),
keras.layers.Dense(64, activation='relu'),
keras.layers.Dense(n_classes, activation='softmax')
])
model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])
model.fit(X_train, y_train, epochs=50, validation_split=0.2)Requires large data, GPU, and careful tuning. Overkill for small tabular datasets.
Which algorithm should you start with?
Always start with Logistic Regression or Random Forest as your baseline. They are fast, interpretable, and work well on most problems. Only move to XGBoost or Neural Networks after establishing a strong baseline.