Supervised learning uses labeled data to make predictions, while unsupervised learning finds hidden patterns or structures in unlabeled data.

Bias refers to error resulting from simplifying assumptions in the model, which may cause underfitting.

Variance refers to error introducted by a model’s sensitivity to fluctuations in the training data, often resulting in overfitting.

The bias–variance tradeoff describes the relationship between two primary sources of error in predictive modeling:

Classification and regression are both supervised learning tasks but differ in the type of output they predict. Classification models assign inputs to discrete categories (e.g., fraud vs. non-fraud), while regression models predict continuous numerical values (e.g., sales forecasts or housing prices). Understanding this distinction guides the selection of appropriate algorithms and evaluation metrics.

Regularization is the model’s version of self-control. It prevents the model from memorizing noise by penalizing complexity.
Two common types:

L1 (Lasso): Encourages sparsity

L2 (Ridge): Shrinks coefficients smoothly

Regularization is crucial for keeping models robust, stable, and less prone to overfitting. If you say this confidently in an interview, nod slowly, and add “to improve generalization,” you’ll look like you know exactly what you’re doing.

Gradient descent updates model parameters by moving in the direction that reduces the loss function, following the negative gradient step by step until convergence.

At each step, you compute the gradient (slope), then move in the opposite direction to reduce the loss. Step by step, the algorithm converges toward a minimum.

Bagging and boosting are ensemble learning techniques that improve model performance using multiple learners:

Bagging (Bootstrap Aggregating) trains multiple models independently on different bootstrapped samples of the dataset and aggregates their predictions. This approach reduces variance and enhances stability.

Boosting, on the other hand, trains models sequentially. Each subsequent model focuses on correcting the errors of the previous one. Boosting reduces bias and can achieve high predictive accuracy but may be more prone to overfitting if not properly regularized.

Cross-validation is a model evaluation technique that assesses how well a model generalizes to unseen data.

Cross-validation mitigates the risk of performance estimates being overly optimistic or pessimistic due to a single train-test split and provides a more reliable assessment of model robustness.

A confusion matrix is a tabular representation of a classification model’s performance. It summarizes predictions into four categories: true positives, false positives, true negatives, and false negatives. These values allow practitioners to compute key evaluation metrics such as precision, recall, specificity, and the F1-score. The confusion matrix is particularly useful when dealing with class imbalance or when accuracy alone fails to provide sufficient insight.

Handling missing data depends on the amount, pattern, and significance of the missingness. Common approaches include:

Deletion: Removing rows or columns with excessive missing values.

Imputation: Replacing missing values using statistical methods (mean, median, mode) or more advanced techniques such as model-based imputation.

Algorithmic approaches: Leveraging models capable of handling missing values natively.

Feature engineering involves transforming raw data into meaningful input features that enhance model performance. This may include encoding categorical variables, scaling numerical values, creating interaction terms, selecting important features, or extracting domain-specific attributes. Effective feature engineering often has a larger impact on model accuracy than modifying or tuning algorithms themselves.