HomeCheat sheetsMachine Learning

# Supervised Machine Learning Cheat Sheet

In this cheat sheet, you'll have a guide around the top supervised machine learning algorithms, their advantages and disadvantages, and use-cases.
Dec 2022  · 5 min read

When working with machine learning models, it's easy to try them all out without understanding what each model does and when to use them. In this cheat sheet, you'll find a handy guide describing the most widely used supervised machine learning models, their advantages, disadvantages, and some key use cases.

Have this cheat sheet at your fingertips

## Supervised Learning

Supervised learning models are models that map inputs to outputs, and attempt to extrapolate patterns learned in past data on unseen data. Supervised learning models can be either regression models, where we try to predict a continuous variable, like stock prices—or classification models, where we try to predict a binary or multi-class variable, like whether a customer will churn or not. In the section below, we'll explain three popular types of supervised learning models: regression-only models, regression and classification models, and classification-only models.

### Regression Only Models

 Algorithm Description and Application Advantages Disadvantages Linear Regression Linear Regression models a linear relationship between input variables and a continuous numerical output variable. The default loss function is the mean square error (MSE). Fast training because there are few parameters. Interpretable/Explainable results by its output coefficients. Assumes a linear relationship between input and output variables. Sensitive to outliers. Typically generalizes worse than ridge or lasso regression. Polynomial Regression Polynomial Regression models nonlinear relationships between the dependent, and independent variable as the n-th degree polynomial. Provides a good approximation of the relationship between the dependent and independent variables. Capable of fitting a wide range of curvature. Poor interpretability of the coefficients since the underlying variables can be   highly correlated. The model fit is nonlinear but the regression function is linear. Prone to overfitting. Support Vector  Regression Support Vector Regression (SVR) uses the same principle as SVMs but optimizes the cost function to fit the most straight line (or plane) through the data points. With the kernel trick it can efficiently perform a non-linear regression by implicitly mapping their inputs into high-dimensional feature spaces. Robust against outliers.   Effective learning and strong generalization performance. Different Kernel functions can be specified for the decision function. Does not perform well with large datasets. Tends to underfit in cases where the number of variables is much smaller than the number of observations. Gaussian Process  Regression Gaussian Process Regression (GPR) uses a Bayesian approach that infers a probability distribution over the possible functions that fit the data. The Gaussian process is a prior that is specified as a multivariate Gaussian distribution. Provides uncertainty measures on the predictions. It is a flexible and usable non-linear model which fits many datasets well. Performs well on small datasets as the GP kernel allows to specify a prior on the function space. Poor choice of kernel can make convergence slow. Specifying specific kernels requires deep mathematical understanding. Robust Regression Robust Regression is an alternative to least squares regression when data is contaminated with outliers. The term “robust” refers to the statistical capability to provide useful information even in the face of outliers. Designed to overcome some limitations of traditional parametric and non-parametric methods. Provides better regression coefficient over classical regression methods when outliers are present. More computationally intensive compared to classical regression methods. It is not a cure-all for all violations, such as imbalanced data, poor quality data. If no outliers are present in the data, it may not provide better results than  classical regression methods.Tree-based models

### Both Regression and Classification Models

 Algorithm Description and Application Advantages Disadvantages Decision Trees Decision Tree models learn on the data by making decision rules on the variables to separate the classes in a flowchart like a tree data structure. They can be used for both regression and classification. Explainable and interpretable. Can handle missing values. Prone to overfitting. Can be unstable with minor data drift. Sensitive to outliers. Random Forest Random Forest classification models learn using an ensemble of decision trees. The output of the random forest is based on a majority vote of the different decision trees. Effective learning and better generalization performance. Can handle moderately large datasets. Less prone to overfit than decision trees. Large number of trees can slow down performance. Predictions are sensitive to outliers. Hyperparameter tuning can be complex. Gradient Boosting An ensemble learning method where weak predictive learners are combined to improve accuracy. Popular techniques include XGBoost, LightGBM and more. Handling of multicollinearity. Handling of non-linear relationships. Effective learning and strong generalization performance. XGBoost is fast and is often used as a benchmark algorithm. Sensitive to outliers and can therefore cause overfitting. High complexity due to hyperparameter tuning. Computationally expensive. Ridge Regression Ridge Regression penalizes variables with low predictive outcomes by shrinking their coefficients towards zero. It can be used for classification and regression. Less prone to overfitting. Best suited when data suffers from multicollinearity. Explainable & Interpretable. All the predictors are kept in the final model. Doesn't perform feature selection. Lasso Regression Lasso Regression penalizes features that have low predictive outcomes  by shrinking their coefficients to zero. It can be used for classification  and regression. Good generalization performance. Good at handling datasets where the number of variables is much larger than the number of observations. No need for feature selection. Poor interpretability/explainability as it can keep a single variable.  from a set of highly correlated variables. AdaBoost Adaptive Boosting uses an ensemble of weak learners that is combined into a weighted sum that represents the final output of the boosted classifier. Explainable & Interpretable. Less need for tweaking parameters. Usually outperforms Random Forest. Less prone to overfitting as the input variables are not jointly optimized. Sensitive to noisy data and outliers.

### Classification Only Models

 Algorithm Description and Application Advantages Disadvantages SVM In its simplest form, support vector machine is a linear classifier. But with the  kernel trick, it can efficiently perform a non-linear classification by implicitly  mapping their inputs into high-dimensional feature spaces. This makes SVM one  of the best prediction methods. Effective in cases with a high number of variables. Number of variables can be larger than the number of samples. Different Kernel functions can be specified for the decision function. Sensitive to overfitting, regularization is crucial. Choosing a “good” kernel function can be difficult. Computationally expensive for big data due to high training complexity. Performs poorly if the data is noisy (target classes overlap). Nearest  Neighbors Nearest Neighbors predicts the label based on a predefined number of samples closest in distance to the new point. Successful in situations where the decision boundary is irregular. Non-parametric approach as it does not make  any assumption on the underlying data. Sensitive to noisy and missing data. Computationally expensive because the entire set of n points for every execution  is required. Logistic Regression  (and its extensions) The logistic regression models a linear relationship between input variables and the response variable. It models the output as binary values (0 or 1)  rather than numeric values. Explainable & Interpretable. Less prone to overfitting using regularization. Applicable for multi-class predictions. Makes a strong assumption about the relationship between input and response variables. Multicollinearity can cause the model to easily overfit without regularization. Linear Discriminant  Analysis The linear decision boundary maximizes the separability between the classes by finding a linear combination of features. Explainable & Interpretable. Applicable for multi-class predictions. Multicollinearity can cause the model to overfit. Assuming that all classes share the same covariance matrix. Sensitive to outliers. Doesn't work well with small class sizes.

Have this cheat sheet at your fingertips

Topics
Related

### Navigating the World of MLOps Certifications

Explore the dynamic world of MLOps certifications: key career benefits, certification vs. certificate insights, and how to choose the right path for you.

10 min

### How to Learn Machine Learning in 2024

Discover how to learn machine learning in 2024, including the key skills and technologies you’ll need to master, as well as resources to help you get started.

15 min

### A Beginner's Guide to CI/CD for Machine Learning

Learn how to automate model training, evaluation, versioning, and deployment using GitHub Actions with the easiest MLOps guide available online.

Abid Ali Awan

15 min

### OpenCV Tutorial: Unlock the Power of Visual Data Processing

This article provides a comprehensive guide on utilizing the OpenCV library for image and video processing within a Python environment. We dive into the wide range of image processing functionalities OpenCV offers, from basic techniques to more advanced applications.

Richmond Alake

13 min

### An Introduction to the Mamba LLM Architecture: A New Paradigm in Machine Learning

Discover the power of Mamba LLM, a transformative architecture from leading universities, redefining sequence processing in AI.

Kurtis Pykes

9 min

### A Beginner's Guide to Azure Machine Learning

Explore Azure Machine Learning in our beginner's guide to setting up, deploying models, and leveraging AutoML & ML Studio in the Azure ecosystem.

Moez Ali

11 min

See MoreSee More