How to Effectively Preprocess and Scale Your Data

We know that some algorithms, like neural networks and SVMs, are very sensitive to the scaling of the data. Therefore, adjust the features so that data representation is more suitable for these algorithms. Sometimes we need to scale our data to 0 to 1, and sometimes we want to remove outliers from the data.

Different Kinds of Preprocessing

StandardScaler:

StandardScaler ensures that for each feature, the mean is 0 and the variance is 1, bringing all features to the same magnitude. However, this scaling does not ensure any particular minimum and maximum values for the features.
Imagine you have data about students with two features.

• Height (cm) → Ranges from 150 to 190
• Weight (kg) → Ranges from 40 to 90

Since height and weight are on different scales, a machine learning model might think weight is more important just because the numbers are bigger.

What StandardScaler Does:

1. Centers the Data (Mean = 0)
2. Example: If the mean height is 170 cm, a height of 180 cm becomes 180 - 170 = 10.
3. Scales to Unit Variance (Std Dev = 1)
4. It is divided by the standard deviation (how spread out the data is).
5. If the standard deviation of height is 10 cm, then 10 / 10 = 1.

Why Use It?

• Helps machine learning models treat all features equally.
• Works well for algorithms like SVM, PCA, and Neural Networks that assume data is centered.

RobustScaler

Works similarly to the StandardScaler in that it ensures statistical properties for each feature that guarantee that they are on the same scale. However, the RobustScaler uses the median and quartiles,1 instead of the mean and variance. This makes the RobustScaler ignore data points that are very different from the rest (like measurement errors). These odd data points are also called outliers and can lead to trouble for other scaling techniques.

MinMaxScaler

It shifts the data such that all features are exactly between 0 and 1. For the two-dimensional dataset, this means all of the data is contained within the rectangle created by the x-axis between 0 and 1 and the y-axis between 0 and 1.

NormalizedScaler

It does a very different kind of rescaling. It scales each data point such that the feature vector has a Euclidean length of 1. In other words, it projects a data point on the circle (or sphere, in the case of higher dimensions) with a radius of 1. This means every data point is scaled by a different number (by the inverse of its length). This normalization is often used when only the direction (or angle) of the data matters, not the length of the feature vector.

Summary
Data preprocessing is crucial for the performance of machine learning algorithms sensitive to data scaling, such as neural networks and SVMs. Common scaling techniques include StandardScaler, which normalizes the mean and variance; RobustScaler, which uses median and quartiles to handle outliers; MinMaxScaler, which scales features to a 0-1 range; and NormalizedScaler, which adjusts the data to unit Euclidean length. These methods ensure that machine learning models interpret all features on equal footing, enhancing model accuracy and reliability.