Implementing Loss or Cost Function in Python from scratch

In this article, we will write our own Python code for different kinds of Loss or Cost functions.

Mean Absolute Error:

Mean Absolute Error (MAE) is a common metric used to evaluate the performance of a regression model. It measures the average magnitude of the differences between the predicted and true values of a continuous variable. MAE provides a straightforward and intuitive measure of how close the predictions are to the actual values.

Below is a simple Python function for calculating the MAE:

def mae(y_predicted, y_true):
    total_error = 0
    for yp, yt in zip(y_predicted, y_true):
        total_error += abs(yp - yt)
    print("Total error is:",total_error)
    mae = total_error/len(y_predicted)
    print("Mean absolute error is:",mae)
    return mae

It can be used in below way.

# Example usage
y_predicted = np.array([1,1,0,0,1])
y_true = np.array([0.30,0.7,1,0,0.5])

mae(y_predicted, y_true)

Total error is: 2.5
Mean absolute error is: 0.5

Mean Squared Error (MSE):

It is a statistical measure used to assess the average squared difference between the observed values and the predicted values in a regression model. By squaring the differences, MSE gives greater weight to larger errors, making it a commonly used metric for evaluating the accuracy and performance of regression models.

def mse(y_true, y_predicted):
    total_error = 0
    for yt, yp in zip(y_true, y_predicted):
        total_error += (yt-yp)**2
    print("Total Squared Error:",total_error)
    mse = total_error/len(y_true)
    print("Mean Squared Error:",mse)
    return mse

Here’s how you can use this function:

y_predicted = np.array([1,1,0,0,1])
y_true = np.array([0.30,0.7,1,0,0.5])
mse(y_true, y_predicted)

Total Squared Error: 1.83
Mean Squared Error: 0.366

Log Loss:

Log loss, also known as logarithmic loss or cross-entropy loss, is a commonly used loss function in machine learning and statistics. It measures the performance of a classification model by quantifying the dissimilarity between the predicted probabilities and the true class labels.

It evaluates the accuracy of the predicted probabilities of the positive class (typically labeled as 1), ranging from 0 to 1, against the true binary labels (0 or 1).

Below is a simple Python function for calculating the Log Loss:

import numpy as np

def log_loss(y_true, y_pred):
    epsilon = 1e-15  # small value to avoid division by zero
    
    y_predicted_new = [max(i,epsilon) for i in y_predicted]
    y_predicted_new = [min(i,1-epsilon) for i in y_predicted_new]
    y_predicted_new = np.array(y_predicted_new)
    return -np.mean(y_true*np.log(y_predicted_new)+(1-y_true)*np.log(1-y_predicted_new))

Here’s how you can use this function:

# Example usage
y_predicted = np.array([1,1,0,0,1])
y_true = np.array([0.30,0.7,1,0,0.5])

loss= log_loss(y_true, y_predicted)
print("Log Loss:", loss)

Log Loss: 17.2696280766844

Conclusion:

Loss functions are crucial components in machine learning model’s performance and guiding the optimization process. By minimizing the loss, we aim to improve the accuracy and reliability of our models. Understanding different loss functions and selecting the appropriate one for the problem at hand is essential for achieving optimal results.

Thanks for reading.