In this post, we will see how to implement the perceptron model using breast cancer data set in python.
A perceptron is a fundamental unit of the neural network which takes weighted inputs, process it and capable of performing binary classifications. This is a follow up to my previous post on the Perceptron Model.
If you want to skip the theory and jump into code directly click here.
Disclaimer: The content and the structure of this article is based on the deep learning lectures from One-Fourth Labs — Padhai.
Perceptron Recap
In the perceptron model inputs can be real numbers unlike the Boolean inputs in MP Neuron Model. The output from the model will still be binary {0, 1}. The perceptron model takes the input x if the weighted sum of the inputs is greater than threshold b output will be 1 else output will be 0.
Fig 1— Mathematical RepresentationLearning Algorithm
The main goal of the learning algorithm is to find vector w capable of absolutely separating Positive P (y = 1) and Negative N(y = 0) sets of data. Perceptron learning algorithm goes like this,
(Fig 2— Perceptron Algorithm)
To understand the learning algorithm in detail and the intuition behind why the concept of updating weights works in classifying the Positive and Negative data sets perfectly, kindly refer to my previous post on the Perceptron Model.
Lets Code
The data set we will be using is breast cancer data set from sklearn. The data set has 569 observations and 30 variables excluding the class variable. The breast cancer data is an imbalanced data set, that means the classes ‘0’ and ‘1’ are not represented equally. In this example, we are not going to perform any sampling techniques to balance the data because this is a simple implementation of the perceptron model.
(Class Imbalance)
Before start building the Perceptron Model, first we need to load the required packages and the data set. The data set is present in the sklearn datasets module. Once we load the data, we need to grab the features and response variables using breast_cancer.data and breast_cancer.target commands.
#import packages
import sklearn.datasets
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from sklearn.model_selection import train_test_split
#load the breast cancer data
breast_cancer = sklearn.datasets.load_breast_cancer()
#convert the data to pandas dataframe.
data = pd.DataFrame(breast_cancer.data, columns = breast_cancer.feature_names)
data["class"] = breast_cancer.target
data.head()
data.describe()
#plotting a graph to see class imbalance
data['class'].value_counts().plot(kind = "barh")
plt.xlabel("Count")
plt.ylabel("Classes")
plt.show()
from sklearn.preprocessing import MinMaxScaler
#perform scaling on the data.
X = data.drop("class", axis = 1)
Y = data["class"]
mnscaler = MinMaxScaler()
X = mnscaler.fit_transform(X)
X = pd.DataFrame(X, columns=data.drop("class",axis = 1).columns)
#train test split.
X_train, X_test, Y_train, Y_test = train_test_split(X,Y, test_size = 0.1, stratify = Y, random_state = 1)(Perceptron Preprocessing)
After fetching the X and Y variables, we will perform Min-Max scaling to bring all the features in the range 0 — 1. Before building the model, we will split the data so that we can train the model on training data and test the performance of the model on testing data. We will use sklearn’s train_test_split function to split the data in the ratio of 90:10 for training and testing respectively. Now that we are done with preprocessing steps, we can start building the model. We will build our model inside a class called perceptron.
In the perceptron class, we will create a constructor function def__init__. The constructor initializes the weights vector w and threshold b to None.
class Perceptron:
#constructor
def __init__ (self):
self.w = None
self.b = None
#model
def model(self, x):
return 1 if (np.dot(self.w, x) >= self.b) else 0
#predictor to predict on the data based on w
def predict(self, X):
Y = []
for x in X:
result = self.model(x)
Y.append(result)
return np.array(Y)
def fit(self, X, Y, epochs = 1, lr = 1):
self.w = np.ones(X.shape[1])
self.b = 0
accuracy = {}
max_accuracy = 0
wt_matrix = []
#for all epochs
for i in range(epochs):
for x, y in zip(X, Y):
y_pred = self.model(x)
if y == 1 and y_pred == 0:
self.w = self.w + lr * x
self.b = self.b - lr * 1
elif y == 0 and y_pred == 1:
self.w = self.w - lr * x
self.b = self.b + lr * 1
wt_matrix.append(self.w)
accuracy[i] = accuracy_score(self.predict(X), Y)
if (accuracy[i] > max_accuracy):
max_accuracy = accuracy[i]
chkptw = self.w
chkptb = self.b
#checkpoint (Save the weights and b value)
self.w = chkptw
self.b = chkptb
print(max_accuracy)
#plot the accuracy values over epochs
plt.plot(accuracy.values())
plt.xlabel("Epoch #")
plt.ylabel("Accuracy")
plt.ylim([0, 1])
plt.show()
#return the weight matrix, that contains weights over all epochs
return np.array(wt_matrix)(Perceptron Model)
The function model takes input values x as an argument and perform the weighted aggregation of inputs (dot product between w.x) and returns the value 1 if the aggregation is greater than the threshold b else 0. Next, we have the predict function that takes input values x as an argument and for every observation present in x, the function calculates the predicted outcome and returns a list of predictions.
Finally, we will implement fit function to learn the best possible weight vector w and threshold value b for the given data. The function takes input data(x & y), learning rate and the number of epochs as arguments.
perceptron = Perceptron()
#epochs = 10000 and lr = 0.3
wt_matrix = perceptron.fit(X_train, Y_train, 10000, 0.3)
#making predictions on test data
Y_pred_test = perceptron.predict(X_test)
#checking the accuracy of the model
print(accuracy_score(Y_pred_test, Y_test))(Perceptron Model Execution)
Once we have our class ready, we initialize a new perceptron class object and using that object we will call fit method on our training data to learn the best possible parameters. We will evaluate the model performance on the test data by calculating the testing accuracy.
The entire code discussed in the article is present in this GitHub repository. Feel free to fork it or download it.
Continue Learning
If you want to learn more about Artificial Neural Networks using Keras & Tensorflow 2.0 (Python or R). Check out the Artificial Neural Networks by Abhishek and Pukhraj from Starttechacademy. They explain the fundamentals of deep learning in a simplistic manner.
Further Improvements
You can try out a few possible improvements to increase the accuracy of the model,
- Vary the train-test size split and see if there is any change in accuracy.
- Choose larger epochs values, learning rates and test on the perceptron model and visualize the change in accuracy.
- Take random weights in the perceptron model and experiment.
Conclusion
In this article, we have seen how to implement the perceptron algorithm from scratch using python.
Connect with Me
GitHub: https://github.com/Niranjankumar-c
LinkedIn: https://www.linkedin.com/in/niranjankumar-c/
Disclaimer — There might be some affiliate links in this post to relevant resources. You can purchase the bundle at the lowest price possible. I will receive a small commission if you purchase the course.