What is "Perceptron" in Deep Learning..?

A perceptron is a basic building block of a neural network. We will explore this concept in simple terms in today’s article.

History:

The perceptron was invented by American psychologist and researcher Frank Rosenblatt in 1957 (or 1958). It is nothing but an algorithm, or we can say a simple mathematical model that could learn to classify patterns by adjusting the strength (weights) of its inputs.

Frank actually said he invented a thing that can actually learn, and it is very similar to the human brain’s neuron. It was a big thing at that time, and everyone started to think, “Now we can build Artificial human intelligence.“

Frank Rosenblatt was very confident in his invention and stated this emphatically.

"embryo of an electronic computer that will be able to walk, talk, see, write, reproduce itself, and be conscious of its existence"

Intuition:

Although the perceptron is quite simple, its design integrates it into deep learning. Now, let’s see what the perceptron looks like.

You provide inputs from the connections on the left side (e.g., X1, X2, Xn), and each input is associated with its own unique weight, along with a bias for each connection (e.g., W1, W2, Wn). These inputs are then sent to the Weighted Sum block (∑), where the summation of all the inputs multiplied by their respective weights is performed using a dot product. We denote the output of this operation as Z. After the summation, Z is passed into the Activation Function. The role of the activation function is to transform Z into a specific range (e.g., 0 to 1 or -1 to 1) using a step function.

A perceptron uses two main equations:

The weighted sum: 𝑧 = ∑ ( 𝑤 𝑖 𝑥 𝑖 ) + 𝑏, to calculate the net input, and an activation function f(x) (often a step function) to produce the final binary output (like 0 or 1)

The perceptron is primarily used in supervised machine learning tasks, particularly in classification problems like Logistic Regression. It is designed based on the structure of a human brain neuron and resembles our neurons in functionality.

The illustration you see on the left represents a neuron, a fundamental component of the nervous system often studied in elementary biology. Neurons play a crucial role in transmitting information throughout the body. When comparing a neuron side by side, you can observe that they take in signals through structures known as dendrites (Just like our perceptron takes input). These dendrites resemble tree roots and are responsible for receiving inputs from other neurons and sensory cells.

Once the signals are received, they travel through the neuron to the nucleus located at the center, where essential cellular processes occur (It’s like the sigma function block that performs summation of those inputs). The nucleus houses the genetic material and regulates various functions, ensuring the neuron operates effectively.

Neurons are interconnected through structures called synapses. These synapses enable communication between adjacent neurons by transferring neurotransmitters, which are chemical signals (Just like our activation function, as we saw in the previous image). This intricate network of connections allows for the complex processing and transmission of information, essential for everything from basic reflexes to higher cognitive functions.

Although the overall structure and function of neurons are much more elaborate, with various types and subtypes serving different purposes in the nervous system, this overview highlights the basic components and their roles in neural communication and processing. Scientists continue to research these complex structures to gain a deeper understanding of how they work and their implications for health and disease.

Our discussion isn’t focused on learning biology; let’s get back to our main topic.

A perceptron is a machine learning algorithm that acts as a linear classifier, distinguishing between two classes by using a straight line defined by an equation. While it works well in two-dimensional, three-dimensional, and four-dimensional spaces for linear or nearly linear data, it has significant limitations. One major drawback is its poor performance on non-linear datasets, often resulting in much lower accuracy in those cases. Knowing these limitations is important for choosing the right classification method in machine learning.

If you’d like a Python explanation of how a perceptron works, just comment below, and I’ll share a blog with easy-to-understand code.

Let's conclude today's topic here. Please subscribe to my blog channel and continue reading more posts.

Thank You For Reading.