A perception about perceptrons


In machine learning, the perceptron is used for supervised learning of binary classifiers. It is a type of linear classifier. It is one of simple type of artificial neuron.It was developed in 1950-60 by Frank Rosenblatt. A perceptron takes multiple binary input say x1,x2… and gives a single binary output.

In above perceptron we have x1,x2,and x3 as input. Weights w1,w2.. (real numbers) are used to express the importance of respective input to output. The output of perceptron is determined by comparing weighted sum (jwjxj ) with threshold value. 

And that’s how perceptron works.                         

We can change weights and threshold value to get different models of decision making.

Let’s further simplify description of perceptron. First, we can write  jwjxj as w.x where ‘w’ and ’x’ are vectors of weight and input values. Second, we can move threshold to another side of inequality and redefine it as Bias, b ≡ −threshold.


x (and a +1 intercept term), and outputs hW,b(x)=f(WTx)=f(∑3i=1Wixi+b)hW,b(x)=f(WTx)=f(∑i=13Wixi+b), where f:RRf:ℜ↦ℜ is called the activation function.

Bias represents how easily we can get '1' as output. For example , if bias is big , it is quite easy to get '1'. But , if the bias is negative, then

it is difficult. We can easily implement some functions like AND, OR and NAND. here , we have a perceptron with two inputs, each with weight -2 , and an overall bias of 3.

We can see that,

And so out perceptron implements NAND gate. In similar way , we can implement any computational function.

Let’s see how we can implement a XNOR gate.

We know that XNOR gate can implemented using AND, NOT and OR gates.

As x1(XNOR)x2= x1(AND)x2 + (NOT x1) AND (NOT x2) 

From above image we can see implementation of x1(AND) x2, (NOT x1) AND (NOT x2), and x1 OR X2.

Now, we can combine these individual models   to form a multilayer neural network and thus, implanting XNOR. As in below image.


This XNOR gate example demonstrates that how we can combine simple neural network to build a large and complex network.

It turns out that we can devise learning algorithms which can automatically tune the weights and biases of a network of artificial neurons. This tuning happens in response to external stimuli, without direct intervention by a programmer.These learning algorithms enable us to use artificial neurons in a way which is radically different to conventional logic gates. Instead of explicitly laying out a circuit of NAND and other gates, our neural networks can simply learn to solve problems, sometimes problems where it would be extremely difficult to directly design a conventional circuit.


Reference :