# Archive for category Neural Network

### Online learning and Batch learning

Posted by Bikash Karmokar in Neural Network on December 30, 2011

**On line learning or stochastic** **learning**:

Stochastic learning occurs when the neuron weights are updated after each individual piece of data is passed through the system. The Feed Forward Neural Network therefore changes with every piece of data and is in a constant state of change during training.

**Batch learning:**

Batch Learning on the other hand stores each neuron weight change when it occurs, and only at the end of each epoch does it update the weights with the net change over the training set. This means the neural network will only update once at the end of each epoch.

### Back Propagation Algorithm

Posted by Bikash Karmokar in Neural Network on August 7, 2011

The generalized delta rule also known as back propagation algorithm is explained here briefly for feed forward Neural Network (NN). The explanation here is intended to give an outline of the process involved in back propagation algorithm.

See the figure carefully and try to match with the explanation given below.

The NN explained here contains three layers. These are input, hidden, and output Layers. During the training phase, the training data is fed into to the input layer. The data is propagated to the hidden layer and then to the output layer. This is called the forward pass of the back propagation algorithm. In forward pass, each node in hidden layer gets input from all the nodes from input layer, which are multiplied with appropriate weights and then summed. The output of the hidden node is the non-linear transformation of the this resulting sum. Similarly each node in output layer gets input from all the nodes from hidden layer, which are multiplied with appropriate weights and then summed. The output of this node is the non-linear transformation of the resulting sum.

The output values of the output layer are compared with the target output values. The target output values are those that we attempt to teach our network. The error between actual output values and target output values is calculated and propagated back toward hidden layer. This is called the backward pass of the back propagation algorithm. The error is used to update the connection strengths between nodes, i.e. weight matrices between input-hidden layers and hidden-output layers are updated.

During the testing phase, no learning takes place i.e., weight matrices are not changed. Each test vector is fed into the input layer. The feed forward of the testing data is similar to the feed forward of the training data.