There are two basic types of learning: batch and on-line. Most neural networks use a hybrid combination: pattern learning.

Batch learning is associated with a finite design set of cases. Weight updates are accumulated after the presentation of each design case. However, the updates are not applied until all cases have been presented. This determines the end of an epoch. Then the process is repeated until a stopping criterion is satisfied. The state of the network at any time does not depend on the order in which the design cases are presented.

On-line learning was originally associated with a stream of random design inputs over which the designer had little control. Weight updates are applied immediately after each design case is presented. The process continues until a stopping criterion is satisfied. The state of the network at any time does depend on the order in which the design cases were presented. The length of an epoch is at the discretion of the designer (e.g., test the stopping criterion every N cases). Pattern learning is on-line learning recursively applied to a finite design set of N cases. It is prudent to randomize the order in which design cases are presented each epoch. It is possible to use batch learning in the on-line random stream scenario. It is also possible to alternate batch learning with on-line learning.