Here we take a simple feed-forward architecture and use some of its output (or intermediate) states to augment input of the next pattern. This form of partial recurrence may be used to learn time varying or sequential patterns -- the feedback units are referred to as context units.
An example, in which the hidden layer is being used to augment the input vector, might be;
The weights from the context units can be trained in exactly the same manner as the others, and thus the standard back propagation algorithm may be used.
