We have reviewed the functional analytic approach to learning in neural networks in Chapter 2 and laid out the framework for learning based on an analysis at the level of function spaces. Various optimization criterion for learning were considered and importance of using a criterion which reduces error in the original function space rather than the sampled space was emphasized in accordance with the work of Ogawa et al.

We have discussed the problems of model selection and the bias variance dilemma which comes up during the selection of the function search space in Chapter 3 and have shown that this framework is well suited for use with standard model selection strategies.

The definition and need for

During the course of the dissertation, we have emphasized the importance and need of not just using the given training data but dynamically selecting the training data with a view to improving generalization ability. Chapter 6 provides a method of selecting the optimal training data, referred to as the

Techniques described under exact incremental learning work surprisingly well under the presence of strong apriori knowledge and are particularly useful in problems where the dimensionality is not too high. But when we consider learning in high dimensional spaces without strong apriori knowledge, this and most other parametric as well as non-parametric methods break down due to the complexity. However, we found evidence that in problems of motor control, the data generated by biological and artificial movements systems, though being high dimensional and sparse globally, usually are distributed densely on a low dimensional hyperplane

The LWPCA dimensionality reduction is used on top of a locally weighted regression to implement the Local Adaptive Subspace Regression (LASS). LASS is an incremental learning algorithm that uses no apriori knowledge, is completely incremental in allocating resources as well as in incorporating new data and avoids competition between local modules to prevent negative interference. LASS was tested in Chapter 8 and for artificial, robot and human motion data, it was shown to effectively detect and exploit low dimensional local manifolds from high dimensional data and achieve excellent learning results.

To summarize, in the presence of strong apriori knowledge and for optimal generalization, we recommend using the exact incremental learning technique developed in Part I of the dissertation. Active data selection techniques can be used to obtain optimal generalization with minimal training data. However, in the absence of apriori knowledge and especially, for learning in high dimensional spaces, LASS is a very effective alternative for obtaining good approximations to the real solution.

Click here to download a compressed (gzip-ed) version of the thesis (180 pages). Formatted for double side book style printing.