Local Dimensionality Reduction

Stefan Schaal, Sethu Vijayakumar and Chris Atkeson

Abstract of paper published in Neural Information Processing Systems 10, MIT Press.

If globally high dimensional data has locally only low dimensional distributions, it is advantageous to perform a local dimensionality reduction before futher processing the data In this paper, we examine several technique for local dimensionality reduction in the context of locally weigted linear regression. As possible candidates, we derive local versions of factor analysis regression, principal component regression on joint distributions, and partial least squares regression. After outlining the statistical bases of this method, we perform Monte Carlo simulations to evaluate their robustness with respect to violations of their statistical assumptions. One surprising outcome is that locally weighted partial least squares regression offers the best average resukts, thus outperforming even factor analysis, the theoretically most appealing of our candidate techniques.
Click here to download an gzip-ed version of the paper (7 pages). Click here for a pdf version.