Fisher linear discriminant and dataset transformation

Bob Fisher

The Fisher Linear Discriminant (FLD) gives a projection matrix W that reshapes the scatter of a data set to maximize class separability, defined as the ratio of the between-class scatter matrix to the within-class scatter matrix. This projection defines features that are optimally discriminating.

Let tex2html_wrap_inline71 be a set of N column vectors of dimension D. The mean of the dataset is
There are K classes tex2html_wrap_inline81. The mean of class k containing tex2html_wrap_inline85 members is:
The between class scatter matrix is
The within class scatter matrix is

The transformation matrix that repositions the data to be most separable is the matrix W that maximizes

Let tex2html_wrap_inline95 be the generalized eigenvectors of tex2html_wrap_inline97 and tex2html_wrap_inline99. Then tex2html_wrap_inline101. This gives a projection space of dimension D. A projection space of dimension d < D can be defined by using the generalized eigenvectors with the largest d eigenvalues to give tex2html_wrap_inline109. The projection of vector tex2html_wrap_inline111 into a subspace of dimension d is tex2html_wrap_inline115.

The generalized eigenvectors are eigenvectors of

Bob Fisher
Friday June 15 18:23:13 BST 2001