Bob Fisher
The Fisher Linear Discriminant (FLD) gives a projection matrix W that reshapes the scatter of a data set to maximize class separability, defined as the ratio of the between-class scatter matrix to the within-class scatter matrix. This projection defines features that are optimally discriminating.
Let be a set of N column vectors of dimension D.
The mean of the dataset is
There are K classes .
The mean of class k containing
members is:
The between class scatter matrix is
The within class scatter matrix is
The transformation matrix that repositions the data
to be most separable is the matrix W
that maximizes
Let
be the generalized eigenvectors of
and
.
Then
.
This gives a projection space of dimension D.
A projection space of dimension d < D can be defined
by using the generalized eigenvectors with the largest d eigenvalues
to give
.
The projection of vector
into a subspace of dimension d
is
.
The generalized eigenvectors are eigenvectors of