Second Layer Kohonen Nets
The second layer Kohonen net takes as its input the activation of the
first net for each extracted unit. Each node of the second net is a
49 long vector, each value corresponding to one node of the first
layer net.
At present, the similarity function used is a distance measure. Only
a subset of the input node (the highest x and the lowest x values,
where x is small) is used to calculate this during training.
The labellings shown below are not 'raw', i.e. they are not simply
produced by assigning each unit to the node that it activates most
highly. Some attempt at context sensitivity has been introduced so
that the similarity score for each unit for each node is the sum of the raw
similarity score, plus a proportion (currently 0.3) of the similarity
scores for the immediately preceding and succeeding units.
4 node Kohonen net produced from activation of wrapper first layer net:
Number of units mapping to each node:
147 90 163 126
Matrix showing number of one-step transitions. The top-left to
bottom right diagonal shows the number of times a unit is followed by
another of the same type.
108 21 16 2
19 27 35 9
12 32 95 24
8 11 16 91
8 node Kohonen net produced from activation of wrapped first layer net:
Number of units mapping to each node:
41 16 49 34 136 78 62 110
Pairwise transition matrix, as before:
10 15 2 1 6 3 1 3
11 0 0 1 2 0 2 0
0 0 30 5 7 6 0 1
1 0 2 13 13 2 2 1
6 0 7 10 78 6 28 1
5 1 5 2 1 58 5 1
5 0 3 2 25 3 9 15
3 0 0 1 3 0 15 88
The following matrix shows the mapping between nodes in the 4-node and 8-node nets:
28 0 8 5
7 4 5 0
1 17 6 25
1 30 3 0
1 4 131 0
2 0 4 72
0 33 5 24
107 2 1 0
As you can see, the 8th node of the 8-node net has taken most of the
units that mapped to the 1st node of the 4-node net. This makes sense
if you look at the pictures. Equally, the 2nd node of the 4-node net
has split, mostly into the 4th and 7th nodes of the 8-node net.
Labelling of the first 100 units of the song by the 4-node net
with context calculations:
Labelling of the first 100 units of the song by the 4-node net
without context calculations:
Labelling of the first 100 units of the song by the 8-node net
with context calculations:
Labelling of the first 100 units of the song by the 8-node net
without context calculations:
Labelling of the first 100 units of the song by a manual
classification (n.b. numerical labels are for plotting convenience
only and should not be taken to imply that certain classes are more or
less similar).
Node counts and transition matrix for the manual classification:
85 55 130 6 37 107 68 15
48 27 0 1 0 2 7 0
26 1 15 2 3 7 0 1
1 15 94 0 4 0 16 0
2 1 1 0 0 2 0 0
1 5 1 1 29 0 0 0
0 7 0 2 1 95 2 0
6 0 17 0 0 0 41 4
0 0 2 0 0 1 2 10
Rough matrix showing how the 8-node mapping and the manual mapping
relate to each other. The 8-node net is down the and manual
labelling is across.
At first glance, it appears that the correspondence is not as strong as it could be:
10 5 6 0 10 8 1 1
7 0 1 0 6 2 0 0
5 2 10 0 1 6 24 1
4 2 8 0 3 3 8 6
41 17 29 3 8 7 25 6
20 8 1 1 0 48 0 0
11 11 16 2 3 11 8 0
9 11 59 0 6 22 2 1