Classification Rates
We will report here on classification rates for clean ICA sources (eye artifacts removed). We will present mean rates and standard deviation across 5 subjects.

Two-class classification was performed along a single dimension at a time. These rates thus comprise 1200 (4*300) trials per class per subject. The chance classification rate for two classes is 50.0%. Grouping by figure (all penguins vs all buildings) produced both the highest mean rate and the smallest standard deviation. Grouping by size (all large against all small) was the next most successful, and grouping by color (all red against all green) produced a rate considerably lower than did the other two groupings, though this was still above chance level.

Grouping by figure classifies all of the penguins against all of the buildings (shown above).

 

Four-class classification was performed, with the stimuli grouped by two dimensions at a time. The dimensional groupings are as follows: 1) Size and color (large red images vs small red images etc.); 2) Size and figure (large penguins vs small buildings, etc.); and 3) Color and figure (red buildings vs green buildings, etc.). Chance level is now 25.0%, and each class includes 600 trials per subject. Based upon the rates obtained from the two-class classification, we predicted that size and figure would be most successful, followed by color and figure, with size and color performing the least successfully. Indeed, this turned out to be the case, with size and figure outperforming size and color, as well as color and figure by over ten percent in the mean rate, though with significant standard deviation. All rates were well above chance level.

 

Grouping by color and figure: Green penguin vs red penguin vs green building vs red building.

 

Pair rates were obtained by running all 28 combinations of single-stimulus pairs as two-class classifications. Chance level is 50.0% for each pair, and there are 300 trials per stimulus per subject. The highest pair rates (above 90.0%) are highlighted in yellow in the figure below. The pair earning the highest mean rate overall (large green penguin vs large red building) also produced the highest rate of the entire expermient of 98.8 for one subject. The lowest pairs overall (lower than 70.0%). Both of these pairs comprise small images which differ only by color.

Mean pair rates and standard deviations (in grey)

 

Finally, the mean eight-class classification rate was 50.78%, with standard deviation 10.99. This is significantly higher than the chance rate of 12.5%. Classification rates are shown in the conditional probability matrix below. The leftmost figure in each row represents the stimulus that was presented. The figure corresponding to each column represents the stimulus it was identified as (each row sums to 1). Thus, values on the diagonal show the percentage of correct identifications, while other values indicate degrees of confusion. Based on the group rates and pair rates, we hypothesized that the highest non-diagonal value in each row will be for the stimulus that varies from the correct choice only by color. Also, we conjuectured that the highest non-diagonal values overall will reflect what was found previously from the pair rates - that the most confusion will occur between those small images that differ only by color.

Each row's diagonal holds the largest value for that row, indicating that for each stimulus presented, the correct one was chosen more than any incorrect one. However, note that many of the highest variances are along the diagonal. The highest value along the diagonal is highlighted in green, and we may interpret this stimulus (large green penguin) as being the strongest classifier overall. With regard to the confusion that takes place in each row, we were mostly correct in guessing that the stimulus most confused with the presented one would be the one that varied only by color (these elements are highlighted in blue). The one exception to this is in the first row, where the two most confused elements were the smaller-sized versions of the large red penguin, in both green and red (highlighted in pink). Finally, the most confused pairs overall corroborates our previous results from the pair rates in that they are from the pairs of small images differing only by color. This can be seen in the second row, sixth column; fourth row, last column; sixth row, second column; and last row, fourth column (all highlighted in blue).

8-class mean conditional probablility matrix and standard deviations (in grey)

 

 

ERP Analysis of ICA Sources
We also plotted ERPs of the highest- and lowest-performing ICA sources for each of the two-class groupings (groupings by only one dimension). We hoped that this would provide some insight into what features of the waveform are more or less conducive to sucessful classification rates. Since the ICA components group by source rather than by location, it is hoped that more of the significant features used in the classification will concentrate into fewer ICA compenents than would be necessary in the channel representation. The ICA sources are displayed below using MATLAB EEGLAB's "topoplot" function. While these displays do render the ICA source as a linear combination of channel sources, we will not at this time draw conclusions between the appearance of the plots and the significance of the parts of the brain being highlighted.

We performed the two-class classification on each of the 124 ICA sources separately, using the data from the subject with the highest rates overall (S1). We begin with the classification by size.

 

Best and worst ICA sources for S1, grouping by size

 

We can see from the above figure that the rate from the top-performing ICA source alone approaches the classification rate acheived using all of the ICA sources, while the lowest source is around chance level. We will not attempt a detailed analysis of the peak timings and amplitudes at this time, but it is already apparent from the following figures that the worst-performing component is much noiser than the most successful sources, especially when considering that each waveform comprises 1200 trials from this subject.

 

 

The ICA components and ERPs from the color grouping prove the most interesting in our analysis. We have included the three bottom ICA components for this dimension due to the considerable differences between the waveforms for the best components.

 

Best and worst ICA sources for S2, grouping by color

 

Note first of all that this subject's two highest single ICA source rates exceed the rates obtained by both all 124 sources, and all of the clean sources. This fact, combined with the fact that the lowest rates for this grouping are in fact below chance level, may provide stong evidence that this classification algorithm could be further optimized by using only those sources whose rates are above a certain threshhold for classification.

It is notable that for the three best sources, the waveforms corresponding to the red stimuli lack the distinct positive and negative peaks present for the green stimuli. While the green stimuli's waveforms vary from source to source, those for the red stimuli prove to be fairly consistent.

The top sources for the figure grouping are near, but not in excess of, the rates achieved by using all sources. Note that this dimension shares two top-rated sources with the dimension of size. The main point of interest for these plots concerns the degree of noisiness for the bottom-rated source.