EEG-based detection of modality-specific visual and auditory sensory processing

A brain–computer interface (BCI) is a system that provides a direct communication pathway between the brain and an external device. ‘Active BCIs’ are based on consciously modulated brain activity, and allow users to intentionally control devices like computers or wheelchairs without movement. ‘Passive BCIs,’ on the other hand, derive their outputs based on brain activity that occurs without intentional, active control and that reflects the user’s current mental state [1]. The purpose of a passive BCI is to enrich a human–machine interaction. The implicit information regarding the user’s cognitive or emotional state would allow the BCI to adjust the given human–machine interaction in some useful way, in real-time. An example would be a neurofeedback system that passively monitors a driver’s brain activity in order to estimate level of fatigue in real-time and alarms the driver if a state of drowsiness is detected [2, 3].

Passive BCIs have been shown to be useful for enhancing human–machine interactions in terms of efficiency, efficacy, and usability [1]. Using data obtained from portable neuroimaging technologies like electroencephalography (EEG), passive BCIs incorporate information about the user’s state beyond what could be feasibly acquired in real-world scenarios using self-reported or behavioral data.

There has been a growing body of literature investigating the potential for passive BCI based on automatic mental workload detection, e.g. [413]. Evaluation of mental workload using neurophysiological signals would allow us to quantify the mental cost involved in performing a task, enabling the system to predict and enhance operator and system performance [14]. Developing passive BCIs for mental workload detection could have a significant industrial and economic impact by preventing operator error-related accidents. Many studies have shown the potential to classify mental workload associated with distinct levels of task difficulty/demand in various laboratory tasks (e.g. n-back task [15, 16], mental arithmetic [1719], the Sternberg memory task [4], auditory oddball target paradigms [20, 21], visual search tasks [21], Multi-Attribute Task Battery [13, 2123]) as well as in more realistic task scenarios like flight simulation [21, 2427] and driving [25, 28].

The emphasis of mental workload studies to date has been almost exclusively on detecting and predicting the level of mental workload. However, it has been suggested that determining the ‘type’ of mental workload would also be useful [29, 30]. Multiple resource theory (MRT; [3134]) provides a framework to start understanding the types of mental workload it might be useful to detect in passive BCI systems. MRT asserts that information processing capacity consists of three separate pools of attentional resources, or dimensions, each with different levels. Broadly, these dimensions can be regarded as (a) stage of processing (levels: perceptual–central or response-related), (b) code (levels: verbal or spatial), and (c) modality (levels: visual or auditory perception) [34]. Two tasks will compete more for cognitive resources to the extent that they occupy the same levels along each of the three dimensions [34]. Therefore, to avoid cognitive overload and maintain performance efficacy in multi-tasking scenarios, it is desirable that different subtasks fall, to the extent possible, into different levels within the three dimensions.

In real-world scenarios, regardless of the particular passive BCI application, users would almost certainly perform activities requiring various, divergent tasks involving different levels along all three cognitive processing dimensions. Having information regarding the specific dimension(s) and level(s) of attentional resources being used at a given time would allow the BCI to apply more suitable adaption techniques than would be possible knowing just the overall workload level. For example, if a passive BCI determines that an air-traffic controller is using a significant amount of auditory attentional resources, the BCI could adjust the system to provide some information visually rather than auditorily. Indeed, it has been shown that dual-task interference can be reduced by off-loading some information channels from the visual to the auditory modality, [35] and vice versa [36, 37].

As a first step toward developing a passive BCI capable of detecting both the type of attentional resources being used and the level of cognitive workload, this research will focus on the sensory modality dimension. The aim of this work is to develop algorithms to distinguish tasks involving visual and auditory processing based on EEG signals.

1.1. Related work and objectives of current study

Although many studies have shown the ability to classify workload levels using EEG, very few studies have focused on detecting modality-specific workload. To the best of our knowledge, there has been only one other published study investigating the single-trial classification of tasks involving primarily visual and auditory processing [29]. In this study, the authors explored 12-channel EEG and functional near-infrared spectroscopy both individually and together in a hybrid-BCI paradigm. They developed their EEG-based classifiers based on induced band power changes and the event-related potential (ERP) waveform. They showed that it was possible to differentiate an auditory task (‘audiobook listening’) from a visual task (‘silent video watching’) with 94% accuracy. The auditory task could be distinguished from an idle state with 72% and 91% accuracy (using power and ERP features, respectively), and the visual task could be distinguished from an idle state with 91% and 82% accuracy (using power and ERP features, respectively). While the results of this study are promising, some questions remain to be answered. First of all, the silent video viewing and audiobook listening tasks were quite different from one another beyond merely in terms of the type of sensory processing required and therefore, as the authors acknowledge, ‘factors like different memory load or increased need for attention management due to multiple parallel stimuli for visual trials may contribute to the separability of the classes’. To conclude with confidence that it is the visual vs. auditory processing that is underlying the task separability, the tasks would need to be identical in all ways except for the sensory modality. Secondly, in this study, each single sensory perception task contained only stimuli from the target sensory domain. It is not clear, then, if the task separability is based on the actual attention to/perception of the visual and auditory stimuli or simply due to the passive sensation of/exposure to these stimuli. In a real-world environment where an individual may be, for example, performing a primarily visual task while being exposed to task-irrelevant auditory stimuli that they are not attending to (and therefore that are not using up attentional resources), will the classifier be able to correctly classify this as a visual (not auditory) task? To conclude with confidence that task separability is due to sensory perception/attention and not merely sensation, the sensory processing tasks should ideally include passive, task-irrelevant stimuli from the opposite modality. Finally, in [29], just a single level of processing demand was included for each sensory modality. Therefore it is not clear what the effect of differing attentional requirements would be on the ability to distinguish auditory from visual processing. We aim to address some of these unknowns in this work.

The main objectives of the present study are therefore to:

(a)  

investigate if EEG can be used to distinguish between auditory and visual processing tasks, even when the tasks include task-irrelevant stimuli from the opposite sensory modality that the individual is sensing but not attending to, and

(b)  

investigate the effect of the level of sensory processing demand on the ability to distinguish between auditory and visual processing tasks under the conditions mentioned above.

2.1. Participants

Fifteen healthy adults (9 female, 6 male) with average age = 28.7 years (SD = 5.7) participated in this study. Participants were included if they had no history of neurological disease, disorder, injury, or cognitive impairment, and had normal or corrected-to-normal visual acuity and normal auditory acuity. The study was approved by the Interdisciplinary Committee on Ethics in Human Research at Memorial University of Newfoundland. The data from one male subject were excluded from the analysis due to a reported lack of concentration during the session, as well as excessive body movement which significantly affected signal quality.

2.2. EEG data acquisition

For each subject, scalp EEG signals were recorded via an ActiCHamp (Brain Products GmbH, Gilching, Germany) system using 64-channel active electrodes, referenced to FCz. The 10–10 international standard electrode placement was used [38, 39]. The impedance threshold was kept below 10 kΩ, and the EEG signals were recorded at a sampling rate of 500 Hz. Appropriate measures were taken to set up the equipment properly and educate participants on best practices for acquiring clean (e.g. high signal-to-noise ratio) data (e.g. limiting movement during recording).

2.3. Experiment

A version of this protocol was first described in [30].

2.3.1. Auditory and visual tasks

In selecting appropriate tasks, four main criteria were considered. First, the tasks had to primarily involve auditory or visual processing, and therefore other task requirements (e.g. motor, cognitive) had to be minimized (e.g. visual and auditory versions of the n-back task, which have a significant working memory component, were not considered suitable). Secondly, the auditory and visual tasks had to be as similar to one another as possible, with the only difference between tasks being the type of sensory processing involved (i.e. the visual and auditory task requirements had to be identical along the other cognitive resource dimensions). Thirdly, the auditory and visual tasks had to contain task-irrelevant stimuli from the opposite modality. Moreover, it had to be possible to modify the attentional demands of the task in terms of the level of sensory processing required (the change in demand could not be due to significantly altering other sensory-independent elements of the task, like working memory).

Since no tasks from the literature satisfied these requirements, we designed an original task. Specifically, we designed a simple monitoring task that allowed for visual and auditory processing conditions at both a high- and low-level of attentional demand. Both the visual and auditory task conditions (within a demand level) contained nearly identical visual and auditory stimuli, and the conditions differed only with respect to which type of stimuli the individual was asked to attend to.

The different task conditions are described in detail in the following sections. The experimental task was designed using the Cogent 2000 toolbox in MATLAB. Subjects completed the tasks on a desktop computer.

2.3.1.1. Active stimuli in sensory modality of interest

In each task trial, stimuli (letters A–Z from the English alphabet and numbers 0–9) were presented in random order, one after another, and the subject was instructed to attend to these ‘active’ stimuli. To ensure that participants were indeed attending to the active stimuli and processing the information, they were asked to respond when specific targets (i.e. specific letters or numbers) were presented by pressing the keyboard’s space bar. During the trial, targets were presented at a rate of $30 \pm 3$%. For Visual trials, the active stimuli were presented visually via the computer monitor as white characters in the center of a black screen. For the Auditory trials, the active stimuli were presented auditorily via speakers located on the desk in front of the subject. The target stimuli were different for each trial, and the subject was told what they would be prior to starting a trial.

2.3.1.2. Level of demand

For each sensory modality, two levels of sensory processing were induced: low demand (L) and high demand (H). The level of processing required was varied by changing: (a) the stimulus presentation speed and (b) the number of target letters/numbers. In the low-demand condition, there was just one target, and the stimuli were presented slowly. There were two targets in the high-demand condition, and the stimuli were presented more quickly. In all task trials, each stimulus was presented for a fixed interval (1500 ms for low demand, 500 ms for high demand condition), followed by a fixed inter-stimulus interval with an all-black screen (750 ms for low demand condition, 250 ms for high demand condition), followed by the next stimulus. Each trial was 30 s in duration.

2.3.1.3. Passive stimuli in opposite sensory modality

In each trial, along with the active stimuli (which were presented in the sensory modality of interest for that trial) a set of ‘passive stimuli’ were presented in the opposite sensory modality. These passive stimuli were presented to ensure that any differences we observed between the auditory and visual trials were due to the sensory processing requirements of performing the task and not merely due to passive exposure to sensory stimuli. As a result, both visual and auditory trials included both visual and auditory stimuli, and the only difference between them was the type of sensory stimuli that the subject had to pay attention to during the trial. So that they were similar to the active stimuli yet clearly unrelated to the task and thus easily ignored, the passive stimuli were characters from the Greek alphabet. These stimuli changed at the same rate as the active stimuli for a given trial. No target letters/numbers were given for the passive stimuli, and the subject was not required to respond to them in any way. Some of the Greek letters were excluded due to their similarity to English letters (e.g. alpha, A). For auditory trials, subjects were instructed to keep their eyes open and look at the passive visual stimuli on the screen, but they were told not to pay attention to or monitor them in any way. For visual trials, the passive auditory stimuli were played through the speakers the same way that the active stimuli were in the Auditory trials, but subjects were told not to pay attention to or monitor them in any way.

To summarize, there were four different single-modality task conditions: high demand auditory (AudH ), high demand visual (VisH ), low demand auditory (AudL ), and low demand visual (VisL ).

2.3.2. Baseline trials

Three types of baseline trials were also collected. One reflected a ‘true baseline’ (BLT ) condition where participants looked at a black computer screen with a constant ‘+’ symbol in the center, with eyes open. No other visual or auditory stimuli were presented.

The other two types of baseline trials contained both auditory and visual passive stimuli that participants were told not to attend to. These passive stimuli were again from the Greek alphabet and were identical to the passive stimuli used during the task trials. Participants sat facing the screen with eyes open, so they could see the visual stimuli, and they could hear the auditory stimuli played through the speakers, but they were told not to pay attention to or monitor these stimuli. Within this condition, there were high and low speed versions (BLH and BLL ), where the stimuli were presented at the same speeds as in the high demand and low demand task conditions, respectively.

To summarize, there were three different baseline conditions: true baseline (BLT ), baseline with passive stimuli changing at high speed (BLH ), and baseline with passive stimuli changing at low speed (BLL ).

2.3.3. Dual-modality trials

Four different dual-modality conditions were collected. In each case, there were both visual and auditory active stimuli, and the participant had to attend (and respond) to the stimuli from both modalities. In the first case, both the visual and auditory stimuli were at a faster speed (as in the high demand single-modality cases). In the second case, both the visual and auditory stimuli were at the slower speed (as in the low demand single-modality case). In the third and fourth cases, the auditory stimuli were at the high speed while the visual stimuli were at the lower speed, and vice versa.

To summarize, there were four different dual-modality conditions: AudH VisH , AudL VisL , AudH VisL , and AudL VisH . These conditions are not analyzed in the present study and will be left for future work.

Table 1 summarizes the types of task and baseline trials in terms of the modalities of the target and passive stimuli, and the stimulus presentation speed.

Table 1. Different trial types, including task and baseline trials. Trials are defined based on the modalities of the target and passive stimuli, and the stimulus presentation speed.

 Active stimuli (attend to)Passive stimuli (do not attend to) AuditoryVisualAuditoryVisualAudH Fast——FastAudL Slow——SlowVisH —FastFast—VisL —SlowSlow—AudH VisH FastFast——AudL VisL SlowSlow——AudH VisL FastSlow——AudL VisH SlowFast——BLH ——FastFastBLL ——SlowSlowBLT ————2.3.4. Experimental procedure

The experiment was completed in a single session of approximately 2 h duration. The protocol began with one eyes-closed and one eyes-open baseline trial (no stimuli presented), each 60 s in length, followed by a practice block for subjects to become familiar with the tasks and the experimental procedure.

The main part of the experiment consisted of four blocks. Note that the first five participants completed six blocks, but this was determined to be too tiring for the participants and subsequent participants only completed four. The final two blocks for the first five participants were excluded from analysis. Each block comprised 18 trials: two trials of each of the four single-task conditions (AudH , VisH , AudL , VisL ), one trial for each of the four dual-task conditions (AudH VisH , AudL VisL , AudH VisL , and AudL VisH ), and two trials for each of the three baseline conditions (BLH and BLL , (BLT ). Each trial was 30 s in length. The trial order, which was different for each block, was near random (we ensured that no two trials of the same condition appeared back-to-back). Subjects initiated each trial by pressing the space key and thus were able to progress through trials/blocks at their own pace, taking breaks as needed. Before pressing the space key to begin each trial, the subject was told what type of trial they were about to complete and reminded of the instructions for that trial type via text explanation presented on the computer screen. For all non-Baseline trials, the subject was then given the target stimulus/stimuli for that trial.

In order to motivate participants to get more engaged in the experiment, their response accuracy was shown at the end of each trial. The response accuracy was also used as an objective performance measure to validate the task’s demand level for the subject. Accuracy was calculated as:

Equation (1)

where:

TP (true positive) = key press when the stimulus is the targetTN (true negative) = no key press when the stimulus is not the targetFP (false positive) = key press when the stimulus is not the targetFN (false negative) = no key press when the stimulus is the target.

The participant was also asked to rate the mental effort required to do the trial using a modified version of the Rating Scale for Mental Effort (RSME) (see figure 1).

Figure 1. The modified Rating Scale for Mental Effort (RSME).

Standard image High-resolution image

The experiment concluded with one eyes-closed and one eyes-open baseline trial (no stimuli presented), each 60 s in length.

Figure 2 illustrates the timing of the experiment overall, as well as at the block and trial levels.

Figure 2. Experimental procedure. The experiment was completed in a single session. The protocol began and concluded with one eyes-closed and one eyes-open baseline trial, followed by a practice block. In the main part of the experiment, blocks were similar in terms of the arrangement of trials, and the only difference was the trials’ order. There were two trials for each single-modality condition in each block.

Standard image High-resolution image 2.4. Data analysis2.4.1. EEG pre-processing and artifact removal

The EEG data analyses were carried out in EEGLAB 2021.0 [40] with plugin tools running under Matlab 2021a (The MathWorks, Inc.). Custom code was written as necessary. After some steps in pre-processing, there were two central signal processing techniques: artifact subspace reconstruction (ASR) [41] and independent component analysis (ICA) [42, 43]. These two approaches are complementary in that ASR uses a sliding window and principal component analysis (PCA) and rejects linear components of non-stationary artifacts such as short-lasting bursts, while ICA captures stationary brain and non-brain (artefactual) source activity like blinks, eye movement, and facial and neck muscle activation by using more sophisticated, physiologically valid assumptions than PCA [44].

2.4.1.1. Pre-processing

For each subject, the whole session data were first down-sampled to 250 Hz (an anti-aliasing filter was applied). To remove potential baseline drifts and line noise, data were then bandpass filtered with a 0.5–55 Hz anti-aliasing Hamming windowed sinc FIR filter with the transition bandwidth of 1 Hz.

2.4.1.2. ASR

The ASR algorithm is explained in detail in [41] and is available as part of the open source EEGLAB plugin clean_rawdata(), which is an offline version of data cleaning suites from BCILAB [45]. EEGLAB plugin clean_rawdata() was used to de-noise the continuous channel data. The process included:

(a)  

Removing channels that were poorly correlated (r < 0.7) with adjacent channels.

(b)  

Rejecting all inter-trial intervals to remove any noisy components that may contribute to the next step.

(c)  

Removing non-stationary high-amplitude bursts. The standard deviation cut-off for removal of bursts was set to 50 to be very conservative and avoid losing potentially valuable EEG. This process improved data stationarity, which is an assumption for the later ICA.

(d)  

Interpolating any removed channels.

(e)  

Referencing data to the common average.

2.4.1.3. ICA

Adaptive mixture ICA (AMICA) [46, 47] was applied. AMICA assumes that PDF of the source activations are mixture of multiple Gaussians. The parameters used for AMICA were: number of models, 1 (i.e. single model for faster analysis); number of times to perform rejection of unlikely data based on initial samples, 15; iteration interval between rejections, 1 (recommended by Makoto’s preprocessing pipeline from sccn.ucsd.edu). All other parameters were set to default.

Subsequently, equivalent current dipoles were estimated using dipfit plugin [48] for scalp projections of the independent components (ICs). Then, ICs with bilaterally near-symmetrical projection pattern were modeled with equivalent dipoles using the fitTwoDipoles EEGLAB plugin [49]. The criteria for selecting ICs were: (a) being localized inside the brain [50]; (b) the residual variance to be less than 25%; [51] (c) ICs with their PSD follows a 1/f curve, scored via ICLabel as brain ICs [52].

The processing of EEG data, including the three above-mentioned steps (Pre-processing, ASR, ICA), was performed in a single script for each subject’s whole session data, and none of the steps were done manually, although, in the process of selecting parameters for each plugin, necessary manual inspections were done to confirm that, without losing much data, artifacts are removed as expected.

2.4.2. Feature extraction

The clean EEG data were used to calculate two sets of features to be used in the classification analysis.

2.4.2.1. Band power

We first calculated a set of signal power features. The frequency bands of interest were the standard delta (1–4 Hz), theta (4–8 Hz), alpha (8–12 Hz), beta (12–30 Hz), and gamma (30–50 Hz) bands.

For each trial, average power was calculated (in each of the five frequency bands, for each of the 63 electrodes) over 10 s windows/epochs with a sliding window of 1 s (i.e. overlapping 9 s). Power spectra were computed based on the fast Fourier transformation of the segmented data with the application of a DPSS taper and specified frequency smoothing of 0.5 Hz. The function ft_freqanalysis() in the FieldTrip toolbox [50] was used.

2.4.2.2. Coherence

Coherence was also calculated in order to capture any oscillatory coupling between neuronal oscillations at similar frequencies in different electrodes. To extract this measure, with the Fourier output from the above-mentioned ft_freqanalysis() function, we calculated the pairwise coherence at the sensor level. That is, for each of the standard frequency bands, a $63 \times 63$ matrix of coherence was calculated over 10 s epochs with a sliding window of 1 s. Thus, for each epoch, a feature set of $5\times63\times63$ was generated. In order to reduce the dimensionality of the feature set, for each electrode, the calculated coherences were averaged across all other electrodes. This resulted in 63 features for each of the five frequency bands, yielding a total of 315 features per epoch. For each task condition, if no data was removed due to artifacts, a total of 20 (epochs) × 8 (trials) = 160 samples were generated.

2.4.3. Classification

In this study, our main objectives were (a) to investigate whether visual and auditory processing tasks can be distinguished using EEG and (b) to investigate what, if any, effect the level of sensory processing demand has on the ability to distinguish the auditory and visual processing tasks using EEG.

2.4.3.1. Classification problems

We first tried to classify the auditory and visual tasks in the high demand condition (i.e. AudH vs. VisH ) and then in the low demand condition (i.e. AudL vs. VisL ). To provide further insight and help with the interpretation of our results, we also explored several other classification problems. We investigated the ability to classify auditory and visual processing tasks from an idle state, both when the idle state involves no sensory stimuli (i.e. AudH vs. BLT , VisH vs. BLT , AudL vs. BLT , VisL vs. BLT ) and when it involves exposure to passive sensory stimuli similar to the task stimuli (i.e. AudH vs. BLH , VisH vs. BLH , AudL vs. BLL , VisL vs. BLL ). The latter condition ensures that we are classifying sensory processing differences between the tasks and baselines, and not merely differences related to the sensation of stimuli.

2.4.3.2. Classification methods

Three different classifiers have been employed for each of the two calculated feature sets (band power features and coherence features). All features in each case were standardized to zero mean and unit standard deviation (z-normalization) for entering to the classifiers. The three classifiers were: (a) regularized linear discriminant analysis (r-LDA) with a shrinkage factor of 0.5, (b) linear support vector machine (SVM), and (c) a meta-classifier incorporating results from both the r-LDA and SVM classifiers. The meta-classifier uses the result from all four classifiers (r-LDA$_}$, SVM$_}$, r-LDA$_}$, SVM$_}$) and selects the output based on the classifier with the highest class probability, i.e. highest confidence in the predicted class.

2.4.3.3. Classifier performance metric

For each classification problem, 8-fold trial-wise cross-validation was used. To begin, each of the eight trials from one of the conditions to be classified were randomly paired with one of the eight trials of the other condition, resulting in eight pairs of trials (each containing one trial from each condition). Then, for each fold of the cross-validation, the epochs from the trials in one of the pairs were held out to form the test set, while the epochs from the remaining six pairs of trials were used as the training set. In the end, each pair of trials was used to form the test set just once. Because we extracted overlapping 10 s epochs from the 30 s trials of each condition, employing trial-wise cross-validation (rather than randomized k-fold cross-validation) ensures that there is no data leakage between the test and training sets. This process (starting with the random pairing of the eight trials from the two conditions) was repeated 10 times (i.e. 10 runs of 8-fold trial-wise classification were performed).

Overall classifier performance was estimated via the mean of the balanced accuracy (calculated as the average of sensitivity and specificity) over all runs and folds of the cross-validation. To improve classification accuracy by reducing incorrect predictions resulting from sudden changes in the EEG signals, a sliding window classification step was performed. Within each fold, the final predicted class for each epoch was determined by the majority vote of the output of the corresponding classifier for that epoch and the two previous epochs. This step was possible due to the trial-wise cross-validation scheme, which preserved the chronological ordering of the epochs.

2.4.4. Validation of different levels of sensory processing demand

In order to confirm that the low demand and high demand conditions of the monitoring tasks did indeed require different levels of processing, we performed Wilcoxon signed rank tests to compare the RSME ratings and response accuracies between the two conditions, within each sensory modality. The RSME ratings and response accuracies represented subjective and objective indicators of the level of demand, respectively. As an additional objective indicator that different levels of demand were induced, we also performed neurophysiological validation via classification of EEG signals. Specifically, we performed classification of AudH vs AudL and VisH vs VisL using the same classification approach described in section 2.4.3 (with power features and r-LDA classifier).

3.1. Validation of different levels of sensory processing demand

Figure 3 shows the result of RSME ratings and response accuracies averaged over all subjects. The results of Wilcoxon signed rank tests show that subjects found high demand trials significantly harder than the low demand trials in both the auditory (p < 0.001) and visual (p < 0.001) tasks. Furthermore, the subjects’ response accuracies also confirm the significant difference between high and low demand trials in both auditory (p < 0.001) and visual (p < 0.001) tasks, as anticipated.

Figure 3. Objective and subjective indicators of task demand. (a) Grand average participant trial response accuracies. (b) Grand average participant RSME ratings. Significant differences indicated with asterisk (at α = 0.05).

Standard image High-resolution image

The neurophysiological data further validates that different levels of demand were induced in the high and low conditions within each modality. Based on EEG power features and an r-LDA classifier, AudH vs AudL and VisH vs VisL could be classified with accuracies of $81.5 \pm 7.2$% and $83.4 \pm 7.6$%, respectively.

3.2. Classification results3.2.1. Auditory vs. visual processing tasks

Table 2 summarizes the classification accuracies for the discrimination of the auditory vs. visual sensory processing tasks in the high and low demand conditions (the accuracies shown are averaged across all participants). We see that for discriminating the auditory vs. visual processing tasks when the sensory processing demand is high, classification accuracies well exceeds chance (which is 50% for binary classification) were achieved for all five classifiers. The combination of power-based features with the r-LDA classifier yielded an accuracy of $77.1 \pm 9.3$%, which was almost identical to the result of the meta-classifier ($77.2 \pm 9.9$%) that incorporated the results of all four other classifiers (one-way repeated measures ANOVA with post-hoc Tukey–Kramer test; $t = -0.6, p = 1.0$). The performance of the three remaining classifiers were significantly lower than the power-based r-LDA classifier ($t \gt 3.56, p \lt 0.001$). For the latter classifier, individual participant accuracies ranged from 59% to 89%.

Table 2. Classification accuracies for auditory vs. visual monitoring trials.

 ClassifierHigh demand (AudH vs. VisH )Low demand (AudL vs. VisL )Band power featuresr-LDA77.1 ± 9.3 $49.1\pm11.3$ SVM $70.6\pm6.3$ $47.9\pm8.8$ Coherence featuresr-LDA

留言 (0)

沒有登入
gif