The current ERP-study investigated whether double emotional face-voice cues attract attention to their location more strongly than single emotional cues or purely neutral cues, respectively. To test, bimodal face-voice cues that varied concerning their degree of emotional content; i.e., fearful in face and voice (both fearful), fearful voice/neutral face or fearful face/neutral voice (single emotional cue), or both neutral (both neutral cue) preceded a visual target. Considering reaction times, the number of cue components related to fear (both, one, none) interacted with cue validity. This was mainly due to a pronounced validity effect in the both fearful condition. Target related ERP-effects started with a contra-versus-ipsi- lateralized P1 increase for valid versus invalid conditions which was largest for the both fearful condition. The early P3a showed a main effect of increased activity in invalidly cued compared to validly cued trials, independently of presence and quantity of emotional content. In contrast, the P3b component was, again, differentially affected by the type of cue, displaying significantly larger amplitudes after valid than after invalid cues (i.e., reversing the P3a pattern) for all cueing conditions except for the bimodal fearful condition. The former pattern of results (i.e., P3a larger after valid than after invalid cueing, P3b larger after invalid than after valid cueing) thus resembles the course of reaction times over the cue-target interval, that is, a regular validity effect after short intervals which turns into a reversed validity effect after longer periods. Viewed from this perspective, the absence of the reversal effect in the P3b for the both fearful condition would be consistent with the assumption of a reduced IOR mechanism. Finally, yet importantly, only in the both fearful condition, there was an increase of LPC for invalid compared to the valid condition, possibly indicating that evaluation of the emotional content is enhanced by a mismatch of spatial positions.

The present behavioral results indicated that emotional content interacted with spatial validity. Regarding reaction times, this interaction was mainly driven by a strong validity effect in the both fearful condition, contrasting with a descriptively slightly reversed validity effect of the neutral condition. Although the hit rate analysis did not yield any statistically significant interactions (which might not be surprising given the overall high level of accuracy), the result displayed a trend of improved performance after double fear cues for valid targets, and for invalid targets after neutral cues. Therefore, the interaction trend in the hit rates resembled the result pattern observed in reaction times. Enhancement of the cueing validity effect in the both fearful condition corresponds to previous findings using unimodal fearful cues. As previously shown, fearful facial expressions [72] as well as fear-related voice cues [74] enhanced performance differences in responding to targets presented at validly versus invalidly cued positions. Importantly, this cueing validity effect was found not only with short SOA for face cues (117 ms) but also with long SOA for voice cues (1200 ms), as long as the ISI was kept short, suggesting that the occurrence of IOR depends on a long ISI. The results of the current study seem in line with this assumption as there was no significant reversal of the validity effect—and a significant regular validity effect in the both fearful condition—despite the SOA exceeded 1000 ms.

Further, effects of bimodal cues have so far only been tested lacking emotional content [47, 63]. In the study by Santangelo et al. [63], bimodal cues evoked a statistically non-significant behavioral enhancement of target processing compared to unisensory cues. In the study of Mastrobernadino et al. [47], enhanced validity effects were observed for bimodal cueing (i.e., correspondence of the picture of a laterally presented animal face occurring alongside another animal face on the opposite side with a centrally presented sound) but these effects were confined to conditions of high task difficulty. While this may be the case for non-emotional cues, the results of our study show that this is not the case for emotional material. Our task consisted of an easy discrimination of a lateralized arrow pointing up- or downwards. According to the overall high accuracy rates of our participants (98.3%), this is a task of rather low perceptual demands. Instead, the results of the current study suggest that bimodally presented fear-related content of spatial cues may be sufficient to enhance the deployment of spatial attention to the cued position compared to bimodal non-emotional cues.

Cues of both neutral content indicated a slight non-significant tendency for an opposite cueing effect of improved performance for invalid compared to valid targets. In the current study, the sound duration was set to 1000 ms to ensure the discrimination of the auditory content. Together with the current ISI of 190–240 ms, the SOA-time was therefore 1190–1240 ms. Considering neutral sound cues of similar psychophysical structure [75], we previously found typical spatial validity effects (i.e., improved performance to valid compared to invalid targets) with an interstimulus interval (ISI) of 50–150 ms, but SOA of 1050–1150 ms due to the long sound duration. Importantly however, neutral cueing effects inverted in an IOR-like manner (invalid improved compared to valid) with an ISI of 650–750 ms, while an ISI of 350–450 ms indicated no cueing effects in neither direction [75]. In the current study, the ISI was 190–240 ms, thus outside the time-range of typical validity effects for neutral cues as found in our previous study. It would thus seem straightforward to assume that the interval administered in the current study was too short to yield reliable IOR, while, on the other hand, sufficiently long for substantial undoing of the initial attention shift towards the cued location, except for the bimodal fearful cueing condition, in which disengagement of attention from the cued location was less feasible.

Time-locked to targets, contra-versus-ispi-lateralized P1-activity indicated an enhancement of valid versus invalid activity for all conditions. Importantly however, targets after bimodal fearful cues indicated a specifically large P1-validity difference due to a massive activity increase for validly cued targets. One the one hand, the general input of P1 for all valid vs. invalid conditions fits well with a spatial bottom-up theory where early target processing is enhanced at positions preoccupied validly by spatial cues independent of awareness or SOA duration [27, 54, 72]. For example, in a spatial cueing paradigm, Giattino and colleagues [27] used lateralized cues of short 17 ms duration hidden in a multiobject-presentation. Targets consisted of neutral quadratic shapes at valid or invalid positions of the cues. Independently of whether participants consciously perceived the cues or not, validly cued targets always evoked higher P1-activity over occipital areas than invalidly cued targets. In addition, in a spatial cueing study with facial anger cues and neutral targets, Liu et al. [44] found that target-related P1-activity increased when validly versus invalidly cued. Similarly, Brosch and colleagues [8] found increased P1 for valid vs. invalid targets lateralized to side of the fearful facial expression when double facial cues with one fearful and one neutral expression were presented. In addition, P1-valdity differences were also found in crossmodal spatial cueing when an auditory anger voice cued a visual neutral target [7]. All these P1-validity differences occurred early (around 100 ms) and showed a strong responsiveness of the spatial location of the target in relation to the preceding position of the emotional cue [8, 72]. These effects might therefore mirror a bottom-up process of sensory processing [8, 72]. Our current finding supports this hypothesis by extending previous data with a specifically contra-vs-ipsilateral P1-validity difference and its enhancement by bimodally double fearful stimulation.

ERP-data time-locked to targets, P3a-activity increased for invalid versus valid double combinations independent of condition. Thus, independent of emotional presence or absence, spatial validity corresponded inversely to the P3a-activity. With spatial cueing the P3a has been so far mainly investigated with predictive designs (i.e., 75% valid vs. 25% invalid conditions), where the low probability of invalid stimuli increases their novelty character of spatial position [16, 24, 28, 29]. However, while the P3a validity differential activation might be larger in predictive cueing, it was still present in the current unpredictive cueing design (50% valid/invalid). The reason might be that even in unpredictive cuing an invalid target position still needs to be spatially updated compared to the preceding cue position, thus leading to increased P3a activation for invalid versus valid targets. Similarly, Lasaponara and colleagues [40] found increased invalid P3a activity when using only a 50% ratio of valid stimulation compared to the sum of invalid and no-cue conditions. Considering the modulation by emotional content in spatial cueing, there is often mentioned only a general P3-activity without subdivisions of P3a and P3b [21, 44, 73, 75]. These emotional ERP-studies mainly revealed P3 validity differences that increased in positive activity for invalid versus valid target stimuli when cued with fearful faces [21, 44]. Importantly, when targets were preceded by disgust sounds, such P3 effects inverted independently of the ISI-length, indicating an increased valid versus invalid activity, which corresponded to early shifts of attention away from the disgust evoking stimulus and may therefore relate to an automatic early bottom-up process of emotional context [73, 75]. As fearful like neutral stimulation supports processing of targets at the valid versus invalid position, the early spatial P3a-effects may also point to automatic early bottom-up processes.

From 325 to 375 ms, the both neutral condition and the fearful face with neutral voice condition (VN/FF) revealed validity effects with increased valid versus invalid activity. Firstly, considering the above-mentioned P3a, these P3b validity-effects were inverted in polarity as they now indicated more activation for valid than invalid targets. This might indicate a disturbance by the valid instead of invalid spatial position and fits well with the current behavioral data indicating an IOR-tendency for the fully neutral condition and VN/FF condition. Secondly, IOR-effect with neutral stimuli can be found over occipital-parietal electrodes at P3-time intervals [48, 58, 75], indicating that attention needed to be boosted at the validly instead of shifting away to the invalidly cued and IOR-conform position. Thirdly, some recent research [39, 62] suggested that a late P3-activity mirrors conscious perception. Thus, the relatively long delay of target-onset after the neutral and mixed VN/FF-cues might support an expectancy effect of target processing at the invalid position. In conclusion, the validity inversion of the late P3, respectively P3b, seems to fit well with the behavioral occurrence of a probable IOR-effect that is missing when full emotional bimodal integration is required.

At around 500 ms time-locked to targets, a positive slow wave (late positive component; LPC) showed validity differences exclusively in the bimodal both fearful combination again with an enhanced invalid versus valid activity. This finding seems to combine different views on the LPC (late positive component); a component that is sometimes thought to include the late positive potential (LPP; [12] for a review). Firstly, the LPC might be considered as a slow wave conflict potential [13, 18, 50]. For example, the LPC-activation appears in non-emotional Stroop-tasks as well as non-emotional Flanker-tasks and mirrors the conflict in semantic meaning (Stroop) of an incongruent color-word-combination relative to congruent color-word-combination respectively the perceptive conflict (Flanker) of incongruent versus congruent target-flanker combinations [1, 13, 18, 42, 70]. Secondly, the LPC/LPP is handled as an indicator of negative emotional processing [30, 36, 51, 59]. In this sense, the LPC/LPP increased over centro-parietal scalp regions for negative versus neutral emotional stimuli, when ERP-studies required a simple emotional discrimination of a centrally presented picture [30, 36, 51, 59]. In addition, a central cueing study [33] indicated validity effects in the same direction as the current study, e.g., increased LPP-activity after invalidly compared to validly cued negative emotional pictures. In the aforementioned studies, the conflicting or negative enhanced LPC was time-locked to the onset of the emotional stimulus itself. In contrast, the LPC difference in the present data was found time-locked to the presentation neutral target, importantly however when following double negative cues and indicating spatial conflict. In addition, perception of stimulus conflict might yield aversive states ([17], for a review), and fits therefore well to the just cued fearful bimodal perception. More specifically, the LPP/LPC complex is thought to be influenced by top-down processes (e.g., reinterpretation of negative stimuli) and reflecting endogenous spatial attention (cf. [12], for a review). Thus, our data might integrate and extend present LPC-theories by showing that targets, invalidly cued by fearful bimodal stimuli, are associated with enhanced spatial conflict.

Time-locked to cues, the facial related N170 component was always enhanced when the facial cue expressed fear—independent of the accompanying sound expression. Investigating bimodal effects of neutral face-voice combinations, Latinus and colleagues [41] found that the N170 did not differ between purely facial and combined facial-voice stimuli. Considering emotional influence, purely visual spatial cueing studies found that N170 activity increased for fearful versus neutral facial spatial cues independently of whether their emotional content was task-relevant [11, 69] or task-irrelevant ([15]; see also [31], for a review). In contrast, some facial studies (e.g., [8, 55]) used a double facial cue with one emotional and one neutral face. In this case, emotional enhancements at N170 might be rather hard to detect or at least weakened by necessary subtraction analysis to extract fear-related activation. In addition, asking for overt discrimination of facial emotion or gender, emotional discrimination also revealed an enhanced N170 for fearful compared to neutral expressions [3, 4, 61], but see [19] for no N170 differences). So far, bimodal emotional differences of N170 have only been investigated with matching emotional face-voice combinations (e.g., bimodal fearful or bimodal neutral), revealing that N170 was enhanced for bimodally fearful compared to bimodally neutral face-voice combinations [68]. Only an increase in the N170 amplitude for angry versus neutral faces, however independently of voice, was found. Therefore, the present finding extends the results of previous studies, by emphasizing the importance of facial expressions for the N170, and its independence of the accompanying voice content. A possible explanation might relate to the visual discrimination demands of the experimental setting, where movement of the attentional focus is dominated by visual information and an added voice might unspecifically enhance attention in general, at least at this early time of processing.

Contrasting the spatial effects of fear and neutral content of bimodal cues, this study consisted of a 4 × 2 stimulus paradigm and thus eight conditions (e.g., even 16 conditions when splitting into ipsi- and contralateral activity). To ensure a relatively high signal-to-noise ratio of the ERP-data (see [6]), we decided to confine experimental manipulations to a single ISI and a single type of emotional content. This resulted in two noteworthy caveats. First, adding a second even longer ISI might have yielded stronger evidence for our interpretation of a delay of disengagement of attention from the cued location in the both fearful condition (i.e., if consistent IOR for all conditions could be observed with a sufficiently long SOA). Second, it should be stressed that our experiment was confined to the investigation of one particular emotion (i.e., fear) and that the results may not transfer to visual-auditory cues related to other emotions. As noted in the Introduction, studies of spatial cueing featuring disgust-related cues yielded a reversed validity effect (i.e., superior performance in invalidly cued trials), consistent with a repelling or dispersing effect on spatial attention (e.g., [44, 72, 74]). Assuming that the validity effect in the bimodal fearful condition of the current study reflects a reduction in attentional disengagement counteracting IOR does not seem to allow predictions for situations in which attentional engagement is absent in the first place.