Introduction

Understanding the neural basis of brain functioning requires knowledge about the temporal and spatial aspects of information processing. Electrophysiology, and specifically electroencephalography (EEG), constitutes to be a major tool for studying neural basis of brain functioning. Contrary to imaging techniques, EEG provides high temporal resolution and records the electrical activity of the brain in the order of milliseconds, using electrodes positioned on the scalp (Amoozegar et al., 2013; Ebrahimzadeh et al., 2013, 2018b; Mirbagheri et al., 2019). EEG signal processing methods are applied to determine the quantitative parameters on the spectrum of frequencies, amplitudes, and coherence.

Computed tomography (CT), and mainly MRI (Nikravan et al., 2016; Ebrahimzadeh et al., 2018a), on the other hand, represents a morphological view of brain with a significant spatial resolution. It provides a multiparametric evaluation of the brain tissue with respect to both its structural and functional properties. In this context, similar to EEG but at different temporal scales (milliseconds vs. seconds), functional Near Infrared Spectroscopy (fNIRS) (Mirbagheri et al., 2019, 2020a,2020b) measures the hemodynamic response, that is the change of oxygen in the blood when a brain region becomes active and functional MRI (fMRI) (Ebrahimzadeh et al., 2019, 2019a; Sadjadi et al., 2022) offers the possibility for examining the brain functional activation non-invasively both during resting state and task execution, expanding the panel of parameters obtainable by MRI [e.g., structural connectivity evaluated by diffusion tensor imaging (DTI), metabolites concentrations evaluated by magnetic resonance spectroscopy (MRS), and perfusion valuated by arterial spin labeling (ASL)] (Mele et al., 2019). This complementarity of information is highlighted in multimodal recording methods that are designed to overcome the single modality shortcomings and to enhance the patient’s treatment experience. Both in preclinical and clinical settings (Grassi et al., 2009; Greco et al., 2013; Auletta et al., 2017; Aiello et al., 2018; Russo et al., 2018), first multimodal imaging techniques attempted to combine functional information obtained from nuclear medicine modalities (positron emission tomography—PET, and single photon emission computed tomography—SPECT) with structural data achieved by CT and MRI, were performed in order to complement diagnostic and prognostic procedures (Bruno et al., 2011). In neurology, simultaneous PET/MRI paved the way for a more comprehensive investigation of the organization and physiology of the brain and to explore, within a single integrated exam, the cerebral connectivity in terms of structural, functional, and metabolic connectome (Salmon et al., 2015; Aiello et al., 2016).

Recently, novel tools have been designed to acquire fMRI and EEG at the same time in attempts to investigate the brain function thoroughly (Mele et al., 2019). Recording these two signals simultaneously combines the optimal spatial and temporal data of both modalities which overcomes the restrictions of single modalities (Ebrahimzadeh et al., 2019b,2021a,2021b). This approach shows promising potential in providing a more thorough understanding of the underlying mechanisms of neural activities, addressing brain function and dysfunction (Mele et al., 2019).

This paper aims to review the existing literature on the integration of fMRI and EEG. We briefly discuss the complementary features of the two modalities, then move on to methods of acquiring them at the same time. We also describe the positive and negative sides of EEG-fMRI simultaneous acquisition and highlight the strategies employed to ensure their optimal combination.

We have reviewed a large strand of literature which focuses on functional neurological assessment. The articles were categorized according to their analysis method and reviewed in each part by their publication date. This helps to identify the trend of works in all the studied methods. First, we review the primary concepts and methods for analyzing EEG inside the MRI scanner presented in the corresponding studies. Then, we cast light on simultaneous EEG-fMRI, elaborating its uses, in both resting state and task completion studies, and measuring the capability of each method in cognitive assessment against other methods. Finally, we address the future perspectives. The current paper tries to strike a balance between task-based studies and those with resting state data. The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) flowchart below demonstrates the organization of the extracted articles (Figure 1). We have strived to include all EEG-fMRI studies focused on functional neurological assessment, which we found through searching for the related keywords in PubMed, Google-scholar, Research-gate, Scopus, and other sources.

FIGURE 1

Figure 1. PRISMA flowchart showing the classification of the extracted articles related to the functional neurological assessment by simultaneous EEG-fMRI recording.

(i): Web of science, Google-scholar, Pub Med, Scopus, Research-gate, and the references of the research and review articles; (ii): Not relevant to research questions aims and objectives/or old articles; (iii): Intracranial studies, wrong setting, wrong patient population, wrong intervention, etc.

Primary concepts Electroencephalography

One of the most extensively utilized procedures for studying brain electrical activity is EEG. Berger, who used radio equipment to record brain electrical activity, performed the first electroencephalograph acquisition over 50 years ago. The discovery of EEG, and therefore of brain electrical activity, opened the doors to extensive studies of brain functions, and it became a standard instrument in clinical and research settings over time (Beres, 2017). The electrical activity of the brain is generated by the synchronous activity of a pool of cortical neurons, specifically of pyramidal cells. These cells have a different electrical charge along the neuron, with dendrites being negative and the rest of the cell being positive. This difference creates an electric dipole, which EEG electrodes can detect and display as a series of positive and negative waves. However, a single pyramidal cell’s electric field is insufficient to produce a discernible EEG signal. As a result, the electrodes capture a pool of pyramidal cells that are oriented parallel to one another and produce radial and tangential dipoles (Olejniczak, 2006; Jackson and Bolger, 2014; Nikravan and Ebrahimzadeh, 2021).

The acquisition of EEG is conducted through the electrode placement on the scalp based on the international 10–20 system which considers four major reference point: inion, nasion, and the two particular points (Klem et al., 1961). The electrodes are attached to the scalp using conductive paste and record a variety of brain oscillations, including delta (0.5–4 Hz), theta (4–8 Hz), alpha (8–13 Hz), beta (13–30 Hz), and gamma (above 30 Hz) (Abreu et al., 2018; Figure 2). Furthermore, evoked potentials can be recorded during task performance, allowing researchers to analyze various neural processes (Sur and Sinha, 2009). The evoked potentials can be divided according to latency. In fact, the potentials that emerge within the first 100 ms after a stimulus are usually attributable to the stimulus nature, whereas the later components reflect the cognitive processes involved in the stimulus perception. For Quantitative EEG (QEEG) and brain connection investigations, novel technologies have led to the development of high-density EEG systems with a large number of channels/electrodes (Astolfi et al., 2007). Currently, EEG is used to characterize a variety of diseases in a clinical setting (configuration with about 20 electrodes), including metabolic or drug alternations, sleep disorders, epileptic syndromes, neurodegenerative diseases, traumatic brain injury, and tumor lesions, as well as the characterization of comatose patients and brain death (Aminoff, 2005).

FIGURE 2

Figure 2. An EEG power spectrum of a 23-year-old healthy volunteer. It presents a topographic representation of each power-band activity in detail. The data obtained from 19-channel MitsarEEG-202 device. The activations are shown in jet color map.

In a variety of physiological conditions and disorders including Alzheimer’s diseases (AD) (Briels et al., 2020), Multiple Sclerosis (MS) (Seraji et al., 2021), epilepsy (Amoozegar et al., 2013; Gholipour et al., 2022), and depression (Peng et al., 2019), EEG has been used to evaluate brain functional connectivity.

Functional magnetic resonance imaging

The fMRI modality is one of the most important non-invasive tools for evaluating brain function. The blood oxygen level dependent (BOLD) effect, which defines the variation in the magnetic state of red blood cells according to hemoglobin oxygenation, is the mechanism that subtends the fMRI signal. Indeed, deoxyhemoglobin is paramagnetic whereas oxyhemoglobin is diamagnetic. The balance between the concentrations of these two elements in the vascular system produces a signal that is distinguishable from the surrounding parenchyma in resting conditions (Mele et al., 2019).

Applying a stimulus causes the hemoglobin balance in specific areas of the brain to change, primarily in favor of deoxyhemoglobin concentration, which shows a signal decrease, and if it is in favor of oxyhemoglobin concentration, it shows an increase in the fMRI signal (Buxton, 2002).

Following the execution of specific tasks, the identification of these signal changes results in a series of images that may be studied to indicate the activations of specific brain areas. It is notable that an indirect measurement of neuronal activation is the BOLD effect, which depends on neurovascular coupling and on different interplay, such as alteration in blood flow and volume and complex interactions between the activated neurocircuitry with astrocytic and vascular targets. In the so-called tripartite synapse, neuronal activation induced by the stimulus determines neurotransmitter release in the synaptic cleft and reabsorption (recycling) by the astrocytic process (Jueptner and Weiller, 1995; Menon and Kim, 1999). The production of vasoactive peptides by secondary astrocytic activation causes intracellular Ca2+ fluctuations in astrocyte end-feet, which cause cellular biochemical and hemodynamic alterations that are observed by fMRI (Figley and Stroman, 2011).

This complex cascade of events subtends neurovascular coupling and the BOLD effect. It also controls the time delay between neuronal activation and BOLD signal fluctuation that distinguishes fMRI from direct measurement of neural activity.

While task-related fMRI has been used in many studies to investigate specific functions and/or brain areas (Ebrahimzadeh et al., 2021a), the resting-state fMRI approach is emerging to analyze spontaneous physiological fluctuations without the need for patient compliance, pathway integrity, or command following, which can be difficult in some patients (Soddu et al., 2011; Cavaliere et al., 2018a). fMRI has been applied to characterize brain functional connectivity in several physiological conditions (Aiello et al., 2015; Marchitelli et al., 2016) and many diseases, including brain tumors (Metwali et al., 2019), MS (Pareto et al., 2018), AD (Liu et al., 2008; Marchitelli et al., 2018), epilepsy (Ebrahimzadeh et al., 2018b,2019, 2021a,2021b; Omidvarnia et al., 2019; Sadjadi et al., 2021), and also psychiatric disorders (Mwansisya et al., 2017; Cavaliere et al., 2018b; Figure 3).

FIGURE 3

Figure 3. The illustration of three epileptic foci (in each row) for three different patients with refractory focal epilepsy which are localized by functional connectivity analysis on fMRI data. The patient in the top had their seizure focus localized in frontal right; the one in the middle showed epileptic activation originated in fronto-temporal left; and the one in the bottom was predicted to have their epileptic focus in parietal right. The activation regions with t-values from 3.1 to 4.5 are shown in red-yellow color map. A, Anterior; I, inferior; L, Left; P, Posterior; R, Right; S, Superior (Sadjadi et al., 2022).

Simultaneous and non-simultaneous functional magnetic resonance imaging and electroencephalography data acquisition

Research on integrated EEG and fMRI can be performed with either non-simultaneous or simultaneous data acquisition. Using the first method, the acquisition of EEG is conducted outside the MR scanner in a separate session, and the sessions are arranged randomly across the patients (George et al., 1995; Menon et al., 1997); while with the second method, fMRI and EEG are recorded inside the MR-scanner (Ebrahimzadeh et al., 2021b; Sadjadi et al., 2021).

Non-simultaneous electroencephalography-functional magnetic resonance imaging acquisition

Many research projects have integrated EEG and fMRI through data using that is acquired separately in different sessions (Heinze et al., 1994; George et al., 1995; Snyder et al., 1995; Menon et al., 1997; Ball et al., 1999; Opitz et al., 1999). A benefit of separate recording EEG and fMRI is that the signal-to-noise ratio of EEG acquired outside the MR scanner is usually much better than that of the EEG acquired inside the scanner (Figure 4).

FIGURE 4

Figure 4. “Primary sensorimotor area activation. (a) Rendering of the segmented head of subject 3, view from left and above. The segmented left and right central sulci are shown in gray. Current density map (maximum in yellow, minimum in black, linear scale shown on the left) at EMG onset is shown embedded in the section of the segmented head. fMRI-activated volume is shown in turquoise. The lateral maximum of cortical current density corresponds to the fMRI activation of the hand area of the left primary sensorimotor cortex and the medial current density maximum corresponds to the fMRI activation of the frontal medial wall motor areas. (b) Magnification of the hand area of the left primary sensorimotor area as shown in (a). The individual current vectors of the CCD source are displayed, whereas in the current density map in (a) only the absolute values of the currents are visualized. In their sum they are equivalent to the current density source at EMG onset shown in (a). The currents point in an anterior direction, consistent with activation of MI in the anterior bank of the central sulcus (Kristeva-Feige et al., 1994). (c) Current vectors in the same area as in (b) are shown 70 ms after EMG onset. The currents point in a posterior direction, consistent with activation of SI in the posterior bank of the central sulcus (Kristeva-Feige et al., 1994).”(Ball et al., 1999)—reprinted from (Kristeva-Feige et al., 1994) with permission.

Although data are recorded in two separate sessions, differences between the sessions can be minimal, especially those involving ERPs. This is certainly the case for several standard ERP paradigms, such as the oddball (“P300”) and the semantic mismatch (“N400”), in both of which ERPs may, in fact, be better recorded outside the scanner. Techniques for fMRI and EEG acquisition follow standard protocols and therefore, are not mentioned further here. Acquiring EEG and fMRI non-simultaneously has a main drawback (Snyder et al., 1995; Menon et al., 1997). The data of single trial fMRI and EEG cannot be integrated to investigate emergent responses of brain which may not be time locked to the response or stimulus (Heinze et al., 1994).

There also may be major empirical and subjective differences for ERPs between the two separate sessions. For example, the subjects’ level of vigilance, attention, familiarity with the task, and motivation may be different. The data extracted from the simultaneous EEG-fMRI demonstrated that arousal levels are a significant determinant of brain activation over the cognitive tasks (George et al., 1995; Ball et al., 1999).

Moreover, it should be considered that the environment of the two separate sessions are not the same. Unlike fMRI that is acquired in a noisy room, EEG is recorded in a quiet and comfortable environment. This is a notable issue for integrated fMRI and EEG investigations of auditory processing (Snyder et al., 1995).

In the past, when it was not easily possible to have a simultaneous record, most studies would use interleaved EEG–fMRI acquisition protocol, in which EEG data was continuously collected and fMRI data was intermittently acquired. Following stimulus presentation, 1–2 s of EEG data was collected without fMRI scanning (the MRI scanner was silent throughout this time interval), followed by a few seconds of fMRI data collection. This is analogous to the “clustering” procedure employed recording fMRI data in auditory experiments (Hall et al., 1999), where auditory stimuli must be heard without interference from scanner noise. Because the neurophysiological response begins a few milliseconds after the stimulus initiation and lasts no more than 1–2 s, depending on the cognitive processes involved in the task, the majority of the valuable stimulus-related EEG is not contaminated by scanner noise.

Simultaneous electroencephalography-functional magnetic resonance imaging

The correlation between hemodynamic mutation and electrical brain activity is evaluated using EEG-fMRI data. Due to the slow BOLD response (in the order of seconds), fMRI with high spatial resolution does not provide enough temporal sampling, but EEG does (in the order of milliseconds), albeit with poor signal source localization (Iannaccone et al., 2015; Ebrahimzadeh et al., 2018b).

Among the available imaging modalities, analysis of fMRI and EEG data and the simultaneous recording is considered to be the cutting-edge multimodal imaging method and has received considerable attention (Herrmann and Debener, 2008; Huster et al., 2012; Manganas and Bourbakis, 2017; Mele et al., 2019). This notion can be substantiated by the large number of related scientific papers published in this field since 2000, as illustrated in Figure 5.

FIGURE 5

Figure 5. Number of papers published per year on simultaneous EEG-fMRI. These data came from Pubmed site by searching for “Simultaneous EEG-fMRI” in Title/Abstract.

Through the integration of these two techniques in a hybrid simultaneous acquisition, it would be possible to overcome the limitations of each modality. It also increases the plethora of analyses that can be conducted, and in turns, of the data that can be obtained (Marawar et al., 2017). Simultaneous acquisition also guarantees an identical registration, as regards the mental state of the subject, the execution of the task and the inference of the recording environment (Fleury et al., 2019).

Acquisition of EEG-fMRI simultaneously, also allows for an identical registration, with respect to the mental state of the subject, the task execution and the acquisition environment inference. This does not happen by recording the two methods separately, especially if the recording takes place in different environments from cognitively unstable patients (Debener et al., 2006).

Unlike simultaneous acquisition, recording the two modalities separately does not provide identical registration, especially if the acquisition takes place in different environments and involving patients with cognitive instability. As for technological issues, simultaneous EEG/fMRI acquisition involves the use of specialized EEG hardware that is safe and compatible with the MRI environment and comfortable for the subject. Using the equipment improperly chosen may result in considerable risks. Concerning safety, there is a potential risk for the subjects coming from electrodes and heating of conducting leads during MR radio frequency transmission, resulting in discomfort or even burns (Kugel et al., 2003). There are several precautions in order to reduce the risk of participant discomfort or injuries, for instance, fMRI sequences should be based on gradient echo-echo planar imaging (GE-EPI); for anatomical reference scans, low specific absorption rate (SAR) sequences, particularly GE-T1 weighted sequences, should be used; and for all sequences in the EEG-fMRI protocol, the SAR should not exceed the SAR of the GE-EPI sequence. Otherwise, it is necessary to perform extensive safety testing with temperature sensors. Staff performing EEG-MRI studies must have received appropriate training, as injuries due to MR-compatible EEG equipment cannot be ruled out if the equipment is accidentally used out of specifications, especially in the case of body coil transmission (Nöth et al., 2012). The adoption of these guidelines is especially important in vigilance-reduced subjects (sleeping or sedated subjects) or, generally, in subjects who cannot give notice of any discomfort reliably, such as children. When employing EEG-fMRI, it is critical to ensure that the participants have a thorough comprehension of all processes involved, that they are comfortable with all steps, and that there are no accidents that could cause discomfort, resulting in movement and, consequently, failure of the experiment (Cavaliere et al., 2018a). Participants should be aware that nothing will be painful, even if some stages may be mildly unpleasant, such as minor scalp abrasion during EEG electrodes setup; this improves the participant’s calmness which is necessary for the experiment to be completed successfully and securely.

Artifacts greatly influence the data obtained from simultaneous EEG-fMRI (Scrivener and Reader, 2022). On one hand, wearing the helmet causes a variation in the magnetic field’s homogeneity, resulting in a variation in image quality; on the other hand, the magnetic field itself causes broad-band artifacts that almost entirely cover the electroencephalographic signal (Steyrl and Müller-Putz, 2019). Another artifact that is observed on the EEG recorded inside the scanner is the ballistocardiogram (BCG) artifact originated from electrodes movement caused by pulse related in the static magnetic field that is influenced by the spatio-temporal variability of cardiac cycles during recording (Figure 6; Marino et al., 2018).

FIGURE 6

Figure 6. A sample of EEG signals recorded inside the scanner: (Left) primary signal; (Middle) after eliminating the MR gradient artifact; (Right) after removing the BCG artifact (Ebrahimzadeh et al., 2021b).

Researchers have devised a variety of artifact removal techniques, including independent component analysis (ICA), which is widely regarded as the best method for removing the BCG artifact (Srivastava et al., 2005). In order to cover studies with simultaneous acquisitions of EEG-fMRI both during task performance and in resting state, we have collected scientific literature published since 2014 on the PubMed website using the key word “Simultaneous EEG-fMRI” (Figure 7). In Figure 7, the cognitive functioning refers to multiple mental abilities, including learning, thinking, reasoning, remembering, problem solving, decision making, and attention.

FIGURE 7

Figure 7. A bar chart illustrating the proportion of simultaneous EEG-fMRI studies in relation to neuropsychological impairments compared with each other in a way that shows the contribution of each neurological disorder in EEG-fMRI studies.

Simultaneous resting-state electroencephalography-functional magnetic resonance imaging

Even while in a state of rest, the brain is a dynamic mechanism that generates activity. This can be demonstrated by fMRI through the estimation of different resting state networks linked to specific cerebral functions and by EEG recording through the detection of varied frequency and amplitude neural waves. The simultaneous recording of rsfMRI and EEG allows researchers to think of the brain as a collection of interconnected networks or systems (Deligianni et al., 2014; Tsuchimoto et al., 2017). The interactions are influenced by the brain’s electrical activities as well as the concurrent variation of BOLD fluctuations. The simultaneous recording of EEG and rsfMRI data has a wide range of applications. Table 1 refers to a summary of the resting state EEG-fMRI studies since 2014. Seven different application areas are presented to cover an acceptable range.

TABLE 1

Table 1. A summary of the resting state EEG-fMRI studies since 2014.

The first research looked at methodological concerns in healthy people, investigating the reconstruction of EEG signal sources using fMRI data and primarily focusing on connectivity analyses. However, in order for the theory to be validated, the study sample is necessary (Deligianni et al., 2014). In seventeen adult volunteers, the authors demonstrated that a simultaneous approach using a 64 channel MRI-compatible EEG cap was useful in validating whole-brain connectomes generated by each method and developing a prediction model of dynamic functional connectivity (Deligianni et al., 2014).

Another study (Marawar et al., 2017) correlated temporal lobe theta and delta EEG frequencies and simultaneous fMRI acquisition in fourteen healthy sleep-deprived adults in drowsy and awake states. The recent study discovered a new brain regional source for theta and delta frequencies for the first time, while they also looked at the fastest frequencies like alpha, beta, and gamma (Marawar et al., 2017). This type of approach decreases the localization of the sources generating different EEG bands but has an advantage of producing a greater differentiation of the slow frequencies. The electrical BOLD correlation appeared to be larger for frequencies below 1 Hz, and is influenced by the spatial relationship between the investigated resting state networks and recording zones (Keinänen et al., 2018).

The basis of some specific electrical oscillations, such as the mu frequency, has also been studied using this relationship. EEG-fMRI was used to identify a positive correlation between the strength of mu frequency and the BOLD signal in regions such as the anterior insula and the anterior cingulate cortex in a research of 36 healthy people, confirming the numerous origins of this specific frequency (Yin et al., 2016). In terms of neurological illnesses, a study on 18 individuals with juvenile myoclonic epilepsy demonstrated the utility of EEG-fMRI acquisition in revealing the disease’s pathogenesis, with a focus on the relationship between frontal networks and epileptic discharges (Dong et al., 2016). Authors of Brueggen et al. (2017) found a lower correlation between occipital alpha band power and BOLD signal fluctuation in the frontal and temporal cortices, as well as the thalami, in fourteen Alzheimer’s patients. Other researchers in psychiatry discovered a connection between the temporal dynamics of the default mode network and the severity of post-traumatic stress disorder (PTSD) in 36 veterans compared to twenty combat-exposed controls (Yuan et al., 2018). It has become clear that simultaneous EEG-fMRI can provide significant information on the relationships between neuronal activity and BOLD response, and that the resting state acquisition, in particular, can be crucial for understanding the variability of brain activity and, more importantly, for identifying the structures involved in triggering EEG waves in the resting state (Figure 8). In this context, expanding the sample size and using different methods of analysis can help to corroborate prior findings and disentangle controversial or inconsistent findings.

FIGURE 8

Figure 8. Simultaneous EEG-fMRI analysis on a patient with focal epilepsy in order to localize the seizure onset zone (SOZ). (A) Simultaneously recorded raw EEG data. (B) The EEG data after artifact removal. (C) The epilepsy-related component time series obtained using Independent Component Analysis (ICA) algorithm. (D) The 3D map of the candidate component. The active area is marked by yellow-red color. (E) The result of dipole localization of the identified generator in deep brain structures. (F) The final result of localizing the SOZ applying simultaneous analysis of EEG–fMRI (Ebrahimzadeh et al., 2019, 2019a).

Simultaneous task electroencephalography-functional magnetic resonance imaging

Researchers can determine which cerebral areas are assigned to specific activities based on the cognitive domain investigated by executing tasks. Simultaneous EEG-fMRI tests using a recognition memory task have shown that theta-alpha low frequency oscillations (4–13 Hz) are connected with the functional activation of a network involving the striatum, prefrontal cortex, and hippocampus. This finding is backed with the hypothesis that the hippocampus works as a brain activity modulator via low frequency oscillations (Herweg et al., 2016). The hippocampus appears to play an important role during sleep as well. In fact, studies have shown that hippocampal activity rises during light sleep and correlates with alpha activity (Knaut et al., 2019). This observation could support the hypothesis that memory consolidation can occur during light sleep stages (Knaut et al., 2019). A simultaneous method has been used to explore neural foundations for perceptual decisions and evidence collection in decision making evaluation, indicating a common involvement for the posterior medial frontal cortex in both processes (Figure 9; Pisauro et al., 2017). The execution of tasks allows to establish, according to the cognitive domain studied, which cerebral areas are assigned to each specific task (Table 2). Since there is a large literature in this field, we have clustered the articles based on different tasks and from each cluster we have displayed one representative published in recent years in Table 2.

FIGURE 9

Figure 9. “EEG-informed fMRI and connectivity analyses in the value-based task. (A) The fMRI GLM model included an EEG-informed regressor capturing the electrophysiological trial-by-trial dynamics of the process of evidence accumulation (EA) in each participant. Three actual single-trial EEG traces are shown. The traces cover the entire trial excluding the time intervals accounting for stimulus processing and motor execution (see inset and Methods for details). To absorb the variance associated with other task-related processes we included three additional regressors: VSTIM—an unmodulated stick function regressor at the onset of the stimuli, VD—a stick function regressor at the onset of stimuli that was parametrically modulated by the value difference between the decision alternatives and RT—a stick function regressor aligned at the time of response and modulated by RT. (B) Hypothetical EA traces in response-locked EEG activity ramping up with different accumulation rates. Convolving these traces with a hemodynamic response function (HRF) leads to higher predicted fMRI activity for longer compared to shorter integration times (that is, the predicted fMRI response scales with the area under each EA trace). (C) The EEG-informed fMRI predictor of the process of EA revealed an activation in pMFC. (D) PPI analysis using the pMFC cluster identified in c as a seed revealed an inverse coupling with a region of the vmPFC and the STR. All activations represent mixed-effects and are rendered on the standard MNI brain at |Z| 42.57, cluster-corrected using a resampling procedure (minimum cluster size, 76 voxels)” (Pisauro et al., 2017).

TABLE 2

Table 2. A summary of the task EEG-fMRI studies since 2014.

It has been demonstrated that an increase in theta band power associated with a choice with negative feedback corresponds to the activation of fronto-parietal areas; on the other hand, an increase in the power of the beta band associated with positive feedback corresponds to the activation of subcortical areas involved in the reward network (Andreou et al., 2017). Using a gambling task paradigm, concurrent activation of significant areas connected to reward and punishment, such as the ventral stratum, posterior cingulate, and medial prefrontal cortex, has been shown (Guo et al., 2017).

Neurofeedback is another use of EEG-fMRI, which allows for the modulation of brain processes, despite the fact that the information returned to the patient so far has only been EEG (Zich et al., 2015) or fMRI related (Guo et al., 2017; Zotev et al., 2018a,b). Several scientists have studied brain activation during movement execution and motion imaginations in healthy volunteers using EEG neurofeedback, suggesting that this approach could be useful in the rehabilitation of patients with post-stroke paralysis (Zich et al., 2015). In terms of fMRI neurofeedback, two studies have looked into the relationship between the BOLD signal and the EEG signal after behavioral modulation. The first study found that modulating thalamic nuclei activation during retrieval of pleasant autobiographical memories can affect both the alpha frequency and the BOLD signal in a group of 34 healthy people (Zotev et al., 2018a). The second study, which was conducted by the same researchers in a group of PTSD patients, found that emotional control training improved alpha frequency and functional connectivity between the prefrontal cortex and the amygdala, and that this improvement was correlated to improved clinical performance (Zotev et al., 2018b). Only one publication has so far described the use of a novel simultaneous real-time fMRI and EEG neurofeedback system (Zotev et al., 2014). The authors demonstrated that emotional self-regulation training based on recall of joyful autobiographical experiences could modulate both BOLD fMRI activation of the amygdala and beta band EEG power asymmetry in healthy subjects (Zotev et al., 2014).

To summarize, task EEG-fMRI provides significant evidence for cognitive and affective processes. This is an excellent place to start learning about and discovering everything there is to know about mental and neurological diseases that are still a mystery. Despite the fact that the multimodal approach identifies several issues that can complicate the research process, simultaneous EEG-fMRI acquisition remains one of the most popular approaches, as it allows for a comprehensive view of brain activity while minimizing the impact on the subjects involved in the study.

Clinical applications in integrating functional magnetic resonance imaging and electroencephalography

There is growing interest in combining single trial, or non-phase-locked, EEG with fMRI to evaluate the clinical applications. Integrating EEG and fMRI or simultaneous EEG-fMRI recording makes it possible to look at a phenomenon from different perspectives (Mulert and Lemieux, 2009). Each of these approaches represents some of the hidden dimensions of reality in clinical studies (Moeller et al., 2020). These signals are related to perceptual and cognitive processing, but they may or may not be locked to an external stimulus. Combined EEG–fMRI methods have been used to identify the neural correlates of clinically or behaviorally important spontaneous EEG activity, such as interictal spikes (Sadjadi et al., 2022), the alpha rhythm, and sleep waves. The area where great progress has been made is epileptic-seizure localization. The common method of integrating EEG and fMRI data here is to transform EEG data into a physiologically meaningful covariate to be used in a voxel-based general linear model. The EEG-fMRI combined method improves and contributes to the literature through substantially increasing the unimodal studies yield to localize the epileptic foci in a non-invasive manner, enabling a more thorough and reliable evaluation prior to surgery, especially in patients with refractory epilepsy (Ebrahimzadeh et al., 2018b,2019, 2019a,2021b), or placement of intracranial electrodes (Tehrani et al., 2021). This allows for providing more patients with the option of surgery while increasing the likelihood of a successful and life-improving operation (Ebrahimzadeh et al., 2019b).

Epileptic focus localization

The ability to record EEG during fMRI has opened up new possibilities for epilepsy research. Until recently, the earliest uses of single-trial EEG and fMRI were limited to seizure localization. On scalp EEG, interictal discharges underlying epileptogenic neuronal activity can be easily seen; nevertheless, the source of this activity can only be inferred in terms of lobes and hemispheres. The epileptic focus localization is relatively different from other EEG–fMRI applications since each patient’s epileptogenic activity is distributed differently spatially and temporally, and signatures of this activity must be determined with a high degree of precision and efficiency (Beers et al., 2015; Ebrahimzadeh et al., 2019, 2019a,2019b,2021b).

Several groups have employed simultaneous EEG and fMRI measurements to study interictal activity in individuals with epilepsy in recent years (Krakow et al., 1999; Lemieux et al., 2001; Aghakhani et al., 2004; Federico et al., 2005; Pittau et al., 2012; An et al., 2015). Although early research (Krakow et al., 1999; Pittau et al., 2012; An et al., 2015) employed EEG spikes to trigger fMRI acquisition, it is now typical to use continuous EEG and fMRI data (Rosenkranz and Lemieux, 2010). In either case, the comparatively large amplitude of epileptogenic activity, which is detectable on scalp EEG at around 100 μV, has aided these studies. This is as opposed to perceptual and cognitive ERPs, which have amplitudes of about 5–10 μV after averaging over several trials; although in some people with epilepsy, the events are in fact infrequent and of short duration [from a fraction of a second to (rarely) more than 10 s] (Bagshaw et al., 2006). In these cases, studying interictal epileptiform episodes is extremely difficult. The localization of task-correlated language and memory function, as well as the localization of ictal and paroxysmal phenomena, are two major applications of fMRI in epilepsy (Detre, 2004). For example, research from a number of labs has revealed that fMRI language lateralization produces results that are equivalent to intracarotid amobarbital testing (Waites et al., 2005). Recent research has also revealed that generalized spike and wave (GSW) or polyspike and wave bursts in patients with idiopathic generalized epilepsy are caused by extensive regions of the cerebral cortex and thalamus (Aghakhani et al., 2004). Interestingly, both activation and deactivation were observed in relation to the GSW; in the thalamus, activation predominated over deactivation, whereas in the cerebral cortex, the converse was observed. Most human patients have a thalamic BOLD response, which matches the thalamic involvement reported in animal models. These findings further assure the practicality of the combined EEG-fMRI approach for better understanding of the brain networks underlying various types of epilepsy (Lemieux, 2004).

Advantages and other clinical applications

The fundamental benefit of gathering EEG and fMRI data in the scanner is that they both reflect the same neural processes. Simultaneous acquisition ensures that people employ the same strategy for both types of input; this is especially crucial for activities that require complicated cognitive processing. Simultaneous EEG and fMRI are clearly helpful for epileptic seizure localization (Acharjee et al., 2015; Ebrahimzadeh et al., 2019, 2019a,2019b; Ebrahimzadeh et al., 2021b). Other essential clinical concerns necessitate simultaneous EEG and fMRI, particularly in cases where symptoms fluctuate over short periods of time. During hallucinating periods, for example, some people with schizophrenia have difficulty separating self-produced percepts from externally generated ones, which might wax and wane unpredictably (Scrivener, 2021).

Simultaneous EEG and fMRI recording is becoming increasingly helpful outside of the therapeutic realm. It aids studies of single-trial EEG and fMRI in which researchers want to better understand neural processing that is not always time bound to external events (Makeig et al., 2002; Ahmad et al., 2015). Simultaneous EEG and fMRI recordings are widely used in studies of the resting state and attempts to better understand brain dynamics underpinning intrinsic EEG rhythms (Goldman et al., 2002) and the default mode of brain operation (Laufs et al., 2003a,b, 2006; Raichle and Snyder, 2007).

Simultaneous acquisition is preferred in clinical and developmental studies since it reduces the overall amount of time required to collect data. It is also required in studies where continuous EEG tracks distinct stages of sleep (e.g., REM/non-REM) (Czisch et al., 2004). Simultaneous recordings may be important for practical reasons (Bridwell and Calhoun, 2019). Non-simultaneous recordings may not be feasible in clinical studies, as well as studies of youngsters and the elderly. The majority of these individuals are unwilling to participate in multiple sessions over long periods of time—we’ve seen this across a variety of subject groups. Furthermore, conducting trials across numerous sessions may not be possible, reliable, or practical in clinical investigations involving medicine. Thus, while normative studies on college-aged individuals can readily be completed in two or more sessions, clinical and developmental studies are far more challenging. Memory and learning paradigms can include extended stimulus exposure, which can interfere with encoding and retrieval, even in normal healthy persons, particularly in children.

This is all why several brain-imaging facilities have developed and refined approaches for simultaneous acquisition of EEG and fMRI data (Bonmassar et al., 1999, 2001b; Krakow et al., 1999; Lemieux et al., 2001; Laufs et al., 2003b; Salek-Haddadi et al., 2003; Yu et al., 2016) and used them to detect EEG spikes, define resting-state EEG, and examine the neurological basis of ERPs (Bonmassar et al., 1999, 2001a,2001b,2002; Lazeyras et al., 2001; Christmann et al., 2002; Liebenthal et al., 2003; Thees et al., 2003; Mulert et al., 2004; Nagai et al., 2004; Mulert and Lemieux, 2009; Mulert, 2022) over the last years.

Characterization of structure and dynamics of brain networks

Oscillation patterns in distinct frequency bands carry out functional coupling in the brain, which may be directly measured using methods like EEG and magnetoencephalography (MEG) (Buzsáki and Watson, 2022). For example, in the visual domain (Fisch et al., 2009) or for cognitive processes (Herrmann et al., 2010), high-frequency oscillations in the gamma band (> 30 Hz) play a significant role in the coupling of neuronal ensembles during perceptual processing. Other frequency bands, such as theta oscillations (4–8 Hz), have been found to be crucial for hippocampus-prefrontal cortex communication (Colgin, 2011; Fujisawa and Buzsáki, 2011). Improved understanding of the causes of disrupted connectivity in the brains of mental patients could lead to new therapeutic strategies: disrupted gamma band oscillations, for example, are a typical finding in schizophrenia (Spencer et al., 2003; Leicht et al., 2010).

The intact function of the N-Methyl-D-aspartate (NMDA) receptor is required for gamma-band oscillations (Carlen et al., 2012). Improvement of glutamatergic neurotransmission at the NMDA receptor has been proposed as a viable new therapy method for schizophrenia patients (Javitt, 2012). EEG-fMRI can be utilized to describe network topology and relevant coupling mechanisms such as gamma oscillations during cognitive processing (Figure 10; Mulert et al., 2010). As a result, it is crucial to characterize both the network topology and the neurophysiological mechanisms involved, and simultaneous EEG-fMRI provides the methodological foundation for this (Debener et al., 2006; Mulert and Lemieux, 2009).

FIGURE 10

Figure 10. “EEG-informed fMRI analysis in an auditory choice reaction task.” (A) EEG single-trial data of a typical data set at electrode Cz. In many trials, a GBR between 30 and 100 ms post-stimulus is present. However, the amplitudes of the GBR are variable over time. This variability can be used for specific predictions of the related BOLD signal. In the lower part of the figure, the corresponding averaged GBR is shown. (B) GBR-specific BOLD activation based on the single-trial coupling of GBR amplitude variation and the corresponding BOLD activation. Activations can be seen in the left auditory cortex (gyrus temporalis superior, Brodmann area 41/22, highest t-value 8.3), the thalamus (highest t-value 7.6) and the anterior cingulate cortex (Brodmann area 24, highest t-value 10.1). In the lower right corner, a glass brain view is provided, demonstrating a 3D view of the three abovementioned clusters. Activations are shown at P. (Mulert et al., 2010).

Brain oscillations and networks

Several intrinsic brain networks, such as the default-mode network (DMN), the dorsal attention network (DAN), and the salience network, have been found based on the spatial patterns of correlated time series that are reliably recognized in resting state BOLD signals (SN). In time periods of seconds to minutes, brain areas within these networks demonstrate increasing functional connectivity. Several neuropsychiatric diseases, such as Alzheimer’s disease, are associated with changes in resting state networks (Franciotti et al., 2013).

Early resting-state EEG-fMRI research attempted to identify BOLD correlations of certain frequency patterns like alpha oscillations, which will further be investigated in next section (Goldman et al., 2002; Laufs et al., 2003a; Moosmann et al., 2003). However, in order to define network dynamics, it was proposed that the EEG signal be linked to functional connectivity within and between networks. Hlinka et al. (2010) for example, found that delta and beta oscillations account for 70% of the DMN variance in functional connectivity.

When alpha power increases, BOLD connectivity between the primary visual cortex and occipital regions decreases, as does negative coupling between visual areas and DMN regions, according to Scheeringa et al. (2012).

Chang et al. (2013) looked at the DMN, DAN, and SN’s functional connectivity. The connectivity between DMN and DAN was discovered to be inversely related to alpha power. Furthermore, the spatial extent of anticorrelation between DMN and DAN was related to alpha power.

While these studies were conducted with the goal of connecting known fMRI resting state networks and investigating the relationship between EEG power of different frequency bands, another method is to link fMRI patterns with more sophisticated EEG organization patterns. The topographic representation of the EEG, for example, remains consistent during durations of roughly 100 ms. The term “microstates” was used to describe these quasi-stable, one-of-a-kind distributions (Lehmann et al., 1987).

Microstates are the sum of concurrent neuronal activity throughout brain areas, rather than activity restricted to a single frequency range. Microstate changes have been observed in a variety of psychiatric illnesses, including schizophrenia (Lehmann et al., 2005). Several publications have recently described the link between EEG microstates and BOLD resting-state networks using simultaneous EEG-fMRI (Huang et al., 1996; Britz et al., 2010; Musso et al., 2010). The study of the relationship between EEG coherence patterns and fMRI connectivity will be another fascinating approach. The link between interareal BOLD correlations and brain oscillations was recently studied in monkeys using the combination of fMRI and invasive electrophysiological measures (Wang et al., 2012). Although gamma oscillations (30–100 Hz) have previously been demonstrated to be notably firmly connected to the BOLD signal, coherence in low frequencies (20 Hz) contributed primarily to fMRI connectivity patterns in the task-free state. Simultaneous EEG-fMRI and EEG source analysis in humans may be used in the future to do comparable analyses incorporating EEG coherence analyses with fMRI connectivity analyses.

EEG-fMRI is now commonly conducted in many MRI centers after several years of development, and both safety and signal quality issues are well addressed (Mulert, 2022). As mentioned earlier the fact that both EEG and fMRI have overlapping sensitivity to synaptic processing means that brain function can be measured using simultaneous EEG-fMRI with high temporal and spatial resolution. The characterization of brain network structure and dynamics is one of the most promising applications for EEG-fMRI nowadays.

Relationships between alpha activity and functional magnetic resonance imaging response

EEG-fMRI studies primarily focused on fMRI responses in relation to EEG alpha band (e.g., 8–12 Hz). This was because alpha activity, which can be detected by an untrained experimenter in unprocessed EEG, had continued to appear in individual recordings. The presence of alpha activity is especially important in concurrent EEG-fMRI as the scanner environment can cause significant artifacts in EEG (for review see Moosmann et al., 2003; Thees et al., 2003). The origins and characteristics of the EEG alpha band attracted a lot of research attention over the decades. It was therefore reasonable to place the prime focus of EEG-fMRI studies on alpha activity (Bridwell and Calhoun, 2019).

Alpha oscillations tend to appear at occipital electrodes and show an increase once individuals close their eyes, are sleepy, or engage in mental arithmetic (Klimesch et al., 2007). Since such tasks involve a lesser degree of visual cortical activity, increased occipital alpha activity is associated with cortical inactivity. This inactivity limits the ability of visual areas to affect areas of the brain that are responsible for current cognitions or tasks. For instance, increases in alpha correspond to decreased resting-state connectivity between early visual areas and the rest of the brain (Scheeringa et al., 2012). Increased visual inactivity is also interpreted as increased synchrony across visual areas, increased dependence across areas, and an overall reduction in visual complexity (Patel et al., 2015). These processes could also indicate reduced cortical metabolism. Since BOLD fMRI is sensitive to metabolic processes, this theory has been put to a test: EEG-fMRI studies have shown negative relationships between alpha activity and occipital, parietal, temporal, and fontal fMRI responses, and positive correlations between alpha and the thalamus (Goldman et al., 2002; Liston et al., 2006; Bridwell and Calhoun, 2019). The negative relationship is in line with the idea that increased alpha activity mirrors reduced cortical metabolism and a subsequent reduction in the BOLD fMRI response. Also, increased metabolism reflects increased fMRI responses and a reduction in alpha. Moosmann et al. (2003) went on to provide more proof by finding a negative relationship between changes in deoxy hemoglobin [measured by near infrared spectroscopy (NIRS)] and alpha EEG. The majority of studies also confirm a negative correlation between fMRI and EEG alpha activity; therefore, it is regarded as one of the reliable and consistent results in the literature. It can serve as a useful “sanity check” in EEG-fMRI.

Electroencephalography-functional magnetic resonance imaging analysis methods

For simultaneous multimodal acquisitions and especially EEG-fMRI, the way to analyze the data plays a significant role in order to get valuable results. Two macro-areas can group the variety of analyses used in the literature: symmetric analyses and integrated analyses (Ahmad et al., 2016). Overall, the symmetrical approach is concerned with analyzing the outcome of two modalities at the same time, whereas the integrated analysis uses the data obtained from one of the modalities to validate the data acquired from the other one. Accordingly, generating a unique model to facilitate the investigation of the brain activity is not impossible (Huster et al., 2012). The integrated analysis can be done in one of these two practical ways: fMRI-informed EEG analysis (Lei et al., 2015) and EEG-informed fMRI analysis (Andreou et al., 2017). In the first one, the activation areas on fMRI data are used as a base to analyze and correct the EEG sources (Babiloni et al., 2002), while in the second one, the electrical activities and EEG parameters are used to predict the hemodynamic variations in the fMRI data (Figure 11; Abreu et al., 2018). Nevertheless, there is still an open issue for choosing the optimal way to analyze the simultaneous EEG-fMRI data. Revealing actually meaningful quantitative biomarkers to use for characterizing the physiological and pathological activity of the brain requires further investigation.

FIGURE 11

Figure 11. Source analysis using EEG-informed fMRI approach in a 26-year-old woman with frontal lobe epilepsy (FLE) which demonstrates a neocortical activation in the inferior frontal gyrus. Top, the component identified on scalp EEG is shown which is located in the left frontotemporal lobe (left) and the dipole localization of the identified generator in deep brain structures (right). Middle, scalp recorded EEG with marked events on F3, F5, and F7. Bottom, Localization of the generator applying simultaneous analysis of EEG-fMRI. The active area is marked with a yellow-red color (Ebrahimzadeh et al., 2021b).

Discussion and future perspectives

Using a combination of EEG and functional magnetic resonance imaging (fMRI), researchers can gain a better understanding