Activations by faces, scenes, tools and body parts

The activations to each of faces, scenes, tools, and body parts are shown for each as the difference from the mean activation to all four stimulus types in Fig. 2 on the HCP-MMP left hemisphere. The results are shown for the right hemisphere in Fig. S2. A list of the abbreviations for the cortical regions is provided in Table S1. The results are shown for the 0-back condition to minimize the memory load, so as to reveal differences in the activations for these different types of stationary visual stimuli. The results reported below shown in Fig. 2 are supported by an analysis in which the baseline for each stimulus type was the mean of the activations to the other three stimuli (Fig. S3D), a type of baseline that has been used previously (Grill-Spector et al. 1998; Stigliani et al. 2015; Natu et al. 2019; Nordt et al. 2021). The results are also supported by an analysis in which the activation to each stimulus type (e.g. scenes) was compared to the prestimulus baseline, which enables activation to e.g. scenes to be shown with no effect related to other stimuli (Fig. 3). The advantage of what is provided in Figs. 2 and S2 is that these show activations selective to each category (e.g. scenes) compared to the same mean baseline computed across all of the stimuli shown in the investigation.

Fig. 2

Brain regions in the left hemisphere with significant differences in the average BOLD signal for the faces, scenes, body parts, and tools compared to the mean of these conditions in the 0-back working memory task, after Bonferroni correction (α = 0.05). Panels A, C show the top 30% of brain regions with significant differences for the 0-back faces and 0-back body parts conditions contrasted with the mean of the four conditions, respectively. Panels B, D display all the brain regions with significant differences for the 0-back scenes and 0-back tools conditions compared to the mean of these four conditions, respectively. The selection of the top 30% of cortical regions in A and C allows the main differences between the four stimulus type, faces, places, body parts, and tools to be easily visualised, but for completeness Fig. S3C shows the same figure as this but without any selection of the top 30%. The corresponding figure for the right hemisphere is in Fig. S2

Fig. 3

The cortical regions exhibiting significant differences in the average BOLD signal between the baseline prestimulus period preceding the 0-back blocks (shown in Fig. 1) before the BOLD signal had responded to the stimuli, and the last 20 timepoints within the 0-back blocks (when the BOLD signal response to the visual stimuli was occurring) for each of the four stimulus types (faces, places, body parts, and tools) after Bonferroni correction (α = 0.05) across 956 participants. The effect size as indicated by Cohen’s d is indicated. The activations are shown in red to yellow. The top 50 cortical regions with significant increases in the BOLD signal are shown out of the 180 cortical regions in the left hemisphere. The baseline prestimulus period was for the last 5 s of the 15 s fixation time and the initial 15 timepoints with a TR of 0.72 s starting when the cue was shown in a run (see Fig. 1)

Faces

Figure 2a (see also Table S2, and Fig. S2 panel A) shows the highest selective activation for the sight of faces in region FFC, with activations too in adjacent and connected TF which is the lateral parahippocampal cortex with connectivity to the hippocampal memory system (Huang et al. 2021; Ma et al. 2022; Rolls et al. 2022b). Interestingly, visual inferior parietal regions PGi, PGs and PFm which have connectivity with anterior temporal lobe visual and semantic regions (Rolls et al. 2023e) were also activated, as consistently were language-related cortical regions TPOJ1, TPOJ2, PSL (the PeriSylvian Language region), STV (the Superior Temporal Visual), and 55b (Rolls et al. 2022a). Also interestingly, inferior parietal high order somatosensory regions PF, PFt, PFop, PFcm and earlier somatosensory regions in the insula (RI), operculum (OP1) (Rolls et al. 2023d) and somatosensory cortical regions 1 and 2 were selectively activated by the sight of faces. Comparable activations were found for the right hemisphere as illustrated in Fig. S2 panel A (see also Table S4), with the FFC region strongly activated by faces, and also the superior temporal sulcus (STS) cortical regions in which we discovered that neurons respond to faces (Baylis et al. 1987; Hasselmo et al. 1989a, b; Rolls 2024a).

Scenes/places

Figure 2b (see also Table S2, and Fig. S2 panel B) shows selective activations to viewed scenes (termed ‘places’) in the ventromedial visual regions VMV1, VMV2, VMV3, VVC, and their forward connected medial parahippocampal cortical regions PHA1, PHA2, PHA3 [where the parahippocampal place or scene area is located (Sulpizio et al. 2020)] which in turn have connectivity into the hippocampal memory and navigation system (Rolls et al. 2023b; Rolls et al. 2023a, b, c, d, e, f, g, h). Activations in DVT and the ProStriate Cortex ProS [where the retrosplenial scene area is located (Sulpizio et al. 2020)], and which projects to the VMV regions (Rolls et al. 2023b; Rolls et al. 2023a, b, c, d, e, f, g, h; Rolls et al. 2024a, b), are also evident, as is activation in POS1 which has high effective connectivity with ProS (Rolls 2024a). Strong selective activation was also found in inferior parietal region PGp, which has connectivity to the hippocampal memory and navigation system and may be involved in self-motion update of scene representations (Rolls 2023a; Rolls et al. 2023e). Activation in earlier cortical regions (V1, V2, V3, V4) is also evident with this contrast (see also Fig. 3 panel B). Interestingly, these scene visual stimuli also selectively activated the medial orbitofrontal cortex (OFC, 13 l and pOFC) and related anterior cingulate regions s32 and 25 which represent reward value (Rolls 2023b; Rolls et al. 2023c) when the baseline was the activation to all 4 stimuli. Comparable activations were found for the right hemisphere as illustrated in Fig. S2 panel B (see also Table S4).

Body parts

Figure 2c (see also Table S2, and Fig. S2 panel C) shows selective activations to the sight of body parts in visual inferior parietal regions PGi, PGs and PFm, and in visual posterior inferior temporal visual cortical regions FFC, PH, PHt, TE1p and TE2p (Rolls et al. 2023e). Visual motion regions such as MT, MST and FST were also activated by the sight of (stationary) body parts. Parietal regions AIP, IP2, and LIPd involved in eye movement control and visually guided actions in space (Rolls et al. 2023b) were also selectively activated by the sight of body parts. Inferior parietal somatosensory region PF at the top of the somatosensory hierarchy (Rolls et al. 2023e) was also activated. Language-related cortical regions TPOJ1, TPOJ2, TPOJ3 and TGd (Rolls et al. 2022a) were also activated. The perirhinal cortex, a route for object information to reach the hippocampal memory system (Rolls et al. 2023b), and also parts of the posterior cingulate cortical division (31pv, 7 m) were also activated. Lateral orbitofrontal cortex regions a47r, p47r and 47 l were also activated by the sight of body parts, which may be related to these stimuli being somewhat unpleasant as some look like dismembered limbs (see Fig. 1), as the lateral orbitofrontal cortex is activated by unpleasant stimuli (Grabenhorst and Rolls 2011; Rolls 2019a, 2023b). Comparable activations were found for the right hemisphere as illustrated in Fig. S2 panel C (see also Table S4), though were less evident in the inferior parietal cortex when selecting only the top 30% of regions with significant activations.

Tools

Figure 2d (see also Table S2, and Fig. S2 panel C) shows selective activations to the sight of tools in the lateral parts of the ventromedial visual regions VMV3, VVC, and PHA3, and in visual motion regions V6, V6a, FST and PH. There is also activation evident in earlier cortical visual regions V1, V2, V3 and V4, and also some posterior cingulate division regions including RSC, v23ab, and d23ab. Tools also activated the medial orbitofrontal cortex and related reward regions, perhaps reflecting that tools are associated with goal/reward-related actions. Comparable activations were found for the right hemisphere as illustrated in Fig. S2 panel D (see also Table S4), with again the lateral parts of the ventromedial visual cortical stream (Rolls 2024a), including VMV3, VVC and PHA3 strongly activated by the sight of tools.

Activations for faces, scenes, body parts and tools shown against a pre-stimulus baseline

Figure 2 shows the selective activations for each stimulus type, faces, places (scenes), body parts, and tools, using as a baseline the mean of the activations across all four stimulus types. To complement this analysis, and in order to show the cortical regions activated by each stimulus type independently of any other stimulus type, Fig. 3 shows the cortical regions with significant responses separately for faces, places, tools and body parts, where the responses are measured as a significant difference in the BOLD signal between the response period (the last 20 timepoints in the timeseries), compared to the lower signal in the prestimulus baseline during the last 5 s of the 15 s fixation period and the first 15 timepoints of each timeseries for each run (see Figs. 1 and S3B). The contrasts in this analysis were thus for faces—the prestimulus baseline, places—the prestimulus baseline, etc., and thus the activations shown are those produced only by faces, or by places, or by tools, or by body parts. There was in this analysis little influence of other stimuli on the activations to faces, places, etc. Given the TR of 0.72 s and the haemodynamic response function, use of the last 5 s of the fixation period and the first 15 time points starting when the cue for a start of a run was shown was appropriate and no activation was evident in this time period, as shown in the timecourse in Fig. S3B. Given that there were typically 39 timepoints in each run, the last 20 did show clear activations to the stimuli, as shown in Fig. S3B. Similar results were found if bins 6–16 in the timecourse in Fig. S3B were used as the prestimulus baseline.

The results shown in Fig. 3 help to confirm the findings shown in Fig. 2. For example, Fig. 3a shows that faces activate strongly FFC; moderately V4, V8, VVC, and TF; and to some extent TF, PeEc, TE2p, STSda, STSdp, TPOJ1-3, and the somatosensory cortex (3b, 1, 2).

Figure 3b shows that places (scenes) activate strongly V4, VMV1, VMV2; moderately PHA1, PHA2, PHA3, VVC, PH and PGp; and to some extent regions where the retrosplenial scene area is located ProStriate (ProS), and the dorsal visual transitional area DVT (Sulpizio et al. 2020).

Figure 3c shows that body parts strongly activate FFC, moderately activate PH, TE2p, TE1p, MT, MST and FST; and to some extent regions STSdp and STSvp, PF, PGi and PGs, and TOPJ1-3.

Figure 3d shows that tools strongly activate VVC, VMV2, V8, V4 and PH; moderately activate PHA3 and FST; and activate to some extent somatosensory 1, 2 and PFt.

Figure 3 helps to emphasise a gradient of efficacy of stimuli from medial to lateral in the ventral temporal lobe, with scenes most medial, then moving laterally tools, then faces, and then most lateral body parts.

Functional connectivities for faces, scenes, tools and body parts

The functional connectivity differences for each of faces, scenes, tools, and body parts are shown for each as the difference from the mean functional connectivity across all four stimulus types in Figs. 4, 5, 6 and 7 on the HCP-MMP left hemisphere, with the corresponding results for the right hemisphere in Figs. S4–S7. As the results are shown for each stimulus type relative to the mean across all stimuli, the higher functional connectivities in each of these figures show what is selective for each of the 4 stimulus types. The results are shown for the 0-back condition to minimize the memory load, so as to reveal differences in the functional connectivities for the different types of stimuli.

Fig. 4

The lower left triangle shows the matrix of functional connectivity differences between 0-back faces and the mean of all 0-back conditions with the Cohen’s d values showing the effect size of the differences. The matrix is for the functional connectivities in the left hemisphere, as listed in Table S1, with V1, V2, V3 … at the top of the y axis and the left of the x axis. The upper triangle matrix shows the Cohen’s d values of positive significant links after FDR correction (α = 0.05). These results were from 956 participants in the HCP dataset. All the values shown in the matrix were limited to the range from − 0.5 to 0.5. The covariates regressed out in this analysis were sex, age, drinker status, smoking status, education qualification and head motion

Fig. 5

The lower left triangle shows the matrix of functional connectivity differences between 0-back scenes and the mean of all 0-back conditions with the Cohen’s d values showing the effect size of the differences. The matrix is for the functional connectivities in the left hemisphere, as listed in Table S1, with V1, V2, V3 … at the top of the y axis and the left of the x axis. The upper triangle matrix shows the Cohen’s d values of significant positive links after FDR correction (α = 0.05). These results were from 956 participants in the HCP dataset. All the values shown in the matrix were limited to the range from − 0.6 to 0.6. The covariates regressed out in this analysis were sex, age, drinker status, smoking status, education qualification and head motion

Fig. 6

The lower left triangle shows the matrix of functional connectivity differences between 0-back body parts and the mean of all 0-back conditions with the Cohen’s d values showing the effect size of the differences. The matrix is for the functional connectivities in the left hemisphere, as listed in Table S1, with V1, V2, V3 … at the top of the y axis and the left of the x axis. The upper triangle matrix shows the Cohen’s d values of significant positive links after FDR correction (α = 0.05). These results were from 956 participants in the HCP dataset. All the values shown in the matrix were limited to the range from − 0.5 to 0.5. The covariates regressed out in this analysis were sex, age, drinker status, smoking status, education qualification and head motion

Fig. 7

The lower left triangle shows the matrix of functional connectivity differences between 0-back tools and the mean of all 0-back conditions with the Cohen’s d values showing the effect size of the differences. The matrix is for the functional connectivities in the left hemisphere, as listed in Table S1, with V1, V2, V3 … at the top of the y axis and the left of the x axis. The upper triangle matrix shows the Cohen’s d values of significant positive links after FDR correction (α = 0.05). These results were from 956 participants in the HCP dataset. All the values shown in the matrix were limited to the range from − 0.5 to 0.5. The covariates regressed out in this analysis were sex, age, drinker status, smoking status, education qualification and head motion

A key point to emphasise is that the functional connectivities to these four stimulus types are different to each other, and different from the functional connectivities in the resting state (Rolls et al. 2023b, 2023e, 2023b), providing evidence that functional connectivities change depending on the task, and providing evidence about pathways especially involved in particular tasks. Although resting state functional connectivity is a useful measure of the basic framework of brain connectivity, the connectivity is not in the short term fixed like anatomical connectivity, but provides evidence about cortical pathways involved in particular functions. An indication of how the functional connectivities can change when visual stimuli are presented is provided in Figs. S8 and S9, which show which functional connectivities increase when visual stimuli are shown compared to the pre-stimulus period.

Faces

The functional connectivities selective for faces are those significantly greater than the mean across all visual stimuli shown in the upper right triangle of Fig. 4, with the positive Cohen’s d values showing the effect size. Higher functional connectivities for faces are found for STS and related regions (such as STGa, STSda, STSdp, STSva, STSvp) with some early visual cortical regions, including V2, V3, V4; with some ventral stream regions including V8, and VMV3 just medial to the FFC; some dorsal stream regions including IPSI, V3A, V3B, and V6; some MT + complex regions including LO1 and V3CD; intraparietal LIPv, MIP and IP0; and inferior parietal PGp. These connectivities are very interesting, for many neurons in the cortex in the macaque STS respond preferentially to moving faces that make or break social contact (Hasselmo et al. 1989a, b), and inputs from both ventral stream face/object and dorsal visual stream movement-related regions are likely to implement this (Baylis et al. 1987; Hasselmo et al. 1989a, b; Hasselmo et al. 1989a, b). Many of the visual regions just noted also have significantly high functional connectivities when viewing faces with inferior parietal regions (such as PGi, PGs and especially PGp) (Rolls et al. 2023e); with posterior cingulate division regions (including PCV—the precuneus visual region, POS1, POS2, 23d, 31a, 31pd) (Rolls et al. 2023b); and with region 45, part of Broca’s area. Connectivity in the right hemisphere was similar, with in addition higher connectivity of some of these visual cortical areas with temporo-parietal junction semantic regions PSL, STV, TPOJ1 and TPOJ2; and with some somatomotor regions including 3b and 4; and of V3CD with A1 and belt auditory regions (Fig. S4).

Scenes/places

The functional connectivities selective for places (i.e. views of spatial scenes) shown in Fig. 5, and especially Fig. S5 for the right hemisphere, were higher for ventromedial visual regions VMV2 and VMV3 with ventral visual stream regions V3, V4, FFC, PIT and V8. The parahippocampal scene area (or parahippocampal place area PPA) is at the junction of the VMV and medial parahippocampal PHA regions (Sulpizio et al. 2020), and this connectivity finding supports the theory that in humans and other primates scene representations are built using ventral stream feature combination mechanisms (Rolls 2023a; Rolls and Treves 2024). Consistent with this, the parahippocampal regions PHA1, PHA2 and PHA3 regions have high functional connectivity with similar ventral stream regions (V3, V4, FFC, PIT, V8) during the visual presentation of scenes (places) (Fig. S5). Interestingly, higher functional connectivity during scene viewing was also found for connectivities of VMV2, VMV3 and VVC with PHA1-3 regions and with some visual motion-r