Introduction

There has been accumulated psychophysical evidence since a seminal report by Melcher and Morrone (2003) that visual information across saccades is integrated in a spatiotopic manner (Melcher and Morrone, 2003; Burr and Morrone, 2012; Zimmermann et al., 2013). Such transsaccadic integration could be achieved assuming a craniotopic coordinate system. Real craniotopic neurons, whose receptive fields (RFs) were fixed to head-centered coordinates regardless of eye positions in the orbit, were found in the anterior bank of the parieto-occipital sulcus (POS; area V6A; Galletti et al., 1993) and in the ventral intraparietal (VIP) area (Duhamel et al., 1997). These craniotopic neurons are believed to utilize the activities of parietal neurons whose retinotopic responses are modulated according to eye positions relative to the head (Andersen and Mountcastle, 1983; Andersen et al., 1985; Zipser and Andersen, 1988). In a more recent study, VIP neurons were shown to represent the direction of a moving object in the world coordinate by integrating visual motion signals with self-motion signals (Sasaki et al., 2020).

Recent psychophysical studies have shown that our brain automatically encodes an object location relative to the background in a scene (Boi et al., 2011; Lin and He, 2012; Uchimura and Kitazawa, 2013; Inoue et al., 2016b; Tower-Richardi et al., 2016; Chakrabarty et al., 2017; Nishimura et al., 2019). For example, a target position for reaching movement is automatically and instantaneously encoded relative to a large rectangle in the background (Uchimura and Kitazawa, 2013; Inoue et al., 2016b; Nishimura et al., 2019). This body of literature points to the existence of nonegocentric coordinates anchored to the background of the scene, distinct from both retinotopic and the still egocentric craniotopic coordinates.

In search of the neural basis of the background coordinate, we previously demonstrated in a functional imaging study of humans that the precuneus, a major hub of the default-mode network, was involved in encoding a target location relative to a large rectangle in the background (Uchimura et al., 2015). Two points are worth noting. First, the task in the imaging study was target discrimination (differentiating a circle from an apple), meaning that the target location was completely irrelevant to the task. Second, the involvement of the precuneus ceased when the large rectangle was replaced with a salient but smaller rectangle.

In this study, we searched for background-centric visual neurons in the monkey precuneus, whose RFs were fixed to the background. We hypothesized the simplest process for calculating a stimulus position relative to the background (Fig. 1A): retinal images are separated into the dot and the background in the retinal coordinate (dot/ret and bkg/ret) and then combined to represent the dot location relative to the background (dot/bkg). Ultimately, we found background-centric neurons in the precuneus whose RFs were fixed to the rectangle. Furthermore, we found that the two types of retinal information (dot/ret and bkg/ret) were represented in a time-division multiplexing manner.

Figure 1.

Design of the study and experiments. A, A hypothesis of direct background-centric coding from retinotopic representations. A dot can be represented relative to the background (background coordinate) by relying on retinotopic representations of the dot (dot/ret) and the background (bkg/ret) without resorting to any intermediates. In this study, we examined whether and how the precuneus is involved in these processes. B, Approximate locations of 942 isolated neurons around the POS are shown by dots on a typical sagittal slice of an MRI image of Monkey 1, taken 3 mm from the midline. Recordings were made in four hemispheres of three monkeys along parallel tracks that were spaced by 1 mm apart. Note that the tracks are compiled across different sagittal planes and different hemispheres of the three monkeys to show the rough locations of recordings. See Figure 2 for detailed locations. The size of each dot represents the number of isolated neurons. The anatomical subdivision (Saleem and Logothetis, 2012) of this particular slice is shown in the middle panel. PEc, area PE caudal; 7m, area 7; medial subdivision (also known as area PGm); V2, second visual area; V6A, visual area 6A. A scale on the middle panel shows the dorsoventral coordinate from the ear canal. In the right panel, arrows show MRI-detectable elgiloy deposit markings on the most anterior track. C–F, A sequence of events in one trial. An FP was presented at a random location within a square zone (D, 20 × 20°). If the monkey fixated on the marker within 100 ms, a rectangular frame (30 × 20°, background, bkg) was presented at a random location within a rectangular zone (E, 60 × 40°). Then, after an interval of 200 ms, a red dot was presented sequentially at 12 different locations with stimulus onset asynchronies of 150 ms. Red dots were presented with an even probability within a zone (F, thick red broken line) that was defined as a union of three rectangular zones of 60 × 40° regions around the frame, center of the monitor, and the FP. Thin red broken lines show 12 equiareal subregions of the zone. Note that the edge of the monitor was invisible to the monkeys in the dark environment throughout the task period. G–I, Distortions of imaginary retinotopic zones in retinal coordinate. G, Imaginary 12 regions for calculating dot/ret information. These regions are rectangular on the display (black) but distorted on the retina (red). The distortion was simulated assuming that the monkey fixated on the center of the display (0, 0). H, Typical fixation locations, including the center (red), and four median positions in each quadrant of the fixation target zone: (5, 5), (−5, 5), (−5, −5), and (5, −5). I, Borders of retinotopic regions under typical conditions. The colors of the borders correspond to the fixation positions shown in H.

ResultsSome precuneus neurons show increases, but others show decreases in their firing rates in response to visual stimuli

We recorded neural activities from 942 neurons, mostly in the precuneus but some in the cuneus around the POS, across four hemispheres of three monkeys (Fig. 1B). When the animals fixated on a point (Fig. 1C, FP) for 100 ms, a large rectangle (30 × 20°) was presented at a random location, which served as the background (Fig. 1C, Bkg). Then, a red dot was presented sequentially at 12 different locations against the background (Fig. 1C). It is worth noting that the animals were not required to remember any positions of the dot or the background but just to fixate on the FP for ∼2 s until they were rewarded by a drop of juice. Most neurons showed significant changes in their activity compared with the baseline in response to the presentation of a background (68%; Fig. 3D, Bkg) or a dot (73%; Fig. 3E). Interestingly, some neurons showed an increase from the baseline (Fig. 3A, top panel, 3D,E, #75), but others showed a decrease (Fig. 3A, middle panel, 3D,E, #41). An exemplified decreasing neuron showed a dramatic decrease from 150 to 5 spikes/s after the background presentation, and the mean discharge rate remained low during the period of dot presentation (Dots 1, 2, …, 12). However, by expanding the y-axis, we noticed that the neuron responded to each dot presentation by doubling its activity from 5 spikes/s to above 10 spikes/s (Fig. 3B, middle panel). Accordingly, when we compared neural activity in response to dot presentation (40–115 ms after dot presentation; Fig. 3B, bottom panel, magenta) against a control period from −20 to 20 ms (cyan, dot baseline), the neuron significantly increased its activity (Fig. 3F, #41). It is worth emphasizing that the precuneus neurons still conveyed dot information even if their mean discharge rate dropped close to zero.

Retinotopic dot neurons

Figure 4A shows a neuron representing dot positions in the retinotopic coordinate. This neuron responded with a discharge rate above 8 spikes/s when a dot was presented in a contra-bottom region relative to the FP (Fig. 4C, peak at [−4°, −14°]) during a 50 ms period from 40 to 90 ms (Fig. 4A, shaded period). By observing the high discharge rate of the neuron, we obtained some information as to where the dot was presented in the retinal coordinate. This neuron represented 1.6 bits of information during the 50 ms period (31 bits/s) when we divided the retinal field into 12 equiareal regions (maximum information, 3.6 bits). A permutation test showed that the information was significant (p < 0.001). In contrast, this neuron did not encode significant dot position information relative to the background (p = 0.37; Fig. 4E). By moving the 50 ms time window along the time axis with steps of 10 ms, we found that significant retinal dot position information (dot/ret) appeared at 50 ms and lasted for 60 ms until 110 ms (Fig. 4B, blue solid trace) with a peak at 90 ms (Fig. 4B, arrow). In total, 149 (of 942, 16%) neurons represented significant dot position information in the retinal coordinate (dot/ret information). The mean of the dot/ret information across the 149 neurons started to increase at 30–40 ms, reached its peak at 80 ms, and subsided thereafter (Fig. 4F). The full-width at half-maximum information (FWHM) was 65 ms, from 50 to 115 ms (horizontal broken line).

Background-centric dot neurons

Figure 5A shows a neuron that represented significant information on the dot position relative to the background (dot/bkg information). This neuron responded above 10 spikes/s when a dot was presented in the contra-top corner of the background rectangle (Fig. 5E) with an information transmission rate of 2.2 bits/s (p < 0.001). In contrast, this neuron did not encode significant information in the retinal coordinate system (Fig. 5C, dot/ret). The neuron showed a peak of dot/bkg information at 140 ms (Fig. 5B, red arrow). In total, 58 (of 942, 6.2%) neurons represented significant dot/bkg information. The mean of the dot/bkg information started to increase after 30 ms and reached its peak at 100 ms, and transmission lasted until after 200 ms (Fig. 5F). The FWHM was 100 ms, from 65 to 165 ms (horizontal broken line), which lagged the FWHM of the dot/ret information by 10 ms. The results show that it takes ∼10 ms to calculate the background-centric dot/bkg position after representing the dot/ret position.

Figure 5.

Background-centric dot neurons (dot/bkg). A, B, Firing and information profiles of a pure dot/bkg neuron representing the background-centric dot location relative to the background. C–E, RFs at 140 ms yielding the peak dot/bkg information (arrow in A). The neuron had a background-centric RF in the left-top corner of the background (E) but not in retinotopic coordinates (C, D). F, Mean information on the dot/bkg location across 58 dot/bkg neurons. The FWHM extended from 65 to 165 ms.

RFs of the retinotopic and background-centric dot neurons

We then compared the spatial characteristics of the RFs in the retinal coordinate (dot/ret) with those in the background (dot/bkg). We fitted a two-dimensional Gabor function to the RF of each neuron, as shown in Figure 6, A and B. It is worth noting that a peak is often associated with a trough (Fig. 6A, third row) and that some neurons (∼15%) had a negative dominant RF (Fig. 6A, bottom row). The Gabor model captured these essential RF characteristics, as shown by the mean determination coefficients >0.5 and as large as 0.96 (dot/ret, 0.78 ± 0.17; dot/bkg, 0.72 ± 0.19 mean ± SD). We defined the RF using the concentration ellipse of the Gaussian function in the Gabor function (Fig. 6A,B, ellipses). Interestingly, the ellipses covered the bottom and contralateral quadrant (the third quadrant) in both the retinotopic (Fig. 6C) and background (Fig. 6E) coordinates. Furthermore, when we plotted the size of the RF (the geometric mean of the ellipse's axes) against its eccentricity (distance between the center of the ellipse and the origin of the coordinate), we found that the size of the RF increased linearly with eccentricity in dot/ret (Fig. 6D; r = 0.39; p = 2.9 × 10−6). In contrast, the size of the background-centric (dot/bkg) RF did not correlate with the eccentricity, which ranged from 0 to 40° (Fig. 6F; r = 0.23; p = 0.13). The general effect of the cortical magnification seemed to be maintained for the retinotopic RF but was somehow eliminated during the process of calculating the background-centric information.

Figure 6.

RFs of the retinotopic and background-centric dot neurons. A, B, Examples of retinotopic (A) and background-centric (B) RFs. RFs of four retinotopic neurons are shown in A and those of other four background-centric neurons are shown in B. Two panels in each row show the actual data from a single neuron in the left and its approximation by the Gabor function in the right. White ellipses in the right panels show the size of “one-SD” of the two-dimensional Gaussian distributions of the Gabor functions. Note in A that the size of the ellipse increases as the RF moved from the center toward the periphery. C, E, Distributions of the retinotopic (C) and background-centric (E) RFs. D, F, The size of the RF plotted against eccentricity. Note that the correlation was significant in the retinotopic (C) but not in the background-centric (E) RFs. Crosses in D show outliers with residual errors >3 SD, which were excluded from regression. G, I, The vertical location of the RF plotted against the ventrodorsal location of neurons. Note that the correlation was significant both in the retinotopic (G) and background-centric (I) RFs: RFs moved lower as the neuron locations moved to more ventral regions. H, J, The size of the RF plotted against ventrodorsal location.

We additionally analyzed relationships between the location and size of the RF and the stereotactic coordinates of the recording sites. We found that the location of the RF moved lower as the recording site moved in the ventral direction in both dot/ret (Fig. 6G) and dot/bkg (Fig. 6I) neurons. On the other hand, the size of the RF did not correlate with the ventrodorsal location (Fig. 6H,J).

Some precuneus neurons initially represent dot positions in the retinotopic coordinate but later relative to the background

We have shown that some neurons in the precuneus represent retinal (Fig. 4, dot/ret) information and others represent background-centric (dot/bkg) information (Fig. 5). However, these two types of information are not mutually exclusive, and some represented both (45/942, 4.8%). For example, one such neuron initially showed a clear RF in the retinotopic coordinate in particular (Fig. 7C, 60 ms) but later developed an RF relative to the background (Fig. 7H, 190 ms). The information curves (Fig. 7B) showed that the dot/ret information dominated from 50 to 120 ms (blue trace), but the dot/bkg information (magenta trace) took over at 120 ms and lasted until 190 ms. Furthermore, the neuron represented significant information on the background location in the retinal coordinate (bkg/ret, green trace). It is worth noting that the bkg/ret information made two peaks as if it alternated with the dot/ret information (Fig. 7B). These features remained when we averaged three types of information across 39 neurons that represented all three types of significant information (Fig. 7I–K). The initial dot/ret information (Fig. 7J) was followed by the background-centric dot/bkg information (Fig. 7I), and the bkg/ret information peaked before and after the dot/ret information (Fig. 7K).

Figure 7.

Some precuneus neurons initially represented dot positions in the retinotopic coordinate but later relative to the background. A, B, Firing and information profiles of a neuron with all three types of information (dot/ret, bkg/ret, and dot/bkg; a star in the triple intersection of the Venn diagram). It initially encoded bkg/ret information (green) and then dot/ret information (blue) and finally bkg/ret (green) and dot/bkg (magenta) information. The bkg/ret information is multiplied by 10 for the sake of visibility. C–H, RFs in the initial (C–E, at 60 ms) and later periods (F–H, at 140 ms). Note that the neuron had a clear retinotopic RF at 60 ms (C) but later yielded a clear background-centric RF (H). I–K, Mean information curves of 39 neurons in the triple intersection. The dotted line in red in I was obtained by multiplying two retinotopic information, dot/ret (J) and bkg/ret (K), and adding a delay of 20 ms. Note the similarity with dot/bkg information. The FWHMs extended from 65 to 165 ms for dot/bkg (I) and 50 to 115 ms for dot/ret (J).

Two types of retinotopic information, dot/ret and bkg/ret, are represented in a multiplexing manner

In Figure 7, we observe an inverse relationship between dot/ret and bkg/ret information: as information on dot/ret emerges, there is a corresponding decline in bkg/ret information and vice versa. This phenomenon suggests two potential interpretations regarding the temporal dynamics of these complementary information streams.

The first interpretation posits that the information pertaining to the dot (figure) and that relating to the background (ground) are temporally distinct, processed in a multiplexed fashion. This notion (multiplexing of information) was proposed by previous studies (Kitazawa et al., 1998; Caruso et al., 2018; Jun et al., 2022). Essentially, this perspective suggests that the retinal information from the figure and the ground are not simultaneously processed but are instead represented separately over time. The second interpretation is more straightforward, proposing that the constant retinal input from the background (bkg/ret) is merely suppressed by the transient prominence of the dot's retinal input (dot/ret).

To discern between these two theories, we introduced a variant in our experimental design where the background element was entirely absent (Fig. 8A). Under this setup, if the simpler suppression hypothesis holds, we expect the temporal dynamics of the dot/ret information to simply increase in amplitude, maintaining its original profile. Conversely, if figure and ground information are processed in a temporally multiplexed manner, dot/ret information would emerge first as figure-specific information. Subsequently, during the phase initially occupied by bkg/ret information (when background elements were present), it would reappear, this time substituting for the ground information.

Figure 8.

Two types of retinotopic information, dot/ret and bkg/ret, were represented in a multiplexing manner. A, A sequence of events in the no-background condition. B, Mean information on the retinotopic dot location in the background and no-background conditions summed across 107 dot/ret neurons. Note the increase in dot/ret information (broken line) in the absence of the background. C, Temporal profile of the increase in dot/ret information in the no-background condition. Note two blocks, one up to 80 ms and another from 100 to 250 ms, with a peak at 180 ms (arrow).

We conducted recordings from 256 neurons under two different conditions: one presenting a background (Fig. 1C) and another without any background (Fig. 8A). Notably, the information profile that displayed a single peak in the presence of a background (Fig. 8B, solid line) evolved into a double-peak profile in the absence of the background (dotted line). The difference between the two profiles revealed two distinct phases: an initial, stable low-activity phase spanning up to 80 ms, followed by a more pronounced phase characterized by a surge in activity from 100 to 250 ms, reaching a peak at 180 ms (Fig. 8C, arrow).

These findings provide compelling evidence in favor of the multiplexing hypothesis. They suggest a sequential processing of information where dot/ret (figure-specific) information prevails during the initial phase up to 80 ms, followed by the dominance of bkg/ret (ground-specific) information in the 100–250 ms timeframe. This pattern is indicative of time-division multiplexing, where distinct types of retinal information are processed in separate, nonoverlapping temporal segments.

Hybrid coding of dot/bkg and dot/ret information in dot/bkg neurons

We have shown that some precuneus neurons initially represented dot/ret information and subsequently represented dot/bkg information in a multiplexing manner (Figs. 7, 8). To corroborate these findings from a different perspective, we employed a hybrid coding analysis (Mullette-Gillman et al., 2005, 2009; Caruso et al., 2021). We evaluated dot/ret coding and dot/bkg coding of the precuneus neurons in a temporally sequential manner by sliding a 50 ms time window along the time axis (Fig. 9). The coding strength was quantified by the mean correlation coefficient (r) across four different background locations on the retina, under the assumption that pure dot/ret or dot/bkg responses would be independent of background location (see Materials and Methods for detail). The coefficient takes a maximum value of one, indicating utmost stability in coding, and drops to zero when the coding is absent.

Figure 9.

Temporal profiles of hybrid coding in dot/ret, dot/bkg, and bkg/ret neurons. A, Correlation coefficients representing dot/ret (ordinate) and dot/bkg (abscissa) information in the hybrid coding analysis, plotted for individual neurons at 0, 70, 140, and 280 ms from the dot onset. Different symbols represent neurons with significant bkg/ret (green), dot/ret (blue), and dot/bkg (magenta) information, each calculated independently of the hybrid coding analyses at each time period. At 70 ms, most dot/ret neurons (blue) were distributed above the diagonal, showing the dot/ret dominance, while at 140 ms some dot/bkg neurons (magenta) crossed the diagonal into the dot/bkg dominant zone. The histogram shows the number of dot/ret and dot/bkg neurons above and below the diagonal. B, The mean temporal profiles of bkg/ret (n = 510), dot/ret (n = 69), and dot/bkg (n = 17) neurons on the hybrid coding plane. These neuron groups were defined at 140 ms as those with significant bkg/ret information (n = 510), with dot/ret information and above the diagonal (n = 69), and with dot/bkg information and below the diagonal (n = 17). Different symbols show at 0 (square), 70 (circle), 140 (diamond), and 280 (triangle) ms. Colored ellipses show “standard errors of the mean” of two-dimensional Gaussian distributions estimated every 10 ms along the timeline. Note that dot/bkg neurons make a right turn at 90 ms toward the dot/bkg dominant zone, followed by a circular path deep into the zone over 120–200 ms. The circular path made a marked contrast with the linear path of the dot/ret neurons.

Consider, for instance, the 50 ms window set from −50 to 0 ms, just prior to the onset of dot presentation (Fig. 9A, 0 ms). During this interval, 485 out of 942 neurons exhibited significant bkg/ret information according to our information analysis. These 485 bkg/ret neurons clustered near the origin of the dot/ret and dot/bkg coding plane, confirming that they did not encode either dot/ret or dot/bkg information. In the subsequent time window from 20 to 70 ms, many neurons (n = 96) began encoding dot/ret information, depicted by blue circles. These dot/ret neurons distributed mostly along the y-axis (dot/ret axis) and over the triangular region above a diagonal line (y = x), indicating a predominance in dot/ret coding over dot/bkg coding. At 140 ms (the time window from 90 to 140 ms), the number of dot/bkg neurons increased from 19 at 70 ms to 30 (red circles). Many of them (n = 17) transitioned across the diagonal line into the dot/bkg dominant zone. Lastly, at the 280 ms time window, most neurons ceased coding for either dot/ret or dot/bkg information.

To compare the mean trajectories of bkg/ret, dot/ret, and dot/bkg neurons, we defined three groups at 140 ms (Fig. 9A, 140 ms): bkg/ret neurons (n = 510, green dots), dot/ret neurons (blue circles) in the dot/ret dominant zone above the diagonal (n = 69), and dot/bkg neurons (red circles) in the dot/bkg dominant zone (n = 17). The mean trajectory of the bkg/ret neurons, shown in the left panel of Figure 9B, was anchored to the origin. In contrast, the mean trajectory of the dot/ret neurons rose straight up to the peak point (0.2, 0.6) at ∼90 ms and returned along the same linear path back to the origin. The mean trajectory of the dot/bkg neurons moved linearly up to 90 ms along a slightly shallower path but then made a right angle turn to cross the diagonal border into the dot/bkg dominant zone. It reached the furthest point into the dot/bkg zone at 140 ms and returned to the diagonal at ∼200 ms. The abrupt turn of the trajectory at ∼90 ms appeared to mark a change from a dot/ret dominant period to a period dominated by dot/bkg, during which dot/ret information is combined with bkg/ret information to yield dot/bkg information. The results of these hybrid coding analyses provided further evidence supporting the multiplexing hypothesis.

Dot/ret and dot/bkg RFs generally do not depend on the gaze position

We further examined whether the dot/ret and dot/bkg RFs was modulated by gaze location (Fig. 10). Some dot/bkg neurons (13/58, 22%) and dot/ret neurons (16/149, 11%) showed significant changes, but the majority did not. However, the mean dot/bkg information of the neurons with modulation (gain neurons) was twice as large as that of no-gain neurons (Fig. 10).

Figure 10.

“Gain” neurons whose activities were modulated by the fixation location (eye position relative to the orbit). A, An example of a dot/bkg “gain” neuron. The top panel shows the dot/bkg RF obtained using all trials. Each of the four panels below shows RF obtained using trials in which fixation fell in one of the four quadrants (I–IV). Note that activation was enhanced when the monkey viewed ipsilateral quadrants (I, IV). P values and χ2 statistics (df = 11) of comparison with the original are indicated above the respective panels. B, An example of a dot/ret “gain” neuron. Activation was enhanced when the fixation fell in the bottom quadrants (III, IV). C, D, Mean information of “gain” neurons (top) and “no-gain” neurons (bottom). Note the similarity of the temporal profiles. Gain neurons were a minority in both dot/bkg neurons (22%, 13/58) and dot/ret neurons (11%, 16/149).

RFs of dot/bkg neurons are independent of background location in half of the cases

We assessed whether the RFs of dot/bkg neurons were modulated by the background location, which varied widely across the four quadrants of the retina (