CORTICAL REGIONS RESPONDING TO LONG-RANGE SYMMETRY PATTERNS
Christopher W. Tyler, Heidi A. Baseler* and Brian A. Wandell*
Smith-Kettlewell Eye Research Institute, 2318 Fillmore Street, San Francisco, California 94115, USA, and *Stanford University, Department of Psychology, Building 420, Stanford, California 94305, USA
We report on a bilateral visual area of occipital cortex that shows strong hemodynamic activation by the presence of multiple symmetries in the image viewed. The stimulus consisted of random dots organized in mirror symmetric or repeating patterns, against control stimuli of purely random dots. The results show negligible or weak activation of visual projection area V1 or most other medial occipital projection areas. However, there was a strong symmetry-specific response around the middle occipital gyrus, a region of lateral occipital cortex whose function is otherwise little known. The high level of activation in the middle occipital gyrus may represent part of a general class of computations that require integration of information across a large span of the visual field.
Following the projection of the visual image onto the retina, visual perception involves a series of synaptic transformations through which visually responsive neurons in the brain process information over areas of the visual scene more than a thousand times greater than the individual photoreceptors. The large spatial extent of these cortical receptive fields raises questions about the types of visual computations undertaken in these portions of the brain. What analysis might higher cortical neurons perform?
Several fundamental types of visual computation require information that is distributed across large portions of the visual field. These computations may be useful for a variety of visual functions, including perceptual rescaling (color constancy), estimating object properties (size estimation, perceptual grouping), and the processing of specific cues that are important for pattern recognition and object classification. Symmetry is an example of an object property that requires long-range integration of scene features. In many cases symmetry information is not present in the local features and can be found only by comparing information distributed across long distances in the visual field.1 The reason for the perceptual salience of symmetry is unclear, but it has been argued that symmetry is a useful cue for discriminating living organisms (friend or foe) from inanimate objects. Symmetry has been shown to be a significant determinant of mate selection.2-4 Its importance as a visual cue to humans is evident from the widespread occurrence of symmetric patterns and designs in the constructed environment of architecture and art throughout human history.
There is some neuropsychological evidence of a cortical specialization for large-scale visual computations. Patients suffering from certain visual agnosias, for example simultanagnosia, identify local elements of the visual field but have difficulty discerning the relations among elements in the image. The patients act as though they can see only one object at a time. Brain injury in these patients is often associated with localized lesions in the occipital-parietal portions of the brain.5 More specifically, certain patients with posterior cerebral artery infarcts of the lateral occipital cortex show a specific deficit in detecting bilateral symmetry in random-dot patterns presented to the contralesional visual field, while pattern and motion discrimination in the deficient regions of the field are relatively normal (L. Vaina, personal communication).
We have used functional magnetic resonance imaging (fMRI) to investigate activity in the visual pathways while observers view patterns containing large-scale symmetry. The object of the study was a proof of principle in a small sample of observers as to whether the property of symmetry would elicit specialized responses or activate known retinotopic regions of cortex. For this initial communication, we have not attempted a survey of the generality of the effects in a large population. The results showed an increase in fMRI signal in a bilateral region of occipital cortex when the observers viewed fields of random dots organized in symmetric or repeating patches in comparison with the fMRI signal elicited by control stimuli of purely random dots. The activity caused by the pattern symmetries was mainly present outside the early visual areas, which can be identified by retinotopic mapping with traveling-wave stimuli.6-9
Figure 1. Examples of individual test stimuli. a) Four-axis reflection symmetry of a random dot base pattern. b) Purely random dot control pattern. c) Translational repetition in x and y of a random dot tile (wallpaper symmetry).
We use the term multiple symmetries in the physicists' sense of any regular similarity in pattern across regions of an image. It thus includes not just reflection symmetry around various axes but translational symmetry and other transformations (although we have not yet been able to test more than the first two.) As examples of the symmetry stimuli, Fig. 1a shows a sample of four-axis symmetry, one of the most perceptually salient of our stimuli. Note that very little of the symmetry can be detected by local spatial analyses spanning the size of classical receptive field sizes found in primary visual cortex, typically less than 0.5 deg in diameter.10,11 A purely random dot pattern of the type used for control stimuli is shown in Fig. 1b. Although replete with contours and blobs, its pattern structure does not resonate visually like a symmetric pattern. A second type of symmetry, repetition symmetry, is shown in Fig 1c. Around the fixation center there is literally no information that the pattern deviates from random. It is only by a comparison across one repetition cycle, a long-range pattern match, that information about the pattern structure becomes available.
Figure 2. Axial MRI slices comparing retinotopic fMRI activation regions (left panels) with four-axis symmetry activation (right panels). Slice orientation and position: CT-15 deg declined from line between anterior and posterior commissures (AC/PC line), 2.6 cm ventral to posterior commissure (PC); HB-10 deg declined from AC/PC line, 1.6 cm ventral to PC; AP-5 deg declined from AC/PC line, 2.3 cm ventral to PC. a) Colored regions indicate locations of known visual areas: V1 (red), V2 (green), V3 (blue), V3a (yellow), V4v (magenta) and V5/MT+ (cyan). The center of V5/MT+ is located approximately 1 cm above the slice shown in observers CT and AP. Therefore, the lateral position and extent of V5/MT+ in each observer has been outlined (dashed cyan line) and projected onto this slice. Note that the presence of symmetry evokes negligible activation in any of the retinotopic areas or V5/MT+. b) Distribution of fMRI responses to symmetry/random alternation. To meet the significance criterion, voxel responses must respond within a 6-sec window following the onset of the symmetry phase of the stimulus, and reach a stimulus correlation level of 0.40 - 1.00 (orange > yellow > white).
In Figure 2, symmetry activation is compared with the location of the retinotopic visual areas (see Methods). A single axial brain slice passing through the pons, cerebellum, occipital and temporal lobes is shown for each of the three observers. The slice was chosen to pass through regions of symmetry activation and through retinotopic areas such as V1. The left panels of Figure 2 show the retinotopic areas in the medial portion of the occipital lobe, extending about 2 cm around the posterior surface. The right panels show fMRI responses in the same slices to four-axis symmetric vs. random dot patterns. Significant activation was defined by a stimulus/response correlation threshold of 0.40 (see Methods). The right panels of Figure 2 show that significant activation correlating with the symmetry/random alternation is almost completely absent from the defined retinotopic areas and the human motion sensitive region, V5/MT+. Instead, it clusters in bilateral patches of lateral occipital cortex. Since the activity of the individual voxels in fMRI is uncorrelated (as indicated by the few isolated locations of activation), the clustering of adjacent activated voxels represents a significant response (see Methods).
Figure 3. Three-dimensional renderings of the left occipital lobe in one subject (CT) depicting a) regions that responded to the presence of four-axis symmetry (fMRI activation shown in orange) compared with b) the positions of the retinotopic visual areas: V1 (red), V2 (green), V3 (blue), V3a (yellow), V4v (magenta) and V5/MT+ (cyan). Left, middle and right panels depict lateral, posterior and medial views, respectively. Images are rendered near the boundary between the cortex and white matter.
The computed spatial distribution of fMRI symmetry activation across slices is projected with the same orange > yellow coding onto the 3D brain renderings of one occipital lobe in Fig. 3. This activation forms a vertically extended region around and including much of the middle occipital gyrus (MOG), reaching up to near the parietal-occipital border. The 3D images illustrate that there was virtually no activation in response to the symmetry condition on the medial surface of the occipital lobe on which are represented the major portions of visual projection areas V1, V2 and V3 (medial rendering at right in Fig 3b). Compared with the positions of known visual areas, the symmetry activation is lateral to the retinotopically organized areas (V1, V2, V3, V3a, V4v), but medial and posterior to the motion-specific area, V5/MT+, shown in cyan.12 The symmetry-specific activation that we report is primarily dorsal with respect to the location of the putative object-specific lateral occipital area, LO.13,14 Object-responsive area LO has been reported to fall mostly on the ventrolateral surface of the cortex, between areas V4v and V5/MT+.
Figure 4. Total number of voxels significantly activated by the symmetry/random alternation within 4 square cm of cortex (400 voxels) in several occipital regions. The 4 square cm area with the largest number of significant activations was chosen by an automated procedure within equal areas of medial retinotopic (Med) and lateral (Lat) regions of cortex. For areas V3A and V4v, located dorsally and ventrally in the occipital lobes, respectively, the 4 square cm regions of greatest activation were chosen but with no attempt to equate the selection areas. Note that the greatest activation density occurred in lateral cortex in five of the six hemispheres analyzed. ). The error bars represent one standard error based on a binomial distribution. See Methods for details.
To quantify this comparison, we needed to define anatomical zones that were comparable for the retinotopic and non-retinotopic regions of occipital cortex. To make a fair comparison, we specified medial and lateral occipital zones to be of exactly the same size for each observer (see Methods). We then searched for the 4 square cm area in each zone that contained the largest number of significantly activated voxels. Thus, the retinotopic and non-retinotopic zones had an equal chance of exhibiting symmetry responses on a random basis. In fact, this statistical comparison revealed that each of the six hemispheres evaluated showed significant activation in the lateral, MOG region (Fig. 4). For good measure, we also report the best symmetry activation in any 4 square cm region of retinotopically defined areas V3a and V4v at the dorsal and ventral extremes of the retinotopic areas, although anatomical boundary limitations precluded matching the sizes of these areas.
Using the automated, objective analysis, the activation was significantly stronger to the four-axis symmetry property in (non-retinotopic) lateral than in (retinotopic) medial occipital cortex in five cases out of six cases. In the sixth case (the right hemisphere of observer ABP), the strong locus of symmetry activation was derived from retinotopic area V3d. In this case, areas V1 and V2 again showed almost no significantly activated voxels, and V4v was essentially silent (as in the other cases when it was sampled). Thus, activation in V3d in one hemisphere of one observer is the only violation of the finding that the lateral occipital cortex in the region of the MOG is the site of strongest symmetry activation in our observers. The responses also suggested that symmetry was more effective in activating V3a than V1-3, but less effective than in the lateral occipital lobe (primarily in the MOG).
The data are consistent with the idea that regions in and near the MOG are involved in the processing of long-range stimulus structure. Because there is little symmetry-related activity in the early, retinotopically organized visual areas, the responses cannot be explained by the long-range connections present in area V1 (see Ref. 15 for a review), area V216,17 or other retinotopic areas.18 Instead, the symmetry-specific responses imply the existence of neurons with larger receptive fields that are driven by patterns of activity spread across the mosaic of neurons in earlier visual areas. The role of the MOG in long-range visual processing is supported by studies that found a nearby and perhaps overlapping occipital region with a large representation of the ipsilateral visual field.19 Occipital regions including the MOG have also been implicated in the processing of illusory contours20,21 and at least a subset of the MOG (the KO region) may be involved in the detection of motion boundaries.22 The detection of both types of stimuli benefits from the long-range integration of visual information.
Based on the stimulus properties, we conjecture that the symmetry information is unavailable to neurons that receive input from only a small portion of the visual field. Still, it is possible that the signals we observe are driven mainly by neurons with small receptive fields near the main axes of symmetry. To press this hypothesis and to determine whether symmetry is processed retinotopically within the MOG, we performed four additional experiments. First, we compared fMRI responses when the orientation of a single symmetry axis was either horizontal or vertical. Second, we compared activity when the symmetry was restricted either to the central visual field or to the periphery. Third, we measured responses to repetition symmetry (Figure 1c), which has long-range structure, but no foveal focus. Fourth, we replaced a 2 deg band of points falling along the axes of symmetry with purely random dots, thereby maintaining global symmetry while eliminating any local symmetry structure around the axes.
Figure 5 summarizes the fMRI responses in the lateral occipital lobe of one observer for a number of symmetry patterns. Figure 5e shows the fMRI response in a baseline scan in which non-symmetric, random dot patterns were alternated with a blank (dark) field. Significant activity is present both in retinotopic areas such as V3a (see same subject in Fig. 3b) and in the mid-occipital region adjacent to the MOG. Figures 5a-d,f-h show fMRI responses when the various types of symmetric patterns derived from random dots are alternated with purely random dots. Figures 5b and 5f compare the activity when a single symmetry axis is oriented either horizontally or vertically. The spatial distributions of the activity largely overlap, implying that the responses are not retinotopically organized with respect to axis orientation. The distributions are also quite similar to the activation region for four-axis symmetry (Fig. 5a), indicating that the inclusion of three more axes after the first confers only a small advantage. The lack of
Figure 5. Comparison of significant activation of the lateral left occipital lobe of one observer (CWT) by eight varieties of pattern stimulation. (Similar results were seen in both hemispheres and in the other two observers.) a) Responses to four-axis symmetry stimuli (Fig. 1a) vs. purely random dots (Fig. 1b). b) Responses to one-axis symmetry around the horizontal meridian. c) Responses when four-axis symmetry is restricted to the central part (0 to 6.5 deg eccentricity) of the stimulus. d) Responses to translational symmetry of repetitive random-dot tiles (wallpaper stimulus, Fig. 1c). e) Responses to the control stimulus, blank (dark) field vs. random dot patterns (Fig. 1b). f) Responses to one-axis symmetry around the vertical meridian. g) Responses when four-axis symmetry is restricted to the peripheral part (6.5 to 13 deg eccentricity) of the stimulus. h) Response to flipping the random dot field around a randomly-chosen axis every 1.5 seconds.
retinotopic organization is confirmed in Figs. 5c and 5g, which compare fMRI responses when the symmetry of the random texture is restricted to the central 1/4 or peripheral 3/4 of the stimulus field. The two conditions again activate largely overlapping regions of the MOG, although the central symmetry stimulus activated the ventral LO region more strongly in this observer.13,14 Figure 5d shows the response to repetition or translational (wallpaper) symmetry. In this stimulus, there is no local information concerning symmetry at all, but only long-range structure (Figure 1c) at the distance of one repetition cycle. Again, the significant activity is found in a region of cortex, i I ncluding the MOG, that overlaps with that for one-axis (Fig. 5b,f) or four-axis (Fig. 5a,c,g) reflection symmetry. Results similar to those in Fig 5 were obtained in both hemispheres and in the other two observers, although not all conditions were tested in all observers.
Fig. 5h provides an important control against the idea that the responses represent the activity of attending to or looking for symmetry rather than a direct response to the presence of the symmetry pattern. For this condition, the texture was completely random, but the pattern on each 1.5 s frame was derived from the previous by flipping it around a horizontal, vertical, left- or right-oblique axis. The observer often found it difficult to detect the symmetry transform in this condition, and hence was making maximum effort to determine which axis had been selected for each flip. Nevertheless, the response is much reduced in this condition relative to any others, indicating that it is the presence of pattern symmetries rather than the effort to observe them that is the basis for the signal we record. However, it is noteworthy that the small focus of significant response for the axis flip condition again lies in the MOG region, suggesting that the weak symmetry signal of the temporal sequence is again processed in the same brain region as for the spatial symmetries.
In conclusion, this initial evaluation of cortical processing of symmetry established that symmetry/random alternation is a sufficient stimulus for significant fMRI activation in a little-understood region of the human occipital lobe in and around the middle occipital gyrus. The activation did not appear to be retinotopically organized. The high level of activation in the MOG may represent part of a general class of computations that require integration of information across a large span of the visual field. These observations provide an approach to the exploration of the corresponding processing in homologous areas of other species by classic neurophysiological and neuroanatomical techniques.
Stimuli were 26º in diameter and consisted of a field of white dots, each 15' in diameter, at 25% density on a dark background. The stimuli for testing long-range symmetry processing consisted of a sequence of fields of random dots organized in patches of symmetry structure alternated with a sequence of control stimuli of purely random dots. Examples of the symmetry stimuli we used are shown in Figure 1. A particularly effective version was the four-axis reflection symmetry pattern exemplified in Figure 1a, alternating with a purely random pattern such as Figure 1b. Note that the evident symmetry structure in Figure 1a is not concentrated at the horizontal, vertical and oblique axes of symmetry but at some arbitrary position in the image determined by the perceived contrast energy of the pattern, which has the same range of values as in the random pattern of Figure 1b. The control stimulus consisted of a blank, dark field alternated with purely random dot patterns (Figure 1b). The stimulus sequence consisted of an 18 s period of blank or symmetric patterns which changed every 1.5 sec, followed by 18 s of purely random dot patterns, also updated every 1.5 sec. The alternation sequence was repeated 5-6 times per scan.
Three observers were tested in a 1.5T GE Signa scanner. The stimuli were rear-projected onto a translucent screen inside the bore of the scanner by means of an LCD projector controlled by a Macintosh computer. The observer's head was stabilized on a bite bar with the eyes looking into a 45º mirror to fixate the front of the projection screen. The observer's task was to maintain fixation at the center of the stimulus and concentrate on the stimulus pattern. No motor task was imposed, in order to limit the differential brain response to sensory processing signals. Functional magnetic resonance (T2*-weighted, blood oxygenation level dependent, or BOLD) images were collected in eight planes through the occipital lobes using a 2D spiral sequence (two spiral interleaves, TR=1500 ms, TE=40 ms, flip angle=90 deg, voxel size = 1x1x4 mm).
Data analysis and visualization:
The fMRI (BOLD) response was analyzed by extracting the Fourier fundamental of the time series at every voxel at the stimulus alternation rate of 1/36 Hz. The analysis was limited to a phase window of 0-60 deg (0-6 sec delay post-stimulus onset). A statistical correction for multiple occurrences was applied. A correlation level of 0.4 provided a protection level of p < 0.01 over the entire set of analyzed voxels (i.e., only 1% of voxels would produce spurious signals at this correlation level). Note that this means that the probability of adjacent voxels being activated would be p < 10-6, so that any clustering of the responses in adjacent voxels represents a highly significant signal. Since the activity of the individual voxels in fMRI is uncorrelated (as indicated by the few isolated locations of activation in Fig. 2), the clumping of responses into clusters of adjacent activated voxels surrounded by a sea of non-activated ones represents a highly significant spatial association of cortical responses. (For example, the probability of finding 6 coadjacent voxels by chance alone is p < 10-35).
A high-resolution anatomical (T1-weighted) 3D MRI volume scan of the entire brain was also run on each observer (voxel size=1x1x1.2mm). Gray (cortex) and white matter were segmented using publicly available software.23 The differential fMRI activity profile was then mapped directly onto the cortical manifold, to allow visualization of the response properties over complete cortical areas. The boundaries of the retinotopic projection areas V1, V2, V3, V3a, V4v were established as described in Engel et al.7 A motion vector field of expanding and contracting white dots on a black background, alternating with static dots, was used to identify area V5/MT+.
To quantify the comparison of medial and lateral occipital activation for Figure 4, we needed to define anatomical zones that were comparable for the retinotopic and non-retinotopic regions of occipital cortex. This is inherently difficult because the non-retinotopic regions are, a fortiori, unresponsive to the retinotopy stimulus, but to define them by the boundary of the significant symmetry responses means that 100% of the voxels would respond to symmetry, by definition. To make a fair comparison, we compared symmetry activation in medial and lateral occipital regions specified to be of the same size on a flattened reconstruction of the cortex.7,23 The medial area was defined retinotopically as the sum of the V1, V2 and V3 from the foveal representation to 13º eccentricity, the size of our stimuli. The matched lateral area was then defined for each observer as a square region of the flattened cortex extending parallel to the border of V3a and separated from it by a minimum of 1 cm. The ventral corner of this square was set 1 cm lateral to the ventral border of V1, and the area of the square was equated to the summed area of the medial (V1, V2, V3) regions. The cortical geometry of V3a ensures that this square region lies entirely in the lateral occipital cortex. We did not attempt to encompass the entire lateral occipital cortex because our main concern was to enforce a fair comparison with the medial retinotopic activation.
Within the two zones, we then searched for the 4 square cm area that contained the largest number of voxels that reached the significant activation criterion. The same search was applied to areas V3a and V4v at the dorsal and ventral extremes of the retinotopic region. Thus, the retinotopic and non-retinotopic zones had an equal chance of exhibiting symmetry responses on a random basis. This allowed an unbiased determination of whether the symmetry response fell predominantly within retinotopic or non-retinotopic cortex. The total number of significant voxels within the 4 square cm region was reported in a bar graph (Fig. 4). The error bars represent one standard error based on a binomial distribution, i.e. the square root of the quantity (np(1-p)), where n is the number of significant voxels, and p is the proportion of significant voxels within the 4 square cm region.