Christopher W. Tyler.
A worthwhile type of probe stimulus is one that is designed to be null with respect to the known responses of classes of cortical neurons. If such a stimulus is visible, it is a challenge to explain its neural substrate. The classic Necker cube may be generated in the novel form of a null contrast image, gray bars against a high contrast background. Regions of zero contrast produce no response in known cortical neurons, so the brain has no obvious mechanism to identify the gray regions in the image and unify them into a perceptual whole. Nevertheless, observation of this stimulus reveals that it is not only perceived as a unitary figure, but one that can be interpreted three-dimensionally and is subject to the well-known depth reversals of the high-contrast Necker cube. This new class of stimuli therefore represents a challenge for visual neuroscience, to explain the spatial integration for regions of no neural signal.
Advances in neuroscience may be made by probing the system with stimuli designed to test the limits of the neural response properties. One type of probe involves stimuli that are null with respect to the known classes of cortical cell sensitivities. An example of such a probe is shown in Fig. 1a. It is a version of the classic Necker cube generated in the unusual form of an inverse contrast image. The background is high-contrast random noise, while the figure consists of gray bars of zero contrast (and hence of null stimulation). The gray is set to be equiluminant with the mean of the black and white elements. This configuration inverts the normal structure of images, in which the figure has high contrast relative to a low contrast background. In this sense, the figure corresponds to a hole in the background, reminiscent of the aperture in the sky depicted in Magrittes well known picture "Grande Famille" (Fig. 1b). Nevertheless, the null-contrast Necker cube is readily perceived as an object framework rather than gaps in the texture. Indeed, the percept is so strong that it is impossible for most observers to reverse it and see the gray strips as gaps in the texture. On the contrary, it is seen as a three dimensional object where some gray strips overly other strips in depth interpolation, and which reverses in depth in much the same fashion as the traditional black-on-white Necker cube. When shown to 60 observers, all reported seeing a 3D depth interpretation that exhibited depth reversals of the two interpretations as a cube. The reversals illustrate that the null-bar configuration is sufficient to activate the monocular depth processing system based on linear perspective cues. The gray bars support object perception to such an extent that most observers are surprised to realize that they are, indeed, simply regions of the absence of texture. On the other hand, this percept loses its robustness if the bars are made thinner, similar to the original black bars of the Necker cube (Fig. 1b). When viewed from about 1 m, the cube in this second version disappears, illustrating the invisibility that was expected in this null stimulus.
a b c d
Fig. 1a. A Necker cube represented by bars of zero contrast in a filed of high-contrast random dots. b. A second version of the null Necker cube with thin gray bars. The cube easily disappears when viewed from a distance. C. An artistic representation (Magrittes "Grande Famille") of an object as a region cut out of its background, though not at reduced contrast in this case. This picture illustrates the tension between figure and ground interpretations of the 'object'. d. A modification of Magritte's figure to provide the dove in the null-contrast representation of (a). The interior of the dove is set at the mean luminance and color values of the surrounding regions.
What cortical mechanism can encode the null stimuli?
The standard explanation of low-level vision is that patterns such as the classic Necker cube in Fig. 2a are encoded by arrays of neurons with oriented receptive fields1, which can be modeled in the form of one-cycle Gabor profile2-4 such as those shown in the inset at the left of Fig. 2a. To illustrate this explanation, the Necker cube stimulus of Fig. 2a may be convolved with such receptive fields to estimate the response over the arrays. Examples of the responses of such arrays are shown for a (odd-symmetric) vertical 'edge' detector in Fig 2b, an (even-symmetric) vertical 'bar' detector in Fig. 2c, and a (rectifying) vertical complex cell in Fig 2d. In each case, the receptive field is depicted in the inset at lower left, and the estimated neural firing level is indicated in the main image by the lightness above the black background. The signals available from such arrays are then passed on to higher-level visual processes for further encoding.
a b c d e
Fig. 2. Representation of classic low-level neural responses to a line stimulus. Receptive fields of the Gabor type shown enlarged in the inset at left of a are convolved with the line Necker cube (a). The resulting firing rates of the receptive field arrays are shown for even-symmetric (b), odd-symmetric (c), complex rectifying (d) and hypercomplex field types. In each case, only the region of positive response is depicted, since neurons do not fire in regions of negative response.
The lines in the gray Necker cube of Fig. 1a constitute a midlevel gray set in a field of random black and white elements. This type of line is a null stimulus, in the sense that typical cortical cells have balanced on- and off-responsive regions and will fail to respond when the gray lies over their receptive field. Such cells will have variable responses from a low to a high level in the region of the high contrast random dots, depending on the appropriateness of its configuration to their receptive field structure. However, the gray bar represents a region of null response. In the presence of the gray bar, they will send no information up to subsequent levels of neural processing. Elsewhere, they will respond sporadically throughout the background region. It seems that the gray bar is would be an invisible null stimulus to classical receptive fields. To validate this assertion, we may evaluate the response to this stimulus in arrays of typical cortical cells of the simple, complex or hypercomplex types1. In this determination, the background activity of the cells is assumed to be zero.
To illustrate the lack of cortical response, the stimulus of Fig. 1 a is convolved with arrays of model simple-cell receptive fields in the form of one-cycle Gabor profiles. The outputs of arrays of odd even and rectifying receptive fields to the null-contrast bar is shown in Figs. 3 b and c for a field size scaled to the size of the bars. Black represents no response, with increasing brightness representing increasing levels of response. The area of the gray bar is indistinguishable in all cases, illustrating that none of the cell types respond in the region of the gray bar, which is therefore invisible all of these cell types of either polarity (to the extent that they are represented by this linear computational model).
a b c d e
Fig. 3. Representation of low-level neural responses to the null stimulus (a), in the same format as for Fig. 2. Receptive fields of the same Gabor type as in Fig. 2 are convolved with the null Necker cube of Fig. 1a. The resulting firing rates of the receptive field arrays are shown for even-symmetric (b), odd-symmetric (c) complex rectifying (d) and hypercomplex (e) field types. There is no distinguishable response for any of these neural types at this coarse scale of analysis.
?Perhaps the scale selection of Fig. 3 is non-optimal for revealing the null-contrast bar. To address this concern, the same convolution is performed for receptive fields on the scale of the pixels of the background noise, scaling down by a factor of 4 to address the other extreme of plausible size scales. Figs. 4 b and c show that the situation is somewhat different, in that the cells respond variably throughout the noise background but not at all in the region of the gray bar. (Not shown is the response profile for circularly symmetric cells of the on- and off-type. The response profiles are similar to those of Fig. 3 a and 4 a except that the local activations are isotropic rather than elongated.) Since black in these figures encodes regions where the cells have no response, the gray bar is invisible to cells of a fine scale, just as for the coarse scale. ?A further option is to rely on the response of complex cells. Their response corresponds to the phase-independent contrast energy of the image, shown for the same two cell sizes as for the simple cell plots in Fig. 3 c and 4 c. The response to the background is now denser (and isotropic) by virtue of the inclusion of all response phases, but the lack of response to the gray bar is equally clear. At the coarse scale (Fig. 3 c), there is no distinguishable difference between the background and gray bar regions. At the fine scale, the response profile is entirely black (no response) within the region of the gray bars and shows no special activity along their edges. The gray bars are just as invisible to the complex cell as it was to the simple cells.
Grosof, Shapley and Hawken5 have measured complex cell responses to grating edges. They find that these responses are nonlinear, responding to the illusory contour orientation at the edge of the grating, and might therefore be able to encode the null-contrast Necker. Similarly, von der Heydt and Peterhans6 report hypercomplex cells in monkey visual area V2 that respond to illusory contours and the edges of grating orthogonal to the bars. Again, it seems that such cells should respond to the passage of a noise edge across their receptive field and hence provide a basis for encoding the null-contrast structure. Heitger et al.7. propose a model of end-stopped cell responses in which there is a second-order unit pooling the responses of oriented cells across their orientation to detect second-order edges of the type in cross-cut gratings. One might think that this type of mechanism would be sensitive to the orientation of the edges of the noise stimulus.
However, the issue is not whether the cell responds at the edge, but whether it responds more strongly at the edge than elsewhere. If a cell responds throughout in the background, including the edge, it cannot be said to be exhibiting a selective edge response. The edge of a noise field differs from the edge of a grating in that the local textural properties are continuously varying in the noise field. Any given receptive field will therefore be stimulated at points throughout the background. Just as for the models of Fig. 3, the hypercomplex responses will drop to zero in the region of the gray bar. It will simply be a region of more extensive null response than other regions. Summing across the local detectors in the direction of the gray bar orientation produces no greater response at the edge of the noise than does the original local responses. In fact, the V2 cells reported by Peterhans and von der Heydt8, which respond to gaps in collinear lines bridging the receptive field, will even tend to fill in across the gray gaps as though they did not exist.
a b c d e
Fig. 4. Representation of low-level neural responses to the null stimulus (a), in the same format as for Figs. 2 & 3, but for a small-scale receptive field. The resulting firing rates of the receptive field arrays are shown for even-symmetric (b), odd-symmetric (c) complex rectifying (d) and hypercomplex (e) field types. All these cell types respond at or close to zero for in the regions of the gray bars. The visible black figures in these response plots correspond to regions of no signal transmission to the next stage of analysis.
Extracting significant signals from the population response.
The implicit (or explicit) model of most concepts of brain processing is that there is a histogram of cell responses at some level of processing on which one can select the appropriate cells by setting some response criterion. Fig. 5 shows examples of histograms for the (even-symmetric) simple, complex and hypercomplex cells for the stimuli with an embedded gray region, compared with those for the background region alone and. It is clear that the background region alone contains a large number of locations of zero response. Adding the gray bars increases the proportion of zero responses, and also of the lowest level of non-zero responses. However, in order for subsequent processing levels to isolate the gray stimulus regions, they would need to set a criterion level above which the responses would indicate the presence of the relevant stimulus. No criterion set on the histogram of any of the cell responses will select for the gray regions because the response proportions are equal to or less than those for the background field alone at all higher response levels. Selecting all the responses above a criterion of, say, level 5 will select only background locations, since all the gray bar locations give responses at a level close to zero. Only for null or very low responses are the proportions distinguishable, but here there is no (or minimal) response to carry the information to higher levels. Thus, the outputs of the three cell types cannot provide the information required for perception of the Necker figure.
Fig. 5. Histograms of the responses to the background alone (cyan) and the embedded null Necker stimulus (magenta) for three cortical cell types of Fig. 4. The first pair of bars refers to the null response level, which is depicted on a broken scale in order to make the other response levels visible. Note that the only significant differences in the response proportions are at or close to the level of zero response for all three cell types.
Generating an invisible stimulus is hardly news. The striking point is that the gray bar is readily visible to us as human observers. We see it as an integrated whole, as an organized form with a highly identifiable orientation and as having sharp edges. It is so readily perceivable that most observers struggle with the concept that it is invisible to cortical cells. It seems that this claim cannot be correct, and yet the illustrations of Figs 3-5 clearly imply that our present understanding of basic cortical cells implies a complete lack of response to this stimulus. Note that the black regions in Fig. 4 are readily visible to us as an object, but the black is not the stimulus, it is a representation of no response from the cells. No signal is being sent from this region to subsequent cortical layers.
In summary, the analyses presented combine to imply that no known cortical cell can signal the presence of an equiluminant gray bar embedded in a noise background. The null-contrast stimulus therefore represents a challenge to cortical neurophysiology and computational vision to explain the response of the human perceptual system to the structural interpretation of the gray bars. It is not hard to generate a computational model that could exhibit the bar response by providing a mechanism for subtractive normalization and sign reversal following the rectification stage of the fine complex and hypercomplex cell responses in Fig. 4. The challenge is to characterize empirical cortical cell responses corresponding to such hypothetical response behaviors. The concept of the null-contrast stimulus represents a new way to probe the neurophysiological organization of visual processing.
1. Hubel DH, Wiesel TN. (1968) Receptive fields and functional architecture of monkey striate cortex. J Physiol. 195(1):215-43.
2. Field DJ, Tolhurst DJ (1986) The structure and symmetry of simple-cell receptive-field profiles in the cat's visual cortex. Proc R Soc Lond B Biol Sci. 228(1253):379-400.
3. Stork DG, Wilson HR. (1990) Do Gabor functions provide appropriate descriptions of visual cortical receptive fields? J Opt Soc Am A7:1362-73.
4. Watson, AB. (2000) Visual detection of spatial contrast patterns: Evaluation of five simple models Optics Express 6(1):12-34.
5. Grosof DH, Shapley RM, Hawken MJ (1993) Macaque V1 neurons can signal 'illusory' contours. Nature 365:550-2.
6. von der Heydt R, Peterhans E. (1989) Mechanisms of contour perception in monkey visual cortex. I. Lines of pattern discontinuity. Journal of Neuroscience 9:1731-48.
7. Heitger F, Rosenthaler L, von der Heydt R, Peterhans E, Kubler O. (1992) Simulation of neural contour mechanisms: from simple to end-stopped cells. Vision Research. 32:963-81.
8. Peterhans E, von der Heydt R. (1989) Mechanisms of contour perception in monkey visual cortex. II. Contours bridging gaps. Journal of Neuroscience 9:1749-63.