Given that many otherwise puzzling aspects of the sensation of light intensity (lightness/brightness) shown in this Demonstration can be understood in terms of a wholly empirical conception of how the visual system generates percepts, it is only logical to ask whether the color sensations elicited by different spectra arise according to the same scheme. After all, the spectral qualities of a stimulus are ambiguous for exactly the same reasons as is spectral intensity, to wit the conflation of illumination, reflectance and transmittance in the spectral return (Figure 1). Indeed, if the empirical theory outlined for the perception of luminance has merit, it should apply not only to color, but to all categories of visual sensation.

A useful starting point in any exploration of the genesis of color on an empirical basis is simultaneous color contrast, a phenomenon that shares many similarities with simultaneous brightness contrast. The standard stimulus for eliciting color contrast is two targets with the same spectral composition on differently chromatic backgrounds. As in Demonstrations, the two identical targets look different as a result of contextual differences; in the case of Demonstrations, however, the perceptual distinctions are based on differences in the apparent hue and saturation of the targets rather than lightness/brightness alone (color sensations are generally described in terms of hue, saturation and brightness, all three of which are appreciably different in comparing the two targets).

The percepts elicited by the standard color contrast stimulus in Demonstration and similar stimuli in which the same spectral targets elicit different color sensations are usually ascribed to 'adaptation' of the color system to the average spectral content of the overall stimulus (typically at the input stages), and/or to computations of spectral ratios across chromatic contrast boundaries (e.g., Land, 1986) (see Demonstration). Both these hypotheses, however, fail to account fully for all the perceptual consequences of such stimuli. Moreover, they are really mathematical descriptions rather than explanations, and provide only a limited biological rationale for color contrast (the truism usually provided is that it makes sense to see an object as having more or less the same color in different illuminants, and that color contrast anomalies are the price that must be paid for the supposed benefit of 'color constancy').

An explanation of color contrast (and constancy) can, however, be given in fully empirical terms. The sources of the target and surround in the standard color contrast stimuli shown in this Demonstration are, as all visual stimuli, profoundly ambiguous: much like the achromatic targets in this Demonstration, the same spectral patterns could have been generated by many combinations of reflectances, conditions of illumination and influences of transmittance (Figure 2). The visual system must nevertheless generate appropriate behavioral responses to the enormous variety of the spectral patterns returned to the eye. As in the case of brightness percepts, the visual system appears to solve this problem by using feedback from the success or failure of these responses to progressively instantiate patterns of neural connectivity that promote ever more appropriate reactions to the stimuli. In this largely phylogenetic process, the neuronal activity elicited by spectral stimuli comes to link spectral profiles of inevitably uncertain provenance their empirical significance. In this scheme, then, the particular pattern of neuronal activity elicited in response to a given stimulus is ultimately dictated by the relative frequencies of occurrence of the real-world combinations of reflectances, illuminants and transmittances that have given rise to successful behavior in response to that spectral stimulus in the past.

If perceptions of color are indeed generated in this wholly empirical way, then the same spectral target on two differently chromatic backgrounds should give rise to different chromatic sensations. The reason is that, in addition to requiring behaviors appropriate to the same reflectances in the same illuminant (i.e., the stimulus on the screen), such stimuli will in other instances have required behaviors appropriate to targets that arise from different reflectances in different illuminants. Consequently, the pattern of spectral returns elicits a pattern of neuronal activity that incorporates these possible underlying sources in proportion to their past occurrence in human experience with spectral stimuli. As in the case of brightness, these ideas have been confirmed by analysis of natural scenes statistics.

Evidently the colors we see are, like brightness, linked to the stimuli that generate them by the historical success and/or failure of the interactions of human observers and their predecessors with objects and illuminants (sources) in the world.