The TheoryLightness/BrightnessColorMotionGeometry & SpaceSound and Music

Research

Research

Research Overview

A Primer on Probabilistic Approaches to Visual Perception

Figure 1 View »


Figure 2 View »


Figure 3 View »

The Puzzle

If the physical reality that the observer must deal with is neither perceived nor accessible from retinal activity generated by light, how does the visual system routinely map retinal images onto successful behavior?
For decades, work aimed at deciphering the underlying mechanisms of the visual system has been predicated on the idea that vision is analytic. The general idea has been that what we see is a consequence of reconstructing physical reality using features of retinal images.
While attractive, this approach has failed to address two important facts. First, several studies have shown that what we see does not accord with the physical properties of sources, or the features of retinal images. Thus, the brightness of a light source does not vary according to its light intensity (Figure 1). (See demonstrations on the website for more examples).

Figure 1
« Close

Figure 1 / The puzzling discrepancy between perception and the physical measurements of sources. The solid line represents judgments of brightness elicited by sources of increasing light intensity with a dim background (redrawn from Nundy and Purves, 2002). The dashed line is the expected trend of perceived brightness if vision were analytic.

Second, retinal images do not specify physical sources. Referred to as the inverse problem. As George Berkeley (1709) pointed out several centuries ago, sources underlying visual stimuli are unknowable in any direct sense. In modern terms, since the light returned to the eye from any scene conflates the contributions of reflectance, illumination and transmittance (and a host of other factors that affect these parameters), the provenance of the spectral contrast at any point in a retinal stimulus (and therefore its significance for visually guided behavior) is profoundly and inevitably ambiguous (Figure 2). The same ambiguity pertains to the positional origin of light rays, since size, distance and orientation are also conflated in the retinal projection (Figure 3).

Figure 2
« Close

Figure 2 / The fundamental factors that determine the luminance of any stimulus (or component thereof) are illumination, reflectance, and transmittance. Because behavior in response to the stimulus will be successful only if the relative contributions of each of these factors are in some sense known, seeing lightness or brightness according to the physical intensities (luminances) in the stimulus as such would be a poor strategy of vision.


Figure 2
« Close

Figure 3 / The inherent ambiguity of any three dimensional object projected onto a plane. As indicated in this diagram, the same retinal projection can be generated by objects of different sizes at different distances from the observer, and in different orientations with respect to the observer.

These fundamental facts raise a puzzling question. Successful behavior in a complex and potentially hostile environment clearly depends on responding appropriately to the physical sources of visual stimuli. However, if the physical reality that the observer must deal with is neither perceived nor accessible from retinal activity generated by light, how does the visual system routinely map retinal images onto successful behavior?

An Alternative Interpretation

Using the only information available on the retina, a wholly empirical strategy gives rise to percepts that incorporate experience from trial and error behaviors in the past. Percepts generated on this basis do not correspond with the measured properties of the stimulus or the underlying objects.

A plausible answer to this puzzle is to simply abandon the long-held assumption that vision involves seeing or estimating physical properties. In this alternative interpretation, vision works by having patterns of light on the retina trigger reflex patterns of neural activity that have been shaped entirely by the past consequences of visually guided behavior over evolutionary and individual life time. Using the only information available on the retina (i.e. frequencies of occurrence of visual stimuli, light intensities), this strategy gives rise to percepts which incorporate experience from trial and error behaviors in the past. Percepts generated on this basis thus correspond only coincidentally with the measured properties of the stimulus or the underlying objects. In the following sections we discuss various perceptual phenomena using the proposed approach. For those who are interested in thinking about this in more formal terms, see the primer that compares the present approach (called empirical ranking theory) with Bayesian decision theory.

References

Purves D, Wojtach WT, Lotto RB (2011) Understanding vision in wholly empirical terms. Proc Natl Acad Sci (doi:10.1073/pnas.1012178108, March 7).

Howe, CQ, Lotto RB, Purves D (2006) Empirical approaches to understanding visual perception. J Theor Biol 241: 866-875.

Howe, CQ, Purves, Dale (2005) Perceiving Geometry: Geometrical Illusions Explained by Natural Scene Statistics. New York, NY: Springer Publishing.

Purves D, Williams MS, Nundy S, Lotto RB (2004) Perceiving the intensity of light. Psychological Rev. Vol 111: 142-158.

Purves D, Lotto RB (2003) Why We See What We Do: An Empirical Theory of Vision. Sunderland, MA: Sinauer Associates.

Nundy, S, Purves D (2002) A probabilistic explanantion of brightness scaling. Proc Natl Acad Sci 99(22): 14482-14487.

Purves D, Lotto RB, Williams SM, Nundy S, Yang Z (2001) Why we see things the way we do: evidence for a wholly empirical strategy of vision. Phil Trans Roy Soc London B-Bio Sci 356:285-297.

Berkeley G (1709/1975) Philosophical works including works on vision. (Ayers MR ed) London: Everyman/ J.M. Dent.