New gamut relativity paper in PloS ONE

The latest gamut relativity paper has now been published in the open-access journal PloS ONE. The paper is entitled “A Unified Account of Perceptual Layering and Surface Appearance in Terms of Gamut Relativity” and is co-authored by Mark D. McDonnell.

This article gives the first unified theoretical account of the famous Adelson checkerboard and Anderson-Winawer lightness effects (see below), two of the most powerful and dramatic illustrations of the dissociation between surface appearance and light intensity at a given image point. See our article for detailed analyses of these fascinating phenomena.

journal.pone.0113159.g001

Two dramatic effects of perceptual layering and surface appearance. (A) Adelson checkerboard: Checks labelled A and B (depicted as appearing in bright and dim illumination) have the same point-to-point luminance but check B appears light gray and check A dark gray. Checks B and D are seen through a ‘transparent shadow layer’, whereas checks A and C are seen in ‘plain view’ (without an accompanying transparent layer). Variations in illumination intensity level produce multiplicative changes in the luminance values depicted as being reflected from the checks in bright and dim illumination. (B) Anderson-Winawer effect: Chess pieces in the upper and lower rows have the same point-to-point luminance but appear white and black, respectively. The white pieces are seen through a blackish transparent ‘atmosphere’ whose transparency varies across space, while the black pieces are seen through a transparent whitish atmosphere. Variations in atmospheric transmittance levels produce additive changes in the luminance values depicted as being reflected from the black and white chess pieces. This article develops a model that aims to quantitatively predict surface lightness through transparent layers, irrespective of the physical source of the transparent layer.

Advertisements

Gamut relativity article featured in highlights column

I just came across this commentary, published in the “News from the field section” of the journal “Attention, Perception, & Psychophysics” (April 2013, Volume 75, Issue 3, pp 383-387), on my recent Journal of Vision article that introduced the theory of gamut relativity. Overall I think the writer does an okay job of summarizing the main findings, although I honestly don’t know what they are trying to say in the final paragraph:

LIGHTNESS
White light

Vladusich, T. (2013). Gamut relativity: A new computational approach to brightness and lightness perception. Journal of Vision, 13(1):14.

You might think a comparison between the intensities of two light sources would be, if not the easiest possible perceptual task, at least among the easiest. Well, it may be easy, but modelling it sure is not. Particularly puzzling are comparison data from conditions in which the two light sources resemble differently illuminated surfaces. (Defining exactly what might be necessary for that resemblance is, itself, a can of worms.) In a recent paper, Vladusich (2013) models data such as these outside the context of conventional notions regarding light intensity.

Rather than impressions of luminance (‘brightness’) or reflectance (‘lightness’) Vladusich finds that the intensities of two light sources appear most similar when they are as close as they can be in the ‘blackness’/‘whiteness’ plane. Although others have posited multiple dimensions for the apparent intensity of individual light sources, blackness and whiteness are different.

Increases in the luminance of any object on your computer’s screen will result in greater whiteness and less blackness, as long as the viewer’s interpretation of the scene doesn’t change. Decreases in the object’s luminance will result in less whiteness and greater blackness. All possible combinations of whiteness and blackness for that object will lie on a line in the blackness/whiteness plane. Blacknesses and whitenesses of objects, subject to different apparent illumination, will occupy a different line (or gamut) within the plane.

Vladusich offers a detailed explanation of how to compute the blackness and whiteness of any arbitrary ‘achromatic colour,’ and with his model he reproduces points of subjective equality (PSE, or perhaps points of least dissimilarity) from many intensity comparisons in the literature.

Scene interpretation is of paramount importance in the assignation of blackness and whiteness values, and different interpretations (and consequently different degrees of ‘scission’) are necessary for something that appears to have greater brightness, at other times appear to have less lightness than a comparison stimulus.

The fact that observers can judge one group of pixels to be more intense than another is irrefutable. Ironically, Vladusich’s model can predict the luminances for which these judgments will be least reliable (that is, at the PSE) without assigning apparent intensities to either group. How that is done remains a mystery.—J.A.S.

Related articles

Why is Grey-Scale Matching Hard?

One of the most pervasive phenomena in color vision science is the difficulty encountered in matching pairs of color or grey-shade samples that appear within different contexts. An attempt to depict this matching difficulty is shown below in terms of two series of grey-scale disks seen against either black or white rings. People generally find it impossible to choose a disk surrounded by black rings to establish a “satisfactory” match with the “Reference” disk surrounded by a white ring. The disk chosen here as “Match”, for example, leaves the distinct impression that “something is missing” from the physical grey scale—that the range of disk luminance values is alone somehow insufficient to capture the full range of grey-scale experience. (Note that the difficulty persists even when the person is allowed to freely adjust luminance to any value, not just the discrete samples shown below.) Why should matching difficulties arise? And what do they tell us about vision?

Slide1

The theory of gamut relativity was originally introduced as a means to account for the impossibility of satisfactory grey-scale matches when targets are viewed against different backgrounds. It seems appropriate to revisit the issue here in order to address the degree to which the theory can, in fact, account for the empirical observations. One can think of this post as providing background material and additional commentary to supplement my 2012 Vision Research article entitled “Simultaneous contrast and gamut relativity in achromatic color perception.”

One often reads in the literature that grey-scale matches are impossible to make when one sample is an increment and one a decrement (increments have higher luminance than the immediate backround and vice versa). Yet the experimental subject almost always has the choice to match an increment to an increment, or a decrement to a decrement, but often rejects this choice in order to make increment-decrement or decrement-increment matches. Furthermore, increment-increment or decrement-decrement matches are often also unsatisfactory, as shown below.

Slide2

An answer to the question of why subjects sometimes match increments to decrements and vice versa was provided in my Vision Research paper. I proposed a theory (gamut relativity) in which subjects compromise between matching the luminance and contrast of samples. As samples seen against different backgrounds cannot generally be simultaneously matched in terms of both luminance and contrast, the subject compromises between the two, as shown below.

Slide3

The model also suggested why luminance and contrast do not add together to produce a satisfactory match, as would be predicted by the classical one-dimensional (1D) theory of the grey scale. The proposed answer in gamut relativity was that the perceptual space of grey shades is two-dimensional (2D). The hypothesis that grey shades are represented in a 2D blackness-whiteness space has, in particular, provided a cogent quantitative account of key aspects of grey-scale matching difficulties in which targets are viewed against different backgrounds. The key idea was that grey-scale “matches” actually represent minimal perceptual mismatches between points lying on different black-to-white gamut lines associated with different background luminance values. The theory also accounted for perceptual mismatches between targets seen under different illumination levels, in addition to a wide body of related data on surface perception under variable illumination, as reported in this article. These findings lent strong empirical support for gamut relativity.

One issue that gamut relativity does not currently address, however, is why matching is particularly difficult with low-contrast targets. The theory incorrectly predicts that  observers should find such matches no less satisfactory than any other match, yet the perceptual data presented in my paper suggests that such matches are rated as less satisfactory than those made with high-contrast targets.

Slide4

It is interesting, in this context, to note the hypothesis of Ekroll and Faul relating low-contrast matching difficulties to the so-called “crispening effect” (see above). Crispening refers to the enhanced perceptual difference between samples as their luminance approaches that of the background against which they are shown (compare the difference between disks indicated in blue and yellow above to those closer to either end of the display). The idea is that the grey shade associated with a low-contrast target “splits” into separate figure and ground components, leading to a relatively larger perceptual difference between figure and background (and other figures). According to this view, the same process may also ensure that grey shades associated with targets seen against different backgrounds are perceptually unique (and therefore unmatchable), thereby explaining the difficulty in matching low-contrast targets.

Slide5

Supporting this idea, the “scission” phenomenon that occurs when a low-contrast target is viewed against a uniform background is similar to, but not as strong as or as obvious as, the scission that occurs during perceptual transparency. As shown in the example above, where black rectangular tabs appear to continue below a white transparent surface, the low-contrast portions of the tabs split into figure and ground components, each belonging to one or the other perceptual layer. The view espoused by Ekroll and Faul—namely, that scission underlies matching difficulties in situations where pairs of targets are viewed against differently colored uniform backgrounds—does not account for a range of perceptual data, such as the observation of matching difficulties with high-contrast targets and under different illumination levels. It remains to be seen, in this respect, whether the explanation of matching difficulties provided in gamut relativity can be combined with the theory’s account of scission, transparency and layered surface representation to quantitatively explain low-contrast matching difficulties and crispening associated with uniform backgrounds.

A Brief Introduction to Gamut Relativity

This blog is about gamut relativity, which is a new theory of surface perception and computational vision that I recently introduced into the scientific literature (read about this blog and its author here). I am currently further developing the theory in my capacity as Research Fellow in the Computational and Theoretical Neuroscience Laboratory at the Institute for Telecommunications Research, University of South Australia. My research was recently featured in a story posted at ABC Science News online. You can find links to several published research articles on gamut relativity at my university homepage here. Below I explain some key aspects of the theory in a manner intended for a general audience.

  • What does gamut relativity aim to explain?

When we look at the world, we perceive things like a frosted white glass, a glossy black car, or a brightly lit moon through a grey cloud bank. That is, we experience surfaces as varying in appearance along different dimensions, such as black to white (lightness), matte to glossy (glossiness), and opaque to transparent (transparency). These experiences are partly based on the way that physical surfaces reflect and transmit light and partly on the way the brain interprets the patterns of light reaching the eye. How the brain constructs our experience of different surface dimensions from these light patterns has long remained a mystery. Gamut relativity aims to explain the nature of the brain representations and computations that underlie these experiences.

  • What does the term gamut relativity mean?

A gamut in the theory is a range of perceivable grey shades between a shade of black and a shade of white. These grey shades are represented as ‘blackness’ and ‘whiteness’ co-ordinates in a two-dimensional perceptual space that I have termed blackness-whiteness space (see picture below). The relativity part of the theory describes the dependence of the grey-scale gamut on how transparent or glossy a surface appears. Different gamuts correspond to different ‘slices’ of blackness-whiteness space, each represented by a different line joining points on the blackness and whiteness axes. These lines correspond to different levels of surface transparency and/or gloss.

glossy_transparent_gr

  • Why is the theory potentially revolutionary?

Gamut relativity questions the widespread assumption in psychology and neuroscience that the dimensions of visual perception correspond to the dimensions of the physical world. This assumption is called reification and is related to a well known logical fallacy. Classical theories of surface perception suppose that the brain rather literally ‘represents’ the physical dimensions of surfaces as dimensions, like building a miniature version of the world inside the brain. According to these theories, lightness, gloss and transparency constitute perceptual dimensions corresponding to the physical dimensions of matte reflectance (the tendency of surfaces to reflect certain amounts of light equally in all directions), specular reflectance (the tendency of surfaces to reflect more light from certain viewing angles) and transmittance (the tendency of surfaces to transmit light rather than reflect it), respectively. If gamut relativity is correct, and the dimensions of perception do not correspond to the dimensions of physics, the theory is likely to have wide ranging ramifications for psychology and neuroscience.

  • How then does gamut relativity explain lightness, transparency and gloss?

The new insight in gamut relativity is to show how brain computations can give rise to the experience of surfaces as varying along different physical dimensions, without literally representing those dimensions in the brain. Gamut relativity thus avoids some of the philosophical traps that have plagued conventional theories of vision, and in so doing provides the first unified account of how we see glossy and transparent surfaces. Gamut relativity predicts that the properties of lightness, gloss and transparency all emerge naturally from brain computations that function to represent surfaces, illumination and atmospheric media. According to the theory, glossy and transparent surfaces form a type of ‘mirror image’ representation in blackness-whiteness space: gamut lines lying below a ‘standard’ gamut line represent different surface transparency levels and gamut lines lying above the ‘standard’ gamut line represent different surface gloss levels. Lightness emerges in terms of invariant relationships between points lying on different gamut lines, enabling the familiar experience of lightness constancy. Illumination properties (shadows, shading and highlights) are represented in a similar manner. Until now, different theories had been needed to explain the apparently different properties of lightness, gloss and transparency. Gamut relativity now provides the first unified description of how the brain represents these surface properties. This type of unification is familiar in the ‘harder’ sciences, such as physics and chemistry, but is rare in computational vision science.

  • If reification is wrong, what is the right way to think about perceptual dimensions?

The theory predicts that whiteness and blackness compose the perceptual dimensions underlying surface perception, and that these dimensions are related to the organizational feature of the visual brain known as the ON and OFF channels. These channels transmit signals encoding increases and decreases in light intensity levels from the eye to the brain. The visual ON and OFF channels have been studied for more than 50 years but their functional purpose has yet to be fully elucidated. The theory thus sheds new light on the functional purpose of these channels in terms of the computations that underlie surface perception. More generally, the theory suggests that perceptual dimensions are more closely related to the computational organization of the visual brain (blackness-whiteness space) than to the dimensions of the physical world.

  • How does the theory work?

Blackness-whiteness space can be understood as a specialized form of representation that enables the computation of surface, illumination and atmospheric properties. These computations take the form of the familiar operations of vector arithmetic, such as vector addition and decomposition. Such vector computations can be thought of as natural expressions of the computational organization of the visual brain (blackness-whiteness space).

  • What’s the current status of the theory?

Gamut relativity suggests novel solutions to many outstanding problems of surface perception, explaining a wide range of perceptual phenomena and data not previously explained in a unified way by existing theories. The theory also makes a wide range of quantitative predictions about surface perception in humans, and is likely to lead to applications in the design of engineered vision systems, such as computer, robotic and bionic vision systems. A great deal of work needs to be done, however, to fully account for the huge variety of perceptual phenomena pertaining to human surface perception and to develop algorithms capable of computing surface properties given an arbitrary image.

  • Can I get involved in this exciting new research area?

Yes. I am currently recruiting motivated students to undertake Honours or PhD research on gamut relativity or related topics in computational vision science. I’m also interested in collaborating with experienced researchers across several disciplines, ranging from psychology and neuroscience to engineering and the arts.