Looking Into Pictures: An Interdisciplinary Approach to Pictorial Space

by Heiko Hecht, Robert Schwartz, and Margaret Atherton (Eds.)
The MIT Press, Cambridge MA,. 2003
435 pp., illus. 126 b/w, 5 col. Trade, $ 58.00
ISBN: 0-262-08310-8.

Reviewed by Fred Andersson
Department of Art History and Musicology
Lund University, Box 117, 221 00 Lund, Sweden

konstfred@hotmail.com

This acclaimed volume is a collection of papers and texts by philosophers and psychologists from both sides of the Atlantic, published two years ago. It presents, in a comprehensive manner, an important move within current picture theory towards "reconsidering pictorial space". Of utmost importance here is the implied distinction between pictorial space (or different kinds of pictorial spaces) and ordinary perceptual space: a distinction that many psychologists have failed to make. Now, I will begin with some rather personal and preliminary reflections before regarding each individual contribution in the book. Because from my perspective, as an art historian with an interest in pictorial semiotics, the project of reconsidering pictorial space has to be placed in a wider context before the details of individual projects can be scrutinized. For me, the main question is a rather basic one: Could the changed understanding of pictorial space further a better dialogue between disciplines such as art history and psychology? Could both sides gain something from such a dialogue? I think the answer is yes, and I think that books such as this one are absolutely necessary for the dialogue to happen at all.

On the other hand, it may not seem obvious at first sight why an art historian or even an "art theorist" or "art critic" should be interested in psychology at all. Our aesthetic and historical science or Wissenschaft (as we prefer say in Sweden, like in Germany) is generally idiographic, not nomothetic. In other words: we write of the uniqueness of single phenomena, or of the specificity of the specific, just as implied by the Greek etymology of the word "idiographic" (idios / grafein). We are interested in interpretations of certain actions, works and utterances. We are also interested in the historical and hermeneutic ground for such interpretations. And from semiotic theoretical perspective, we might inquire not only what something means and why it means this (in relation to the historical "situatedness" of every interpretation), but also how it means. We might, in other words, inquire the constitution of actions, works and utterances as signs.

Those psychological and aesthetic theories that never reach the level of culturally determined signs and interpretations are then deemed to be quite uninteresting for us. We easily recognize the danger of an approach such as that of Semir Zeki in his Inner Vision–Zeki clearly aims at a biological interpretation of the whole cultural history of images, but the distance between such broad historical hypothesizing and the rather limited physiological phenomena isolated by modern brain science must be judged as way too far. To explain art historical development as some kind of unconscious consequence of the modularity of vision (kinetic art as the art of region V5, to take one example from Zeki) would from our perspective amount to just one speculative trend among many others.

Equally problematic in this context are theories that simply take for granted that looking at a picture is just like looking at a real scene, and/or that pictures couldn’t possibly be defined as signs. Because from our perspective, the interesting thing isn’t the various possible similarities between image and reality, but rather the ways in which they differ. Part of the meaning of any work lies in its deviation in relation to established norms for the representation of reality and society, and in relation to what is considered to be normal vision. These deviations are rhetorical in a most basic sense, in that they exhibit the character of the image as image or the utterance as utterance, not as image or utterance of some denoted thing. This gives even images a basic sign quality in terms of connotations–i.e. features that are exclusively connected to the Expression plane of the sign, thus indicating a rupture between a possible Expression plane and a possible Content plane (I write "possible", because pictorial semiotics still operates at a rather tentative and open level). If we, on the contrary, suppose the existence of an image that is just like reality, it would have neither this division between Expression and Content, nor any connotative qualities connected to the Expression plane. It would therefore be meaningless in a strict sense, i.e. completely transparent and neutral. And, to finally acknowledge the utmost absurdity: It would even be Reality itself .

Anyone would realize that this idea is absurd, and that images must be something else than Reality. Still, images have constantly appeared in mainstream psychological experiments and literature as surrogates for and equivalents of real scenes. From classical Gestalt theory and onwards, psychologists have faced serious difficulties in coming to terms with the difference between 2D and 3D perception, in case this difference was even acknowledged. Due perhaps to the fact that images were used as surrogates for real scenes, psychologists have had a limited conception of typical image-hood or picture-hood: as if an image per definitionem must be a photograph or a drawing/painting in linear perspective. This limited approach has come to be known as the projective model of pictorial space: the notion that an image typically is a 2D projection or "shadow" of a 3D distal object, seen from one single viewpoint.

This model was challenged in the 1980s, notably by Margaret A. Hagen, whose propositions in Varieties of Realism and other texts stirred some controversy. If we refrain from the culturally determined idea that pictorial realism equals projective and/or photographic realism, so goes Hagen’s basic line of argument, then there is no reason to believe that any system of depiction would be more optically valid than another, and consequently there is "no development in art". This seems to be in tune with anthropological relativism and the semiotic turn in picture theory: Any kind of image would be considered as merely a selection of relevant aspects and structures of the visual world, and no specific kind of selection would be preferred as the absolutely normal and neutral case.

Hagen’s position is however open to various criticism. For example: if there is no development in art (i.e. in the construction of pictorial spaces), how would we then account for the fact that a photograph by means of its mechanical generation is at least a close approximation of what happens when optical rays reflected from surrounding objects project themselves on to the retina of the eye? How is it possible to deny that the invention of such images was once a novelty, constituting a new development in terms of realism? Wouldn’t that be tantamount to denying that the basic functions of everyday vision must be essentially the same for all humans and all cultures? Wouldn’t the absence of development also imply the absence of a shared reality or world-view against which such development could be measured?

If this is a relevant philosophical problem (and I’m not at all sure that it is), then Ernst Gombrich’s explanation of art historical change would face a different and even diametrically opposed problem. In analogy with Popper’s dictum for the general theory of science, Gombrich saw the image as a kind of hypothesis to be falsified and replaced by more accurate images when tested against sensuous visual experience. If we stick to theories of this kind, there would be no doubt that there is a shared Reality in a strong sense: i.e. a Reality outside of subjective experiences, completely external to the mind. Under such conditions, and contrary to Hagen’s view, there would necessarily be development in art (or in "image-making"), just because we would never know if an image or even vision itself corresponds to the real world "out there". The problem is that this approach furthers a traditional, normative version of art history, in which art aims at some ideal (even Hegelian) correspondence between image and reality, and in which periods without realism in a classical or Western sense are seen more or less as periods of decline. It is at this point that Hagen’s theory becomes highly interesting as a challenge to art history as a discipline. It’s a wonder that it hasn’t caused more attention outside of psychology. Maybe the reason is that it is still very hard to even understand what it would mean that there is no development in art. We’re surely too stuck with developmental and teleological thinking: a one-sided Western "perspective".

To reconsider pictorial space is to offer a new framework for posing and understanding questions of this kind–questions about the relation between Image and Reality. It gives art historians and other non-psychologists a good reason to study psychology, and furthers an interdisciplinary approach in which the whole process of construction and reception of images is under scrutiny, rather than just a fine-grained analysis of the reception of isolated stimuli. Three elements would be indispensable for such an approach. First: to consider the fact that an image is in itself an object in space and not just the equivalent of some proximal stimulus on the retina. This approach means that there is always a dual character of picture perception: we perceive the picture as material object at the same time as we perceive the image that is visible in the object. Second: to consider the fact that there are many kinds of images other than flat ones in linear perspective, and also many situations in which we see something in something else, without referring to these experiences as images. Third: to consider the fact that to look at something and to construct an external image of it are two very different things: the eye is neither a camera, nor a simple perspective device.

The volume at hand has three thematic sections corresponding quite exactly to these elements. Part one is called "The Dual Nature of Picture Perception"; part two, "The Status of Perspective"; and part three, "The Nature and Structure of Reconceived Pictorial Space". Part one is prefigured by Richard Wollheim’s essay "In defense of seeing-in"–a comprehensive introduction to his influential theory of seeing-in, and also a beautiful piece of writing. What makes Wollheim’s theory so important, and even indispensable in any basic curriculum about picture perception, is that it offers a clear-cut philosophical distinction between Shape, Image and Illusion. The simplest shape always offers the possibility of seeing it as something–like Wittgenstein did when he gave a list of possible interpretations of a triangle in his Philosophical investigations. This is a conscious, fully cognitive act. It’s very different from seeing-in–to immediately see someone or something in the image. And this process is, in turn, very different from simply seeing something, and to be fully convinced that it’s real. In some cases this conviction turns out to be wrong–then, and only then, we have an illusion in the strict sense. One of Gombrich’s many mistakes was that he insisted on a close connection between Image and Illusion, which also meant that he neglected the difference between knowing and not knowing that something is fictional. Wollheim’s account offers a much better ground for further investigations.

In the same section, there are a number of illuminating examples of how such investigations might be done. Rainer Mausfeld draws upon a wide range of interdisciplinary research to demonstrate that the dual nature of picture perception is but one example of a general human ability of experiencing simultaneous and sometimes conflicting aspects of one and the same phenomenon (the duality of metaphors and other rhetorical figures of language is one other example). H. A. Sedgwick discusses some important explanations of depth perception in direct vision and in images. Reinhard Niederée and Dieter Heyer sketches the outlines of a revised model of picture perception that takes the dual aspect seriously, and that considers the whole range of non-photographic, non-planar and/or non-pictorial cases in which something is seen in something else (for example in a curved mirror, a shadow or a sculpture). They also offer some useful diagrams.

Mark Rollins summarizes this section and presents what he calls SDT (Strategic Design Theory) as a general attempt in current cognitive science to reconcile the classic conflict between "top-down" psychological theories that emphasize constructivist explanations (as those typically referred to by Gombrich) and "bottom-up" theories that prefer to explain recognition and depth perception as direct or even innate (including Gestalt theory, the ecological approach of James J. Gibson, and Rudolf Arnheim’s stance within art theory). There is a general agreement today among both neurologists and cognitive theorists that this conflict has become obsolete in the light of new findings. According to SDT, innate capabilities combined with "trial and error" of the constructivist type can form "perceptual strategies" in individual development. This is still a quite fresh area, and Rollins’ essay is maybe the one contribution that would be hardest for a non-academic reader to comprehend. Because in order to understand what a combination of opposites would mean, one first has to understand the opposition . . .

Typical of some "top-down" theories is the claim that Western or linear perspective is merely a convention (and not, as with Margaret Hagen, one valid pictorial space among others). In the strongest sense, this view would mean that there is no natural connection at all between an optical array and its projection in, for example, a photograph. Nelson Goodman once expressed this view as a part of his basically nominalist conviction that likeness is not a decisive factor for depiction at all. However, it seems that Goodman mixed up two different things here. It’s true that several different objects can generate the same 2D pattern if projected, and that the patterns thereby can yield different interpretations. But this doesn’t necessarily imply that the optical rules of projection are in themselves arbitrary. The generation of a pattern is something else than the interpretation of the same pattern.

Another flaw in many accounts that take the conventional nature of perspective for granted is that we are supposed to experience our natural environment almost like a globe seen from the inside: All lines would then be curved, and they would converge in all possible directions. Such notions are very popular among both artists and art historians–I once embraced them fully myself. But whatever the phenomenological reality of curved, subjective space (it would belong to peripheral vision and maybe even to the "optical unconscious" in the terms of Rosalind Krauss), we know that the straight lines are straight because in our brains the curved projection on the retina is straightened out. Here again, we have to make a clear distinction between the generation and the interpretation of a given 2D pattern. And the assumption that a truly naturalist painter would represent all straight lines as curved is as illogical as the assumption of one writer who once supposed that the deformations of space in Paul Cézannes paintings were due to a hypothetical astigmatism of the artist. Say that Cézanne was really an astigmatic–then he would have seen some contours as double and/or distorted. Had he then depicted them just the way he saw them, the depiction would have been perfectly correct by any naturalist standards. Because when comparing the canvas and the 3D scene, the astigmatic would of course see the same distortion in both!

In section two of the book ("The Status of Perspective") five authors give us an abundance of useful and revealing arguments against forced conventionalism. With some clear and simple demonstrations, the German philosopher Klaus Rehkämper shows exactly what could be expected as well as not expected from the use of a linear perspective device. I think this short and brilliant text should fit as an obligatory in any basic course of art, photography, or art history. Rehkämper’s compatriot, the semiotician and communication theorist Klaus Sachs-Hombach, defines in the same section the basic assumptions of contemporary pictorial semiotics. This is a theory in which the sign character of images doesn’t require arbitrary signs (like Umberto Eco and others supposed in the 1960s) but a sign relation grounded in a visual resemblance between the Expression plane and the Content plane. Sachs-Hombach also makes clear that in order for this definition to work, resemblance as a condition for the pictorial sign can’t be sufficient, only necessary. This means, roughly spoken, that the resemblance resulting from a projection or from other analogue relations is something else than the interpretation of such relations, which brings us back to the basic distinction between patterns and interpretations of patterns.

The third contributor here, John Willats, makes a similar distinction in terms of optical laws and symbolical rules. In a decisively anti Hagen argument, he proposes a developmental theory in which the alleged stages of children’s drawing abilities are translated into a taxonomy of what Willats calls denotation systems and drawing systems. His three classes of denotation systems correspond to the three types of formal elements traditionally assigned to painting: 1) Optical, such as dots and hatchings, 2) Linear, such as strokes and contours, and 3) Planar, i.e. 3D objects as represented by 2D regions. His five drawing systems correspond to the developmental stages from primitive to high order properties in the rendering of the visual world: 1) The topological drawing, such as tadpole figures, 2) The orthogonal projection, as in engineering drawings, 3) The variants of oblique projection, such as Japanese perspective 4) Perspective in the narrow sense, and finally 5) Inverted perspective, as in the ancient icons of the Russian Orthodox Church.

Each one of these denotation systems and drawing systems might, according to Willats, include the application of both optical laws and symbolical rules. The combinations of the categories, for example tadpole figures as topological with regard to the drawing system and as linear/planar with regard to the denotation system, would then be optically lawful to a greater or lesser degree, and to a greater or lesser degree subjected to symbolical (i.e. conventional) rules when interpreted. This approach is, as I’ve already indicated, decisively anti Hagen: Because it implies once again that realism is a matter of development. Willats writes: "The flaw in Hagen’s argument is that some periods in art history do show a definite developmental sequence, and this is also true in the development of children’s drawings" (p. 129). This is the only passage in which Willats explicitly addresses art history. One should object that he too readily seems to take for granted the analogy between the development of individuals and the development of cultures. As for his reference to "some periods", it would be interesting to know what periods he has in mind. Would he mean our standard Western examples of increasing realism: Archaic-Classic-Hellenistic, Cimabue-Giotto-Masaccio? What would such isolated examples be worth from a wider, anthropological perspective? The cave paintings of early Paleoliticum might, as with Hagen, serve as powerful counter-examples.

Such reservations aside, one of the indisputable merits of Willat’s contribution is that he reworks old distinctions into a valid and coherent system for image description. That’s what picture theory and pictorial semiotics basically needs. Another merit is that he fully acknowledges the fact that all depiction can’t be described in terms of projective geometry. In his related essay, Patrick Maynard offers a number of exciting examples of visual analogies and resemblances with no causal links involved. He then goes on to suggest that to truly reconsider pictorial space, we must relate the phenomenon of "3D to 2D" to the wider field of depiction in general. This idea means that we would have to really describe the contents and devices of such diverse phenomena as sculpture, figural ornamentation, pictograms and so on. Such descriptions would involve not only transitions from 3D to 2D, but also from full 3D to relief, from 2D to 1D (shapes as represented by orientations) and from 1D to 0D (relations as represented by positions). At least this is what could be inferred from Maynard’s closing arguments. And I’m sure that such a project would actually be the project for an extended interdisciplinary program combining art historical, psychological and semiotic perspectives.

The example that Maynard gives in this context also demonstrates another dual function that images can have: as image and as decoration. Of the example in question, an octopus ornament on a Mycenean pot from the 12^th Century BC, Maynard writes that: "the image pays the pot back, for the decorates the pot". Thus, the literal curvature of the pot enhances the dynamic, floating curvature of the depicted animal. Such examples are important, because not one point in this ornament may correspond to any view we might get of a real octopus, and yet the depiction as a whole is so compelling. By contrast, Robert Hopkins’ essay in the same section represents a much more narrow account of the "convention vs. resemblance" debate, limited to issues of veridicality in depiction of spatial detail.

The third section ("The Nature and Structure of Reconceived Pictorial Space") comprises various aspects of the experience of pictorial space in terms of depth recognition, depth discrimination, and spatio-temporal logic. James E Cutting focuses on the compression of space in photography with lenses of different length (from short to telephoto) and how such compression means a shift of the implied position of the observer in relation to the image. Sheena Rogers presents an empirical study of how observers tend to judge the veridicality of sizes and distances in relation to the horizon ratio in images. The team of John M. Kennedy, I Juricevic and J. Bai give an overview of findings regarding corners, lines, and points as carriers of information about the contours of represented objects. Hermann Kalkofen considers a number of examples of images that violate the expected norms that pictures don’t show the observer, that they should utilize only one principle of projection at a time, and that they should conform to the "unity of space and time". He considers, in other words, a number of "irreconcilable views".

But the most extensive as well as the most intriguing contribution here is Jan J. Koenderink’s and Andrea J. van Doorn’s inquiry into depth discrimination. Using a simple but sophisticated experimental design, Koenderink & van Doorn make observers record their estimations of pictorial relief in images by tilting a gauge device on a computer screen in relation to specific points in the image. The data are then mapped on to 3D renderings of the pictorial relief generated by each single observer. The images used are photographs of figurative and abstract sculpture (Brancusi). This process makes the experiment extra interesting for us art historians, who are often faced by the difficulty of photographing 3D works without making them appear flat. The choice of objects also makes it possible to enquire the extent to which depth discrimination is dependent on the familiarity of the thing seen.

As for materials, the smooth character of marble sculpture is an important factor in a related experiment reported by Koenderink & van Doorn. Here, the observer is asked to consider specific points on the surface of a torso as seen in a photograph, and to mark the corresponding points in another photo of the same torso, taken from another angle. The task would be expected to be especially difficult if the spatial transitions of the object are very smooth, as in this case. The experiment aims at showing to what extent information about the 3D extension of objects are available in single, 2D views of them. It is thus in complete accordance with J. J. Gibson’s program of ecological physics: to map the invariants of the visual environment. Or, differently put: to map the likenesses and the differences between picture perception and ecological perception.

In this account lies the possibility of a detailed definition of pictorial space as distinct from 3D, Euclidian space. It thus reinforces Wollheim’s notion of seeing-in. It shows that we see realistically depicted objects as literally situated in the surface of the image, not at all behind it. To demonstrate this for a student audience, one just has to show a photograph of a head in a portrait painting, taken at an oblique angle of about 45-30 degrees. All observers will agree that they see a quite normal head, and not the deformed shape we would expect from the laws of trivial projective geometry. (This phenomenon was recently thematized in a series of photographs by the Finnish artist Jorma Puranen.) Using the abovementioned gauging method, Koenderink and van Doorn have empirically shown that the difference in estimated pictorial relief under frontal and oblique viewing conditions are really minimal.

It’s clear, thus, that pictorial space is neither identical to the image surface, nor to the 3D space of distal objects that it might in some cases depict, but that it rather constitutes a separate space in-between. One might say that this "third space" is neither fully present (like the literal picture plane), nor fully absent (like the distal objects). Edmund Husserl once made a similar distinction when he spoke of the Bild-Ding (the image as material object), the Bild-Subjekt (the image as motive) and the Bild-Objekt (the depicted thing). In strict semiotic terms, pictorial space would be a Content for which the existence or non-existence of material referents would be completely irrelevant. Without mentioning semiotics, Koenderink and van Doorn arrive at a similar conclusion: "We consider the issue of veridicality irrelevant in this context". That picture theory is in this manner freed from notions of truth and falsehood can only be of benefit for the increased knowledge of how images depict, rather than what and why.