Remko Scha

Institute of Artificial Art, Amsterdam

Artificial Art

Computation and Cognition.

Could the digital computer have an essential influence on the art of the future, and be more than just another new technical tool? It is hardly possible to raise this question without thinking rightaway about the fantastic promises of cognitive science and artificial intelligence. These new branches of science and technology are committed to the development of precise mathematical models of human cognition -- models which are inspired by the digital computer and which, in their turn, can be implemented on such a computer. These promises, taken at face value, suggest the ultimate possibility of "artificial artists": completely automatic computer art, produced by carefully calibrated simulations of human emotion and human intelligence, equipped with extensive databases about for instance everyday life, international politics, art history, and the properties of paint.

Whether this would in principle be possible is a question which often gives rise to passionate philosophical discussions. More interesting, however, is the question about the practical feasibility of this idea: is the state of technology such that it makes sense to start to work on this? To set up, for instance, a research program spanning a few decades in order to develop the various algorithms and data bases that together will constitute the artificial artist? This practical question must be answered negatively. Artificial Intelligence research has failed to produce plausible models of human cognition so far, and there is no reason to expect that it will become much more successful very soon. The complexity of existing computational models can be gauged by looking at the "expert systems" which have recently become popular: relatively simple algorithms with a limited, precisely demarcated task. The amount of knowledge that a human expert can bring to bear on a problem, and the versatility that such an expert displays in applying this knowledge, are gigantic when we compare them to the capacities of current "expert systems".

For the time being, the project of emulating human cognition on digital computers is frought with fundamental problems. The results of A.I. research suggest that there are two kinds of cognitive processes, which differ in essential respects. On the one hand there are perceptual processes, which aim at the classification of sensory data; on the other hand there are the formal symbol manipulations that we employ in arithmetic, board games, legal reasoning, and other formally defined situations. The data structures and algorithms that are required to perform these two kinds of cognitive processes, turn out to be very different. Today's digital computer is primarily equipped for symbol manipulation; if one wants to implement, on this kind of machine, the associative thought processes that constitute perception, one must spell out these processes in an extremely cumbersome way. For human cognition, it is the other way around: humans are capable of both kinds of cognition as well, but in this case the perceptual processes are the more basic ones.

The most important challenge of A.I. research may be to implement, on digital computers, the human cognitive processes for which that kind of hardware is not optimally suitable. So far, this hasn't been very successful. A.I. rules in the areas where symbol manipulation plays a crucial role; but the computer's perceptual capabilities remain extremely meager. We may predict, for instance, with reasonable certainty that when the world champion chess will be a computer program, there will not yet be a program that can reliably distinguish twenty human faces, if lighting and camera angle are allowed to vary a little.

For this reason, emulating the decision processes of the human artist is not a realistic goal -- and that is not because artistic processes are necessarily more difficult to mimic than other mental processes, but because the existing models of all cognitive processes are inadequate. Even if  'creativity' would be a completely mythical notion, and genuine conceptual innovation were a negligible element in artistic practice -- even then, the artificial artist is not in the offing, and neither are the artificial engineer, the artificial pastry-cook or the artificial kindergarten pupil.

This conclusion may not seem very surprising. It resonates with a popular prejudice: that the artistic role of the computer should obviously be limited to that of a technical tool, because the inspiration that underlies all true art happens to be the prerogative of the human artist. Even artists who largely rely on the computer to design their work, and who are inspired by the computer's possibilities and limitations, often pride themselves on their creativity, and view the computer as "only a tool, like the sculptor's chisel". This received opinion, however, underestimates the unique new opportunities that the computer offers.

In certain kinds of cognitive skills, computers outperform people consistently by a large margin: for instance in making extensive calculations, in systematically checking long lists of possibilities, in manipulating intricate algebraic formulas, and in drawing random samples from large sets of items generated by complex definitions. These capabilities are not enough to emulate human thinking, but their implementation embues the computer with an independent, inhuman style of thinking, which is interesting because it gives rise to unforeseeable results. This becomes apparent if we take a closer look at the successes of Artificial Intelligence. These are mostly due to programs that solve limited, formally defined problems, which do not require the perceptual capabilities that are only possessed by higher animals. Examples are programs that play simple games, and expert systems for well-defined design problems, such as the lay-out of integrated electronic circuits. A brief glance at the behavior of such programs yields an interesting insight: they employ a rather different procedure than the human expert who solves the same problem. The computer program finds the best solution by systematically investigating all possibilities. The human expert, who is unable to face the full richness of the whole search space, thinks in terms of a limited repertoire of prototypes and conventions, and explores minimal variations on these if necessary.

The procedural differences also lead to different results. In dealing with a technical problem which allows different solutions that can be compared as being more or less optimal, the computer program will often come up with the better results, because it has looked more systematically at the different possibilities; that is why the electronics industry employs computers to design integrated circuits. Computer-generated solutions differ in interesting ways from human designs: they are often surprising and unconventional. When a computer program plays a demonstrably correct and winning chess end-game, it may very well seem bizarre and incomprehensible to a human observer. In such a case, the program shows that there are many possibilities which the chess game allows, but which the human perspective on chess overlooks.

If we do not care about emulating human behavior, but are interested in unexpected and unconventional results, the computer program may thus be superior. And art production is an obvious case in point. Though many artists have a hang-up about originality, they often produce works which look very much like those of their role-models and colleagues. We therefore propose a more analytic approach to art production: first define a mathematical system which charts all formal and perceptual possibilities as thoroughly as possible, and then implement computer programs which generate random images on the basis of this system. This approach may eventually yield more interesting results then the necessarily more limited method of the traditional individual artist.


Chance Art

To generate artworks by means of algorithms that make random choices from a mathematically defined class of possibilities, is not a new idea. There is a tradition known as "chance art", which aims at the production of art works which are not determined by the artist -- arbitrary artworks, random samples from the space of all possibilities. The attitude behind this resembles the stance of artists who exhibit nothing or who declare the whole world to be an artwork: one avoids choices and positions, one throws the observer back on his inalienable responsibility for his own esthetic experiences, and draws attention to the esthetic interpretability of everything.

So far, the project of making 'arbitrary' paintings has only been realized to a limited extent. In a way, painters like Picabia and Polke come closest, when they mingle intuively selected elements from various domains of life and art. But chance art based on clear, hard mathematical probabilities involves a paradox: more explicitly than any other genre, it displays the conceptual context that frames it. When, for instance, graphic designs are defined as grids of pixels that may be either black or white, drawing a random sample tends to yield a grey-looking plane (Morellet). One gets somewhat more variation by putting a random number of dots with a random size at random locations on the plane (De Vries), or by drawing a random number of lines with a random thickness through random points in random directions (Mandelbrot).

Constructions for making random shapes can also be more complex. For instance, Lévy's construction for random mountainscapes, as described by Mandelbrot: superimposing infinitely many infinitely small 'edges' which run across the plane through random points in random directions. (An 'edge' is a step-function on the plane: on one side of a straight line, the plane is elevated a certain amount.) Cutting a horizontal plane through such a random mountainscape, yields a 'map' with random coastlines.

To conclude this quick enumeration of elementary principles, I want to point to the phenomenon of Lissajous patterns, where a shape is created by multiplying different harmonic oscillations. More complex patterns arise when more complex movements are multiplied (cf. 'Machine Drawings').

So far, chance artists have been content with applying probabilistic operations within such simple systems. That is sufficient if all one wants to do is put forward the very idea of chance. But to really take on the project of the arbitrary painting, we need more; we need a formal language which allows us to assign distinct codes to perceptually different paintings, but also to assign the same code to perceptually equivalent paintings whose details may nevertheless differ considerably (as in the case of the different instantiations of Morellet's random pixels). Such a language is an algebra: it specifies a set of elementary shapes and a set of operations which map shapes onto other, possibly more complex shapes.

Algebras like this have been developed already for characterising specific styles. Harold Cohen, for instance, embued his drawing program AARON with an original style reminiscent of the COBRA painters. Programs which try to mimic existing artists have also been developed, for instance for Miró and Diebenkorn. The 'arbitrary painting' project, however, requires a system with a much richer repertoire of stylistic possibilities, and with the capability to exploit those possibilities in a very flexible way -- so that the degree of stylistic coherence within a painting (or within an exhibition) is itself a parameter whose value can be chosen at random.

From a completely different perspective, the psychology of Gestalt perception has also developed some coding languages which are relevant for our purpose -- for instance, in the work by Leeuwenberg and Buffart in Nijmegen on the mental representation of drawings built up out of straight line segments, and in the work by Lerdahl and Jackendoff in Boston on the perception of music.

It is clear that the machinery needed for the 'arbitrary painting' project would have to be substantially richer and more complex than any of these currently existing systems. New coding languages are needed to begin to conceptualize the range of possible images in a halfway adequate fashion. And drawing random samples from the space defined by such a language, and calculating the detailed execution of the various operations, will not be possible without a computational implementation.


Esthetic Implications

Art is often viewed as a means of communication, employed by high-minded individuals to convey profound ideas. In this view it is self-evident that artworks are designed, made, assembled, or at the very least chosen by artists: the artist is responsible for the artwork; the validity of the artwork is grounded in the artist's sense of life.

The actual production and reception of art in our society is a very different matter, however: works of art constitute material for an unconstrained process of esthetic reflection and interpretation, which is not subject to any rules. In Marcel Duchamp's words: 'the spectator makes the picture'. As a consequence, anything whatsoever can serve as input for the esthetic reflection process; the special status of the artwork is abolished.

Another point to note, is that the esthetic experience is in fact not a particular kind of extraordinary psychological state, but an ingredient of almost all experience --whenever objects, situations and processes which are not elements of a predefined code, are nevertheless employed by our mind as symbols for other (usually vaguer and more complex, perhaps intrinsically more important) things, circumstances and developments. Every object, every situation, every process thus has an infinite potential of meanings.

One of the first unequivocal articulations of this idea about esthetics is provided by Marcel Duchamp's 'readymades' -- simple objects from the real world, which he exhibited as works of art. Duchamp's gesture is sometimes interpreted as a celebration of the sublime autonomous creative power of the artist's Kunstwollen, but I would like to argue for a different interpretation. It is true that Duchamp chose his objects quite judiciously, but one should not be mistaken about the nature of his circumspection. Duchamp has made explicit statements about this, but one can also read it off the objects themselves. They are very ordinary, 'neutral' objects: schoolbook, coat-rack, hat-rack, bicycle-wheel, bottle-rack, snow-shovel, plastic bucket, coffee grinder, typewriter-cover: standard objects, drawing lesson examples.

Duchamp's own words: "It is very difficult to choose an object, because after a few weeks you start to like it or to hate it. One must approach a thing with indifference, as if one has no esthetic emotion. The choice of readymades is always based on visual indifference and, at the same time, on the complete absence of good or bad taste." Like the chairs and tables which always represent 'the object' in philosophical discussions, Duchamp's readymades are 'free variables', schemas that all other objects can substitute for, lacking specific properties which would block unification. [The preponderance of racks and containers might be taken to symbolize exactly this 'non-property'.]

Duchamp's readymades imply the esthetic interpretation of everything. They imply the abolishment of an esthetics in the traditional sense -- of an esthetics which assigns a different value to different things and experiences -- that appreciates paintings by brilliant artists more highly than children's drawings or technical diagrams -- which looks differently at bronze sculptures than at supermarkets or natural phenomena. They imply that esthetic perception is viewed as a cognitive process that is not tied to the art-context -- a process that has its origin and its justification in the observer, and that can be applied to arbitrary material.

The readymade implies that art history is finished: the artwork looses its special status, and art history its incentive. But at the same time it suggests a new beginning: to practise the esthetic interpretation of everything. Important movements in twentieth-century art betray an awareness of this: surrealism, nouveau réalisme, pop art, postmodernism all put forth some version of this idea. But they also take it back rightaway, by defining themselves as a style, with a focused interest in certain aspects of reality, and sometimes with preconceived ideas about how that reality is to be interpreted.

Other artists have thematized the esthetic interpretability of the real world in an explicit, somewhat humorous fashion: a socle with the whole world on it, a signature on a glass pane, paintings with the word "everything". Or, complementarily, they chose the uselessness and superfluousness of art as their topic: paintings with the word "nothing", mirrors, empty canvases, empty frames, empty rooms.

The project of artificial art shows that the awareness of the esthetic interpretability of everything is indeed a new beginning -- the beginning of an activity that is related to art, but at the same time clearly distinguished from traditional art practice. The crucial thing is that we do not know what 'everything' is! In our conventional, habitual thinking we tend to content ourselves with what we encounter already in the existing world, and what looks very much like it. The computer, however, makes it possible now to explore the combinatorics of the space that is defined by our full repertoire of visual elements and operations. Just as photography has been used for 150 years now to find out what we can see in the world we live in, the computer will systematically explore the esthetic possibilities of our cognitive faculties.

Chance and Architecture

Chance art faces the meaninglessness of art and life, and draws constructive conclusions from this state of affairs. It may thus be the only adequate reaction to the current impasse in autonomous visual art. But in the applied arts a similar step is necessary. In architecture, for instance, one must deal with the issue that style is unavoidable but that every style is problematic.

Functionalism is impossible. When the functions are fixed, many choices remain. The architect must confess to a personal esthetic choice or to a social/cultural tradition. In designing buildings for the general public or for unknown future users, such a confession is out of place.

And efficiency is not an objective criterion that defines the optimal solution among different functionally equivalent ones. Every efficiency implies an esthetics. At best, this is a stance concerning the true nature of materials and their proper use, and a normative stylization of daily life. In the worst case (efficiency as an 'economic' criterion: "always choose the cheapest solution"), it boils down to the mean esthetics of money. Architecturecannot avoid to symbolize something. Architecture which is economically efficient in the narrowest sense, symbolizes the Margin of Profit as the highest good.

Postmodernism is a neo-victorianism. The eclectic application of various historical styles, intended as a solution for the dilemma sketched above, results in a narrow, fashionably defined, neo-style. The call for a style-transcending meta-style remains unanswered.

The meta-style that is needed here, can be developed in the context of Artificial Art project described above, which should embrace a mathematical, perceptually oriented analysis of all known styles. After an explicit analysis of these styles, all possible syntheses, interpolations and extrapolations of them can be generated at will by means of computer programs.

Conclusion.

Because the production of art is left to the arbitrary impulses and shortsighted ambitions of individual artists, most art is conventional and predictable. Human creativity is often over estimated. Well-designed generative algorithms can yield more surprising results than toiling individual artists.

For human persons it is very difficult to survey all possible combinations of a set of items. For this reason, many structures that are implicitly given in the visual language of our cognitive system have never been witnessed yet. The algorithmic exploration of explicit visual grammars will make these structures visible. I imagine the following division of labour: people define the elements and operations of the visual algebra, and thereby specify an infinite combinatorial space; computer programs draw random samples from this space. Progress by human abstraction and machine systematicity -- just as in science.

The interpretation of the artwork will be decoupled from all anecdotal information about the artist. Image-generation software will be developed through a collective, analytic, un-expressive effort. Art will not revolve around the individual artist any more. A Copernican turn.