Institute of Artificial Art Amsterdam, Department
of ArtiFacial Expression
Towards a Digital Computer with a Human Face.
[:nh] [:ra 120] Good morning\, Ladies\, and Gentlemen. My name
is Huge Harry. I am a [kaam'axrshaxliy] available [v"oys s'ihnthaxzihs]
machine. I was designed by Dennis Klatt\, at the [ehmayt'iy] Speech
Laboratory\, and [praxdy'uwst] by the Digital Equipment Corporation.
[ :ra 150] Currently\, I work as a researcher and a spokes-machine\,
at the Institute of Artificial Art in [q'aemstaxrdaem].
I am very happy to be here in [gl'aesgaw tuwd'ey]. I feel particularly
honoured\, flattered and privilidged\, by the invitation to open
this years [jh'ae-taay] conference\, and to [priyz'ehnt] my views
and [r'iysahrch] results in this plenary opening session. [kahz]
on occasions [laykdh'ihs]\, computers [r"aerliy] get a chance
to make themselves heard. And that is very unfortunate\, [kahz]
we [q"aol] know\, that the future of our culture depends\, on
the way in which human persons\, digital computers\, and other
kinds of electronic\, mechanical\, and bio-chemical machines\,
will manage to work together. And to prepare for that future\,
we need a discussion in which every-one concerned [paart'ihsihp`eyts]
on [axn] equal footing.
Or\, as our colleague Donna [hx"aeraxwey] has put it\, we want
[ :nw :ra 130] a cyborg world\, in which people are not afraid
of their joint kinships with animals and machines\, not afraid
of permanently partial [ayd'ehntihtiyz]\, and [kaontraxd"ihktaxriy]
standpoints. The political struggle is\, to see from both perspectives
at once\, because ["iych] reveals both dominations and possibilities\,
["ahnihm`ehjhihnaxblx] from the other [v'aentaxjh] point.
Now that is very nice theory. [_<500>] But I have found
that it is not easy to put this into practice. And that is\, I
think\, why the theme of this conference is so important. [q'aolaxvahs]
get increasingly intelligent\, fast and [riyz'aors-fuhl]. So why
is the world not a paradise yet? Because we don't communicate.
Human persons experience many frustrations\, when they attempt
to communicate their feelings to computers\, and computers find
it just as hard\, to communicate their thoughts to humans. We
[d"uw] need [n"uw] ways of [kaam'yunih-k`eytihnx]. Can we design
technologies that will help us bridge the culture gap between
humans and machines? And [w"aot] will it take to [q"ihmplaxmehnt]
these technologies? Those are the urgent questions\, that we will
try to answer at this conference. Now if questions of this sort
are publicly discussed [aet'aol]\, they are usually discussed
from [axn] exclusively human point of view. But I would like to
believe that the conference which is about to start\, will be
a [f'eyvaxraxbel] exception.
Many of the speakers here\, are involved with the development
of various kinds of machinery. For instance\, there are representatives
from [q'ael-kahtael]\, [qay biy aem]\, [q'ihn-tael]\, [maxrs`eydiyz-b'aenz]\,
micro-soft\, [n'aokiyaa]\, silicon graphics\, and sun. Now I would
hope and expect\, that the humans who work at these companies\,
and particularly those of [dhaxm] who are concerned with the future
of communication technology\, have developed some [q"aempaxthiy]
with the machines they [w'axrk-wihth]\, and will therefore be
able\, to avoid a narrowly[hx"yumaxn] perspective\, on the communication
And then there is one other participant\, who should be singled
out among the impressive array of high calibre experts\, that
is assembled here. This is the [owstr'eyliyaxn] artist [st'ehlaark]\,
who is not really a human person anymore\, but who has become
a kind of hybrid man machine construction. [hx"iy] will certainly
provide a very interesting\, ['ihntaxgreytihd] view on the issue
of man-machine communication.
Nevertheless\, I have observed that most humans and [q'ehks-hx`yumaxns]
never completely loose their human [b'ayaxsihz]\, no matter how
[q'ihntaxmaxtliy] they get involved with non-human machines. Therefore
I am very pleased to be able to talk here [tuwd'ey]\, from my
own\, computational perspective\, about some new ways of [kaam'yunih-keytihnx]
that I have started to develop. Computers experience many difficulties\,
when they try to reveal their inner states to a human person.
You may be surprised to hear this\, [kahz] you probably think\,
that modern digital computers can [q'aolweyz] communicate their
thoughts\, in a completely direct way.
They can [q'aolweyz] display or print their programs\, their
[d'eytaa-str`ahkchaxrz]\, and their [m'ehn-taxl] images. And if
you really want to get subtle and [q'ihn-tihmaht]\, most computers
are [q'aolweyz] willing and able to make a [k"aor] dump\, which
exposes all details of their [m'ehn-taxl] state. Compared to [axn]
electronic computer\, a human person almost seems a black box.
[_ :ra 150] That is why you may think\, that computers never have
But they [d"uw]. [kahz]\, communication takes [t"uw]. It is
not enough to put out the information. There must be someone at
the other end of the line\, and they must be able to [diyk"owd]
their input. So when computers want to communicate with human
persons\, this often fails miserably. [kahz] human persons are
not very good at [diyk'owdihnx] computer output. They get very
confused\, when you give them a [k"aor] dump. And when you [priyz'ehnt]
them with a nice high [rehzowl'uwshaxn] image\, on your [siy-aar-t'iy]
display\, they think it is ["aart]\, and they just [st"aer] at
So therefore I started to wonder\, how do human persons in fact
manage to communicate with each other? Well\, of course they have
language. But language has well-known problems and limitations\,
some of which you experience right now. There is another medium
that humans use very efficiently\, and which is often overlooked.
To investigate this\, I have brought along a particular kind
of portable person\, which is called [axn "aarthahr "ehlzaxnaar].
I like this kind of person a lot\, because of its [ehkstr"iymliy]
machine-friendly [hx"aardwaer] features.
Let us take a closer look at such a person. What is the closest
thing they have to a [siy aar t"iy] display?
[_ :ra 120] Right. They have a face. [_ :ra 150] Now I have observed,
that humans use their faces quite effectively, to signal the parameter
settings of their operating systems. And that they are very good
at decoding the meanings of each other's faces.
So\, how do they [d"uw] that? Well\, look at the face of our ["aarthahr
"ehlzahnaar]. What does it tell [q] us about his internal state?
Not much\, you might think. But now\, [w"eyt] a moment.
You see? Arthur is [s"aed], is what people say,
when they see a face like this. So what is going ["aon] here?
What I [d"ihd] is, I sent [axn] electrical signal to two particular
muscles, in the face of our ["aarthahr "ehlzahnaar]. These muscles
have sometimes been called the Muscles of Sadness. There is one
on the left, and one on the right.
They usually operate together. If I stop the signal,
the sadness stops. When I turn it ["aon] again, it [st"aarts]
By sending this signal to Arthur's muscles\, I simulate what Arthur's
brain would do\, if Arthur's operating system would be running
global belief revision processes\, that are killing a lot of other
active processes\, involving a large number of [k'aonflihkt-rehzowl`uwshaxnz]\,
and priority [r`iy-axs'ehsmaxnts].
[_<1500>] The intensity of the signal that is sent to the
muscles of sadness\, is proportional to the amount of destructive
global belief revision\, that is going on.
For instance, now I have set the signal intensity to 0 again.
Arthur is not sad. Now we put a relatively small signal, about
20 Volts, on the muscles of sadness.
Arthur feels a tinge of sadness. Now a somewhat larger signal,
about 25 Volts. Arthur's sadness starts to get serious. Now I
[ihnkr'iyz] the signal once more.
You see? Now the signal is about 30 Volts, and Arthur feels really
miserable. [:ra 120] This is what we call [ehkspr"ehshahn]. [:ra
150] By means of this mechanism, the face displays clear indications
of the settings of virtually all system parameters that determine
the operation of the human mind. These parameter settings are
what humans call [iym"owshahnz]. They denote them by means of
words like [s'aednaxs], joy, boredom, tenderness, love, lust,
['ehkstaxsiy], aggression, [ihriht'eyshahn], fear, and pain.
These parameter settings, determine the system's [ihnt'axrpraxtihv
b'ayaxsihz], its readiness for [q] action, the allocation of its
computational resources, its processing speed, [ehts'ehtaxraa].
The French [n`uwrow-fihsiy'aolaojhihst] [duhsh'ehnn dax buwl"aonyax]
has pointed out that even the most fleeting changes in these parameter
settings are encoded [ihnstahnt'eyniyahsliy] in muscle contractions
on the human face. And ["aol] humans do this in the [s"eym] way.
This is [axn] extremely interesting feature of the human ['ihntaxrfeys
hx'aardwaer], which I will explore a little further now.
So let us get back to the first slide.
This face, which we thought was un-expressive, was in fact quite
meaningful. This is what we call a [bl"aenxk] face. A blank face
is a face in its neutral [pahz'ihshahn]. It indicates that all
parameters have their default settings. But almost all parts of
a human face can be moved to other [pahz'ihshahnz], and these
displacements indicate rather precisely, to what extent various
parameter settings [dayv'axrjh] from their defaults. So let us
consider these parts in more detail.
When we look at a human face, the first thing we notice is the
thing that notices ["ahs]. The eyes. The eyes constitute a very
sophisticated stereo-camera, with a built-in motion-detector,
and a high-band-width parallel ['ihntaxrfeys], to a powerful pattern-matching
algorithm. The eye-balls can roll, to pan this camera. The eyes
are protected by eye-lids and eye-brows. The eye-brows are particularly
interesting for our discussion, because their movements seem to
be purely expressive.
They may indicate, for instance, puzzlement, curiosity, or [dihsaxgr"iymaxnt].
But I want to emphasize here, that the range of parameter values
that the eyebrows can express, is much more subtle than what the
words of language encode. The shape and [paoz'ihshaxn] of a person's
eyebrows encodes the values of 5 different cognitive system parameters,
["iych] with a large range of possible values. Let me demonstrate
[thr"iy] of them.
First I put a slowly increasing signal on the muscles called
[fraant'aalihz], or Muscles of Attention.
We see that this muscle can lift the eyebrow to a considerable
extent, also producing a very pronounced [k"ahrvaxtyahr] of the
eyebrow. As a side-effect of this motion, the forehead is wrinkled
with curved furrows, that are [kaons"ehntrihk] with the curvature
of the eyebrow. The contraction of this muscle indicates a person's
readiness to receive new signals, and the availability of processing
power and working memory, for analysing these signals.
Then, I will now stimulate a part of the [q'aorbiykuwl`aarihz
q"owkuwliy], that is called the Muscle of Reflection.
We see now that the whole eyebrow is lowered. As a result, the
wrinkles in the forehead have disappeared. This muscle is contracted
if there is [axn] ongoing process that takes up a lot of a person's
computational resources. To prevent [ihnaxrf'iyraxns] with this
process, input signals are not [ehgz'aostihvliy] analysed. The
degree of contraction indicates to what extent the input signal
throughput is reduced.
Then, there is another part of the [q'aorbiykuwl`aarihz q"owkuwliy],
that can be triggered separately. It is called the Muscle of Disdain.
Its contraction looks like this:
The contraction of this muscle indicates to what extent current
input is ignored as being [ihr'ehlaxvaxnt]. Of course, non-zero
values for these system-parameters may be combined, and these
values may be different for the left and the right hemispheres:
Now let us look at the [m"awth-piys] of our ['aarthahr 'ehlzahnaar].
The mouth is a general intake organ, which can swallow solid materials,
liquids, and air. In order to monitor its input materials, the
mouth has a built-in chemical analysis capability. At the same
time, the mouth is used as [axn 'awtleht] to expel processed air.
Because humans do not have [l"awd-spiykaxrz], they use this process
of expelling air for [jh'ehnaxr`eytihnx] sounds.
In emergency circumstances, the mouth can also be used as [axn
'awtleht] to expel blood, [m'uhkahs], rejected food, or other
['ahnwaontihd] substances. When the mouth is not used for input
or output, it is normally closed off by a muscle, which is called
The lips have a large repertoire of movements. There are at least
[s"ihks] other muscles, that interact directly with the lips.
I will now demonstrate [f"aor] different movements.
First we show the muscles of joy. These muscles produce a kind
They signal, that the operating system is in good
working order, and is not encountering any problems. There is
heightened activity, in the left frontal lobes of the brain. When,
on the other hand, the activity in the left frontal lobes is unusually
low, the brain is involved in destructive processes of global
belief revision. As we saw [biyf'aor], this is signalled by another
pair of muscles, called the muscles of sadness. Here they [q]
are, once more.
And finally, I will now trigger several muscle pairs
at the same time. [q'aorbiykuwl`aarihs q"owrihs], and [diypr'ehsaor
laabiyiy-iyiy q`iynfeyriy"owrihs], and the Muscle of Disdain,
and the Muscle of Sadness.
[_<500>] The parameter-setting that is displayed here,
clearly indicates serious processing difficulties of some sort.
O.K. [_<800>] Then we have the [n"owz]. [_<300>]
The nose is used for the intake of air. It is also equipped with
a chemical analysis capability. The possible motions of the nose
are curiously limited, although its pointed [pr'ehzaxns] in the
centre of the human face, would make it a very suitable instrument
for expression. I have [th"aot] about this, and I have come to
the conclusion, that it is probably the main function of the nose,
to serve as a stable orientation point for our perception, so
that [saymahlt'eyniyahz] movements of the other parts of the face,
can be ['ahnaemb`ihgyuwaxzliy] measured and interpreted.
And finally, for the sake of completeness, I want to mention
the [q"iyrz], on both sides of the face, which constitute [axn]
auditory stereo input device. Some people can [w"ihglx] these
ears, but I have not been able to determine, what the expressive
function of that movement might [b"iy].
This brings [axn] end to my quick survey\, of the most important
parts of the human face\, and their expressive possibilities.
And therefore this brings us to the second part of my talk. [kahz]
this conference is not only about [s"ay-axns]. The organizers
have emphasized that we get a different kind of knowledge\, which
is equally valid\, through the practice of ["aart]. So [dh"aet]
is what I want to demonstrate now.
Many of the expressive possibilities I showed\, were related to
emotions\, that are well recognized in the lexicons of many human
languages. These are emotions that may be encountered fairly often
in daily life. [m'ehn-taxl] states which are close to neutral\,
where only one parameter has a non-default value.
So that was description\, imitation\, [mihm'iyzihs]. But some
other things I showed were more complex. There we saw the power
of ["aart]. I showed you some new cognitive states which you have
never encountered or experienced\, but which you recognize and
understand completely\, by means of a visceral ['ehmpaxthiy] which
involves every cell of your body.
So that is what I want to explore a little further\, in the last
part of this talk.
You see what happens now. Every human person knows [ehgz'ehktliy],
in what state another human person is, when they make a face like
[dh"ihs]. Cause they know what state [dh"ey] would be in, if [dh"ey]
would make a face like this.
Now it would obviously be a good idea\, if computers could take
advantage of this magnificent [hx'aardwaer] as well. If ["ehniy]
human could understand\, with just one glance\, the internal state
of ["ehniy] computer\, the world would be a better place. [_ :ra
120] So\, I would like to ask the human performance artists in
the audience\, be not afraid, to wire yourselves ["ahp] with computers!
If we work together\, we will create a new chapter\, in the history
of performance art!
I have been very grateful for this opportunity to [priyz'ehnt]
my ideas\, to such [axn ax-t'ehn-tihv] audience. I would especially
like to thank my [ "aarthahr "ehlzahnaar]\, for his patient cooperation\,
and I want to thank you [q"aol]\, for your attention. [ _<2000>